Shopify GraphQL Admin API rate limiting cost and sleep time












1















I am trying to consume Shopify GraphQL API for Admin in PHP ( Laravel ).

Rate limiting and throttling works differently in GraphQL api as compared to REST api, its calculated based on the cost of the query.

Cost of Shopify GraphQL query






Few points to keep in mind:




  • Maximum available cost is 1000 for one api call (query).

  • If you have consumed some points from 1000, every second, 50 points will be restored.

  • If you have less points of cost in your bucket, and you make a query of cost higher than that, it will throttle.


The query that I am passing to api has an estimated cost of 502, represented by requestedQueryCost. Whereas, actualQueryCost represents the actual response returned by the api for a specific shop.


In above snapshot, its the worst case scenario, requestedQueryCost is equal to acutalQueryCost for a store with heavy number of orders.


Now, when this query is executed I have consumed 502 points, 498 left, 1 second elapsed, 50 points added = 548 , and I can make a second api call to fetch second page of data. After second api call I will have less points left, so I will have to put sleep for 1 or 2 seconds to gain the points to make api call.


In the case shown in snapshot, i had to put 10 seconds sleep wait in order to restore 500 points to make next api call.

Problem: How to best decide sleep (wait) time for different shops? We don't want all shops to wait for 10 seconds even if they have less query cost.

Note: For code reference, see my answer below.










share|improve this question





























    1















    I am trying to consume Shopify GraphQL API for Admin in PHP ( Laravel ).

    Rate limiting and throttling works differently in GraphQL api as compared to REST api, its calculated based on the cost of the query.

    Cost of Shopify GraphQL query






    Few points to keep in mind:




    • Maximum available cost is 1000 for one api call (query).

    • If you have consumed some points from 1000, every second, 50 points will be restored.

    • If you have less points of cost in your bucket, and you make a query of cost higher than that, it will throttle.


    The query that I am passing to api has an estimated cost of 502, represented by requestedQueryCost. Whereas, actualQueryCost represents the actual response returned by the api for a specific shop.


    In above snapshot, its the worst case scenario, requestedQueryCost is equal to acutalQueryCost for a store with heavy number of orders.


    Now, when this query is executed I have consumed 502 points, 498 left, 1 second elapsed, 50 points added = 548 , and I can make a second api call to fetch second page of data. After second api call I will have less points left, so I will have to put sleep for 1 or 2 seconds to gain the points to make api call.


    In the case shown in snapshot, i had to put 10 seconds sleep wait in order to restore 500 points to make next api call.

    Problem: How to best decide sleep (wait) time for different shops? We don't want all shops to wait for 10 seconds even if they have less query cost.

    Note: For code reference, see my answer below.










    share|improve this question



























      1












      1








      1








      I am trying to consume Shopify GraphQL API for Admin in PHP ( Laravel ).

      Rate limiting and throttling works differently in GraphQL api as compared to REST api, its calculated based on the cost of the query.

      Cost of Shopify GraphQL query






      Few points to keep in mind:




      • Maximum available cost is 1000 for one api call (query).

      • If you have consumed some points from 1000, every second, 50 points will be restored.

      • If you have less points of cost in your bucket, and you make a query of cost higher than that, it will throttle.


      The query that I am passing to api has an estimated cost of 502, represented by requestedQueryCost. Whereas, actualQueryCost represents the actual response returned by the api for a specific shop.


      In above snapshot, its the worst case scenario, requestedQueryCost is equal to acutalQueryCost for a store with heavy number of orders.


      Now, when this query is executed I have consumed 502 points, 498 left, 1 second elapsed, 50 points added = 548 , and I can make a second api call to fetch second page of data. After second api call I will have less points left, so I will have to put sleep for 1 or 2 seconds to gain the points to make api call.


      In the case shown in snapshot, i had to put 10 seconds sleep wait in order to restore 500 points to make next api call.

      Problem: How to best decide sleep (wait) time for different shops? We don't want all shops to wait for 10 seconds even if they have less query cost.

      Note: For code reference, see my answer below.










      share|improve this question
















      I am trying to consume Shopify GraphQL API for Admin in PHP ( Laravel ).

      Rate limiting and throttling works differently in GraphQL api as compared to REST api, its calculated based on the cost of the query.

      Cost of Shopify GraphQL query






      Few points to keep in mind:




      • Maximum available cost is 1000 for one api call (query).

      • If you have consumed some points from 1000, every second, 50 points will be restored.

      • If you have less points of cost in your bucket, and you make a query of cost higher than that, it will throttle.


      The query that I am passing to api has an estimated cost of 502, represented by requestedQueryCost. Whereas, actualQueryCost represents the actual response returned by the api for a specific shop.


      In above snapshot, its the worst case scenario, requestedQueryCost is equal to acutalQueryCost for a store with heavy number of orders.


      Now, when this query is executed I have consumed 502 points, 498 left, 1 second elapsed, 50 points added = 548 , and I can make a second api call to fetch second page of data. After second api call I will have less points left, so I will have to put sleep for 1 or 2 seconds to gain the points to make api call.


      In the case shown in snapshot, i had to put 10 seconds sleep wait in order to restore 500 points to make next api call.

      Problem: How to best decide sleep (wait) time for different shops? We don't want all shops to wait for 10 seconds even if they have less query cost.

      Note: For code reference, see my answer below.







      php laravel graphql shopify shopify-app






      share|improve this question















      share|improve this question













      share|improve this question




      share|improve this question








      edited Nov 27 '18 at 8:09







      awebartisan

















      asked Nov 25 '18 at 16:38









      awebartisanawebartisan

      732715




      732715
























          2 Answers
          2






          active

          oldest

          votes


















          0














          You get a clean slate of cost calls per shop. If one shop is at zero, you may still have 1000 waiting for you on another shop. You should ensure your calling mechanism is clear on that! Only sleep a thread per shop. You should be able to assign a request to a thread, so that if it sleeps you are still operating with other threads. I would be laughing my ass off if PHP operated with one thread assigned to all requests. That would be so 1982 computing!






          share|improve this answer
























          • With PHP-FPM and Nginx, its new process for each request.

            – awebartisan
            Nov 25 '18 at 18:24






          • 1





            Perfect, so then you have no worries. Sleeping will not affect all stores, but only that thread.

            – David Lazar
            Nov 25 '18 at 18:54



















          0














          Below is my rough draft of a solution, still looking for expert opinion on how to effectively handle it so that each Shop has to wait for the time they deserve, according to the amount of data. Please advise.



          public function getRequiredOrders()
          {
          $firstRequestTimeStamp = now();
          $ordersGraph = $this->shop->api()->graph($this->firstQuery())->body->orders;
          $this->transform($ordersGraph); //transforming to required format

          $previousRequestTimeStamp = $firstRequestTimeStamp;

          while($ordersGraph->pageInfo->hasNextPage) {

          $nextRequestTimeStamp = now();
          $timeElapsed = $nextRequestTimeStamp->diffInSeconds($previousRequestTimeStamp);
          $restoredPoints = $timeElapsed * 50; //50 points are restored every 1 second
          $pointsLeft = $this->shop->api()->getApiCalls('graph', 'left');
          $totalPointsLeft = $pointsLeft + $restoredPoints;

          if($totalPointsLeft >=502){ //one must know the maximum cost of their query
          $lastEdgeCursor = end($ordersGraph->edges)->cursor;
          $nextQuery = $this->nextQuery($lastEdgeCursor);
          $previousRequestTimeStamp = $nextRequestTimeStamp;
          $ordersGraph = $this->shop->api()->graph($nextQuery)->body->orders;
          $this->transform($ordersGraph);
          }else{
          sleep(1);
          continue;
          }

          }
          return $this->allOrders;
          }





          share|improve this answer























            Your Answer






            StackExchange.ifUsing("editor", function () {
            StackExchange.using("externalEditor", function () {
            StackExchange.using("snippets", function () {
            StackExchange.snippets.init();
            });
            });
            }, "code-snippets");

            StackExchange.ready(function() {
            var channelOptions = {
            tags: "".split(" "),
            id: "1"
            };
            initTagRenderer("".split(" "), "".split(" "), channelOptions);

            StackExchange.using("externalEditor", function() {
            // Have to fire editor after snippets, if snippets enabled
            if (StackExchange.settings.snippets.snippetsEnabled) {
            StackExchange.using("snippets", function() {
            createEditor();
            });
            }
            else {
            createEditor();
            }
            });

            function createEditor() {
            StackExchange.prepareEditor({
            heartbeatType: 'answer',
            autoActivateHeartbeat: false,
            convertImagesToLinks: true,
            noModals: true,
            showLowRepImageUploadWarning: true,
            reputationToPostImages: 10,
            bindNavPrevention: true,
            postfix: "",
            imageUploader: {
            brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
            contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
            allowUrls: true
            },
            onDemand: true,
            discardSelector: ".discard-answer"
            ,immediatelyShowMarkdownHelp:true
            });


            }
            });














            draft saved

            draft discarded


















            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53469612%2fshopify-graphql-admin-api-rate-limiting-cost-and-sleep-time%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown

























            2 Answers
            2






            active

            oldest

            votes








            2 Answers
            2






            active

            oldest

            votes









            active

            oldest

            votes






            active

            oldest

            votes









            0














            You get a clean slate of cost calls per shop. If one shop is at zero, you may still have 1000 waiting for you on another shop. You should ensure your calling mechanism is clear on that! Only sleep a thread per shop. You should be able to assign a request to a thread, so that if it sleeps you are still operating with other threads. I would be laughing my ass off if PHP operated with one thread assigned to all requests. That would be so 1982 computing!






            share|improve this answer
























            • With PHP-FPM and Nginx, its new process for each request.

              – awebartisan
              Nov 25 '18 at 18:24






            • 1





              Perfect, so then you have no worries. Sleeping will not affect all stores, but only that thread.

              – David Lazar
              Nov 25 '18 at 18:54
















            0














            You get a clean slate of cost calls per shop. If one shop is at zero, you may still have 1000 waiting for you on another shop. You should ensure your calling mechanism is clear on that! Only sleep a thread per shop. You should be able to assign a request to a thread, so that if it sleeps you are still operating with other threads. I would be laughing my ass off if PHP operated with one thread assigned to all requests. That would be so 1982 computing!






            share|improve this answer
























            • With PHP-FPM and Nginx, its new process for each request.

              – awebartisan
              Nov 25 '18 at 18:24






            • 1





              Perfect, so then you have no worries. Sleeping will not affect all stores, but only that thread.

              – David Lazar
              Nov 25 '18 at 18:54














            0












            0








            0







            You get a clean slate of cost calls per shop. If one shop is at zero, you may still have 1000 waiting for you on another shop. You should ensure your calling mechanism is clear on that! Only sleep a thread per shop. You should be able to assign a request to a thread, so that if it sleeps you are still operating with other threads. I would be laughing my ass off if PHP operated with one thread assigned to all requests. That would be so 1982 computing!






            share|improve this answer













            You get a clean slate of cost calls per shop. If one shop is at zero, you may still have 1000 waiting for you on another shop. You should ensure your calling mechanism is clear on that! Only sleep a thread per shop. You should be able to assign a request to a thread, so that if it sleeps you are still operating with other threads. I would be laughing my ass off if PHP operated with one thread assigned to all requests. That would be so 1982 computing!







            share|improve this answer












            share|improve this answer



            share|improve this answer










            answered Nov 25 '18 at 17:25









            David LazarDavid Lazar

            5,65531426




            5,65531426













            • With PHP-FPM and Nginx, its new process for each request.

              – awebartisan
              Nov 25 '18 at 18:24






            • 1





              Perfect, so then you have no worries. Sleeping will not affect all stores, but only that thread.

              – David Lazar
              Nov 25 '18 at 18:54



















            • With PHP-FPM and Nginx, its new process for each request.

              – awebartisan
              Nov 25 '18 at 18:24






            • 1





              Perfect, so then you have no worries. Sleeping will not affect all stores, but only that thread.

              – David Lazar
              Nov 25 '18 at 18:54

















            With PHP-FPM and Nginx, its new process for each request.

            – awebartisan
            Nov 25 '18 at 18:24





            With PHP-FPM and Nginx, its new process for each request.

            – awebartisan
            Nov 25 '18 at 18:24




            1




            1





            Perfect, so then you have no worries. Sleeping will not affect all stores, but only that thread.

            – David Lazar
            Nov 25 '18 at 18:54





            Perfect, so then you have no worries. Sleeping will not affect all stores, but only that thread.

            – David Lazar
            Nov 25 '18 at 18:54













            0














            Below is my rough draft of a solution, still looking for expert opinion on how to effectively handle it so that each Shop has to wait for the time they deserve, according to the amount of data. Please advise.



            public function getRequiredOrders()
            {
            $firstRequestTimeStamp = now();
            $ordersGraph = $this->shop->api()->graph($this->firstQuery())->body->orders;
            $this->transform($ordersGraph); //transforming to required format

            $previousRequestTimeStamp = $firstRequestTimeStamp;

            while($ordersGraph->pageInfo->hasNextPage) {

            $nextRequestTimeStamp = now();
            $timeElapsed = $nextRequestTimeStamp->diffInSeconds($previousRequestTimeStamp);
            $restoredPoints = $timeElapsed * 50; //50 points are restored every 1 second
            $pointsLeft = $this->shop->api()->getApiCalls('graph', 'left');
            $totalPointsLeft = $pointsLeft + $restoredPoints;

            if($totalPointsLeft >=502){ //one must know the maximum cost of their query
            $lastEdgeCursor = end($ordersGraph->edges)->cursor;
            $nextQuery = $this->nextQuery($lastEdgeCursor);
            $previousRequestTimeStamp = $nextRequestTimeStamp;
            $ordersGraph = $this->shop->api()->graph($nextQuery)->body->orders;
            $this->transform($ordersGraph);
            }else{
            sleep(1);
            continue;
            }

            }
            return $this->allOrders;
            }





            share|improve this answer




























              0














              Below is my rough draft of a solution, still looking for expert opinion on how to effectively handle it so that each Shop has to wait for the time they deserve, according to the amount of data. Please advise.



              public function getRequiredOrders()
              {
              $firstRequestTimeStamp = now();
              $ordersGraph = $this->shop->api()->graph($this->firstQuery())->body->orders;
              $this->transform($ordersGraph); //transforming to required format

              $previousRequestTimeStamp = $firstRequestTimeStamp;

              while($ordersGraph->pageInfo->hasNextPage) {

              $nextRequestTimeStamp = now();
              $timeElapsed = $nextRequestTimeStamp->diffInSeconds($previousRequestTimeStamp);
              $restoredPoints = $timeElapsed * 50; //50 points are restored every 1 second
              $pointsLeft = $this->shop->api()->getApiCalls('graph', 'left');
              $totalPointsLeft = $pointsLeft + $restoredPoints;

              if($totalPointsLeft >=502){ //one must know the maximum cost of their query
              $lastEdgeCursor = end($ordersGraph->edges)->cursor;
              $nextQuery = $this->nextQuery($lastEdgeCursor);
              $previousRequestTimeStamp = $nextRequestTimeStamp;
              $ordersGraph = $this->shop->api()->graph($nextQuery)->body->orders;
              $this->transform($ordersGraph);
              }else{
              sleep(1);
              continue;
              }

              }
              return $this->allOrders;
              }





              share|improve this answer


























                0












                0








                0







                Below is my rough draft of a solution, still looking for expert opinion on how to effectively handle it so that each Shop has to wait for the time they deserve, according to the amount of data. Please advise.



                public function getRequiredOrders()
                {
                $firstRequestTimeStamp = now();
                $ordersGraph = $this->shop->api()->graph($this->firstQuery())->body->orders;
                $this->transform($ordersGraph); //transforming to required format

                $previousRequestTimeStamp = $firstRequestTimeStamp;

                while($ordersGraph->pageInfo->hasNextPage) {

                $nextRequestTimeStamp = now();
                $timeElapsed = $nextRequestTimeStamp->diffInSeconds($previousRequestTimeStamp);
                $restoredPoints = $timeElapsed * 50; //50 points are restored every 1 second
                $pointsLeft = $this->shop->api()->getApiCalls('graph', 'left');
                $totalPointsLeft = $pointsLeft + $restoredPoints;

                if($totalPointsLeft >=502){ //one must know the maximum cost of their query
                $lastEdgeCursor = end($ordersGraph->edges)->cursor;
                $nextQuery = $this->nextQuery($lastEdgeCursor);
                $previousRequestTimeStamp = $nextRequestTimeStamp;
                $ordersGraph = $this->shop->api()->graph($nextQuery)->body->orders;
                $this->transform($ordersGraph);
                }else{
                sleep(1);
                continue;
                }

                }
                return $this->allOrders;
                }





                share|improve this answer













                Below is my rough draft of a solution, still looking for expert opinion on how to effectively handle it so that each Shop has to wait for the time they deserve, according to the amount of data. Please advise.



                public function getRequiredOrders()
                {
                $firstRequestTimeStamp = now();
                $ordersGraph = $this->shop->api()->graph($this->firstQuery())->body->orders;
                $this->transform($ordersGraph); //transforming to required format

                $previousRequestTimeStamp = $firstRequestTimeStamp;

                while($ordersGraph->pageInfo->hasNextPage) {

                $nextRequestTimeStamp = now();
                $timeElapsed = $nextRequestTimeStamp->diffInSeconds($previousRequestTimeStamp);
                $restoredPoints = $timeElapsed * 50; //50 points are restored every 1 second
                $pointsLeft = $this->shop->api()->getApiCalls('graph', 'left');
                $totalPointsLeft = $pointsLeft + $restoredPoints;

                if($totalPointsLeft >=502){ //one must know the maximum cost of their query
                $lastEdgeCursor = end($ordersGraph->edges)->cursor;
                $nextQuery = $this->nextQuery($lastEdgeCursor);
                $previousRequestTimeStamp = $nextRequestTimeStamp;
                $ordersGraph = $this->shop->api()->graph($nextQuery)->body->orders;
                $this->transform($ordersGraph);
                }else{
                sleep(1);
                continue;
                }

                }
                return $this->allOrders;
                }






                share|improve this answer












                share|improve this answer



                share|improve this answer










                answered Nov 27 '18 at 8:09









                awebartisanawebartisan

                732715




                732715






























                    draft saved

                    draft discarded




















































                    Thanks for contributing an answer to Stack Overflow!


                    • Please be sure to answer the question. Provide details and share your research!

                    But avoid



                    • Asking for help, clarification, or responding to other answers.

                    • Making statements based on opinion; back them up with references or personal experience.


                    To learn more, see our tips on writing great answers.




                    draft saved


                    draft discarded














                    StackExchange.ready(
                    function () {
                    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53469612%2fshopify-graphql-admin-api-rate-limiting-cost-and-sleep-time%23new-answer', 'question_page');
                    }
                    );

                    Post as a guest















                    Required, but never shown





















































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown

































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown







                    Popular posts from this blog

                    A CLEAN and SIMPLE way to add appendices to Table of Contents and bookmarks

                    Calculate evaluation metrics using cross_val_predict sklearn

                    Insert data from modal to MySQL (multiple modal on website)