Wait until all data is received before sending another chunk












0















I use GSocket under Windows. For those who don't know, the underlying socket is natively winsock2-type socket handle.



I need fast, asynchronous (multithreaded, event-derived) TCP streaming-oriented file transfer for a project of mine and I need to introduce a sync behavior between the two sides by sending a chunk of data only when the previous chunk has been received by the remote user (i.e socket write buffer is empty). Otherwise sending huge files will force the remote user to first receive them all until he receives and process other packets. I do not want to change my entire program design again by for example changing to multi-socket interface or implementing multiplexing protocols.



A simple sleep call seems to work. It allows me to simultaneously process file incoming chunks while processing other packets in between. Needless to say this is a very robust non-optimized semi-resolved draft sort of non-solution.
I also tried to wait for G_IO_OUT condition to become true using a wrapper similar to the famous select(). The issue with it is that it allows for three chunks to be sent and then it freezes forever as data can not be written anymore.
G_IO_OUT is intended to be masked-in when data can be written using non-blocking sends. This can be helpful to indicate whether the buffer is full, but not whether it is empty.
Also something that worths mentioning I think is that I definitely don't want to send acknowledge from the receiver to the sender that it received chunk. This adds a lot of additional complexity and will ultimately slow down the file transfer further.



So, what can I do to block the sending function until all present data is sent and received by the remote user without providing a significant delay to the receiver?





Here is an illustration of how my architecture works and is meant to work:



Sender:




  • Sends request (FILE_PROPOSAL) with size of file name to expect

  • Sends filename

  • Sends request (FILE_INCOMMING) with size of the file to expect

  • Sends the file in a thread function, per chunk, so that I can introduce synced delay between the send calls. Sends FILE_CHUNK request with type + file ID before every raw chunk.


Receiver:




  • Receives request (FILE_PROPOSAL) with size of file name to expect

  • Receives filename

  • Receives request (FILE_INCOMMING) with size of the file to expect

  • Allocates a structure containing file info (including ID)

  • Returns

  • Receives FILE_CHUNK(S) request(s) with id and chunk size and then the chunk(s) itself and appends it to the allocated buffer in the allocated structure.

  • Returns, giving control to the receiver to be able to process other requests,










share|improve this question





























    0















    I use GSocket under Windows. For those who don't know, the underlying socket is natively winsock2-type socket handle.



    I need fast, asynchronous (multithreaded, event-derived) TCP streaming-oriented file transfer for a project of mine and I need to introduce a sync behavior between the two sides by sending a chunk of data only when the previous chunk has been received by the remote user (i.e socket write buffer is empty). Otherwise sending huge files will force the remote user to first receive them all until he receives and process other packets. I do not want to change my entire program design again by for example changing to multi-socket interface or implementing multiplexing protocols.



    A simple sleep call seems to work. It allows me to simultaneously process file incoming chunks while processing other packets in between. Needless to say this is a very robust non-optimized semi-resolved draft sort of non-solution.
    I also tried to wait for G_IO_OUT condition to become true using a wrapper similar to the famous select(). The issue with it is that it allows for three chunks to be sent and then it freezes forever as data can not be written anymore.
    G_IO_OUT is intended to be masked-in when data can be written using non-blocking sends. This can be helpful to indicate whether the buffer is full, but not whether it is empty.
    Also something that worths mentioning I think is that I definitely don't want to send acknowledge from the receiver to the sender that it received chunk. This adds a lot of additional complexity and will ultimately slow down the file transfer further.



    So, what can I do to block the sending function until all present data is sent and received by the remote user without providing a significant delay to the receiver?





    Here is an illustration of how my architecture works and is meant to work:



    Sender:




    • Sends request (FILE_PROPOSAL) with size of file name to expect

    • Sends filename

    • Sends request (FILE_INCOMMING) with size of the file to expect

    • Sends the file in a thread function, per chunk, so that I can introduce synced delay between the send calls. Sends FILE_CHUNK request with type + file ID before every raw chunk.


    Receiver:




    • Receives request (FILE_PROPOSAL) with size of file name to expect

    • Receives filename

    • Receives request (FILE_INCOMMING) with size of the file to expect

    • Allocates a structure containing file info (including ID)

    • Returns

    • Receives FILE_CHUNK(S) request(s) with id and chunk size and then the chunk(s) itself and appends it to the allocated buffer in the allocated structure.

    • Returns, giving control to the receiver to be able to process other requests,










    share|improve this question



























      0












      0








      0








      I use GSocket under Windows. For those who don't know, the underlying socket is natively winsock2-type socket handle.



      I need fast, asynchronous (multithreaded, event-derived) TCP streaming-oriented file transfer for a project of mine and I need to introduce a sync behavior between the two sides by sending a chunk of data only when the previous chunk has been received by the remote user (i.e socket write buffer is empty). Otherwise sending huge files will force the remote user to first receive them all until he receives and process other packets. I do not want to change my entire program design again by for example changing to multi-socket interface or implementing multiplexing protocols.



      A simple sleep call seems to work. It allows me to simultaneously process file incoming chunks while processing other packets in between. Needless to say this is a very robust non-optimized semi-resolved draft sort of non-solution.
      I also tried to wait for G_IO_OUT condition to become true using a wrapper similar to the famous select(). The issue with it is that it allows for three chunks to be sent and then it freezes forever as data can not be written anymore.
      G_IO_OUT is intended to be masked-in when data can be written using non-blocking sends. This can be helpful to indicate whether the buffer is full, but not whether it is empty.
      Also something that worths mentioning I think is that I definitely don't want to send acknowledge from the receiver to the sender that it received chunk. This adds a lot of additional complexity and will ultimately slow down the file transfer further.



      So, what can I do to block the sending function until all present data is sent and received by the remote user without providing a significant delay to the receiver?





      Here is an illustration of how my architecture works and is meant to work:



      Sender:




      • Sends request (FILE_PROPOSAL) with size of file name to expect

      • Sends filename

      • Sends request (FILE_INCOMMING) with size of the file to expect

      • Sends the file in a thread function, per chunk, so that I can introduce synced delay between the send calls. Sends FILE_CHUNK request with type + file ID before every raw chunk.


      Receiver:




      • Receives request (FILE_PROPOSAL) with size of file name to expect

      • Receives filename

      • Receives request (FILE_INCOMMING) with size of the file to expect

      • Allocates a structure containing file info (including ID)

      • Returns

      • Receives FILE_CHUNK(S) request(s) with id and chunk size and then the chunk(s) itself and appends it to the allocated buffer in the allocated structure.

      • Returns, giving control to the receiver to be able to process other requests,










      share|improve this question
















      I use GSocket under Windows. For those who don't know, the underlying socket is natively winsock2-type socket handle.



      I need fast, asynchronous (multithreaded, event-derived) TCP streaming-oriented file transfer for a project of mine and I need to introduce a sync behavior between the two sides by sending a chunk of data only when the previous chunk has been received by the remote user (i.e socket write buffer is empty). Otherwise sending huge files will force the remote user to first receive them all until he receives and process other packets. I do not want to change my entire program design again by for example changing to multi-socket interface or implementing multiplexing protocols.



      A simple sleep call seems to work. It allows me to simultaneously process file incoming chunks while processing other packets in between. Needless to say this is a very robust non-optimized semi-resolved draft sort of non-solution.
      I also tried to wait for G_IO_OUT condition to become true using a wrapper similar to the famous select(). The issue with it is that it allows for three chunks to be sent and then it freezes forever as data can not be written anymore.
      G_IO_OUT is intended to be masked-in when data can be written using non-blocking sends. This can be helpful to indicate whether the buffer is full, but not whether it is empty.
      Also something that worths mentioning I think is that I definitely don't want to send acknowledge from the receiver to the sender that it received chunk. This adds a lot of additional complexity and will ultimately slow down the file transfer further.



      So, what can I do to block the sending function until all present data is sent and received by the remote user without providing a significant delay to the receiver?





      Here is an illustration of how my architecture works and is meant to work:



      Sender:




      • Sends request (FILE_PROPOSAL) with size of file name to expect

      • Sends filename

      • Sends request (FILE_INCOMMING) with size of the file to expect

      • Sends the file in a thread function, per chunk, so that I can introduce synced delay between the send calls. Sends FILE_CHUNK request with type + file ID before every raw chunk.


      Receiver:




      • Receives request (FILE_PROPOSAL) with size of file name to expect

      • Receives filename

      • Receives request (FILE_INCOMMING) with size of the file to expect

      • Allocates a structure containing file info (including ID)

      • Returns

      • Receives FILE_CHUNK(S) request(s) with id and chunk size and then the chunk(s) itself and appends it to the allocated buffer in the allocated structure.

      • Returns, giving control to the receiver to be able to process other requests,







      sockets asynchronous networking gtk gio






      share|improve this question















      share|improve this question













      share|improve this question




      share|improve this question








      edited Nov 26 '18 at 4:22







      Edenia

















      asked Nov 26 '18 at 3:29









      EdeniaEdenia

      1,163623




      1,163623
























          0






          active

          oldest

          votes











          Your Answer






          StackExchange.ifUsing("editor", function () {
          StackExchange.using("externalEditor", function () {
          StackExchange.using("snippets", function () {
          StackExchange.snippets.init();
          });
          });
          }, "code-snippets");

          StackExchange.ready(function() {
          var channelOptions = {
          tags: "".split(" "),
          id: "1"
          };
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function() {
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled) {
          StackExchange.using("snippets", function() {
          createEditor();
          });
          }
          else {
          createEditor();
          }
          });

          function createEditor() {
          StackExchange.prepareEditor({
          heartbeatType: 'answer',
          autoActivateHeartbeat: false,
          convertImagesToLinks: true,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: 10,
          bindNavPrevention: true,
          postfix: "",
          imageUploader: {
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          },
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          });


          }
          });














          draft saved

          draft discarded


















          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53474440%2fwait-until-all-data-is-received-before-sending-another-chunk%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown

























          0






          active

          oldest

          votes








          0






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes
















          draft saved

          draft discarded




















































          Thanks for contributing an answer to Stack Overflow!


          • Please be sure to answer the question. Provide details and share your research!

          But avoid



          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.


          To learn more, see our tips on writing great answers.




          draft saved


          draft discarded














          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53474440%2fwait-until-all-data-is-received-before-sending-another-chunk%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          Popular posts from this blog

          A CLEAN and SIMPLE way to add appendices to Table of Contents and bookmarks

          Calculate evaluation metrics using cross_val_predict sklearn

          Insert data from modal to MySQL (multiple modal on website)