Wait until all data is received before sending another chunk
I use GSocket under Windows. For those who don't know, the underlying socket is natively winsock2-type socket handle.
I need fast, asynchronous (multithreaded, event-derived) TCP streaming-oriented file transfer for a project of mine and I need to introduce a sync behavior between the two sides by sending a chunk of data only when the previous chunk has been received by the remote user (i.e socket write buffer is empty). Otherwise sending huge files will force the remote user to first receive them all until he receives and process other packets. I do not want to change my entire program design again by for example changing to multi-socket interface or implementing multiplexing protocols.
A simple sleep call seems to work. It allows me to simultaneously process file incoming chunks while processing other packets in between. Needless to say this is a very robust non-optimized semi-resolved draft sort of non-solution.
I also tried to wait for G_IO_OUT
condition to become true using a wrapper similar to the famous select()
. The issue with it is that it allows for three chunks to be sent and then it freezes forever as data can not be written anymore.
G_IO_OUT
is intended to be masked-in when data can be written using non-blocking sends. This can be helpful to indicate whether the buffer is full, but not whether it is empty.
Also something that worths mentioning I think is that I definitely don't want to send acknowledge from the receiver to the sender that it received chunk. This adds a lot of additional complexity and will ultimately slow down the file transfer further.
So, what can I do to block the sending function until all present data is sent and received by the remote user without providing a significant delay to the receiver?
Here is an illustration of how my architecture works and is meant to work:
Sender:
- Sends request (FILE_PROPOSAL) with size of file name to expect
- Sends filename
- Sends request (FILE_INCOMMING) with size of the file to expect
- Sends the file in a thread function, per chunk, so that I can introduce synced delay between the send calls. Sends FILE_CHUNK request with type + file ID before every raw chunk.
Receiver:
- Receives request (FILE_PROPOSAL) with size of file name to expect
- Receives filename
- Receives request (FILE_INCOMMING) with size of the file to expect
- Allocates a structure containing file info (including ID)
- Returns
- Receives FILE_CHUNK(S) request(s) with id and chunk size and then the chunk(s) itself and appends it to the allocated buffer in the allocated structure.
- Returns, giving control to the receiver to be able to process other requests,
sockets asynchronous networking gtk gio
add a comment |
I use GSocket under Windows. For those who don't know, the underlying socket is natively winsock2-type socket handle.
I need fast, asynchronous (multithreaded, event-derived) TCP streaming-oriented file transfer for a project of mine and I need to introduce a sync behavior between the two sides by sending a chunk of data only when the previous chunk has been received by the remote user (i.e socket write buffer is empty). Otherwise sending huge files will force the remote user to first receive them all until he receives and process other packets. I do not want to change my entire program design again by for example changing to multi-socket interface or implementing multiplexing protocols.
A simple sleep call seems to work. It allows me to simultaneously process file incoming chunks while processing other packets in between. Needless to say this is a very robust non-optimized semi-resolved draft sort of non-solution.
I also tried to wait for G_IO_OUT
condition to become true using a wrapper similar to the famous select()
. The issue with it is that it allows for three chunks to be sent and then it freezes forever as data can not be written anymore.
G_IO_OUT
is intended to be masked-in when data can be written using non-blocking sends. This can be helpful to indicate whether the buffer is full, but not whether it is empty.
Also something that worths mentioning I think is that I definitely don't want to send acknowledge from the receiver to the sender that it received chunk. This adds a lot of additional complexity and will ultimately slow down the file transfer further.
So, what can I do to block the sending function until all present data is sent and received by the remote user without providing a significant delay to the receiver?
Here is an illustration of how my architecture works and is meant to work:
Sender:
- Sends request (FILE_PROPOSAL) with size of file name to expect
- Sends filename
- Sends request (FILE_INCOMMING) with size of the file to expect
- Sends the file in a thread function, per chunk, so that I can introduce synced delay between the send calls. Sends FILE_CHUNK request with type + file ID before every raw chunk.
Receiver:
- Receives request (FILE_PROPOSAL) with size of file name to expect
- Receives filename
- Receives request (FILE_INCOMMING) with size of the file to expect
- Allocates a structure containing file info (including ID)
- Returns
- Receives FILE_CHUNK(S) request(s) with id and chunk size and then the chunk(s) itself and appends it to the allocated buffer in the allocated structure.
- Returns, giving control to the receiver to be able to process other requests,
sockets asynchronous networking gtk gio
add a comment |
I use GSocket under Windows. For those who don't know, the underlying socket is natively winsock2-type socket handle.
I need fast, asynchronous (multithreaded, event-derived) TCP streaming-oriented file transfer for a project of mine and I need to introduce a sync behavior between the two sides by sending a chunk of data only when the previous chunk has been received by the remote user (i.e socket write buffer is empty). Otherwise sending huge files will force the remote user to first receive them all until he receives and process other packets. I do not want to change my entire program design again by for example changing to multi-socket interface or implementing multiplexing protocols.
A simple sleep call seems to work. It allows me to simultaneously process file incoming chunks while processing other packets in between. Needless to say this is a very robust non-optimized semi-resolved draft sort of non-solution.
I also tried to wait for G_IO_OUT
condition to become true using a wrapper similar to the famous select()
. The issue with it is that it allows for three chunks to be sent and then it freezes forever as data can not be written anymore.
G_IO_OUT
is intended to be masked-in when data can be written using non-blocking sends. This can be helpful to indicate whether the buffer is full, but not whether it is empty.
Also something that worths mentioning I think is that I definitely don't want to send acknowledge from the receiver to the sender that it received chunk. This adds a lot of additional complexity and will ultimately slow down the file transfer further.
So, what can I do to block the sending function until all present data is sent and received by the remote user without providing a significant delay to the receiver?
Here is an illustration of how my architecture works and is meant to work:
Sender:
- Sends request (FILE_PROPOSAL) with size of file name to expect
- Sends filename
- Sends request (FILE_INCOMMING) with size of the file to expect
- Sends the file in a thread function, per chunk, so that I can introduce synced delay between the send calls. Sends FILE_CHUNK request with type + file ID before every raw chunk.
Receiver:
- Receives request (FILE_PROPOSAL) with size of file name to expect
- Receives filename
- Receives request (FILE_INCOMMING) with size of the file to expect
- Allocates a structure containing file info (including ID)
- Returns
- Receives FILE_CHUNK(S) request(s) with id and chunk size and then the chunk(s) itself and appends it to the allocated buffer in the allocated structure.
- Returns, giving control to the receiver to be able to process other requests,
sockets asynchronous networking gtk gio
I use GSocket under Windows. For those who don't know, the underlying socket is natively winsock2-type socket handle.
I need fast, asynchronous (multithreaded, event-derived) TCP streaming-oriented file transfer for a project of mine and I need to introduce a sync behavior between the two sides by sending a chunk of data only when the previous chunk has been received by the remote user (i.e socket write buffer is empty). Otherwise sending huge files will force the remote user to first receive them all until he receives and process other packets. I do not want to change my entire program design again by for example changing to multi-socket interface or implementing multiplexing protocols.
A simple sleep call seems to work. It allows me to simultaneously process file incoming chunks while processing other packets in between. Needless to say this is a very robust non-optimized semi-resolved draft sort of non-solution.
I also tried to wait for G_IO_OUT
condition to become true using a wrapper similar to the famous select()
. The issue with it is that it allows for three chunks to be sent and then it freezes forever as data can not be written anymore.
G_IO_OUT
is intended to be masked-in when data can be written using non-blocking sends. This can be helpful to indicate whether the buffer is full, but not whether it is empty.
Also something that worths mentioning I think is that I definitely don't want to send acknowledge from the receiver to the sender that it received chunk. This adds a lot of additional complexity and will ultimately slow down the file transfer further.
So, what can I do to block the sending function until all present data is sent and received by the remote user without providing a significant delay to the receiver?
Here is an illustration of how my architecture works and is meant to work:
Sender:
- Sends request (FILE_PROPOSAL) with size of file name to expect
- Sends filename
- Sends request (FILE_INCOMMING) with size of the file to expect
- Sends the file in a thread function, per chunk, so that I can introduce synced delay between the send calls. Sends FILE_CHUNK request with type + file ID before every raw chunk.
Receiver:
- Receives request (FILE_PROPOSAL) with size of file name to expect
- Receives filename
- Receives request (FILE_INCOMMING) with size of the file to expect
- Allocates a structure containing file info (including ID)
- Returns
- Receives FILE_CHUNK(S) request(s) with id and chunk size and then the chunk(s) itself and appends it to the allocated buffer in the allocated structure.
- Returns, giving control to the receiver to be able to process other requests,
sockets asynchronous networking gtk gio
sockets asynchronous networking gtk gio
edited Nov 26 '18 at 4:22
Edenia
asked Nov 26 '18 at 3:29
EdeniaEdenia
1,163623
1,163623
add a comment |
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53474440%2fwait-until-all-data-is-received-before-sending-another-chunk%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53474440%2fwait-until-all-data-is-received-before-sending-another-chunk%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown