why python multithreading doesn't work in socket when sending message to server?
I'm try to grab bullet screens by python. The process is that I will get the response after sending login and some others messages to bullet screens server.
When I plan to grab two rooms's bullet screens, I can only get the response of the first request.
Code is like this:
# coding=utf-8
import multiprocessing
import socket
import time
import re
import signal
import threading
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
host = socket.gethostbyname("openbarrage.douyutv.com")
port = 8601
client.connect((host, port))
def send_req_msg(msg):
"""
wrap msg
"""
pass
def set_msg(roomid):
msg = 'type@=loginreq/roomid@={}/'.format(roomid)
send_req_msg(msg)
msg_more = 'type@=joingroup/rid@={}/gid@=-9999/'.format(roomid)
send_req_msg(msg_more)
if __name__ == '__main__':
pool =
roomid = [123, 666]
for i in range(0, len(roomid)):
t = threading.Thread(target=set_msg(roomid[i]))
t.start()
pool.append(t)
for a in pool:
a.join()
Both multiprocessing and threading can't succeed. As I use wireshard to analysis the tcp. There is only the first request and it's response. Also I can only get the room's screen bullets of 123. So why the second thread/processing doesn't work? And what should I do? Thanks.
python multithreading sockets web-crawler
asked Nov 23 '18 at 16:40
G.yx
135
add a comment |
I'm try to grab bullet screens by python. The process is that I will get the response after sending login and some others messages to bullet screens server.
When I plan to grab two rooms's bullet screens, I can only get the response of the first request.
Code is like this:
# coding=utf-8
import multiprocessing
import socket
import time
import re
import signal
import threading
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
host = socket.gethostbyname("openbarrage.douyutv.com")
port = 8601
client.connect((host, port))
def send_req_msg(msg):
"""
wrap msg
"""
pass
def set_msg(roomid):
msg = 'type@=loginreq/roomid@={}/'.format(roomid)
send_req_msg(msg)
msg_more = 'type@=joingroup/rid@={}/gid@=-9999/'.format(roomid)
send_req_msg(msg_more)
if __name__ == '__main__':
pool =
roomid = [123, 666]
for i in range(0, len(roomid)):
t = threading.Thread(target=set_msg(roomid[i]))
t.start()
pool.append(t)
for a in pool:
a.join()
Both multiprocessing and threading can't succeed. As I use wireshard to analysis the tcp. There is only the first request and it's response. Also I can only get the room's screen bullets of 123. So why the second thread/processing doesn't work? And what should I do? Thanks.
python multithreading sockets web-crawler
asked Nov 23 '18 at 16:40
G.yx
135
When I ran your codesend_req_msg()was called 4 times with different arguments so that's correct. If you think messages aren't sent try to print response from server after sending message if there's any.
– Filip Młynarski
Nov 23 '18 at 16:52
add some print statements at entry to set_msg and send_req_msg, so you can see that they are called and what the parameter value is they are passed. Also add a print statement at the end of each function so you can see when they exit.
– barny
Nov 23 '18 at 17:01
If you are on Windows, multiprocessing always starts a whole new process so trying to share a global likeclientwon't work - the whole python file is re-executed so the second instance will get a different client connection. Look at the help 16.6.3.2 in this page docs.python.org/2/library/multiprocessing.html. AFAICT when using Multiprocessing, properly and completely reading and understanding the manual pages is critical for success.
– barny
Nov 23 '18 at 17:09
@barny I had add many print statements and can only get print statements of first thread. I will read the page ,thanks.(I'm on mac)
– G.yx
Nov 24 '18 at 3:16
@kevin has solved your problem.
– barny
Nov 24 '18 at 15:05
add a comment |
I'm try to grab bullet screens by python. The process is that I will get the response after sending login and some others messages to bullet screens server.
When I plan to grab two rooms's bullet screens, I can only get the response of the first request.
Code is like this:
# coding=utf-8
import multiprocessing
import socket
import time
import re
import signal
import threading
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
host = socket.gethostbyname("openbarrage.douyutv.com")
port = 8601
client.connect((host, port))
def send_req_msg(msg):
"""
wrap msg
"""
pass
def set_msg(roomid):
msg = 'type@=loginreq/roomid@={}/'.format(roomid)
send_req_msg(msg)
msg_more = 'type@=joingroup/rid@={}/gid@=-9999/'.format(roomid)
send_req_msg(msg_more)
if __name__ == '__main__':
pool =
roomid = [123, 666]
for i in range(0, len(roomid)):
t = threading.Thread(target=set_msg(roomid[i]))
t.start()
pool.append(t)
for a in pool:
a.join()
Both multiprocessing and threading can't succeed. As I use wireshard to analysis the tcp. There is only the first request and it's response. Also I can only get the room's screen bullets of 123. So why the second thread/processing doesn't work? And what should I do? Thanks.
python multithreading sockets web-crawler
asked Nov 23 '18 at 16:40
G.yx
135
I'm try to grab bullet screens by python. The process is that I will get the response after sending login and some others messages to bullet screens server.
When I plan to grab two rooms's bullet screens, I can only get the response of the first request.
Code is like this:
# coding=utf-8
import multiprocessing
import socket
import time
import re
import signal
import threading
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
host = socket.gethostbyname("openbarrage.douyutv.com")
port = 8601
client.connect((host, port))
def send_req_msg(msg):
"""
wrap msg
"""
pass
def set_msg(roomid):
msg = 'type@=loginreq/roomid@={}/'.format(roomid)
send_req_msg(msg)
msg_more = 'type@=joingroup/rid@={}/gid@=-9999/'.format(roomid)
send_req_msg(msg_more)
if __name__ == '__main__':
pool =
roomid = [123, 666]
for i in range(0, len(roomid)):
t = threading.Thread(target=set_msg(roomid[i]))
t.start()
pool.append(t)
for a in pool:
a.join()
Both multiprocessing and threading can't succeed. As I use wireshard to analysis the tcp. There is only the first request and it's response. Also I can only get the room's screen bullets of 123. So why the second thread/processing doesn't work? And what should I do? Thanks.
python multithreading sockets web-crawler
python multithreading sockets web-crawler
asked Nov 23 '18 at 16:40
G.yx
135
asked Nov 23 '18 at 16:40
G.yx
135
asked Nov 23 '18 at 16:40
G.yx
135
asked Nov 23 '18 at 16:40
G.yx
135
asked Nov 23 '18 at 16:40
G.yx
135
135
When I ran your codesend_req_msg()was called 4 times with different arguments so that's correct. If you think messages aren't sent try to print response from server after sending message if there's any.
– Filip Młynarski
Nov 23 '18 at 16:52
add some print statements at entry to set_msg and send_req_msg, so you can see that they are called and what the parameter value is they are passed. Also add a print statement at the end of each function so you can see when they exit.
– barny
Nov 23 '18 at 17:01
If you are on Windows, multiprocessing always starts a whole new process so trying to share a global likeclientwon't work - the whole python file is re-executed so the second instance will get a different client connection. Look at the help 16.6.3.2 in this page docs.python.org/2/library/multiprocessing.html. AFAICT when using Multiprocessing, properly and completely reading and understanding the manual pages is critical for success.
– barny
Nov 23 '18 at 17:09
@barny I had add many print statements and can only get print statements of first thread. I will read the page ,thanks.(I'm on mac)
– G.yx
Nov 24 '18 at 3:16
@kevin has solved your problem.
– barny
Nov 24 '18 at 15:05
add a comment |
When I ran your codesend_req_msg()was called 4 times with different arguments so that's correct. If you think messages aren't sent try to print response from server after sending message if there's any.
– Filip Młynarski
Nov 23 '18 at 16:52
add some print statements at entry to set_msg and send_req_msg, so you can see that they are called and what the parameter value is they are passed. Also add a print statement at the end of each function so you can see when they exit.
– barny
Nov 23 '18 at 17:01
If you are on Windows, multiprocessing always starts a whole new process so trying to share a global likeclientwon't work - the whole python file is re-executed so the second instance will get a different client connection. Look at the help 16.6.3.2 in this page docs.python.org/2/library/multiprocessing.html. AFAICT when using Multiprocessing, properly and completely reading and understanding the manual pages is critical for success.
– barny
Nov 23 '18 at 17:09
@barny I had add many print statements and can only get print statements of first thread. I will read the page ,thanks.(I'm on mac)
– G.yx
Nov 24 '18 at 3:16
@kevin has solved your problem.
– barny
Nov 24 '18 at 15:05
When I ran your code
send_req_msg() was called 4 times with different arguments so that's correct. If you think messages aren't sent try to print response from server after sending message if there's any.– Filip Młynarski
Nov 23 '18 at 16:52
When I ran your code
send_req_msg() was called 4 times with different arguments so that's correct. If you think messages aren't sent try to print response from server after sending message if there's any.– Filip Młynarski
Nov 23 '18 at 16:52
add some print statements at entry to set_msg and send_req_msg, so you can see that they are called and what the parameter value is they are passed. Also add a print statement at the end of each function so you can see when they exit.
– barny
Nov 23 '18 at 17:01
add some print statements at entry to set_msg and send_req_msg, so you can see that they are called and what the parameter value is they are passed. Also add a print statement at the end of each function so you can see when they exit.
– barny
Nov 23 '18 at 17:01
If you are on Windows, multiprocessing always starts a whole new process so trying to share a global like
client won't work - the whole python file is re-executed so the second instance will get a different client connection. Look at the help 16.6.3.2 in this page docs.python.org/2/library/multiprocessing.html. AFAICT when using Multiprocessing, properly and completely reading and understanding the manual pages is critical for success.– barny
Nov 23 '18 at 17:09
If you are on Windows, multiprocessing always starts a whole new process so trying to share a global like
client won't work - the whole python file is re-executed so the second instance will get a different client connection. Look at the help 16.6.3.2 in this page docs.python.org/2/library/multiprocessing.html. AFAICT when using Multiprocessing, properly and completely reading and understanding the manual pages is critical for success.– barny
Nov 23 '18 at 17:09
@barny I had add many print statements and can only get print statements of first thread. I will read the page ,thanks.(I'm on mac)
– G.yx
Nov 24 '18 at 3:16
@barny I had add many print statements and can only get print statements of first thread. I will read the page ,thanks.(I'm on mac)
– G.yx
Nov 24 '18 at 3:16
@kevin has solved your problem.
– barny
Nov 24 '18 at 15:05
@kevin has solved your problem.
– barny
Nov 24 '18 at 15:05
add a comment |
1 Answer
1
active
oldest
votes
t = threading.Thread(target=set_msg(roomid[i]))
This does not pass the function set_msg() to the thread. It calls the function and passes the result to the thread (in this case, the value passed is always None because set_msg() does not return anything). If you call the function before creating the thread, then the function will not run in the thread. You need to pass the function (and its arguments) to the Thread constructor without calling it:
t = threading.Thread(target=set_msg, args=(roomid[i],))
answered Nov 23 '18 at 17:09
Kevin
18.6k53559
thanks, this is my fault. And I will try it tonight.
– G.yx
Nov 24 '18 at 3:18
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53450288%2fwhy-python-multithreading-doesnt-work-in-socket-when-sending-message-to-server%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
t = threading.Thread(target=set_msg(roomid[i]))
This does not pass the function set_msg() to the thread. It calls the function and passes the result to the thread (in this case, the value passed is always None because set_msg() does not return anything). If you call the function before creating the thread, then the function will not run in the thread. You need to pass the function (and its arguments) to the Thread constructor without calling it:
t = threading.Thread(target=set_msg, args=(roomid[i],))
answered Nov 23 '18 at 17:09
Kevin
18.6k53559
thanks, this is my fault. And I will try it tonight.
– G.yx
Nov 24 '18 at 3:18
add a comment |
t = threading.Thread(target=set_msg(roomid[i]))
This does not pass the function set_msg() to the thread. It calls the function and passes the result to the thread (in this case, the value passed is always None because set_msg() does not return anything). If you call the function before creating the thread, then the function will not run in the thread. You need to pass the function (and its arguments) to the Thread constructor without calling it:
t = threading.Thread(target=set_msg, args=(roomid[i],))
answered Nov 23 '18 at 17:09
Kevin
18.6k53559
thanks, this is my fault. And I will try it tonight.
– G.yx
Nov 24 '18 at 3:18
add a comment |
t = threading.Thread(target=set_msg(roomid[i]))
This does not pass the function set_msg() to the thread. It calls the function and passes the result to the thread (in this case, the value passed is always None because set_msg() does not return anything). If you call the function before creating the thread, then the function will not run in the thread. You need to pass the function (and its arguments) to the Thread constructor without calling it:
t = threading.Thread(target=set_msg, args=(roomid[i],))
answered Nov 23 '18 at 17:09
Kevin
18.6k53559
t = threading.Thread(target=set_msg(roomid[i]))
This does not pass the function set_msg() to the thread. It calls the function and passes the result to the thread (in this case, the value passed is always None because set_msg() does not return anything). If you call the function before creating the thread, then the function will not run in the thread. You need to pass the function (and its arguments) to the Thread constructor without calling it:
t = threading.Thread(target=set_msg, args=(roomid[i],))
answered Nov 23 '18 at 17:09
Kevin
18.6k53559
answered Nov 23 '18 at 17:09
Kevin
18.6k53559
answered Nov 23 '18 at 17:09
Kevin
18.6k53559
answered Nov 23 '18 at 17:09
Kevin
18.6k53559
18.6k53559
thanks, this is my fault. And I will try it tonight.
– G.yx
Nov 24 '18 at 3:18
add a comment |
thanks, this is my fault. And I will try it tonight.
– G.yx
Nov 24 '18 at 3:18
thanks, this is my fault. And I will try it tonight.
– G.yx
Nov 24 '18 at 3:18
thanks, this is my fault. And I will try it tonight.
– G.yx
Nov 24 '18 at 3:18
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53450288%2fwhy-python-multithreading-doesnt-work-in-socket-when-sending-message-to-server%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
When I ran your code
send_req_msg()was called 4 times with different arguments so that's correct. If you think messages aren't sent try to print response from server after sending message if there's any.– Filip Młynarski
Nov 23 '18 at 16:52
add some print statements at entry to set_msg and send_req_msg, so you can see that they are called and what the parameter value is they are passed. Also add a print statement at the end of each function so you can see when they exit.
– barny
Nov 23 '18 at 17:01
If you are on Windows, multiprocessing always starts a whole new process so trying to share a global like
clientwon't work - the whole python file is re-executed so the second instance will get a different client connection. Look at the help 16.6.3.2 in this page docs.python.org/2/library/multiprocessing.html. AFAICT when using Multiprocessing, properly and completely reading and understanding the manual pages is critical for success.– barny
Nov 23 '18 at 17:09
@barny I had add many print statements and can only get print statements of first thread. I will read the page ,thanks.(I'm on mac)
– G.yx
Nov 24 '18 at 3:16
@kevin has solved your problem.
– barny
Nov 24 '18 at 15:05