How to extract data from incoming HTTP POST using python
up vote
0
down vote
favorite
I have a Ubuntu LAMP webserver and data is being sent to the webserver continuously through HTTP POST method. I need to extract the data from HTTP POST and insert them in a Database. I don't know how to do that. There is a lot of example available on how to handle outgoing HTTP POST request but the incoming HTTP POST request. I want to write a python3 script which will extract the data from incoming HTTP POST request and save them as varible which I will use to insert the data into the database and also return a response to the client.Can anybody help me in this regard?
python http post extraction
asked Nov 17 at 6:14
jyoti
15
add a comment |
up vote
0
down vote
favorite
I have a Ubuntu LAMP webserver and data is being sent to the webserver continuously through HTTP POST method. I need to extract the data from HTTP POST and insert them in a Database. I don't know how to do that. There is a lot of example available on how to handle outgoing HTTP POST request but the incoming HTTP POST request. I want to write a python3 script which will extract the data from incoming HTTP POST request and save them as varible which I will use to insert the data into the database and also return a response to the client.Can anybody help me in this regard?
python http post extraction
asked Nov 17 at 6:14
jyoti
15
Are you using Flask?
– oetoni
Nov 17 at 6:20
No, I am not using any api. I want to make a request to a python script like anuurl.com/cgi-bin/pythonscript.py and then the script executes and extracts the data from HTTP POST request.
– jyoti
Nov 17 at 6:50
hi @jyoti, i updated the answer ;) check below
– oetoni
Nov 19 at 21:10
add a comment |
up vote
0
down vote
favorite
up vote
0
down vote
favorite
I have a Ubuntu LAMP webserver and data is being sent to the webserver continuously through HTTP POST method. I need to extract the data from HTTP POST and insert them in a Database. I don't know how to do that. There is a lot of example available on how to handle outgoing HTTP POST request but the incoming HTTP POST request. I want to write a python3 script which will extract the data from incoming HTTP POST request and save them as varible which I will use to insert the data into the database and also return a response to the client.Can anybody help me in this regard?
python http post extraction
asked Nov 17 at 6:14
jyoti
15
I have a Ubuntu LAMP webserver and data is being sent to the webserver continuously through HTTP POST method. I need to extract the data from HTTP POST and insert them in a Database. I don't know how to do that. There is a lot of example available on how to handle outgoing HTTP POST request but the incoming HTTP POST request. I want to write a python3 script which will extract the data from incoming HTTP POST request and save them as varible which I will use to insert the data into the database and also return a response to the client.Can anybody help me in this regard?
python http post extraction
python http post extraction
asked Nov 17 at 6:14
jyoti
15
asked Nov 17 at 6:14
jyoti
15
asked Nov 17 at 6:14
jyoti
15
asked Nov 17 at 6:14
jyoti
15
asked Nov 17 at 6:14
jyoti
15
15
Are you using Flask?
– oetoni
Nov 17 at 6:20
No, I am not using any api. I want to make a request to a python script like anuurl.com/cgi-bin/pythonscript.py and then the script executes and extracts the data from HTTP POST request.
– jyoti
Nov 17 at 6:50
hi @jyoti, i updated the answer ;) check below
– oetoni
Nov 19 at 21:10
add a comment |
Are you using Flask?
– oetoni
Nov 17 at 6:20
No, I am not using any api. I want to make a request to a python script like anuurl.com/cgi-bin/pythonscript.py and then the script executes and extracts the data from HTTP POST request.
– jyoti
Nov 17 at 6:50
hi @jyoti, i updated the answer ;) check below
– oetoni
Nov 19 at 21:10
Are you using Flask?
– oetoni
Nov 17 at 6:20
Are you using Flask?
– oetoni
Nov 17 at 6:20
No, I am not using any api. I want to make a request to a python script like anuurl.com/cgi-bin/pythonscript.py and then the script executes and extracts the data from HTTP POST request.
– jyoti
Nov 17 at 6:50
No, I am not using any api. I want to make a request to a python script like anuurl.com/cgi-bin/pythonscript.py and then the script executes and extracts the data from HTTP POST request.
– jyoti
Nov 17 at 6:50
hi @jyoti, i updated the answer ;) check below
– oetoni
Nov 19 at 21:10
hi @jyoti, i updated the answer ;) check below
– oetoni
Nov 19 at 21:10
add a comment |
4 Answers
4
active
oldest
votes
up vote
0
down vote
The code I'm using for this:
#!/usr/bin/python3
import socketserver
from http.server import BaseHTTPRequestHandler
import time
import threading
def do_something():
threading.Timer(1.0, do_something).start()
do_other_things
class MyHandler(BaseHTTPRequestHandler):
def do_post(self):
global site_id, first, last, pass1
if self.path == '/do_something':
site_id = self.request.POST.get('m_site_name')
first = self.request.POST.get('m_first_name')
last = self.request.POST.get('m_last_name')
pass1 = self.request.POST.get('m_device_name')
do_something()
self.send_response(200)
try:
httpd = socketserver.TCPServer(("localhost", 9001), MyHandler)
httpd.serve_forever()
finally:
httpd = socketserver.TCPServer(("localhost", 9001), MyHandler)
httpd.server_close()
But It's not working at all. can you tell me why?
answered Nov 19 at 8:49
jyoti
15
add a comment |
up vote
0
down vote
UPDATE
According to the code you posted below, here is a working answer.
#!/usr/bin/python3
import socketserver
from http.server import BaseHTTPRequestHandler
import time
import threading
def do_something(site_id, first, last, pass1):
print(site_id)
print(first)
print(last)
print(pass1)
#just to illustrate the point and print the variables
class MyHandler(BaseHTTPRequestHandler):
def do_POST(self): # !important to use 'do_POST' with Capital POST
global site_id, first, last, pass1 #those are still undefined at the module level ;) remember this for later
if self.path == '/do_something':
request_headers = self.headers
site_id = request_headers["m_site_name"]
first = request_headers["m_first_name"]
last = request_headers["m_last_name"]
pass1 = request_headers["m_device_name"]
do_something(site_id, first, last, pass1)
self.send_response(200)
self.end_headers() #as of P3.3 this is required
try:
httpd = socketserver.TCPServer(("localhost", 9001), MyHandler)
httpd.serve_forever()
finally:
httpd = socketserver.TCPServer(("localhost", 9001), MyHandler)
httpd.server_close()
Calling it with Postman
and the command line output is
C:DevelopmentPythontestvenvScriptspython.exe C:/Development/Python/test/webserver_old.py
1001
jyoti0
127.0.0.1 - - [19/Nov/2018 21:53:45] "POST /do_something HTTP/1.1" 200 -
jyoti1
101
I combined answers from these here:
Reference one, two and third
And this is also very important to read:
https://docs.python.org/3/library/http.server.html
http.server is not recommended for production. It only implements basic security checks.
I believe is ok for a small implementation and some testing around or proof of concept but eventually you'll need to manage this better, maybe I can suggest you to spend some time and use Flask, is actually an excellent & very light framework for Python API building and prototyping.
-
Previous answer (deprecated & updated above)
-
As per a very light and simple reference to this one:
def do_POST(self):
# Doesn't do anything with posted data
content_length = int(self.headers['Content-Length']) # <--- Gets the size of data
post_data = self.rfile.read(content_length) # <--- Gets the data itself
self._set_headers()
self.wfile.write("<html><body><h1>POST!</h1></body></html>")
Update (without and API):
Assuming you are running on or machine at a custom port with a custom trailing part at the URL, then "pure" python would look like this:
import SocketServer
from BaseHTTPServer import BaseHTTPRequestHandler
def doSomething():
print "i did"
class MyHandler(BaseHTTPRequestHandler):
def do_POST(self):
if self.path == '/doSomething':
mail = self.request.POST.get('email')
something = self.request.POST.get('something')
doSomething()
self.send_response(200)
httpd = SocketServer.TCPServer(("", 8080), MyHandler)
httpd.serve_forever()
I assume this way you can reuse the variables freely. Check also this reference here, Brenda's answer.
answered Nov 17 at 6:28
oetoni
730621
I am guessing "post_data = self.rfile.read(content_length)" will save all of the data. Is there a way to save specific data as variables, like: user = username mail = xyz@abc.com place = anywhere so that I can use the variables later?
– jyoti
Nov 17 at 6:57
@jyoti, check again I updated the answer ;) In the first example with .read() you can also check the content during debugging and later understand from what you discover how to extract them as needed
– oetoni
Nov 17 at 7:09
I will implement both of your suggestions and will let you know about the results. I want to access the pythonscript.py file through a Web Browser and when the script is accessed via web browser the script will be executed and as a result data from the HTTP POST will be extracted and inserted into a database. Unfortunately, I wouldn't implement your suggestion today but I surely will do that on Monday and will update you accordingly. Although thanks for the help, I had no clue about where to start or How to begin with in this regard.
– jyoti
Nov 17 at 7:32
don't mention it! I hope it helps... please know that the script itself should run as a web service or under and api, to make life easier you can use django or flask but if you are insisting on using non of them then try the second one and only check the content of post ;) you can control the port 8080 as per your preference and the script will run under localhost:8080 (or any port :) )
– oetoni
Nov 17 at 7:35
I know nothing of Django or flask. That's why I didn't use any of them. I don't know how to use or implement django or flask.
– jyoti
Nov 17 at 8:20
|
show 8 more comments
up vote
0
down vote
@oetoni, I am getting time out error while using:
#!/usr/bin/python3
import socketserver
from http.server import BaseHTTPRequestHandler
import time
import threading
def do_something(site_id, first, last, pass1):
print(site_id)
print(first)
print(last)
print(pass1)
#just to illustrate the point and print the variables
class MyHandler(BaseHTTPRequestHandler):
def do_POST(self): # !important to use 'do_POST' with Capital POST
global site_id, first, last, pass1 #those are still undefined at the module level ;) remember this for later
if self.path == '/do_something':
request_headers = self.headers
site_id = request_headers["m_site_name"]
first = request_headers["m_first_name"]
last = request_headers["m_last_name"]
pass1 = request_headers["m_device_name"]
do_something(site_id, first, last, pass1)
self.send_response(200)
self.end_headers() #as of P3.3 this is required
try:
httpd = socketserver.TCPServer(("localhost", 9001), MyHandler)
httpd.serve_forever()
finally:
httpd = socketserver.TCPServer(("localhost", 9001), MyHandler)
httpd.server_close()
But I am getting the correct response while using this code:
#!/usr/bin/python3
# -*- coding: UTF-8 -*-
import cgi
import cgitb
cgitb.enable()
print('Content-Type: text/html')
print('')
arguments = cgi.FieldStorage()
for i in arguments.keys():
print(arguments[i].value)
and it prints the received data on the web browser.
I am using this script as a cgi script on the apache web server which can be accessed through a web browser. I am not running this script as a service or application.
answered 2 days ago
jyoti
15
@oetoni, I've found out the problem. It's the port 9001. I am running this script as a cgi script so, I need to use port 80. But when I use port 80, I get this error: AH01215: self.socket.bind(self.server_address): /var/www/html/cgi-bin/anything.py AH01215: PermissionError: [Errno 13] Permission denied: /var/www/html/cgi-bin/anything.py End of script output before headers: anything.py
– jyoti
2 days ago
add a comment |
up vote
0
down vote
accepted
#!/usr/bin/python3
# -*- coding: UTF-8 -*-
import cgi
import cgitb
cgitb.enable()
print('Content-Type: text/htmln')
arguments = cgi.FieldStorage()
print(arguments["m_site_name"].value)
print("<br />n")
print(arguments["m_first_name"].value)
print("<br />n")
print(arguments["m_last_name"].value)
print("<br />n")
print(arguments["m_device_name"].value)
print("<br />n")
site = arguments["m_site_name"].value
first = arguments["m_first_name"].value
last = arguments["m_last_name"].value
device = arguments["m_device_name"].value
-----do_other_things_with_the_variables(site,first,last,device)-----
This code solved my problem. Now I can store HTTP POST data into variables with this python cgi script.
my HTTP POST Request:
http://your_server_url_or_IP/cgi-bin/python_script.py?m_site_name=MySite&m_first_name=anyname&m_last_name=anylastanme&m_device_name=anydeviceidorname
answered 2 days ago
jyoti
15
add a comment |
4 Answers
4
active
oldest
votes
4 Answers
4
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
0
down vote
The code I'm using for this:
#!/usr/bin/python3
import socketserver
from http.server import BaseHTTPRequestHandler
import time
import threading
def do_something():
threading.Timer(1.0, do_something).start()
do_other_things
class MyHandler(BaseHTTPRequestHandler):
def do_post(self):
global site_id, first, last, pass1
if self.path == '/do_something':
site_id = self.request.POST.get('m_site_name')
first = self.request.POST.get('m_first_name')
last = self.request.POST.get('m_last_name')
pass1 = self.request.POST.get('m_device_name')
do_something()
self.send_response(200)
try:
httpd = socketserver.TCPServer(("localhost", 9001), MyHandler)
httpd.serve_forever()
finally:
httpd = socketserver.TCPServer(("localhost", 9001), MyHandler)
httpd.server_close()
But It's not working at all. can you tell me why?
answered Nov 19 at 8:49
jyoti
15
add a comment |
up vote
0
down vote
The code I'm using for this:
#!/usr/bin/python3
import socketserver
from http.server import BaseHTTPRequestHandler
import time
import threading
def do_something():
threading.Timer(1.0, do_something).start()
do_other_things
class MyHandler(BaseHTTPRequestHandler):
def do_post(self):
global site_id, first, last, pass1
if self.path == '/do_something':
site_id = self.request.POST.get('m_site_name')
first = self.request.POST.get('m_first_name')
last = self.request.POST.get('m_last_name')
pass1 = self.request.POST.get('m_device_name')
do_something()
self.send_response(200)
try:
httpd = socketserver.TCPServer(("localhost", 9001), MyHandler)
httpd.serve_forever()
finally:
httpd = socketserver.TCPServer(("localhost", 9001), MyHandler)
httpd.server_close()
But It's not working at all. can you tell me why?
answered Nov 19 at 8:49
jyoti
15
add a comment |
up vote
0
down vote
up vote
0
down vote
The code I'm using for this:
#!/usr/bin/python3
import socketserver
from http.server import BaseHTTPRequestHandler
import time
import threading
def do_something():
threading.Timer(1.0, do_something).start()
do_other_things
class MyHandler(BaseHTTPRequestHandler):
def do_post(self):
global site_id, first, last, pass1
if self.path == '/do_something':
site_id = self.request.POST.get('m_site_name')
first = self.request.POST.get('m_first_name')
last = self.request.POST.get('m_last_name')
pass1 = self.request.POST.get('m_device_name')
do_something()
self.send_response(200)
try:
httpd = socketserver.TCPServer(("localhost", 9001), MyHandler)
httpd.serve_forever()
finally:
httpd = socketserver.TCPServer(("localhost", 9001), MyHandler)
httpd.server_close()
But It's not working at all. can you tell me why?
answered Nov 19 at 8:49
jyoti
15
The code I'm using for this:
#!/usr/bin/python3
import socketserver
from http.server import BaseHTTPRequestHandler
import time
import threading
def do_something():
threading.Timer(1.0, do_something).start()
do_other_things
class MyHandler(BaseHTTPRequestHandler):
def do_post(self):
global site_id, first, last, pass1
if self.path == '/do_something':
site_id = self.request.POST.get('m_site_name')
first = self.request.POST.get('m_first_name')
last = self.request.POST.get('m_last_name')
pass1 = self.request.POST.get('m_device_name')
do_something()
self.send_response(200)
try:
httpd = socketserver.TCPServer(("localhost", 9001), MyHandler)
httpd.serve_forever()
finally:
httpd = socketserver.TCPServer(("localhost", 9001), MyHandler)
httpd.server_close()
But It's not working at all. can you tell me why?
answered Nov 19 at 8:49
jyoti
15
edited Nov 19 at 9:38
answered Nov 19 at 8:49
jyoti
15
answered Nov 19 at 8:49
jyoti
15
answered Nov 19 at 8:49
jyoti
15
15
add a comment |
add a comment |
up vote
0
down vote
UPDATE
According to the code you posted below, here is a working answer.
#!/usr/bin/python3
import socketserver
from http.server import BaseHTTPRequestHandler
import time
import threading
def do_something(site_id, first, last, pass1):
print(site_id)
print(first)
print(last)
print(pass1)
#just to illustrate the point and print the variables
class MyHandler(BaseHTTPRequestHandler):
def do_POST(self): # !important to use 'do_POST' with Capital POST
global site_id, first, last, pass1 #those are still undefined at the module level ;) remember this for later
if self.path == '/do_something':
request_headers = self.headers
site_id = request_headers["m_site_name"]
first = request_headers["m_first_name"]
last = request_headers["m_last_name"]
pass1 = request_headers["m_device_name"]
do_something(site_id, first, last, pass1)
self.send_response(200)
self.end_headers() #as of P3.3 this is required
try:
httpd = socketserver.TCPServer(("localhost", 9001), MyHandler)
httpd.serve_forever()
finally:
httpd = socketserver.TCPServer(("localhost", 9001), MyHandler)
httpd.server_close()
Calling it with Postman
and the command line output is
C:DevelopmentPythontestvenvScriptspython.exe C:/Development/Python/test/webserver_old.py
1001
jyoti0
127.0.0.1 - - [19/Nov/2018 21:53:45] "POST /do_something HTTP/1.1" 200 -
jyoti1
101
I combined answers from these here:
Reference one, two and third
And this is also very important to read:
https://docs.python.org/3/library/http.server.html
http.server is not recommended for production. It only implements basic security checks.
I believe is ok for a small implementation and some testing around or proof of concept but eventually you'll need to manage this better, maybe I can suggest you to spend some time and use Flask, is actually an excellent & very light framework for Python API building and prototyping.
-
Previous answer (deprecated & updated above)
-
As per a very light and simple reference to this one:
def do_POST(self):
# Doesn't do anything with posted data
content_length = int(self.headers['Content-Length']) # <--- Gets the size of data
post_data = self.rfile.read(content_length) # <--- Gets the data itself
self._set_headers()
self.wfile.write("<html><body><h1>POST!</h1></body></html>")
Update (without and API):
Assuming you are running on or machine at a custom port with a custom trailing part at the URL, then "pure" python would look like this:
import SocketServer
from BaseHTTPServer import BaseHTTPRequestHandler
def doSomething():
print "i did"
class MyHandler(BaseHTTPRequestHandler):
def do_POST(self):
if self.path == '/doSomething':
mail = self.request.POST.get('email')
something = self.request.POST.get('something')
doSomething()
self.send_response(200)
httpd = SocketServer.TCPServer(("", 8080), MyHandler)
httpd.serve_forever()
I assume this way you can reuse the variables freely. Check also this reference here, Brenda's answer.
answered Nov 17 at 6:28
oetoni
730621
I am guessing "post_data = self.rfile.read(content_length)" will save all of the data. Is there a way to save specific data as variables, like: user = username mail = xyz@abc.com place = anywhere so that I can use the variables later?
– jyoti
Nov 17 at 6:57
@jyoti, check again I updated the answer ;) In the first example with .read() you can also check the content during debugging and later understand from what you discover how to extract them as needed
– oetoni
Nov 17 at 7:09
I will implement both of your suggestions and will let you know about the results. I want to access the pythonscript.py file through a Web Browser and when the script is accessed via web browser the script will be executed and as a result data from the HTTP POST will be extracted and inserted into a database. Unfortunately, I wouldn't implement your suggestion today but I surely will do that on Monday and will update you accordingly. Although thanks for the help, I had no clue about where to start or How to begin with in this regard.
– jyoti
Nov 17 at 7:32
don't mention it! I hope it helps... please know that the script itself should run as a web service or under and api, to make life easier you can use django or flask but if you are insisting on using non of them then try the second one and only check the content of post ;) you can control the port 8080 as per your preference and the script will run under localhost:8080 (or any port :) )
– oetoni
Nov 17 at 7:35
I know nothing of Django or flask. That's why I didn't use any of them. I don't know how to use or implement django or flask.
– jyoti
Nov 17 at 8:20
|
show 8 more comments
up vote
0
down vote
UPDATE
According to the code you posted below, here is a working answer.
#!/usr/bin/python3
import socketserver
from http.server import BaseHTTPRequestHandler
import time
import threading
def do_something(site_id, first, last, pass1):
print(site_id)
print(first)
print(last)
print(pass1)
#just to illustrate the point and print the variables
class MyHandler(BaseHTTPRequestHandler):
def do_POST(self): # !important to use 'do_POST' with Capital POST
global site_id, first, last, pass1 #those are still undefined at the module level ;) remember this for later
if self.path == '/do_something':
request_headers = self.headers
site_id = request_headers["m_site_name"]
first = request_headers["m_first_name"]
last = request_headers["m_last_name"]
pass1 = request_headers["m_device_name"]
do_something(site_id, first, last, pass1)
self.send_response(200)
self.end_headers() #as of P3.3 this is required
try:
httpd = socketserver.TCPServer(("localhost", 9001), MyHandler)
httpd.serve_forever()
finally:
httpd = socketserver.TCPServer(("localhost", 9001), MyHandler)
httpd.server_close()
Calling it with Postman
and the command line output is
C:DevelopmentPythontestvenvScriptspython.exe C:/Development/Python/test/webserver_old.py
1001
jyoti0
127.0.0.1 - - [19/Nov/2018 21:53:45] "POST /do_something HTTP/1.1" 200 -
jyoti1
101
I combined answers from these here:
Reference one, two and third
And this is also very important to read:
https://docs.python.org/3/library/http.server.html
http.server is not recommended for production. It only implements basic security checks.
I believe is ok for a small implementation and some testing around or proof of concept but eventually you'll need to manage this better, maybe I can suggest you to spend some time and use Flask, is actually an excellent & very light framework for Python API building and prototyping.
-
Previous answer (deprecated & updated above)
-
As per a very light and simple reference to this one:
def do_POST(self):
# Doesn't do anything with posted data
content_length = int(self.headers['Content-Length']) # <--- Gets the size of data
post_data = self.rfile.read(content_length) # <--- Gets the data itself
self._set_headers()
self.wfile.write("<html><body><h1>POST!</h1></body></html>")
Update (without and API):
Assuming you are running on or machine at a custom port with a custom trailing part at the URL, then "pure" python would look like this:
import SocketServer
from BaseHTTPServer import BaseHTTPRequestHandler
def doSomething():
print "i did"
class MyHandler(BaseHTTPRequestHandler):
def do_POST(self):
if self.path == '/doSomething':
mail = self.request.POST.get('email')
something = self.request.POST.get('something')
doSomething()
self.send_response(200)
httpd = SocketServer.TCPServer(("", 8080), MyHandler)
httpd.serve_forever()
I assume this way you can reuse the variables freely. Check also this reference here, Brenda's answer.
answered Nov 17 at 6:28
oetoni
730621
I am guessing "post_data = self.rfile.read(content_length)" will save all of the data. Is there a way to save specific data as variables, like: user = username mail = xyz@abc.com place = anywhere so that I can use the variables later?
– jyoti
Nov 17 at 6:57
@jyoti, check again I updated the answer ;) In the first example with .read() you can also check the content during debugging and later understand from what you discover how to extract them as needed
– oetoni
Nov 17 at 7:09
I will implement both of your suggestions and will let you know about the results. I want to access the pythonscript.py file through a Web Browser and when the script is accessed via web browser the script will be executed and as a result data from the HTTP POST will be extracted and inserted into a database. Unfortunately, I wouldn't implement your suggestion today but I surely will do that on Monday and will update you accordingly. Although thanks for the help, I had no clue about where to start or How to begin with in this regard.
– jyoti
Nov 17 at 7:32
don't mention it! I hope it helps... please know that the script itself should run as a web service or under and api, to make life easier you can use django or flask but if you are insisting on using non of them then try the second one and only check the content of post ;) you can control the port 8080 as per your preference and the script will run under localhost:8080 (or any port :) )
– oetoni
Nov 17 at 7:35
I know nothing of Django or flask. That's why I didn't use any of them. I don't know how to use or implement django or flask.
– jyoti
Nov 17 at 8:20
|
show 8 more comments
up vote
0
down vote
up vote
0
down vote
UPDATE
According to the code you posted below, here is a working answer.
#!/usr/bin/python3
import socketserver
from http.server import BaseHTTPRequestHandler
import time
import threading
def do_something(site_id, first, last, pass1):
print(site_id)
print(first)
print(last)
print(pass1)
#just to illustrate the point and print the variables
class MyHandler(BaseHTTPRequestHandler):
def do_POST(self): # !important to use 'do_POST' with Capital POST
global site_id, first, last, pass1 #those are still undefined at the module level ;) remember this for later
if self.path == '/do_something':
request_headers = self.headers
site_id = request_headers["m_site_name"]
first = request_headers["m_first_name"]
last = request_headers["m_last_name"]
pass1 = request_headers["m_device_name"]
do_something(site_id, first, last, pass1)
self.send_response(200)
self.end_headers() #as of P3.3 this is required
try:
httpd = socketserver.TCPServer(("localhost", 9001), MyHandler)
httpd.serve_forever()
finally:
httpd = socketserver.TCPServer(("localhost", 9001), MyHandler)
httpd.server_close()
Calling it with Postman
and the command line output is
C:DevelopmentPythontestvenvScriptspython.exe C:/Development/Python/test/webserver_old.py
1001
jyoti0
127.0.0.1 - - [19/Nov/2018 21:53:45] "POST /do_something HTTP/1.1" 200 -
jyoti1
101
I combined answers from these here:
Reference one, two and third
And this is also very important to read:
https://docs.python.org/3/library/http.server.html
http.server is not recommended for production. It only implements basic security checks.
I believe is ok for a small implementation and some testing around or proof of concept but eventually you'll need to manage this better, maybe I can suggest you to spend some time and use Flask, is actually an excellent & very light framework for Python API building and prototyping.
-
Previous answer (deprecated & updated above)
-
As per a very light and simple reference to this one:
def do_POST(self):
# Doesn't do anything with posted data
content_length = int(self.headers['Content-Length']) # <--- Gets the size of data
post_data = self.rfile.read(content_length) # <--- Gets the data itself
self._set_headers()
self.wfile.write("<html><body><h1>POST!</h1></body></html>")
Update (without and API):
Assuming you are running on or machine at a custom port with a custom trailing part at the URL, then "pure" python would look like this:
import SocketServer
from BaseHTTPServer import BaseHTTPRequestHandler
def doSomething():
print "i did"
class MyHandler(BaseHTTPRequestHandler):
def do_POST(self):
if self.path == '/doSomething':
mail = self.request.POST.get('email')
something = self.request.POST.get('something')
doSomething()
self.send_response(200)
httpd = SocketServer.TCPServer(("", 8080), MyHandler)
httpd.serve_forever()
I assume this way you can reuse the variables freely. Check also this reference here, Brenda's answer.
answered Nov 17 at 6:28
oetoni
730621
UPDATE
According to the code you posted below, here is a working answer.
#!/usr/bin/python3
import socketserver
from http.server import BaseHTTPRequestHandler
import time
import threading
def do_something(site_id, first, last, pass1):
print(site_id)
print(first)
print(last)
print(pass1)
#just to illustrate the point and print the variables
class MyHandler(BaseHTTPRequestHandler):
def do_POST(self): # !important to use 'do_POST' with Capital POST
global site_id, first, last, pass1 #those are still undefined at the module level ;) remember this for later
if self.path == '/do_something':
request_headers = self.headers
site_id = request_headers["m_site_name"]
first = request_headers["m_first_name"]
last = request_headers["m_last_name"]
pass1 = request_headers["m_device_name"]
do_something(site_id, first, last, pass1)
self.send_response(200)
self.end_headers() #as of P3.3 this is required
try:
httpd = socketserver.TCPServer(("localhost", 9001), MyHandler)
httpd.serve_forever()
finally:
httpd = socketserver.TCPServer(("localhost", 9001), MyHandler)
httpd.server_close()
Calling it with Postman
and the command line output is
C:DevelopmentPythontestvenvScriptspython.exe C:/Development/Python/test/webserver_old.py
1001
jyoti0
127.0.0.1 - - [19/Nov/2018 21:53:45] "POST /do_something HTTP/1.1" 200 -
jyoti1
101
I combined answers from these here:
Reference one, two and third
And this is also very important to read:
https://docs.python.org/3/library/http.server.html
http.server is not recommended for production. It only implements basic security checks.
I believe is ok for a small implementation and some testing around or proof of concept but eventually you'll need to manage this better, maybe I can suggest you to spend some time and use Flask, is actually an excellent & very light framework for Python API building and prototyping.
-
Previous answer (deprecated & updated above)
-
As per a very light and simple reference to this one:
def do_POST(self):
# Doesn't do anything with posted data
content_length = int(self.headers['Content-Length']) # <--- Gets the size of data
post_data = self.rfile.read(content_length) # <--- Gets the data itself
self._set_headers()
self.wfile.write("<html><body><h1>POST!</h1></body></html>")
Update (without and API):
Assuming you are running on or machine at a custom port with a custom trailing part at the URL, then "pure" python would look like this:
import SocketServer
from BaseHTTPServer import BaseHTTPRequestHandler
def doSomething():
print "i did"
class MyHandler(BaseHTTPRequestHandler):
def do_POST(self):
if self.path == '/doSomething':
mail = self.request.POST.get('email')
something = self.request.POST.get('something')
doSomething()
self.send_response(200)
httpd = SocketServer.TCPServer(("", 8080), MyHandler)
httpd.serve_forever()
I assume this way you can reuse the variables freely. Check also this reference here, Brenda's answer.
answered Nov 17 at 6:28
oetoni
730621
edited Nov 19 at 21:06
answered Nov 17 at 6:28
oetoni
730621
answered Nov 17 at 6:28
oetoni
730621
answered Nov 17 at 6:28
oetoni
730621
730621
I am guessing "post_data = self.rfile.read(content_length)" will save all of the data. Is there a way to save specific data as variables, like: user = username mail = xyz@abc.com place = anywhere so that I can use the variables later?
– jyoti
Nov 17 at 6:57
@jyoti, check again I updated the answer ;) In the first example with .read() you can also check the content during debugging and later understand from what you discover how to extract them as needed
– oetoni
Nov 17 at 7:09
I will implement both of your suggestions and will let you know about the results. I want to access the pythonscript.py file through a Web Browser and when the script is accessed via web browser the script will be executed and as a result data from the HTTP POST will be extracted and inserted into a database. Unfortunately, I wouldn't implement your suggestion today but I surely will do that on Monday and will update you accordingly. Although thanks for the help, I had no clue about where to start or How to begin with in this regard.
– jyoti
Nov 17 at 7:32
don't mention it! I hope it helps... please know that the script itself should run as a web service or under and api, to make life easier you can use django or flask but if you are insisting on using non of them then try the second one and only check the content of post ;) you can control the port 8080 as per your preference and the script will run under localhost:8080 (or any port :) )
– oetoni
Nov 17 at 7:35
I know nothing of Django or flask. That's why I didn't use any of them. I don't know how to use or implement django or flask.
– jyoti
Nov 17 at 8:20
|
show 8 more comments
I am guessing "post_data = self.rfile.read(content_length)" will save all of the data. Is there a way to save specific data as variables, like: user = username mail = xyz@abc.com place = anywhere so that I can use the variables later?
– jyoti
Nov 17 at 6:57
@jyoti, check again I updated the answer ;) In the first example with .read() you can also check the content during debugging and later understand from what you discover how to extract them as needed
– oetoni
Nov 17 at 7:09
I will implement both of your suggestions and will let you know about the results. I want to access the pythonscript.py file through a Web Browser and when the script is accessed via web browser the script will be executed and as a result data from the HTTP POST will be extracted and inserted into a database. Unfortunately, I wouldn't implement your suggestion today but I surely will do that on Monday and will update you accordingly. Although thanks for the help, I had no clue about where to start or How to begin with in this regard.
– jyoti
Nov 17 at 7:32
don't mention it! I hope it helps... please know that the script itself should run as a web service or under and api, to make life easier you can use django or flask but if you are insisting on using non of them then try the second one and only check the content of post ;) you can control the port 8080 as per your preference and the script will run under localhost:8080 (or any port :) )
– oetoni
Nov 17 at 7:35
I know nothing of Django or flask. That's why I didn't use any of them. I don't know how to use or implement django or flask.
– jyoti
Nov 17 at 8:20
I am guessing "post_data = self.rfile.read(content_length)" will save all of the data. Is there a way to save specific data as variables, like: user = username mail = xyz@abc.com place = anywhere so that I can use the variables later?
– jyoti
Nov 17 at 6:57
I am guessing "post_data = self.rfile.read(content_length)" will save all of the data. Is there a way to save specific data as variables, like: user = username mail = xyz@abc.com place = anywhere so that I can use the variables later?
– jyoti
Nov 17 at 6:57
@jyoti, check again I updated the answer ;) In the first example with .read() you can also check the content during debugging and later understand from what you discover how to extract them as needed
– oetoni
Nov 17 at 7:09
@jyoti, check again I updated the answer ;) In the first example with .read() you can also check the content during debugging and later understand from what you discover how to extract them as needed
– oetoni
Nov 17 at 7:09
I will implement both of your suggestions and will let you know about the results. I want to access the pythonscript.py file through a Web Browser and when the script is accessed via web browser the script will be executed and as a result data from the HTTP POST will be extracted and inserted into a database. Unfortunately, I wouldn't implement your suggestion today but I surely will do that on Monday and will update you accordingly. Although thanks for the help, I had no clue about where to start or How to begin with in this regard.
– jyoti
Nov 17 at 7:32
I will implement both of your suggestions and will let you know about the results. I want to access the pythonscript.py file through a Web Browser and when the script is accessed via web browser the script will be executed and as a result data from the HTTP POST will be extracted and inserted into a database. Unfortunately, I wouldn't implement your suggestion today but I surely will do that on Monday and will update you accordingly. Although thanks for the help, I had no clue about where to start or How to begin with in this regard.
– jyoti
Nov 17 at 7:32
don't mention it! I hope it helps... please know that the script itself should run as a web service or under and api, to make life easier you can use django or flask but if you are insisting on using non of them then try the second one and only check the content of post ;) you can control the port 8080 as per your preference and the script will run under localhost:8080 (or any port :) )
– oetoni
Nov 17 at 7:35
don't mention it! I hope it helps... please know that the script itself should run as a web service or under and api, to make life easier you can use django or flask but if you are insisting on using non of them then try the second one and only check the content of post ;) you can control the port 8080 as per your preference and the script will run under localhost:8080 (or any port :) )
– oetoni
Nov 17 at 7:35
I know nothing of Django or flask. That's why I didn't use any of them. I don't know how to use or implement django or flask.
– jyoti
Nov 17 at 8:20
I know nothing of Django or flask. That's why I didn't use any of them. I don't know how to use or implement django or flask.
– jyoti
Nov 17 at 8:20
|
show 8 more comments
up vote
0
down vote
@oetoni, I am getting time out error while using:
#!/usr/bin/python3
import socketserver
from http.server import BaseHTTPRequestHandler
import time
import threading
def do_something(site_id, first, last, pass1):
print(site_id)
print(first)
print(last)
print(pass1)
#just to illustrate the point and print the variables
class MyHandler(BaseHTTPRequestHandler):
def do_POST(self): # !important to use 'do_POST' with Capital POST
global site_id, first, last, pass1 #those are still undefined at the module level ;) remember this for later
if self.path == '/do_something':
request_headers = self.headers
site_id = request_headers["m_site_name"]
first = request_headers["m_first_name"]
last = request_headers["m_last_name"]
pass1 = request_headers["m_device_name"]
do_something(site_id, first, last, pass1)
self.send_response(200)
self.end_headers() #as of P3.3 this is required
try:
httpd = socketserver.TCPServer(("localhost", 9001), MyHandler)
httpd.serve_forever()
finally:
httpd = socketserver.TCPServer(("localhost", 9001), MyHandler)
httpd.server_close()
But I am getting the correct response while using this code:
#!/usr/bin/python3
# -*- coding: UTF-8 -*-
import cgi
import cgitb
cgitb.enable()
print('Content-Type: text/html')
print('')
arguments = cgi.FieldStorage()
for i in arguments.keys():
print(arguments[i].value)
and it prints the received data on the web browser.
I am using this script as a cgi script on the apache web server which can be accessed through a web browser. I am not running this script as a service or application.
answered 2 days ago
jyoti
15
@oetoni, I've found out the problem. It's the port 9001. I am running this script as a cgi script so, I need to use port 80. But when I use port 80, I get this error: AH01215: self.socket.bind(self.server_address): /var/www/html/cgi-bin/anything.py AH01215: PermissionError: [Errno 13] Permission denied: /var/www/html/cgi-bin/anything.py End of script output before headers: anything.py
– jyoti
2 days ago
add a comment |
up vote
0
down vote
@oetoni, I am getting time out error while using:
#!/usr/bin/python3
import socketserver
from http.server import BaseHTTPRequestHandler
import time
import threading
def do_something(site_id, first, last, pass1):
print(site_id)
print(first)
print(last)
print(pass1)
#just to illustrate the point and print the variables
class MyHandler(BaseHTTPRequestHandler):
def do_POST(self): # !important to use 'do_POST' with Capital POST
global site_id, first, last, pass1 #those are still undefined at the module level ;) remember this for later
if self.path == '/do_something':
request_headers = self.headers
site_id = request_headers["m_site_name"]
first = request_headers["m_first_name"]
last = request_headers["m_last_name"]
pass1 = request_headers["m_device_name"]
do_something(site_id, first, last, pass1)
self.send_response(200)
self.end_headers() #as of P3.3 this is required
try:
httpd = socketserver.TCPServer(("localhost", 9001), MyHandler)
httpd.serve_forever()
finally:
httpd = socketserver.TCPServer(("localhost", 9001), MyHandler)
httpd.server_close()
But I am getting the correct response while using this code:
#!/usr/bin/python3
# -*- coding: UTF-8 -*-
import cgi
import cgitb
cgitb.enable()
print('Content-Type: text/html')
print('')
arguments = cgi.FieldStorage()
for i in arguments.keys():
print(arguments[i].value)
and it prints the received data on the web browser.
I am using this script as a cgi script on the apache web server which can be accessed through a web browser. I am not running this script as a service or application.
answered 2 days ago
jyoti
15
@oetoni, I've found out the problem. It's the port 9001. I am running this script as a cgi script so, I need to use port 80. But when I use port 80, I get this error: AH01215: self.socket.bind(self.server_address): /var/www/html/cgi-bin/anything.py AH01215: PermissionError: [Errno 13] Permission denied: /var/www/html/cgi-bin/anything.py End of script output before headers: anything.py
– jyoti
2 days ago
add a comment |
up vote
0
down vote
up vote
0
down vote
@oetoni, I am getting time out error while using:
#!/usr/bin/python3
import socketserver
from http.server import BaseHTTPRequestHandler
import time
import threading
def do_something(site_id, first, last, pass1):
print(site_id)
print(first)
print(last)
print(pass1)
#just to illustrate the point and print the variables
class MyHandler(BaseHTTPRequestHandler):
def do_POST(self): # !important to use 'do_POST' with Capital POST
global site_id, first, last, pass1 #those are still undefined at the module level ;) remember this for later
if self.path == '/do_something':
request_headers = self.headers
site_id = request_headers["m_site_name"]
first = request_headers["m_first_name"]
last = request_headers["m_last_name"]
pass1 = request_headers["m_device_name"]
do_something(site_id, first, last, pass1)
self.send_response(200)
self.end_headers() #as of P3.3 this is required
try:
httpd = socketserver.TCPServer(("localhost", 9001), MyHandler)
httpd.serve_forever()
finally:
httpd = socketserver.TCPServer(("localhost", 9001), MyHandler)
httpd.server_close()
But I am getting the correct response while using this code:
#!/usr/bin/python3
# -*- coding: UTF-8 -*-
import cgi
import cgitb
cgitb.enable()
print('Content-Type: text/html')
print('')
arguments = cgi.FieldStorage()
for i in arguments.keys():
print(arguments[i].value)
and it prints the received data on the web browser.
I am using this script as a cgi script on the apache web server which can be accessed through a web browser. I am not running this script as a service or application.
answered 2 days ago
jyoti
15
@oetoni, I am getting time out error while using:
#!/usr/bin/python3
import socketserver
from http.server import BaseHTTPRequestHandler
import time
import threading
def do_something(site_id, first, last, pass1):
print(site_id)
print(first)
print(last)
print(pass1)
#just to illustrate the point and print the variables
class MyHandler(BaseHTTPRequestHandler):
def do_POST(self): # !important to use 'do_POST' with Capital POST
global site_id, first, last, pass1 #those are still undefined at the module level ;) remember this for later
if self.path == '/do_something':
request_headers = self.headers
site_id = request_headers["m_site_name"]
first = request_headers["m_first_name"]
last = request_headers["m_last_name"]
pass1 = request_headers["m_device_name"]
do_something(site_id, first, last, pass1)
self.send_response(200)
self.end_headers() #as of P3.3 this is required
try:
httpd = socketserver.TCPServer(("localhost", 9001), MyHandler)
httpd.serve_forever()
finally:
httpd = socketserver.TCPServer(("localhost", 9001), MyHandler)
httpd.server_close()
But I am getting the correct response while using this code:
#!/usr/bin/python3
# -*- coding: UTF-8 -*-
import cgi
import cgitb
cgitb.enable()
print('Content-Type: text/html')
print('')
arguments = cgi.FieldStorage()
for i in arguments.keys():
print(arguments[i].value)
and it prints the received data on the web browser.
I am using this script as a cgi script on the apache web server which can be accessed through a web browser. I am not running this script as a service or application.
answered 2 days ago
jyoti
15
answered 2 days ago
jyoti
15
answered 2 days ago
jyoti
15
answered 2 days ago
jyoti
15
15
@oetoni, I've found out the problem. It's the port 9001. I am running this script as a cgi script so, I need to use port 80. But when I use port 80, I get this error: AH01215: self.socket.bind(self.server_address): /var/www/html/cgi-bin/anything.py AH01215: PermissionError: [Errno 13] Permission denied: /var/www/html/cgi-bin/anything.py End of script output before headers: anything.py
– jyoti
2 days ago
add a comment |
@oetoni, I've found out the problem. It's the port 9001. I am running this script as a cgi script so, I need to use port 80. But when I use port 80, I get this error: AH01215: self.socket.bind(self.server_address): /var/www/html/cgi-bin/anything.py AH01215: PermissionError: [Errno 13] Permission denied: /var/www/html/cgi-bin/anything.py End of script output before headers: anything.py
– jyoti
2 days ago
@oetoni, I've found out the problem. It's the port 9001. I am running this script as a cgi script so, I need to use port 80. But when I use port 80, I get this error: AH01215: self.socket.bind(self.server_address): /var/www/html/cgi-bin/anything.py AH01215: PermissionError: [Errno 13] Permission denied: /var/www/html/cgi-bin/anything.py End of script output before headers: anything.py
– jyoti
2 days ago
@oetoni, I've found out the problem. It's the port 9001. I am running this script as a cgi script so, I need to use port 80. But when I use port 80, I get this error: AH01215: self.socket.bind(self.server_address): /var/www/html/cgi-bin/anything.py AH01215: PermissionError: [Errno 13] Permission denied: /var/www/html/cgi-bin/anything.py End of script output before headers: anything.py
– jyoti
2 days ago
add a comment |
up vote
0
down vote
accepted
#!/usr/bin/python3
# -*- coding: UTF-8 -*-
import cgi
import cgitb
cgitb.enable()
print('Content-Type: text/htmln')
arguments = cgi.FieldStorage()
print(arguments["m_site_name"].value)
print("<br />n")
print(arguments["m_first_name"].value)
print("<br />n")
print(arguments["m_last_name"].value)
print("<br />n")
print(arguments["m_device_name"].value)
print("<br />n")
site = arguments["m_site_name"].value
first = arguments["m_first_name"].value
last = arguments["m_last_name"].value
device = arguments["m_device_name"].value
-----do_other_things_with_the_variables(site,first,last,device)-----
This code solved my problem. Now I can store HTTP POST data into variables with this python cgi script.
my HTTP POST Request:
http://your_server_url_or_IP/cgi-bin/python_script.py?m_site_name=MySite&m_first_name=anyname&m_last_name=anylastanme&m_device_name=anydeviceidorname
answered 2 days ago
jyoti
15
add a comment |
up vote
0
down vote
accepted
#!/usr/bin/python3
# -*- coding: UTF-8 -*-
import cgi
import cgitb
cgitb.enable()
print('Content-Type: text/htmln')
arguments = cgi.FieldStorage()
print(arguments["m_site_name"].value)
print("<br />n")
print(arguments["m_first_name"].value)
print("<br />n")
print(arguments["m_last_name"].value)
print("<br />n")
print(arguments["m_device_name"].value)
print("<br />n")
site = arguments["m_site_name"].value
first = arguments["m_first_name"].value
last = arguments["m_last_name"].value
device = arguments["m_device_name"].value
-----do_other_things_with_the_variables(site,first,last,device)-----
This code solved my problem. Now I can store HTTP POST data into variables with this python cgi script.
my HTTP POST Request:
http://your_server_url_or_IP/cgi-bin/python_script.py?m_site_name=MySite&m_first_name=anyname&m_last_name=anylastanme&m_device_name=anydeviceidorname
answered 2 days ago
jyoti
15
add a comment |
up vote
0
down vote
accepted
up vote
0
down vote
accepted
#!/usr/bin/python3
# -*- coding: UTF-8 -*-
import cgi
import cgitb
cgitb.enable()
print('Content-Type: text/htmln')
arguments = cgi.FieldStorage()
print(arguments["m_site_name"].value)
print("<br />n")
print(arguments["m_first_name"].value)
print("<br />n")
print(arguments["m_last_name"].value)
print("<br />n")
print(arguments["m_device_name"].value)
print("<br />n")
site = arguments["m_site_name"].value
first = arguments["m_first_name"].value
last = arguments["m_last_name"].value
device = arguments["m_device_name"].value
-----do_other_things_with_the_variables(site,first,last,device)-----
This code solved my problem. Now I can store HTTP POST data into variables with this python cgi script.
my HTTP POST Request:
http://your_server_url_or_IP/cgi-bin/python_script.py?m_site_name=MySite&m_first_name=anyname&m_last_name=anylastanme&m_device_name=anydeviceidorname
answered 2 days ago
jyoti
15
#!/usr/bin/python3
# -*- coding: UTF-8 -*-
import cgi
import cgitb
cgitb.enable()
print('Content-Type: text/htmln')
arguments = cgi.FieldStorage()
print(arguments["m_site_name"].value)
print("<br />n")
print(arguments["m_first_name"].value)
print("<br />n")
print(arguments["m_last_name"].value)
print("<br />n")
print(arguments["m_device_name"].value)
print("<br />n")
site = arguments["m_site_name"].value
first = arguments["m_first_name"].value
last = arguments["m_last_name"].value
device = arguments["m_device_name"].value
-----do_other_things_with_the_variables(site,first,last,device)-----
This code solved my problem. Now I can store HTTP POST data into variables with this python cgi script.
my HTTP POST Request:
http://your_server_url_or_IP/cgi-bin/python_script.py?m_site_name=MySite&m_first_name=anyname&m_last_name=anylastanme&m_device_name=anydeviceidorname
answered 2 days ago
jyoti
15
edited 2 days ago
answered 2 days ago
jyoti
15
answered 2 days ago
jyoti
15
answered 2 days ago
jyoti
15
15
add a comment |
add a comment |
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53348748%2fhow-to-extract-data-from-incoming-http-post-using-python%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Are you using Flask?
– oetoni
Nov 17 at 6:20
No, I am not using any api. I want to make a request to a python script like anuurl.com/cgi-bin/pythonscript.py and then the script executes and extracts the data from HTTP POST request.
– jyoti
Nov 17 at 6:50
hi @jyoti, i updated the answer ;) check below
– oetoni
Nov 19 at 21:10