Python send POST with header

Question

I try to build a python script who sends a POST with parameters for extracting the result. With fiddler, I have extracted the post request who return that I want. The website uses https only.

POST /Services/GetFromDataBaseVersionned HTTP/1.1 Host: www.mywbsite.fr "Connection": "keep-alive", "Content-Length": 129, "Origin": "https://www.mywbsite.fr", "X-Requested-With": "XMLHttpRequest", "User-Agent": "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/536.5 (KHTML, like Gecko) Chrome/19.0.1084.52 Safari/536.5", "Content-Type": "application/json", "Accept": "*/*", "Referer": "https://www.mywbsite.fr/data/mult.aspx", "Accept-Encoding": "gzip,deflate,sdch", "Accept-Language": "fr-FR,fr;q=0.8,en-US;q=0.6,en;q=0.4", "Accept-Charset": "ISO-8859-1,utf-8;q=0.7,*;q=0.3", "Cookie": "ASP.NET_SessionId=j1r1b2a2v2w245; GSFV=FirstVisit=; GSRef=https://www.google.fr/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&ved=0CHgQFjAA&url=https://www.mywbsite.fr/&ei=FZq_T4abNcak0QWZ0vnWCg&usg=AFQjCNHq90dwj5RiEfr1Pw; HelpRotatorCookie=HelpLayerWasSeen=0; NSC_GSPOUGS!TTM=ffffffff09f4f58455e445a4a423660; GS=Site=frfr; __utma=1.219229010.1337956889.1337956889.1337958824.2; __utmb=1.1.10.1337958824; __utmc=1; __utmz=1.1337956889.1.1.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=(not%20provided)" {"isLeftColumn":false,"lID":-1,"userIpCountryCode":"FR","version":null,"languageCode":"fr","siteCode":"frfr","Quotation":"eu"}

And now my python script:

#!/usr/bin/env python # -*- coding: iso-8859-1 -*- import string import httplib import urllib2 host = "www.mywbsite.fr/sport/multiplex.aspx" params='"isLeftColumn":"false","liveID":"-1","userIpCountryCode":"FR","version":"null","languageCode":"fr","siteCode":"frfr","Quotation":"eu"' headers = { Host: www.mywbsite.fr, "Connection": "keep-alive", "Content-Length": 129, "Origin": "https://www.mywbsite.fr", "X-Requested-With": "XMLHttpRequest", "User-Agent": "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/536.5 (KHTML, like Gecko) Chrome/19.0.1084.52 Safari/536.5", "Content-Type": "application/json", "Accept": "*/*", "Referer": "https://www.mywbsite.fr/data/mult.aspx", "Accept-Encoding": "gzip,deflate,sdch", "Accept-Language": "fr-FR,fr;q=0.8,en-US;q=0.6,en;q=0.4", "Accept-Charset": "ISO-8859-1,utf-8;q=0.7,*;q=0.3", "Cookie": "ASP.NET_SessionId=j1r1b2a2v2w245; GSFV=FirstVisit=; GSRef=https://www.google.fr/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&ved=0CHgQFjAA&url=https://www.mywbsite.fr/&ei=FZq_T4abNcak0QWZ0vnWCg&usg=AFQjCNHq90dwj5RiEfr1Pw; HelpRotatorCookie=HelpLayerWasSeen=0; NSC_GSPOUGS!TTM=ffffffff09f4f58455e445a4a423660; GS=Site=frfr; __utma=1.219229010.1337956889.1337956889.1337958824.2; __utmb=1.1.10.1337958824; __utmc=1; __utmz=1.1337956889.1.1.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=(not%20provided)" } url = "/Services/GetFromDataBaseVersionned" # POST the request conn = httplib.HTTPConnection(host,port=443) conn.request("POST",url,params,headers) response = conn.getresponse() data = response.read() print data

But when I run my script, I have this error:

socket.gaierror: [Errno -2] Name or service not known

There is no good reason to speak HTTP manually. Use urllib or even better, requests. — ThiefMaster
– ThiefMaster, Commented May 26, 2012 at 17:53
Refer this answer for better explanation. stackoverflow.com/a/51124572/7384392 — Nitanshu
– Nitanshu, Commented Jul 1, 2018 at 18:10

simhumileco · Accepted Answer · 2019-02-01 21:39:08Z

Thanks a lot for your link to the requests module. It's just perfect. Below the solution to my problem.

import requests import json url = 'https://www.mywbsite.fr/Services/GetFromDataBaseVersionned' payload = { "Host": "www.mywbsite.fr", "Connection": "keep-alive", "Content-Length": 129, "Origin": "https://www.mywbsite.fr", "X-Requested-With": "XMLHttpRequest", "User-Agent": "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/536.5 (KHTML, like Gecko) Chrome/19.0.1084.52 Safari/536.5", "Content-Type": "application/json", "Accept": "*/*", "Referer": "https://www.mywbsite.fr/data/mult.aspx", "Accept-Encoding": "gzip,deflate,sdch", "Accept-Language": "fr-FR,fr;q=0.8,en-US;q=0.6,en;q=0.4", "Accept-Charset": "ISO-8859-1,utf-8;q=0.7,*;q=0.3", "Cookie": "ASP.NET_SessionId=j1r1b2a2v2w245; GSFV=FirstVisit=; GSRef=https://www.google.fr/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&ved=0CHgQFjAA&url=https://www.mywbsite.fr/&ei=FZq_T4abNcak0QWZ0vnWCg&usg=AFQjCNHq90dwj5RiEfr1Pw; HelpRotatorCookie=HelpLayerWasSeen=0; NSC_GSPOUGS!TTM=ffffffff09f4f58455e445a4a423660; GS=Site=frfr; __utma=1.219229010.1337956889.1337956889.1337958824.2; __utmb=1.1.10.1337958824; __utmc=1; __utmz=1.1337956889.1.1.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=(not%20provided)" } # Adding empty header as parameters are being sent in payload headers = {} r = requests.post(url, data=json.dumps(payload), headers=headers) print(r.content)

Why are you sending the header as the data of the request and why that works?
FYI you should NOT manually send headers like this. It's request's job. The maximum you should have to send is the user agent and custom headers.
@Engin Yapici Usually we Don't need to dump our data as json but in some special case we need it. Such as sending post request in FASTAPI, It's requires to dump our data as json.
@Willysatrionugroho - agreed. I was trying to pass a payload without json.dumps() and it failed, got very frustrated. Found this answer, used json.dumps() and the API works like a charm.

simhumileco · Accepted Answer · 2019-03-12 16:52:45Z

If we want to add custom HTTP headers to a POST request, we must pass them through a dictionary to the headers parameter.

Here is an example with a non-empty body and headers:

import requests import json url = 'https://somedomain.com' body = {'name': 'Maryja'} headers = {'content-type': 'application/json'} r = requests.post(url, data=json.dumps(body), headers=headers)

Source

This reply is correct, worked for me using body to post specific keys that are supported by the server

simhumileco · Accepted Answer · 2019-12-06 17:39:18Z

To make POST request instead of GET request using urllib2, you need to specify empty data, for example:

import urllib2 req = urllib2.Request("http://am.domain.com:8080/openam/json/realms/root/authenticate?authIndexType=Module&authIndexValue=LDAP") req.add_header('X-OpenAM-Username', 'demo') req.add_data('') r = urllib2.urlopen(req)

Collectives™ on Stack Overflow

Python send POST with header

3 Answers 3

10 Comments

1 Comment

Comments

Linked

Hot Network Questions