Saturday, 31 August 2013

python consecutive urllib2 POST gives 404

python consecutive urllib2 POST gives 404

The problem that I have - and try to solve with Python - is to make
consecutive POST requests (completing an online form) for a website
(specifically, a free online demo of an api at
demo.travelportuniversalapi.com). I am not able to acquire the results
page so far - been at this for two days now.
The code I employ is:
import sys
import urllib, urllib2, cookielib
from BeautifulSoup import BeautifulSoup
import re
class website:
def __init__(self):
self.host = 'demo.travelportuniversalapi.com'
self.ua = 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:23.0)
Gecko/20100101 Firefox/23.0'
self.session = cookielib.CookieJar() #session devine o
instanta a obiectului cookielib
pass
def get(self):
try:
url =
'http://demo.travelportuniversalapi.com/(S(cexfuhghvlzyzx5n0ysesra1))/Search'
#this varies every 20 minutes
data = None
headers = {'User-Agent': self.ua}
request = urllib2.Request(url, data, headers)
self.session.add_cookie_header(request)
response = urllib2.urlopen(request)
self.session.extract_cookies(response, request)
url = response.geturl()
data = {'From': 'lhr', 'To': 'ams', 'Departure' :
'9/4/2013','Return' : '9/6/2013'}
headers = {'User-Agent': self.ua, "Content-type":
"application/x-www-form-urlencoded; charset=UTF-8",
}
request = urllib2.Request(url, urllib.urlencode(data),
headers, 20)
self.session.add_cookie_header(request)
response = urllib2.urlopen(request, timeout=30) #HTTP
Error 404: Not Found - aici am eroare
self.session.extract_cookies(response, request)
except urllib2.URLError as e:
print >> sys.stderr, e
return None
rt = website()
rt.get()
The error that i receive at the last urllib2.Request is HTTP Error 404:
Not Found. I am not sure my cookies are working. Monitoring HTTP packets
with an addon in the browser I noticed the following header when the POST
is sent in a broswer: 'X-Requested-With XMLHttpRequest' - is this
relevant?
Thank you, Regards, Nicoara

No comments:

Post a Comment