0

I am trying to scrape a webpage. The page has a select with many options in it. ON selecting one option, the page reloads with a table corresponding to that option. I want to scrape that table. However, the URL stays the same. The only change happens in the form data which on inspection has the value of option selected and a csrf_token. How do I replicate the behaviour in python?

My code:

client = requests.session() url = "https://recruitment.advarisk.com/tests/scraping" client.get(url) csrftoken = client.cookies['csrftoken'] return csrftoken
Improve this question
2
  • As I don't want to flag your post, here's a possible answer : stackoverflow.com/questions/4476373/… Commented Jun 27, 2017 at 9:31
  • @Alteus I did try that. But I get Bad Request. CSRF token missing or incorrect. I got the csrf value from xpath. But when I post it in data parameter it gives that error. Commented Jun 27, 2017 at 9:38
Load 7 more related questions Show fewer related questions

0

Reset to default

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.