Python http client fails to load a page the browser succeeds in loading

• http
• requests
• urllib2
• urllib

Someone using a HTTP client, such as requests, urllib.client, or urllib2 is trying to load a website, and they are seeing different results from what works for them with a browser. Usually they get an HTTP error or the content they wanted to load is missing.

For example:

• requests.get always gets a 404
• Can’t scrape price information from site XYZ

Note that this is not about trying to log into a site!

• python requests.get always get 404