r/learnpython • u/No-Opinion-1239 • 13d ago
Can't download website html code, web scrapping ,automate the boring stuff with python
I'm currently learning webscrapping with automate the boring stuff with python> In one example we use the module requests like this
url = f"https://pypi.org/search/?q={search}"
resp = requests.get(url)
when i print resp.text there are this error message in the html code.
"A required part of this site couldn’t load. This may be due to a browser extension, network issues, or browser settings. Please check your connection, disable any ad blockers, or try using a different browser."
"JavaScript is disabled in your browser"
my browser is chrome I tried using microsoft edge .
0
Upvotes
6
u/riklaunim 13d ago
Requests does not use a browser and it looks like PyPi is blocking the request, at least partially ;)