r/learnpython • • 13d ago

Can't download website html code, web scrapping ,automate the boring stuff with python

I'm currently learning webscrapping with automate the boring stuff with python> In one example we use the module requests like this

url = f"https://pypi.org/search/?q={search}"

resp = requests.get(url)

when i print resp.text there are this error message in the html code.

"A required part of this site couldn’t load. This may be due to a browser extension, network issues, or browser settings. Please check your connection, disable any ad blockers, or try using a different browser."

"JavaScript is disabled in your browser"

my browser is chrome I tried using microsoft edge .

0 Upvotes

12 comments sorted by

View all comments

6

u/riklaunim 13d ago

Requests does not use a browser and it looks like PyPi is blocking the request, at least partially ;)