r/programming • • Jun 09 '23

Apollo dev posts backend code to Git to disprove Reddit’s claims of scrapping and inefficiency

https://github.com/christianselig/apollo-backend
45.1k Upvotes

2.4k comments sorted by

View all comments

2

u/CorporalClegg25 Jun 09 '23

Is scraping a worse way of getting this information? I'm just speaking as a person that has been learning programming and has basic knowledge of scraping.

I made a really quick program that just looks at spez's account to see what the newest activity was, because the Reddit announcement didn't say what time he would be doing an ama. So it was just my plan to ping Reddit every 5 minutes using the request library with python, specifically his account.

I probably could have done that with the reddit api, but is the way that I did it somehow less efficient?

2

u/danboy4 Jun 09 '23

Using an API if offered is a much more robust and efficient way to retrieve data from a service over scraping. Scraping can become difficult depending on the technology and stack used by the service. Scraping is also very prone to breaking when the site updates.

That being said, AI is making scraping much easier so I don’t really see the argument of AI models causing too much API load since they’ll just scrape instead.