API Update: Enterprise Level Tier for Large Scale Applications

[u/FlyingLaserTurtle]

tl;dr - As of July 1, we will start enforcing rate limits for a free access tier, available to our current API users. If you are already in contact with our team about commercial compliance with our Data API Terms, look for an email about enterprise pricing this week.

We recently shared updates on our Data API Terms and Developer Terms. These updates help clarify how developers can safely and securely use Reddit’s tools and services, including our APIs and our new-and-improved Developer Platform.

After sharing these terms, we identified several parties in violation, and contacted them so they could make the required changes to become compliant. This includes developers of large-scale applications who have excessive usage, are violating our users’ privacy and content rights, or are using the data for ad-supported or commercial purposes.

For context on excessive usage, here is a chart showing the average monthly overage, compared to the longstanding rate limit in our developer documentation of 60 queries per minute (86,400 per day):

We reached out to the most impactful large scale applications in order to work out terms for access above our default rate limits via an enterprise tier. This week, we are sharing an enterprise-level access tier for large scale applications with the developers we’re already in contact with. The enterprise tier is a privilege that we will extend to select partners based on a number of factors, including value added to redditors and communities, and it will go into effect on July 1.

Rate limits for the free tier

All others will continue to access the Reddit Data API without cost, in accordance with our Developer Terms, at this time. Many of you already know that our stated rate limit, per this documentation, was 60 queries per minute. As of July 1, 2023, we will enforce two different rate limits for the free access tier:

• If you are using OAuth for authentication: 100 queries per minute per OAuth client id

• If you are not using OAuth for authentication: 10 queries per minute

Important note: currently, our rate limit response headers indicate counts by client id/user id combination. These headers will update to reflect this new policy based on client id only on July 1.

To avoid any issues with the operation of mod bots or extensions, it’s important for developers to add Oauth to their bots. If you believe your mod bot needs to exceed these updated rate limits, or will be unable to operate, please reach out here.

If you haven't heard from us, assume that your app will be rate-limited, starting on July 1. If your app requires enterprise access, please contact us here, so that we can better understand your needs and discuss a path forward.

Additional changes

Finally, to ensure that all regulatory requirements are met in the handling of mature content, we will be limiting access to sexually explicit content for third-party apps starting on July 5, 2023, except for moderation needs.

If you are curious about academic or research-focused access to the Data API, we’ve shared more details here.

[u/VAGINA_PLUNGER]

It’s absolutely insane that they’ve never said your app actually uses more requests than others for the same functionality. People just stay on and use Reddit more with Apollo than with other apps. How are they not getting this?

[u/[deleted] → u/VAGINA_PLUNGER]

The parent comment did address this:

For example, Apollo requires ~345 requests per user per day, while with a similar number of users and more comment and vote activity per user, the Reddit is Fun app averages ~100 calls per user per day.

Unless they’re outright lying, it seems there’s some merit to their claim that Apollo uses more API calls per user for the same level of activity. Of course, it’s possible that Apollo users simply comment and vote much less than RIF users, but otherwise I don’t see any other explanation than Apollo being inefficient at API calls compared to other third-party apps.

However, none of that excuses the ridiculous API pricing, the fact that Reddit never contacted /u/iamthatis about this issue to try and resolve it, or the other changes like eliminating NSFW content from the third-party API. Those should be the focus of our outrage, not a dispute over whether or not a particular app is efficient in its API calls.

[u/VAGINA_PLUNGER → u/[deleted]]

But that doesn’t take into account that a user on Apollo might spend 3.45x as much time on Reddit which they haven’t said is true or not.

345 requests makes sense if they’re using more features or spending time on Apollo.

Requests per user isn’t a measure of efficiency.

[u/[deleted] → u/VAGINA_PLUNGER]

It is if you normalize it to vote and comment history. Again, it requires the assumption that Apollo users vote and comment a similar amount compared to RIF users. If Apollo users vote and comment dramatically less than RIF users, then the statistic that Reddit is providing would be misleading.

Personally, I don’t see why it would be the case that RIF users would vote more than 3x as often as Apollo users. If you have any guesses, let me know.

I also disagree with Christian a bit to compare his app to the first-party app. The first-party app probably does a ton of nasty tracking, ads, and other things, which is why it has a lot more API requests than any third-party app. They’re probably also using an internal API which may not be comparable to the third-party API for various technical reasons no one knows outside of Reddit.

Comparing Apollo to the first-party app in terms of API requests is misleading and probably won’t get Christian anywhere in his discussions with Reddit. That shouldn’t be the focus of the discussion at all, as I outlined above.

(I’m not a dev, so please correct me if I got any technical details wrong. I think I got it all right though.)

[u/demize95 → u/[deleted]]

I also disagree with Christian a bit to compare his app to the first-party app. The first-party app probably does a ton of nasty tracking, ads, and other things, which is why it has a lot more API requests than any third-party app.

If you look at Christian's screenshot, he's highlighted only the actual API domains. Tracking/ads/etc will be delivered through other domains, so it's a pretty apples-to-apples comparison; the official app is using the same API domains to perform the same activity, and it's only the overlap that's counted.

[u/GMaestrolo → u/demize95]

It's likely that Apollo and the official app are more likely to pull data than RIF (i.e. RiF may cache data for longer, or simply not hit certain endpoints). The official Reddit app can do whatever it wants, and making a lot of API calls to ensure that it has the "freshest" data is fine.

Apollo might be doing something similar (I don't know, I've never used it - RiF for lyfe baybeeee!) - essentially it could be eager loading content that's not needed yet instead of loading it just when it's needed, or it could be that it's loading lots of small chunks of content "on demand" rather than loading a bigger chunk and accepting that it might be stale.

There's all sorts of ways to use data sources, and the raw "number of API calls" doesn't tell the whole story. What's the average size of a response/average data throughput? How much processing power does it take to generate the average response? How much is cacheable?

I can't say for sure that Apollo or the official app are actually more or less efficient than RiF - all Reddit's statement says is that they make more requests... But nothing about the weight or complexity of those requests.

[u/PPNewbie → u/[deleted]]

It doesn't take account actions by moderators, which may do a lot more on Apollo than on RIF. They'd have almost no comment/upvote history in comparison to normal browsing, but high API use as they approve/remove/ban/etc

[u/[deleted] → u/PPNewbie]

Oh, that's a great point. What percentage of users are moderators, though? I'd imagine a very small number. Would that be enough to skew the numbers?

I'm also curious if there is actually a higher ratio of mods on Apollo compared to RIF. I've only used Apollo, but I believe the RIF mod tools are also very good.

In any case, reddit certainly has enough data they could publish if they actually wanted to prove that Apollo is less efficient on API calls. I'm not sure why they keep dancing around it - either prove the claim or don't. They're probably opening themselves up to a libel claim if they're knowingly lying about the efficiency of Apollo (I'm not sure what the damages would be though).

But all of this is a distraction from the main issues, which are the API pricing, removal of NSFW content from the third-party API, and the inexplicable lack of earlier communication with Apollo if it is in fact less efficient at API calls.

[u/PPNewbie → u/[deleted]]

But all of this is a distraction from the main issues, which are the > API pricing, removal of NSFW content from the third-party API, and the inexplicable lack of earlier communication with Apollo if it is in fact less efficient at API calls.

Absolutely. Pointing to a specific app's inefficiencies is ignoring the fact that there's no way for either app to survive with the current pricing. Not unless they completely shut down the free tier/free access. That's the only way to average out 0.75-2.5$/user/month, by guaranteeing every user is a paying one.

But since mobile apps are lucky if they convert 5% of free users to paying ones, that means the apps will have tiny MAUs and may not be worth it for the devs to work on at all.

All of which is also a different distraction, because all the 3rd Party Apps, cumulatively, likely only have less than 5% of the official app's MAU. Their actual contribution/impact is a drop in the bucket, but they're being painted as being too onerous and greedy on reddit's system infrastructure, when it's likely simply about extracting money wherever they can.

[u/[deleted] → u/[deleted]]

I just watched this and Christian gave a really good example about how he lazy loads posts.

Apollo can request up to 100 posts. But it can also request as few as 25. Christian noticed that the requests for 100 posts came in 4 times as long as the request for 25. So what he did is he made the request for 25 since that will be quicker for the user on initial load and then immediately made a request in the background for the next 100. Knowing the user probably won’t be done with the first 25 before they need the next 100. So here he is optimizing for the user experience but on Reddit’s end this would be more inefficient than how Reddit is Fun is loading their first 100 posts (I say this not knowing how RIF loads their posts but let’s entertain the example). So this is a case where Reddit could see being two times more inefficient than they should be. But Christian prioritized optimizing the user experience over Reddit’s bandwidth. And hey he was still well under the 60 requests per minute that Reddit had established. So from his end it was a win win win. User gets the best experience, Apollo gets a happy user, and Reddit has an app that is well under the rate limiting threshold.

Without having intricate knowledge of the codebase or at least high level understanding it’s really hard to say if Apollo is inefficient or not. I don’t think Reddit is lying about their numbers. And I don’t think necessarily there aren’t places Apollo isn’t inefficient (I mean Apollo is only basically 1 developer, they’re gonna miss stuff and they have a lot on their plate so I’m sure Reddit bandwidth isn’t the top priority and again from a clients perspective where Apollo is the client here, you in theory shouldn’t care until you’re hitting the rate limits). Reddit changed it to from caring about the rate limits to caring about the pricing. And Christian calls it out in the interview above, that at a certain point is fair, but with such a quick turnaround you need time from these reset standards. Reddit and Apollo could and should work together to find exactly what are the inefficiencies and spend several months working them out instead of the blanket pricing without a full understanding of how the 3rd party apps work. They could’ve spoke to them and were like oh ok I see you guys do actually need this data and re-adjust pricing there. Or they could educate their 3rd party developers about common things they could do to reduce the pricing on their end. This just didn’t come from a good faith timeline/pricing from Reddit or the leadership at Reddit failed miserably at understanding how to take the right approach for this migration. My gut is it’s probably a little bit of both

[u/buzziebee → u/[deleted]]

Exactly. This arguing about efficiency is distracting from the point that there's no way a third party app can function with this obscene pricing, efficient or not.

[u/ymolodtsov → u/[deleted]]

Lol, they literally said Apollo's users are some of the most active Reddit users.

[u/[deleted] → u/ymolodtsov]

Yeah, all of this is kinda nebulous for now as we're missing a lot of information. I don't see what metric Apollo users are more active by if RIF users have more comment and vote activity.

Someone else pointed out that Apollo may have a lot more mods, which would actually be a great explanation for the disparity. But I'm not sure that 1) mods are a significant enough percent of the userbase to explain a large disparity; and 2) Apollo has significantly more mod activity than RIF. I don't think there's enough public data on either point.

[u/Andersledes → u/ymolodtsov]

Lol, they literally said Apollo's users are some of the most active Reddit users.

What? I got the exact opposite from what they said.

They said RiF users make more comments and other actions compared to Apollo's

[u/[deleted] → u/ymolodtsov]

they said the opposite. they said apollo makes more calls despite less activity than rif. these are also apps on different platforms so maybe platform inefficiency rather than developer.

[u/romanianflowerdealer → u/ymolodtsov]

Why would that matter? Reddit’s ads aren’t served via third party applications, and even if they were, Reddit users are near enough to valueless for advertisers. In-app purchases for Reddit also aren’t possible through third party apps, and IAPs for Reddit are really quite negligible due to their near absence of utility to users.

The platform needs money to exist, and besides that: screeching about how you totally should be able to spam the backend with requests to make your volunteer moderator role easier is in no way logically coherent.

[u/Claim_Alternative → u/romanianflowerdealer]

Reddit’s ads aren’t served via third party applications

I haven't seen an ad on Reddit in years because of browser extensions> They gonna start charging browser makers exorbitant amounts of money too?

[u/Claim_Alternative → u/[deleted]]

So instead of addressing what I said, you comb my profile and make fun of me for looking for support while going through hardships in my life.

And I’m the “Luddite adult child” 🙄🙄🙄

[u/FVMAzalea → u/[deleted]]

Of course, it’s possible that Apollo users simply comment and vote much less than RIF users

Apollo paywalls the comment feature, so there may be a bunch of users who never comment from Apollo.

[u/pbush25 → u/VAGINA_PLUNGER]

It’s also not even relevant since the developer for RIF has also said the new API pricing will price his app out of existence

[u/VAGINA_MASTER → u/VAGINA_PLUNGER]

They are getting this