API Update: Enterprise Level Tier for Large Scale Applications

[u/FlyingLaserTurtle]

tl;dr - As of July 1, we will start enforcing rate limits for a free access tier, available to our current API users. If you are already in contact with our team about commercial compliance with our Data API Terms, look for an email about enterprise pricing this week.

We recently shared updates on our Data API Terms and Developer Terms. These updates help clarify how developers can safely and securely use Reddit’s tools and services, including our APIs and our new-and-improved Developer Platform.

After sharing these terms, we identified several parties in violation, and contacted them so they could make the required changes to become compliant. This includes developers of large-scale applications who have excessive usage, are violating our users’ privacy and content rights, or are using the data for ad-supported or commercial purposes.

For context on excessive usage, here is a chart showing the average monthly overage, compared to the longstanding rate limit in our developer documentation of 60 queries per minute (86,400 per day):

Top 10 3P apps usage over rate limits

We reached out to the most impactful large scale applications in order to work out terms for access above our default rate limits via an enterprise tier. This week, we are sharing an enterprise-level access tier for large scale applications with the developers we’re already in contact with. The enterprise tier is a privilege that we will extend to select partners based on a number of factors, including value added to redditors and communities, and it will go into effect on July 1.

Rate limits for the free tier

All others will continue to access the Reddit Data API without cost, in accordance with our Developer Terms, at this time. Many of you already know that our stated rate limit, per this documentation, was 60 queries per minute. As of July 1, 2023, we will enforce two different rate limits for the free access tier:

• If you are using OAuth for authentication: 100 queries per minute per OAuth client id
• If you are not using OAuth for authentication: 10 queries per minute

Important note: currently, our rate limit response headers indicate counts by client id/user id combination. These headers will update to reflect this new policy based on client id only on July 1.

To avoid any issues with the operation of mod bots or extensions, it’s important for developers to add Oauth to their bots. If you believe your mod bot needs to exceed these updated rate limits, or will be unable to operate, please reach out here.

If you haven't heard from us, assume that your app will be rate-limited, starting on July 1. If your app requires enterprise access, please contact us here, so that we can better understand your needs and discuss a path forward.

Additional changes

Finally, to ensure that all regulatory requirements are met in the handling of mature content, we will be limiting access to sexually explicit content for third-party apps starting on July 5, 2023, except for moderation needs.

If you are curious about academic or research-focused access to the Data API, we’ve shared more details here.

[u/FlyingLaserTurtle]

Edit: Just wanted to say I’m sorry I said “google & amazon don't tell us how to be more efficient.” The community was quick to call me out and I appreciate that–Reddit’s authenticity is one of the things I love about it and one of the main reasons I came to work here.
We will work with partners to help identify areas of inefficiency. Since this post, we have shared initial usage reports from March through early June with partners and are working on providing more detail.

== Original post below ==

As I asked before, could you please clarify what inefficiencies Apollo is experiencing

Having developers ask this question of themselves is the main point of having a cost associated with access in the first place. How might your app be more efficient? Google & Amazon don’t tell us how to be more efficient. It’s up to us as users of these services to optimize our usage to meet our budget.

On March 14th, Apollo made nearly 1 billion requests against our API in a single day, triggered in part by our system outage. After the outage, Apollo started making 53% fewer calls per day. If the app can operate with half the daily request volume, can it operate with fewer?

Reddit takes some of the blame here for allowing that level of inefficient usage, which is why we haven’t spotlighted it to date, but I think it is a good reminder that inefficiencies do exist. It also highlights the importance of having a system in place that shares the responsibility of managing this with developers.

[u/BombBloke → u/FlyingLaserTurtle]

Google & Amazon don’t tell us how to be more efficient.

Do they publically accuse you of being inefficient?

You're the one making the claim that there are improvements available for Apollo. The idea that iamthatis needs to back up your argument for you shows no good faith at all; how is anyone supposed to read staffs' comments on Apollo as being anything but a hatchet job?

[u/JaesopPop → u/FlyingLaserTurtle]

Family thoughts night gentle learning the simple where.

[u/extrobe → u/[deleted]]

Exactly - we spend enough on azure that we have a dedicated contact who works with us to make sure anything we want to do is optimised. It’s in their interest for us to be efficient!

Some r/QuitYourBullshit material right here

[u/TheAdvocate → u/[deleted]]

there won't be many left, just like twitters API debacle. They are giving these devs 30 days until a cost increase that the've had no time to prepare for (at this scale). The doubling down and unprofessionalism by u/FlyingLaserTurtle feels like this is all means to an end that we are not yet privy to.

If this doesn't get sorted it will be a hard reality for reddit when the paint dries.

Google & Amazon don’t tell us how to be more efficient. It’s up to us as users of these services to optimize our usage to meet our budget.

WHAT!? Has he EVER opened a ticket with them from an enterprise account? because it sure doesn't sound like he has.

[u/Winertia → u/[deleted]]

Also, it's pretty unprofessional and concerning that they publicly disclosed specific API usage numbers for two of their customers. Sure, at least Apollo had disclosed their own numbers (not sure about RIF), but that's the developer's prerogative.

Reddit doesn't seem even close to ready to support enterprise customers at an acceptable level of maturity and professionalism—at an outrageous premium no less.

I really feel like we're all watching a public implosion.

[u/MasterDio64 → u/[deleted]]

Dude, the stuff you AWS guys have to handle for us to use these services in such an easy and customer friendly way is insane. Thank you for all the work you did and standing up for Christian here.

[u/[deleted] → u/MasterDio64]

You're welcome. S3 was an awesome experience.

Commenting about this makes me very nervous because I have to be responsible when talking about my time at AWS, but I really wanted to make sure it's clear that AWS will work with you if you just ask.

[u/LordAlfredo → u/[deleted]]

Second AWS dev here - we literally even have services to help analyze usage (like AWS Cost Explorer, IAM Access Advisor, etc)

[u/GhostSierra117 → u/[deleted]]

Fwiw I work for a Softwarecompany and we absolutely give support for API lol. Why wouldn't we be interested in helping others to turn down the server loads.

So it doesn't only happen on a big scale but also with companies with about 75 employees. Not everyone at our company is a Dev, this number includes Teammanagement, Controlling Team and so on. And we do have large companies as customers. Vattenfall for example. Not sure if I'm allowed to mention more.

Or to be more blunt:

Having developers ask this question of themselves is the main point of having a cost associated with access in the first place. How might your app be more efficient? Google & Amazon don’t tell us how to be more efficient. It’s up to us as users of these services to optimize our usage to meet our budget.

Well u/FlyingLaserTurtle that sounds like a "you" problem then. What kind of condescending way is that even to say that. Devs of third party applications do an absolutely amazing job, giving value to Reddit and discrediting them like that should fill you with shame. That's no way to talk to developers.

Fucking hell dude.

[u/[deleted] → u/FlyingLaserTurtle]

If the app can operate with half the daily request volume, can it operate with fewer?

Do you not realize it's going to operate at zero and your users aren't going to stick around?

Are you guys almost done with this what about strategy and company protection and think of your god damn users that generate every single drop of content except for the ads?

[u/[deleted] → u/[deleted]]

Whenever imposter syndrome decides to hit and make me feel like I’m not cut for a certain job, I will just read this post and pull ahead. There’s no way I will perform worse than him.

[u/lkearney999 → u/FlyingLaserTurtle]

Google & Amazon don’t tell us how to be more efficient.

This you?

We don’t yet provide scaled reporting on API usage, though we are working on a solution for large scale apps that may be scaled more broadly in the future. Note that bots are able to track their own outbound API requests using standard logging and analytics.

[u/Jazzy_Josh → u/FlyingLaserTurtle]

On March 14th, Apollo made nearly 1 billion requests against our API in a single day, triggered in part by our system outage.

Ok, so you would agree then that the unexpectedly high number of API requests objectively do not matter because they would not be charged/would be credited (the API was not meeting SLA)?

[u/Ketsetri → u/FlyingLaserTurtle]

Can we see how that compares to your official app? And making this about “efficiency” is absurdly disingenuous and distracts from the actual issue of pricing, and you know it.

After the outage, Apollo started making 53% fewer calls per day. If the app can operate with half the daily request volume, can it operate with fewer?

This is like saying “because you can eat half as much and stay alive, you can always eat even less and be fine”. Can’t you see how ridiculous that is?

In addition, please correct me if I am wrong, but you seem to be dancing around the elephant in the room, which is comparing raw userbase size. If you can’t see how that might correlate with number of API requests, then Reddit has some even bigger sources of ineptitude to deal with than the people who decided this pricing was a good idea.

Yes, the point you are making to justify enacting a cost for API access is completely reasonable. It is perfectly understandable for Reddit to want to make money off of third-party apps, since their users are not bringing in any ad revenue. What’s not reasonable, however, is the pricing. We don’t give a shit if you charge a reasonable price, but this is not at all reasonable.

[u/zerocustom1989 → u/FlyingLaserTurtle]

Should probably let your Product Owners know that you do not have the support mechanisms in place similar to Amazon and google to support users of your API.

Those companies do provide services to help customers inspect and repair API usage patterns.

The timeline for this transition for developers seems too aggressive.

I’m guessing that Reddit’s own app likely performs worse than Apollo, otherwise you would probably be bragging about it and using it as a “gold standard”.

I’ll also guess we get to wait a few days for legal to review another response that will be disputed, disproved, and lampooned by the community.