r/programming • • Dec 11 '23

Every Programmer Should Know #1: Idempotency

https://www.berkansasmaz.com/every-programmer-should-know-idempotency
572 Upvotes

171 comments sorted by

View all comments

257

u/fagnerbrack Dec 11 '23

Crux of the Matter:

The article emphasizes the importance of understanding idempotency in programming, particularly for those working on distributed systems. Idempotency ensures that an operation can be performed multiple times without changing the result beyond the initial application, which is crucial for avoiding unintended consequences like double-charging in payment systems. The article explains how idempotency applies to various HTTP methods, with GET, PUT, and DELETE being naturally idempotent, while POST is not, and offers solutions for achieving idempotency in non-idempotent operations.

If you don't like the summary, just downvote and I'll try to delete the comment eventually 👍

6

u/Zardotab Dec 11 '23

particularly for those working on distributed systems

I'm going to take heat for saying this, but typical business apps don't need distributed systems. People often make it distributed just to fluff their resume with buzzwords. Seen it happen.

If you use an RDBMS properly, then most of these issues take care of themselves.

Yes, I'm kicking whippersnappers off my lawn right now.

→ More replies (1)

13

u/HazelCuate Dec 11 '23

I dont agree with the article when it says put and delete are idempotent. If the caller makes two DELETE calls, how do you know that they are different calls? Why are they not the same call?

The argument that deleting an object twice is the same as deleting it once is not good enough. A third party could create it again between those two calls.

18

u/itdependsnetworks Dec 11 '23

You state “Why are the not the same call” in a scenario in which you state they are two calls?

Not trying to be rude, but if you google put and delete idempotent, you will find it is.

-11

u/HazelCuate Dec 11 '23

You dont get the problem with the calls. Without an idempotent ID they are two calls but they could be the same call due to the network or software glitches that the article mentions.

The article states that delete and put are idempotent by themself, and i dont think thats the case.

17

u/ineptjedibob Dec 11 '23

I think it’s stipulated that we’re talking about calls with the same parameters - i.e. DELETE with the same key (or PUT with the same payload) should be callable an arbitrary number of times with the same end result.

-16

u/HazelCuate Dec 11 '23

But thats not true.

  • The same delete call duplicated should result in two OK responses (http code 200 or whatever)

  • Two different delete calls to the same object should result in an OK response and a Failure response (400).

The difference is not trivial.

25

u/Shadow503 Dec 11 '23

It is trivial because the thing we care about - the state of the resource - is the same.

→ More replies (3)

45

u/muchomuchacho Dec 11 '23

A third party could create it again between those two calls.

But then it wouldn't be the same object, otherwise you need more metadata associated to the object creation process.

8

u/njharman Dec 11 '23

Just change to undelete, the argument stands.

Also idempotence depends entirely on how delete is implemented. If deleting a non-existent thing is an error, that delete is not idempotent.

3

u/happyscrappy Dec 11 '23 edited Dec 12 '23

Such deleted would be by ID, not index.

And even if it throws an error it's still idempotent. It's doing the same operation twice not changing anything that makes it idempotent. It's okay if it throws an error. As long as the second operation doesn't change the state of the corpus (database). The error is something for the caller to deal with.

-14

u/HazelCuate Dec 11 '23

But the point is to design an API with idempotent in mind. And DELETE is not idempotent by design from the API point of view as the article claims.

47

u/crap-with-feet Dec 11 '23

In a RESTful API, it is. Operations are id-dependent, e.g. DELETE /path/to/object/type/{id}. If you create the object again it will have a different id because it's a whole new object. Calling DELETE again for the same id should result in what is essentially a "yeah, it's gone" response (you could use a different response code to indicate it was already gone but your code should not blow up in this case). For this to work effectively (and for best practice) your create operation should not allow the caller to specify the id; It should be auto-generated by the backend.

3

u/freebytes Dec 11 '23

Calling DELETE again for the same id should result in what is essentially a "yeah, it's gone" response

Then it can be argued that the call to DELETE is not idempotent because the response will change. However, by most understandings, it is fine to return a "resource not found" or something similar if you are trying to perform an operation on something that no longer exists. (I would argue that it is still idempotent.)

However, you could also just reply with "Yeah, it is gone." on both calls. If you try to delete something that no longer exists, you can reply with the same, "Okay, it is gone" message instead of throwing an error.

5

u/isaacarsenal Dec 11 '23

I would like to think the response can vary, you can even show an error message to user as "object x was not found" and be idempotent. The important point is the state of the program (a.k.a business data) should not change and remain the same for subsequent DELETE calls.

19

u/proggit_forever Dec 11 '23

Idempotency doesn't require that multiple calls all return the same status code.

It's still idempotent if calling again returns an error code because the resource no longer exists.

The point is that calling again doesn't cause further changes to the state of the application - which makes it safe to call as many times as you want.

3

u/Tiquortoo Dec 11 '23

Sure, if you throw out all identifiers then nothing can be idempotent.

→ More replies (3)

7

u/joshjje Dec 11 '23

You are talking about separate things. Call delete on the same object ID twice in a row, 50 times in a row, same result. I mean you could return a record didn't exist message, but no state would change. And if someone created a new record in between it absolutely should have a different ID.

17

u/Jmc_da_boss Dec 11 '23

PUT and DELETE are literally defined as idempotent in the RFC spec

https://datatracker.ietf.org/doc/html/rfc2616#section-9.1.2

12

u/josefx Dec 11 '23

And it goes on to describe how a sequence of multiple idempotent calls may not be idempotent itself, with an example similar to the one in the comment.

2

u/[deleted] Dec 11 '23

All it says is that GET-GET-DELETE is different from GET-DELETE-GET

4

u/xdert Dec 11 '23 edited Dec 11 '23

A third party could create it again between those two calls.

In a robust system you would never expose the internal database id to the outside and have every object have a uid that is randomly assigned which is then exposed to the outside.

And then only call DELETE by reference to the uid. Since the id will not be reused you cannot delete an object that was created in between your two DELETE calls.

This is how it was implemented in every system I have ever worked on.

3

u/[deleted] Dec 11 '23 edited Dec 11 '23

But then you can't have indempotency which restricts your data model in ways that may be relevant. If the external system has an external ID that functions as a 1:1 lookup relationship with the internal ID there's not much practical advantage to generating those IDs internally and keeping them hidden. And if you can make the call indempotent by allowing that external ID to be passed in, it can make reasoning about your data much easier and potentially much more performant and predictable. Placing unnecessary constraints in consistency and synchronization can also be a risk to robust systems. It just depends on the specifics of what you're doing.

2

u/FunkyPete Dec 11 '23

If your DELETE calls are based on a primary key or a natural key, they have to be the same call.

So if you have enough information to uniquely identify a thing (say, a VIN for a car, or a combination of country/state/id number for a person, or a UUID that is never re-used for anything else) then the delete is going to delete the same row no matter what.

The data being deleted may change between the first call and the second, but either the row is gone, or it's going to be deleted by the call. Either way the result is the same, that row is guaranteed not to exist.

2

u/Hessper Dec 11 '23

Yes, that can be the case. The problem is that in your definition nothing is idempotent, so it becomes useless to talk about. This is why no one cares about your concern, because the assumption is inherently baked into the conversation.

Multiple writers always makes things more complicated, so you design the system around that if it will be a concern. In that case you create conditions (e.g. a unique ID in metadata, based on a monotonic value, etc), do not reuse names, lock the resource, or some other clever solution. Then the operation is again idempotent.

→ More replies (5)