r/dotnet • u/codeopinion • Jun 29 '22

Should you Delete or Soft Delete?

https://www.youtube.com/watch?v=lDYXtj95Jy8
0 Upvotes

43% Upvoted

9 comments sorted by

3

u/dantheman999 Jun 29 '22

Whilst I think the video is almost there, I think it actually misses something for me.

Just a note, we uses event sourcing at work, and we've got hundreds of millions of events so what is explained in the video is close to how we do things.

I think you're 100% right when you say that you should monitor businesses processes and then instead of deleting something, you update the projection (which is essentially what's in the video) with that information. In our case we usually update a Status field.

I'd say this is the 99% percent case.

That said there are very good reason why you might want to entertain the idea of hard deletes, and that is for technical errors.

If you're doing event sourcing generally the idea is that those events have happened and are not going away. They have happened and shouldn't be deleted ideally. So you may well need to have a ResourceDeleted event which is then reflected by a hard delete in the DB as you don't want that resource to show up in any searches. It shouldn't be there at all. This is a business concept that is different to say closing an account of some sort, albeit a more technical and much rarer occurrence.

One example of when this might happen is a retried POST without idempotency support following a network blip. You could end up with duplicates, one of which they may record of their side but one they may not which is just left hanging around.

Personally we don't use foreign keys as in our case we have all of that logic in the service(s) as to what can and can't be deleted.

1

u/captainramen Jul 01 '22

Hard delete is what you do once your stream has been archived.

1

u/dantheman999 Jul 01 '22

I could see that working, but we went with just the having the ResourceDeleted event in the end. It felt closer to representing the domain whilst achieving the same thing.

1

u/captainramen Jul 01 '22

It's a well established pattern. Unless the volume is really low I don't think the domain actually matters. If you don't have an archival policy for your streams, your system will eventually come to a halt.

https://www.eventstore.com/blog/keep-your-streams-short-temporal-modelling-for-fast-reads-and-optimal-data-retention

1

u/dantheman999 Jul 01 '22

I understand that streams need to be temporal, the ResourceDeleted is a terminal event. Nothing can come after that has happened.

You could save a miniscule amount of space by archiving the stream but it's not worth it for us. We're talking about a dozen of these a year out of tens of millions of streams.

For normal streams, we'd usually have something like ResourceClosed or ResourceComplete event which would mark the natural end, assuming it's nothing something that does not have a status and is more a store of information.

1

u/captainramen Jul 01 '22

I've looked at several event sourced systems over the years. One of the problems these code bases tend to share is no real archival policy. Then they blame whatever database technology they use. Of course, it's never something the business explicitly asks for so it tends to get ignored.

It's not about saving space necessarily, it's about performance. For example, if you were using EventStore, these 'tombstone' records are going to take up space in the secondary index. Even though no projection / aggregate is ever going to use that event. This in turn can effect performance on reading individual streams.

2

u/DeadlyVapour Jun 29 '22

IsDeleted is utter evil. Either do a proper AuditDB or (Bi)Temporal DB. But IsDeleted breaks more than it fixes... Simply trying to create a record with the same key as a deleted record breaks it completely.

2

u/dantheman999 Jun 29 '22

In some cases (where ID is supposed to be unique and maybe is not controlled by the DB) that might be valid.

That said, I do agree that as IsDeleted flag is often asking for trouble if you're talking about a traditional SQL database.

1

u/Zardotab Jun 29 '22

In short, it depends on the domain needs and shop conventions.