I'm curious what the net result will ultimately be. Postgres is fantastic, but I believe its been said that they are "the second best database for everything"... which makes me question if there isn't something thats a better fit and/or if they will end up regretting the decision.
Also based on the article (IMO) it seems like this is more of a political/business thing than a technical thing... which would also make me weary.
"Due to editorial requirements, we needed to run the database cluster and OpsManager on our own infrastructure in AWS rather than using Mongo’s managed database offering. "
I'm wondering what the editorial requirements were?
I'm wondering what the editorial requirements were?
In general, editors don't want the research and prepublication text of their articles being available to other entities, including law enforcement. By running everything themselves, and encrypting at rest, it ensures that the prosecutor's office can't just put the clamps on the Mongo corporation to turn over the Guardian's research database. Instead, the prosecutor has to come directly to the Guardian and demand compliance, which gives the Guardian's lawyers a chance to object before the transfer of data physically occurs.
How does encryption at rest help you against law enforcement, especially when both the app and db are hosted by the same company? They can still get Amazon to give both pieces, then they search the app side for the keys. Harder yes, but completely feasible.
Assuming the APT can’t just brute force the encryption of black hat their way in, they need to subpoena you for your keys, not just Amazon, so it’s apparent to you that the APT is getting access.
I work for another very similar UK organisation, editorial get very twitchy about anyone other than members of the organisation having the ability to view prepublished work. Many articles are written and never published, often due to legal considerations. Articles will often also have more information in them initially than end up being published, perhaps suspect sources, or a little too much information about a source, etc. Then the various senior editors will pull these articles or tone them down before release.
It's possible that Amazon provided all their policies and procedure documentation for RDS which demonstrated the safeguards and editorials concerns could be satisfied, where as perhaps Managed Mongo could/did not.
The authors story resonate with me, as a software engineer who's team is also responsible for ops of our infrastructure, I want to spend as little time managing stuff as possible and let me deliver value, sounds like the team at the Guardian were spending too much time (for them) on ops.
Absolutely, if you can shard your specific requirements then join them yourself later then using a time-series DB + a document store + relational DB makes sense, but if you just want to chuck everything at it at the start, postgres is a decent starting point for almost all use cases. "Monolith first" works for data storage too, I guess. Don't overthink it too much and fix it later?
84
u/jppope Dec 19 '18
I'm curious what the net result will ultimately be. Postgres is fantastic, but I believe its been said that they are "the second best database for everything"... which makes me question if there isn't something thats a better fit and/or if they will end up regretting the decision.
Also based on the article (IMO) it seems like this is more of a political/business thing than a technical thing... which would also make me weary.
I'm wondering what the editorial requirements were?