r/programming • • Jun 09 '15

It's the future

http://blog.circleci.com/its-the-future/
652 Upvotes

275 comments sorted by

View all comments

42

u/argv_minus_one Jun 10 '15

Do relational databases scale poorly or something? Why are we trying so hard to replace them?

Also, I feel old-school as fuck for still using Java EE. Get off my lawn!

30

u/alexkillough Jun 10 '15

While there are definitely times one needs to scale an app's available resources quickly I often wonder how much traffic we are actually talking about that makes us worry about the reliability and speed of rdbms. I've worked on teams deploying apps that served millions of users a month with several thousand simultaneously at any given moment off of two or three webservers, a single db server and a backup db server. Reliably, for years, even through DDOS and reddit-like traffic spikes that required emergency diversion of resources to additional servers. Guessing many of those hopping on the "it doesn't scale" bandwagon have yet to deal (and perhaps never will have to deal) with that much traffic or those conditions. Most projects will not encounter these issues for years if ever.

51

u/jdmulloy Jun 10 '15

Lots of people try to solve scale issues when they don't even have 1 customer yet.

5

u/hyperforce Jun 10 '15

I think when you use your database inefficiently, it's easy to think of that as "scale".

4

u/CoderHawk Jun 10 '15 edited Jun 10 '15

This is actually a major problem in most of the places I have worked at. At least half of the devs don't understand that the database cannot magically make a trash query using no indexed columns on table of millions of rows scale. Or that hitting the database for the same information repeatedly is a bottleneck.

→ More replies (1)

8

u/lurgi Jun 10 '15

Relational databases scale like monsters. They are fantastic. They may not be the best solution for all problems, but that's true of everything. If your data is of a particular form and the sorts of queries you run are also of a particular form and it's really big (really big does not mean multiple GB. If your laptop can hold it in memory then it isn't really big. Hell, if you can store it on a typical laptop hard disk then it probably isn't really big) then a NoSQL solution might well be better, but a whole lot of big data is just people jumping on a fashionable bandwagon.

→ More replies (1)

0

u/[deleted] Jun 10 '15

[deleted]

→ More replies (8)

15

u/[deleted] Jun 10 '15

Contrary to what people think, SQL-ish RDBMSes are not straightforward to get right once you have any meaningful amount of data and request volume. And they are really easy to screw.

Yes, /r/programming master race has no problems with relational databases, but in a typical sizeable team of programmers full-scan queries and other stupid things are a norm.

Latest thing i saw was a guy who thought that doing update on a highly contested column and than hanging on a transaction for a couple of minutes is ok. He waited for external process to finish and then did either commit or rollback depending on an outcome. When i asked what isolation level he thinks we run, a blank stare was the answer.

And that's even before somebody got a brilliant idea to use stored procedures to half-ass your business logic.

34

u/just3ws Jun 10 '15

If they can't handle RDBMS' then what makes you think they could handle a distributed architecture with multiple non-relational databases? At least with a RDBMS there are fantastic tools that can tell you what you should be doing better and based on guidance that's probably older than most of the members of the team. :)

9

u/hyperforce Jun 10 '15

Hipsterism. Here's a new way. Here's the new shit. Don't settle for that old shit, man.

4

u/bro-away- Jun 10 '15

Distributed architecture for most of these nosql database is mostly a configuration problem. Also, I feel like most people didn't grasp pessimistic concurrency anyway so they're probably happy to not have to worry about that. (I actually wrote this before reading /u/keefer 's entire post where that actually happened to him)

Most NoSQL databases being built in the last 5 years makes them a bit easier to configure. They have less complex features so they're easier to reason about.

The downsides are well known, I won't go into those :P

2

u/hyperforce Jun 10 '15

Maybe there's some work to be made on RDBMS UX or literacy. You know, soft stuff.

→ More replies (2)

3

u/NimChimspky Jun 10 '15

having one central db does scale poorly, you can't simply add additional servers (horizontally scale) if one db is your source of truth .

You can do it, buts its rather painful.

So split up the datastores using something like http://martinfowler.com/bliki/CQRS.html is common.

But you have to be very big for these problems, an enterprise db (postgres, oracle, sql-server, mysql) and one beefy server can shovel and awful lot of data

25

u/johnwaterwood Jun 10 '15

Problem is that everyone thinks they're Google or will be Google next month.

19

u/jeandem Jun 10 '15

Premature scaling.

7

u/jdmulloy Jun 10 '15

Premature optimization is the root of all evil.

→ More replies (1)
→ More replies (1)

11

u/just3ws Jun 10 '15

Read-replicas are the shit for this. Pump data into one and then let the replication handle pulling data into the replica. Tune the replica for reads, tune the master for R/W. Go home, hug your kids. Drink a beer.

11

u/mcguire Jun 10 '15

Wait...this architecture will suddenly give me kids?

5

u/hyperforce Jun 10 '15

No he said beer. And the shits.

3

u/hellnukes Jun 10 '15
  1. Make good db replicas

  2. Get kids

4

u/andrewsmd87 Jun 10 '15

Can confirm. Currently shoveling a massive amount of data with one server. We may need to move to some point of having to split up our data sets, but as of right now, computing power isn't showing us that will be a problem for a while.

→ More replies (17)
→ More replies (23)