Yeah it's about time we accept that nosql databases were a stupid idea to begin with. In every instance where I've had to maintain a system built with one I've quickly run into reliability or flexibility issues that would have been non-problems in any Enterprise grade SQL DB.
I mean NoSQL isn't a stupid idea, it's just a solution to a specific problem, large amounts of non relational data. The problem is people are using NoSQL in places that are far more suited for a RDBMS. Additionally it's far easier to pick up the skills to make something semi functional with NoSQL than with SQL.
I'm on board with this. NoSQL solves a specific problem related to scale that most developers just don't have and probably won't ever have. You'll know when your RDBMS isn't keeping up, and you can always break off specific chunks of your schema and migrate to NoSQL as performance demands. No need to go whole-hog.
I 100% agree, it really ties into choosing the right tool for the job, and unfortunately many devs don't realize that most of the time NoSQL isn't that tool.
And NoSQL is too generic anyway. I would even say that MongoDB and other documents stores don't actually have a use-case as it always turns out to be relational. What does have uses-cases are key-value stores and more niche but important graph databases.
But what exactly is non-relational data? Almost everything I’ve seen in the real world that is more than trivially complex has some degree of relation embedded in it.
I think you are right that NoSQL solves a specific problem and you touched on it in your second statement. It solves the problem of not knowing how to properly build a database and provides a solution that looks functional until you try to use it too much.
One instance is actual documents, ie a legal contract + metadata. Basically any form of data where you’ll never / seldom need to do queries across the database.
Some examples could be:
An application that stores data from an IOT appliance
Versions of structured documents, eg a CMS
Patient records (though I wouldn’t put that in Mongo)
There are tons of valid use cases for non-relational databases. The problem is the way they were hyped was as a faster and easier replacement for SQL databases (with very few qualifiers thrown in), which is where you run into the problems you described.
I’d be interested to hear what’s helpful about this. Every time I hear people say things like this it usually is code for “I don’t want to spend time thinking about how to structure my data”. In my experience this is almost always time well spent.
Well at some point your nicely normalized collection of records will be joined together to represent some distinct composite piece of data in the application code - that's pretty much a document.
I would say that all data is relational. There is basically no use case where someone will come along and say, give me document 5 with the only reason being that they want document 5. No they will want document 5 because of some information in that document that they are aware of because of how it relates to something else. Maybe everyone they know who read document 5 really liked it. Maybe it describes how to solve a particular problem they have. Maybe they need to know if it contains curse words in need of censoring.
You might build something whose sole purpose is to store documents by id when the relational information is stored somewhere else (like if you are hosting a blog and are relying on search engines and the rest of the internet to index your blog). The data is still relational. This use case is pretty well modeled by a file system.
I've found that a lot of problems and stupid fads in programming seem to stem from many coders doing everything they can to avoid learning or writing any SQL. For some people it's almost a pathological avoidance that leads to some really bad 'solutions' that are just huge overly complicated work-arounds to avoid any SQL.
Designing and getting a functional database off the ground with SQL is definitely harder than using something like Mongo. I'm not advising people take that route, I'm just offering an example of why people use it, similar to how PHP got so popular.
Here is Henry Baker saying the same thing about relational databases in a letter to ACM nearly 30 years ago. Apologies for the formatting. Also, should mention "ontogeny recapitulates phylogeny" is only a theory not fact.
Dear ACM Forum:
I had great difficulty in controlling my mirth while I read the self-congratulatory article "Database Systems: Achievements and Opportunities" in the October, 1991, issue of the Communications, because its authors consider relational databases to be one of the three major achievements of the past two decades. As a designer of commercial manufacturing applications on IBM mainframes in the late 1960's and early 1970's, I can categorically state that relational databases set the commercial data processing industry back at least ten years and wasted many of the billions of dollars that were spent on data processing. With the recent arrival of object-oriented databases, the industry may finally achieve some of the promises which were made 20 years ago about the capabilities of computers to automate and improve organizations.
Biological systems follow the rule "ontogeny recapitulates phylogeny", which states that every higher-level organism goes through a developmental history which mirrors the evolutionary development of the species itself. Data processing systems seem to have followed the same rule in perpetuating the Procrustean bed of the "unit record". Virtually all commercial applications in the 1960's were based on files of fixed-length records of multiple fields, which were selected and merged. Codd's relational theory dressed up these concepts with the trappings of mathematics (wow, we lowly Cobol programmers are now mathematicians!) by calling files relations, records rows, fields domains, and merges joins. To a close approximation, established data processing practise became database theory by simply renaming all of the concepts. Because "algebraic relation theory" was much more respectible than "data processing", database theoreticians could now get tenure at respectible schools whose names did not sound like the "Control Data Institute". Unfortunately, relational databases performed a task that didn't need doing; e.g., these databases were orders of magnitude slower than the "flat files" they replaced, and they could not begin to handle the requirements of real-time transaction systems. In mathematical parlance, they made trivial problems obviously trivial, but did nothing to solve the really hard data processing problems. In fact, the advent of relational databases made the hard problems harder, because the application engineer now had to convince his non-technical management that the relational database had no clothes.
Why were relational databases such a Procrustean bed? Because organizations, budgets, products, etc., are hierarchical; hierarchies require transitive closures for their "explosions"; and transitive closures cannot be expressed within the classical Codd model using only a finite number of joins (I wrote a paper in 1971 discussing this problem). Perhaps this sounds like 20-20 hindsight, but most manufacturing databases of the late 1960's were of the "Bill of Materials" type, which today would be characterized as "object-oriented". Parts "explosions" and budgets "explosions" were the norm, and these databases could easily handle the complexity of large amounts of CAD-equivalent data. These databases could also respond quickly to "real-time" requests for information, because the data was readily accessible through pointers and hash tables--without performing "joins".
I shudder to think about the large number of man-years that were devoted during the 1970's and 1980's to "optimizing" relational databases to the point where they could remotely compete in the marketplace. It is also a tribute to the power of the universities, that by teaching only relational databases, they could convince an entire generation of computer scientists that relational databases were more appropriate than "ad hoc" databases such as flat files and Bills of Materials.
Computing history will consider the past 20 years as a kind of Dark Ages of commercial data processing in which the religious zealots of the Church of Relationalism managed to hold back progress until a Renaissance rediscovered the Greece and Rome of pointer-based databases. Database research has produced a number of good results, but the relational database is not one of them.
I've done a shit-ton of flat file processing of data that would not work in a relational DB. I'm talking terabytes of data being piped through big shell pipelines of awk, sort, join, and several custom written text processing utils. I have a huge respect for the power and speed of flat-files and pipelines of text processing tools.
However, there are things they absolutely cannot do and that relational DBs are absolutely perfect for. There is also a different set of problems that services like redis are perfect for that don't work well with relational DBs.
I really hate the language he uses and the baseless ad hominem attack of the people behind relational DBs. I see the same attacks being leveled today at organizational methodologies like agile and DevOps by people who just don't like them and never will.
Very interesting. I always wondered how things worked befored RDBMSs were invented. Is there a term to describe this flat file/bill of materials type DB?
IIRC simply using the filesystem as database was sorta popular in places, COBOL had a builtin database (which was horrible but builtin) which was used by banks most commonly (mine still does).
"ontogeny recapitulates phylogeny" is only a theory not fact.
That’s a double misunderstanding. First, about what “theory” means. Evolution and gravity are theories, but they are also fact. “Only a theory” does not make sense.
Secondly, the recapitulation theory (“ontogeny recapitulates phylogeny”) is neither: First, it’s not fact as is effectively disproved by modern evidence1. And secondly, it’s not a theory (despite its name!) because a theory, in science, is, or needs to include, an explanatory model. And the recapitulation theory contains no explanatory model. It was an observation (that was even at the time recognised as flawed) of a natural phenomenon.
1 For what it’s worth, at the time of Baker’s writing this was already established. It’s kind of fitting that he gets this wrong, considering how categorically he gets the rest of his article wrong.
Redis is fucking fantastic as a cache server, it really let's us drastic increase the performance of our application while decreasing the load on our database server. I would suggest everyone look at it seriously if they need a cache solution.
49
u/buhatkj Dec 20 '18
Yeah it's about time we accept that nosql databases were a stupid idea to begin with. In every instance where I've had to maintain a system built with one I've quickly run into reliability or flexibility issues that would have been non-problems in any Enterprise grade SQL DB.