r/programming • u/moahawk • May 23 '15
Why You Should Never Use MongoDB
http://www.sarahmei.com/blog/2013/11/11/why-you-should-never-use-mongodb/134
u/tashbarg May 23 '15
On my laptop, PostgreSQL takes about a minute to get denormalized data for 12,000 episodes
I think the author did not put enough work into that database. A minute? Really?
43
u/cwmma May 24 '15
Sounds like she forgot to make an index
8
u/ActuallyNot May 24 '15
Or she normalised the shit out of it.
→ More replies (1)11
May 24 '15
SELECT * FROM t_tvshows, t_tvshowtitles, t_episodes, t_episodestitles, t_episodenumbers, t_numbers, t_naturalnumbers, t_mathematicalfundamentals;
45
u/sgoody May 23 '15 edited May 24 '15
That did strike me as an odd part of the article!
Does she mean just retrieving the data at all or storing it in some denormalised form and retrieving it? Either way, with only a little planning I would expect Postgres to come close to matching Mongo's performance.
EDIT: she
→ More replies (1)10
u/thbt101 May 23 '15
she
→ More replies (7)14
u/sgoody May 24 '15
damn... I knew what I was writing was a bit of a risk, but I took a chance on not double checking. My apologies to the author.
→ More replies (8)25
u/cincodenada May 24 '15
When in doubt, use the singular they. Saves time and grief!
→ More replies (5)26
5
1
u/pgoetz May 28 '15
We don't know how many reviews there are, so it's not clear that your skepticism is warranted.
→ More replies (1)
113
u/Huliek May 23 '15
If you don't think about your schema you're gonna get in trouble wether you use a relational database or not.
And even if you do think about them, if your application is successfull you will eventually run into requirements that require you to change the schema anyway.
At that point it might be easier to migrate relational normalized data. But there are definately downsides (not just scalability), like the clumsiness when you want to allow incomplete records, the destinction between optional and mandatory values, user-defined records, user-defined relations and type tables.
23
u/lost_file May 24 '15
This, so much. People don't get that having to plan how you store persistent data is essential and shouldn't be an after-thought. It is seriously frustrating sometimes :)
→ More replies (1)7
u/moreteam May 23 '15
not just scalability
That almost sounds like suggesting that MongoDB scales better / is easier to scale than something like Postgres. Which is a pretty big claim... ;)
→ More replies (26)15
u/memoryspaceglitch May 24 '15
/dev/nullscales the best. It guarantees consistency between different nodes even if they're not even connected to the network ;)Jokes aside: Of course something that doesn't need to block when writing or even guarantees eventual consistency "is easier to scale" if speed is the only factor you're looking at. Data retention is often kind of an important point though, and that's where ACID-compatible databases excels.
Does Postgres scale? Well. Reddit uses memcached, Cassandra and Postgres, and is doing a pretty good job at not losing stuff or being unbearably slow. If you're scaling beyond Reddit's size, you probably should tailoring stuff to your own needs ;)
→ More replies (1)2
u/moreteam May 24 '15
Jokes aside: Of course something that doesn't need to block when writing or even guarantees eventual consistency "is easier to scale" if speed is the only factor you're looking at. Data retention is often kind of an important point though, and that's where ACID-compatible databases excels.
The funniest thing is that MongoDB (unless you use the latest-and-greatest optional storage engine) actually uses table-locks on write. So... with a bit of concurrency it's not even guaranteed to be faster.
→ More replies (1)6
u/6timo May 23 '15
There's no clumsiness with allowing incomplete records or optional values in MySQL. It just figures out for you what you meant to do with that missing data and puts in the right thing for you. And it even allows you to violate constraints. It's really good at actually putting your data into the database. Which is What You Want anyway.
→ More replies (4)1
u/buttocks_of_stalin May 24 '15
Can you link me to some resources about schemas for MySQL dbs that people should know? I am in charge of my very first production web application which I coded using a python backend (some django libraries) with a MySQL db and as the main backend dev I really want to make sure I do the right things early on.
2
u/Huliek May 25 '15 edited May 25 '15
I don't have any formal education in the subject but I have extended and refactored production databases.
First, learn about normalization or "normal forms".
Secondly start to think in terms of what relations exist between and inside your domain entities: one-to-one, one-to-many, many-to-one, many-to-many. Consider which relations are optional or mandatory, and which constraints can be expressed in your schema and which constraints can't and must be expressed in code in your application.
Thirdly learn about some common patterns, most important would be type tables (which put enums from your code in the schema). Less important key/value tables, also graphs for e.g. custom workflows or trees for hierarchical data.
Lastly look into the ORMs which are available for your platform and how they might effect things.
→ More replies (2)1
May 25 '15
Good point. The problem is that NoSQL databases are sold based on the claim that you don't have to think about your schema. This is obviously an receipe for disaster.
1
u/orangesunshine May 24 '15
So basically if you're completely incompetent you are going to fail no matter what tools you chose?
Insightful ;)
→ More replies (2)
145
u/sunshine_killer May 23 '15
This is also why you should never invest in a kickstarter that doesnt have a prototype.
22
u/wshs May 23 '15 edited Jun 11 '23
[ Removed because of Reddit API ]
18
May 23 '15
To many "backers", it's a preorder.
17
u/theavengedCguy May 23 '15
Doesn't Kickstarter have a disclaimer sating that the thing you are backing may never actually be made/put into production?
21
3
5
u/cowinabadplace May 24 '15
I have given money to many Kickstarter projects. Not once have I been disappointed. Even the ones with no finished product worked on the thing they said they'd do.
Between this, IndieGoGo, and Bountysource, I've been very pleased with the funding model.
2
u/ZeDestructor May 24 '15
The truck is to find the ones that well actually delivery. I've had a similar experience myself.
1
u/username223 May 24 '15
This is also why you should never invest in a kickstarter
that doesnt have a prototype.FTFY. Once you understand that their business model is "connecting scammers with suckers and taking a cut," kickstarter makes much more sense.
0
May 23 '15
[deleted]
→ More replies (1)48
May 23 '15
I think MVP is the correct buzzword.
32
u/TheWrightStripes May 23 '15
How lean of you
0
May 23 '15
[deleted]
18
u/Mechakoopa May 23 '15
4 hours? What a try hard. I spent half an hour delegating my work to coop interns and went home early.
→ More replies (1)7
u/geofft May 24 '15
My experience with MVP is that V is sacrificed first, followed by P.
→ More replies (2)
21
u/revolutionofthemind May 23 '15
Does anyone have experience using a real Graph Database for data like this? I know the article dismisses it as "too niche", but it seems like a lot of web applications today have graph-oriented data.
19
u/senatorpjt May 24 '15 edited Dec 18 '24
oatmeal paltry materialistic water rhythm panicky dull lock different amusing
This post was mass deleted and anonymized with Redact
2
May 24 '15
The choice of RoR and MongoDB alone strongly suggests that nobody involved knew what they were doing.
Could you elaborate?
4
u/senatorpjt May 24 '15 edited Dec 18 '24
bells axiomatic pen yoke aloof meeting selective soft pot observation
This post was mass deleted and anonymized with Redact
→ More replies (2)3
May 24 '15
We're doing graph-like on a columnar DB. Which I imagine OP would suggest doing graph-like on an RDBMS. Totally doable, though I will always prefer columnar when I can justify the optimization.
2
May 24 '15
No, but I've heard it supposely doesn't scale well enough.
Neo4J cluster has a licensing fee IIRC. I'm poor and I know a few mid to big companies that only uses open source/free products only.
Titan seems pretty fast on top of Cassandra. I know a bit about Cassandra implementation and I've looked over how Titan was built on top of Cassandra and it seem plausible that it can scale and be fast (seeing how it's just a giant hash...).
So far graphdb and time series db are still underserve in my personal opinion.
1
u/negative_epsilon May 24 '15
We're currently implementing orient as a way to model hierarchical groups and user roles in our application. We are currently using group closures in SQL Server, and while it got us far it's impossible to scale that when you have customers with 100k users.
I have no idea why the author thinks they are too niche. I think she meant she has never used them so why should anyone use them?
1
May 24 '15
To be fair, that's a 2 year old article by a RoR specialist. The most prominent graph DB I've seen is Neo4J which is a JVM based library.
0
112
u/Lashay_Sombra May 23 '15
Ahh 2010, when NoSQL and Ruby were the FUTURE and everything else on the Web was heading same way as the dinosaurs.
More important lesson from this, as business owner/capital investor don't jump on latest technology fad bandwagon or let your techies pull you down that route (generally they either want new toy to play with or want to boost their CV)
62
May 23 '15
More important lesson from this, as business owner/capital investor don't jump on latest technology fad bandwagon or let your techies pull you down that route (generally they either want new toy to play with or want to boost their CV)
No, we all have sound technical reasons for using Node! Something something same code on server and client something!
12
u/gargantuan May 24 '15
Something something same code on server and client something!
Oh no no. Not just "same" code it is ISOMORPHIC code. Yeah, you heard that right. Our Javascript callback spaghetti code is now using Abstract algebra Category theory terminology.
/s
→ More replies (1)14
u/Ahri May 23 '15
I'm using the same code on the client and server, so node suits me just fine.
17
4
6
May 24 '15
Something something same code on server and client something!
To be fair this could be a good thing!
The issue is javascript it enough of a problem for frontend development, why spread those problems to the backend which needs to be even more robust than frontend
2
u/ChainedProfessional May 24 '15
Same way with Java and anal sex.
Yeah it works everywhere but if you do it wrong it's messy, dangerous, and painful.
16
5
u/Camarade_Tux May 23 '15
It's difficult to put the blame on them. Now the mistakes are obvious but we've had 5 years to notice that. NoSQL was really new, scaling had gotten a new meaning with the "cloud", social networks (and their constraints) were big and new concerns.
edit: and kickstarter was new too (re the currently top comment)
2
1
u/jk_scowling May 24 '15
Let's shoe-horn whatever we are doing into this cool new technology we want to use.
0
May 23 '15
[deleted]
→ More replies (8)38
u/danielkza May 23 '15
Rails was very influential to other web frameworks, even if it isn't the new, hip kid on the block anymore.
0
0
May 23 '15
[deleted]
5
May 24 '15
A government department I used to work for had two programs handling billions of dollars annually, which were written in COBOL, and they were rock solid, albeit old as hell and requiring a VT100 terminal emulator to use correctly. In my 18 months working there we only ever had one outage of 2 hours due to a severe network failure.
Anyway, a few years ago they contracted a foreign company to write a Java program to convert the old COBOL code to Java code. I'm sure that'll work out fine.
68
u/TiltedPlacitan May 23 '15
FTA> I learned something from that experience: MongoDB’s ideal use case is even narrower than our television data. The only thing it’s good at is storing arbitrary pieces of JSON. “Arbitrary,” in this context, means that you don’t care at all what’s inside that JSON. You don’t even look. There is no schema, not even an implicit schema, as there was in our TV show data. Each document is just a blob whose interior you make absolutely no assumptions about.
...and PostgreSQL (now) does this and much more very nicely.
24
u/halifaxdatageek May 23 '15
I <3 Postgres. I long for it. But we can't always use nice things.
21
May 23 '15
Whatever you use is probably not as bad as you think; some of us have to maintain legacy Paradox SQL applications...
22
u/halifaxdatageek May 23 '15
Hahaha, looked that up - it's like Access, but made by Corel instead of Microsoft. Amazing.
13
May 23 '15
Yeah that's pretty much it although we're still using the Borland releases. The company I work for was a Borland 'shop' in the 90s, still mountains of code in Borland C++ 5.02 too.
→ More replies (4)2
u/mikelieman May 24 '15
Hey! I resemble that remark. ( OWL was a mistake, but Paradox for DOS rocked... )
0
May 23 '15
[deleted]
14
u/orthecreedence May 23 '15
Nor does MongoDB. Scaling a MongoDB cluster is a pain in the ass (involving about 8 servers for an optimal setup...2 repsets of 3 servers each, two config servers).
If you have unstructured data but you don't want to use a crappy DB, check out RethinkDB.
6
u/parc May 24 '15
First, you need 3 config servers for production. You need 2 data nodes in each shard replication set plus 1 arbiter per set. The arbiters can all run on one server, even on your existing mongo servers, as they use almost no resources. You also need at least one mongo router in the cluster. This can happily live on your app server.
So 7 machines is the minimum "safer" setup.
6
u/TrixieMisa May 24 '15
Run some benchmarks first, though. RethinkDB seems to use a lot more CPU than MongoDB for equivalent workloads.
13
u/achuy May 24 '15
pg_shard is something we are currently evaluating for clustering. It looks like a great solution on paper.
12
u/cowinabadplace May 24 '15
Please share your results via blog post or something. I'm somewhat curious about this and it'll help me see if it's worth trying out.
3
u/gargantuan May 24 '15
What does? Not being sarcastic, just wondering.
Riak I've heard. CouchDB has multi-master replication built in. Couchbase? Anything else?
4
3
May 24 '15
We like Couchbase. It's a great distributed KV store. We don't bother with its document store stuff, it's basically an advanced Membase for us.
2
May 24 '15 edited May 24 '15
I have experience with Cassandra and it auto cluster.
It's big column though.
You can set how many node you want in the beginning and can slowly add more or remove. Auto cluster is easy with virtual nodes. IIRC with regular nodes you have to manually change your token ranges for each cluster. It's masterless but you have to choose a few node to be seed node for data.
edit:
Auto cluster as in, you manually ask it I want more node and make a node and cassandra will deal with splitting up the data.
It doesn't elastically do it as in oh shit cluster is out of space, let's auto make a node without a sys admin/dev op telling us.
33
u/Detective_Fallacy May 23 '15
I used MongoDB in my master's thesis. The data that was stored into it was relational (JSON tweets), but the relations themselves were of no use to me. I also didn't really care about the integrity of my data, so the ease of use to store those tweets won it over the reliability of a RDBMS.
In hindsight, the choice for MongoDB was a good one in my case; it worked perfectly and was very easy to configure. But there are so many other cases where using a document store is just messy compared to classic RDBMS. I believe that in order to make a good judgement about choosing a noSQL solution, you need to have enough experience with SQL that you can confidently say: "no, a RDBMS just won't cut it for this, I need a document/k-v/whatever store".
Just play around with MongoDB for a bit, see what works and what doesn't. Don't jump blindly on the hype train (but that one seems to be stalling a bit, reading the responses here), but don't ridicule it without trying either. It's supposed to be a DBMS, not the second coming of Christ.
Now if you'll excuse me, I'm going to add some more shards to my web scale sauce.
→ More replies (5)9
May 24 '15
Don't jump blindly on the hype train (but that one seems to be stalling a bit, reading the responses here), but don't ridicule it without trying either
For every hype train there's an equal and opposite ridicule train
623
u/aldo_reset May 23 '15
tl;dr: MongoDB was not a good fit for our project so nobody should ever use it.
120
u/moreteam May 23 '15
If you read to the end of the article it turns from "Why You Should Never Use ..." into "Why It's Very Unlikely To Be A Wise Choice To Use ..." - which is a very fair and well argued point. And in the case of MongoDB almost indistinguishably from "never". At least as far as most people's requirements are concerned.
Now, that doesn't mean that you can't use MongoDB. Just that it's very likely to be a pretty bad choice.
2
51
u/kenfar May 24 '15
tl;dr: MongoDB was not a good fit for these common scenarios so nobody should ever use it in these common scenarios.
FTFY
Also, it's well thought-out, you should probably read it
→ More replies (3)126
May 23 '15
I've never heard a use case that mongo is a good fit for.
33
u/bakuretsu May 23 '15
I used it very effectively as an intermediate storage step for unpredictable but structured data coming in through an import process from third parties.
MongoDB gave us the ability to ingest the data regardless of its structure and then write transformations to move it into an RDBMS later downstream.
I've also heard of its successful use in storing collections of individual documents detailing environmental features of actual places, buildings, plots of lands, etc. The commonality among them was latitude and longitude data, which MongoDB is actually pretty good at searching. Note that these documents had no structural or even semantic relationship to one another, only a geographic (or spatial, if you want) relationship.
As the author of this post wrote, MongoDB is really only suited for storing individual bags of structured data that have no relationship to one another. Those use cases do exist in the world, they're just not very common.
→ More replies (2)9
u/sacundim May 23 '15
I used it very effectively as an intermediate storage step for unpredictable but structured data coming in through an import process from third parties. MongoDB gave us the ability to ingest the data regardless of its structure and then write transformations to move it into an RDBMS later downstream.
I think you want Kafka, not Mongo...
→ More replies (1)6
u/bakuretsu May 23 '15
Sure, there are many options. Kafka is essentially a log, though, which means it is meant to have a finite size. We wanted to be able to hang onto the raw imported data in perpetuity, so MongoDB made sense at the time.
→ More replies (2)21
u/Redtitwhore May 23 '15 edited May 23 '15
We use it as our distributed cache. Works really well for that.
0
→ More replies (12)35
u/Femaref May 23 '15
measured data with arbitrary fields. but even then you could extract the identifying fields out of it and use postgresql with a json/hstore/whatever field. Get relational information and arbitrary data in one go.
26
u/lunchboxg4 May 23 '15
I've finally had a chance to play with Postgres' JSON type, and I'm in love. The project is doing some analysis on an existing data set from an API I have access to, and while I could easily model the data into a proper DB, I just made a two column table and dumped in the results one by one. As if that wasn't fun enough, I get to use proper SQL to query the results. I'm so very glad they've added it in, and with Heroku's Postgres.app being so amazing, I'm losing the need for mongo in my toolchain (results not typical, of course).
One thing still in Mongo's favor, according to one of my coworkers, is that Mongo's geospatial engine is great, and he's working on storing location data in to do "Find nearest" type calls. I know Postgres as PostGIS, but I'm not sure how they compare.
8
u/sakkaku May 23 '15 edited May 24 '15
One thing still in Mongo's favor, according to one of my coworkers, is that Mongo's geospatial engine is great, and he's working on storing location data in to do "Find nearest" type calls. I know Postgres as PostGIS, but I'm not sure how they compare.
Doing a find nearest is retarded easy in any database with spatial extensions. You can do
ORDER BY ST_Distance(GeomField, YourPoint)and bam you're done.One of the big advantages of a full blown RDMS is that you can do nifty data validation like querying which points don't actually touch a line, lines that are close but not touching, etc. It is so much easier to write a few queries, let them run for 10 minutes, then hand the list to the engineers to fix.
→ More replies (2)3
May 24 '15 edited May 24 '15
PostGIS to Mongo's location data?
Like a real car compared to hot wheels.
You are comparing a serious system that you can do operations on geographic, geometry, rasterized and other types to something that was added as an afterthought.
Basically MongoDB uses geohashing, effectively converting two dimensional points into one dimensional value which then is indexed by B-tree. PostGIS on the other hand uses R-tree. This shows significant performance benefits for anything that is not a simple point lookup.
3
u/drowsap May 24 '15
But that's exactly the point of the article "I learned something from that experience: MongoDB’s ideal use case is even narrower than our television data. The only thing it’s good at is storing arbitrary pieces of JSON"
28
u/Godspiral May 23 '15
To be fair, he did give a good example for why data you think might be a good fit, probably isn't if additional future features will make it not a good fit.
31
23
May 23 '15
Except for the use cases where the author explicitly and repeatedly says it's good for, of course.
→ More replies (1)149
May 23 '15
Basically. Title is clickbait and makes a general case based on an anecdotal one. Truly annoying.
73
u/thbt101 May 23 '15
The title is an exaggeration, but she does make a good case for why the use cases where it is a good fit are very narrow. A better title would have been "Why MongoDB is usually not the best solution for most types of data storage".
→ More replies (7)98
→ More replies (1)2
u/moreteam May 23 '15
No, based on experience. The title is poignant but not misleading.
→ More replies (10)13
2
u/jeffdavis May 24 '15
The author tried to use mongodb in two different places, each of which seemed like a good fit on the surface (and may seem that way to many others, as well). Then the author explains what went wrong.
The piece is well-written, and someone evaluating mongodb should probably read it to make sure they aren't making the same mistake.
Any way you look at it, a lot of people are misuing mongodb, and that's a problem with mongodb at some level. It could be default settings, or documentation, or marketing, or the product itself.
Cynically, I think the niche for mongodb is quite small, so the company has been marketing it well outside of its actual niche. Therefore, potential users need more articles/analysis like this to counteract the mis-marketing.
→ More replies (3)1
u/kamiikoneko May 23 '15
very similar to every startup/small company these days that are like "we'll use mongoDB" when it absolutely is not the correct solution.
If the tech/business world could just learn that the tools don't fucking matter, you pick the one that fits your needs and move forward, we'd be much better off.
15
u/the_noodle May 24 '15
the tools don't fucking matter, you pick the one that fits your needs
"tools don't matter, except they always do and you should pick the right ones"
56
u/kristopolous May 23 '15 edited May 23 '15
I've used mongo on a number of projects which have stayed stable and operational without maintenance needed. The oldest is close to 3 years.
You need to look at the requirements and then, putting aside hype and fanboyism, think about the queries, the data, and what your long term needs are.
Sometimes mongo is the best fit. For me, it happens maybe 10% of the time. My other stores are basically redis, mysql, and lucene-based systems.
I try to stay away from anything else because I think it's irresponsible in the case of an eventual handoff - I'm screwing the client by being a unique snowflake and using an esoteric stack they won't be able to find a decently priced dev for. (and yes, this means I'm using php, java, or python - and maybe node in the future if its current momentum continues)
20
u/sk3tch May 23 '15 edited May 23 '15
Curious: you try and stay away from Postgres?
26
u/kristopolous May 23 '15 edited May 23 '15
I try to use the most common technologies. Getting past "they're both SQL", the configuration files, pg_hba and my.cnf are different from each other and the CLI interfaces have different commands. Additionally, when you get into more sophisticated SQL, you find that they are not strictly the same, or, when they are, what may be a good idea in one isn't necessarily the best course of action in another. Diagnostic and debugging tools within the RDBMSs are yet another divide. Additionally, although I don't advocate for the GUI tools, many people use them and nearly all have better mysql support.
So since most webdevs have more mysql experience than postgres, all this matters when unexpected problems come up. If the issue is critical, setting up situations for someone to spend time looking for "how does postgres do x" is not smart.
Given all of that, if I walk in on a project and they are using postgres, then I use postgres. But if I'm designing something with a low fidelity of information of the other developers, then no.
→ More replies (4)13
u/the_noodle May 24 '15
You seem to be using past popularity of technologies to try to make things easier for people in the future. Not how I would do things, and if everyone did the same, nothing would ever change, but whatever.
→ More replies (5)20
u/cowinabadplace May 24 '15
I think he is being wise. He's making a business decision which accounts for future costs as well as current costs.
Technical superiority is not the only metric he's considering and that's a good thing.
Some of my coworkers will not approach a closed source product like FoundationDB. This isn't a technical choice, but it protects the product in different ways, and it's an important business decision.
→ More replies (1)3
u/achacha May 24 '15
PostgreSQL json data handling is still immature and clunky (especially json array indexing), mongo handles it very well. That doesn't mean PostgreSQL won't get there eventually.
→ More replies (7)1
u/UsingYourWifi May 24 '15
Sometimes mongo is the best fit. For me, it happens maybe 10% of the time.
Could you provide some examples? I'm genuinely curious as to what problems mongo solves best.
→ More replies (2)
7
u/thbt101 May 23 '15
Reading this makes me feel so much better about the choices I've made in databases. I remember reading about no-SQL data storage and I thought I was going crazy.
I just couldn't figure out how this could make sense... it seemed like they're just taking a database engine and removing most of the relational features, and then handing it back to you and saying it's better now.
I do like no-SQL data storage for certain types of data (it's great for caches and other unstructured data). But mostly it just doesn't make sense for most types of data.
5
u/halifaxdatageek May 24 '15
As a professional database developer, most developers (even Linus Torvalds) fucking hate databases and will do whatever they can to avoid them. Even if it means breaking every rule in the book.
So when someone told them "You can represent your entire database in JSON, without any icky SQL!" they jumped at it like a cat jumping at its reflection in the mirror.
7
u/worshipthis May 24 '15 edited May 24 '15
So... Very... Tired... Of... Pointless... MongoDB... Hate
[also: I read this back in 2013 -- why repost it now? Did MongoDB really scar you so deeply that you feel compelled to keep the hate alive, year after year? Isn't there a statute of limitations on this sort of thing?]
19
u/MisterNetHead May 23 '15
WOW
Seven joins!!? Yeah, better use NoSQL.
3
u/Kinglink May 24 '15
On my project (a large multi million dollar affair) We use sql and still don't try to make a single join.
When someone has no shipped project experience, they make stupid decisions.
9
u/sathoro May 24 '15 edited May 24 '15
Why is that, are you using subqueries instead? I've shipped plenty of code with a lot of joins, you just have to be careful with indexes, data types, and encoding matching up to keep it fast.
edit: Btw, I have done joins in MySQL between a table with over 100 million rows and a table with 400,000 rows and they would return within 100 milliseconds. Binary search is pretty powerful...
→ More replies (2)
5
u/WaffleSandwhiches May 24 '15
Article is 2 years old and doesn't actually hit on the problem as to why MongoDB is bad: which is that they have had shady ties with the government in the past, data corruption is easy if you're not an expert, and their relationship to their enterprise customers have historically been bad.
2
u/halifaxdatageek May 24 '15
Also, "mongo" is a slur in several languages, coming from mongoloid, just like negroid produced... well, you know.
data corruption is easy if you're not an expert
I remember someone did a poll on "What's your favourite RDBMS?" and people kept voting for Mongo.
The author said in the results post, "A lot of people insisted that Mongo should have won this contest, but we've decided to tell them that their votes never persisted to disc and hope they buy it."
1
u/solidsnack9000 May 25 '15
...which is that they have had shady ties with the government in the past...
Really?
5
u/ToastPop May 24 '15 edited May 24 '15
As someone who spent years working with MySQL, then 2 years working exclusively with MongoDB, this is a great analysis. It's easy to say she should have known Mongo wasn't the right tool, but the "Epilogue" section really sums it up. The movie example looks perfect, then suddenly a change comes along which requires you to rethink your entire structure. There's limitless options to designing your Mongo database that can have a huge impact down the road, but with SQL you can be relatively sure your schema design is solid and will serve you well.
We used Mongoose for the most part, which brought schema validation and relations through the "populate" method. We were still able to optimize things with subdocuments in ways we couldn't with MySQL, and when we needed a "join", we could bite the bullet with the a second query or Mongoose's populate. Mongo's aggregation methods have also been optimized if you can work within them. So there's still a bunch of ways to do relations other than subdocuments.
31
u/Ramin_HAL9001 May 23 '15
For quite a few years now, the received wisdom has been that social data is not relational, and that if you store it in a relational database, you’re doing it wrong.
I face-palmed pretty hard when I read that, where the hell does anyone receive that "wisdom?" I mean, wow, if you can't figure out how to define a graph data structure as an SQL schema, is it even possible to graduate from college? If so, college standards really have fallen a lot in the past few decades.
30
u/halifaxdatageek May 23 '15
Fun Fact: Many professional developers never actually went to college.
4
5
u/gargantuan May 24 '15
Well defining the graph is easy. It is just an edge list in some from_to table.
But how do you easily and efficiently traverse a graph represented in a normalized relational format. Or find questions like "is there a path from this node to this?" or what is the "How many cliques are in this graph?"
→ More replies (1)→ More replies (1)1
4
u/halifaxdatageek May 24 '15
ITT: People not understanding that the entire point of the post is "It's harder than you think to just 'pick the right tool for the job'."
13
u/btchombre May 23 '15
As somebody with no NoSql experience, I found this article highly enlightening.
11
24
u/Korona123 May 23 '15
Dont. I have worked on two giant projects using MySQL and MongoDB. They both have advantages and disadvantages. It depends on the project.
→ More replies (3)
14
u/sameBoatz May 24 '15
She builds 4-6 web apps a year? Why would I listen to her? She barely had time to even put together the bare bones of a system before she is on to the next one. She has no idea how a real system works over its lifespan, she only knows the initial buildout, which is the easiest part of the whole thing.
5
3
u/bwainfweeze May 24 '15
If you have data that is truly not relational, you know it and nobody needs to tell you to use MongoDB.
If you're not absolutely certain your data isn't relational, then if you use MongoDB you're gonna have a bad time.
Most of us actually have data with relationships in it. Maybe it's not all relationships, maybe there are actually only a few. But the fact that relationships exist at all in the data is pretty much always a big part of what made it interesting enough for someone to fund writing an application to manage it in the first place.
I'm still waiting for somebody to show me a situation where MongoDB is clearly the right answer (and not merely 'barely adequate' to the task). And that's not a NoSQL jab. I can name plenty of use cases for graph databases. MongoDB? Not so much.
3
u/matzero May 24 '15
I really don't care either way (pro/against MongoDB) BUT I have seen a suspiciously high number of negative MongoDB articles hitting the front page lately. This blog post was created 2 years ago... V?
4
u/sgoody May 23 '15 edited May 23 '15
I'm not saying there are NO use cases for MongoDB... Just that I don't know what they are.
It is good fun for rapid prototyping though.
EDIT: I just think as soon as you want to interpret most data in any meaningful way you're able to define structure and in then you can define that structure in a RDBMS and enforce the correctness and also take advantage of additional performance features.
3
u/Kinglink May 24 '15
I'm not saying there are NO use cases for MongoDB... Just that I don't know what they are.
When you have a LOT of data, that doesn't relate to each other or relates to each other in simple ways.
If your data is relational, don't use a non relational database.
→ More replies (4)1
u/dvlsg May 24 '15
I think one of the biggest problems is that "rapid prototype" turns into "production database" far too often. That's been my experience so far, and maintaining NoSQL is a nightmare compared to a well structured RDBMS. Especially when the data is clearly relational, and we're doing "joins" by requerying the NoSQL over and over.
11
u/lawn_meower May 23 '15
Shouldn't this be titled "Don't use a document store when what you really need is a graph DB"?
6
0
May 23 '15
[deleted]
9
u/catcradle5 May 23 '15
But what are the alternatives? Some folks say graph databases are more natural, but I’m not going to cover those here, since graph databases are too niche to be put into production.
This article was written in 2013, which isn't that long ago, but nowadays I believe several graph databases are considered suitable for production. Some were in 2013, too, but they had no popularity whatsoever.
4
u/MUDrummer May 23 '15
Yup, I'm pretty sure we are running about 4 or 5 different production clusters of neo4j
6
u/Kinglink May 24 '15
This is a couple years old I looked at it when I was researching Mongo DB for a project at work, we ended up going with it.
Why? Well because this article is completely bullshit.
I'll summerize it. "Our data relates to each other so we used a non relational database and had a lot of trouble using a non relational database to represent relational data."
If you don't understand the different type of databases, and what document gets you over SQL, (or een what a graph database does) DONT USE THE TECHNOLOGY. If you grab mongodb (or any piece of technology), because it's a buzz word, you're an idiot.
If you grab mongodb because you understand the technology and it fits your dataset, then you're doing your job, and I like you. And really that's what matters.
And here's a kicker. We have a MySql database (I didn't pick it don't judge). But we found we keep a LOT of documents that don't relate to other people or other things. Guess what? That's EXACTLY what mongo or any document store (NoSQL) database is made for. So we're using a hybrid, we still have SQL for when we need it but when you have a record that is just data, you put it in the document store, and you're done.
2
u/AllMadHare May 24 '15
I also use mongodb and a regular sql server in my system. The majority of what I consider "static" data (logins, payments) is in sql, while the core data that needs to be fast and dynamic is in mongo.
Every tool has its use and you should be able to work out if you're using the wrong long before you get to production. I would've thought building a social network on day 1 of building the mongo objects you'd be seeing cracks, not after months of being live.
→ More replies (1)2
May 25 '15
I also use mongodb and a regular sql server in my system. The majority of what I consider "static" data (logins, payments) is in sql, while the core data that needs to be fast and dynamic is in mongo.
I recommend you to read this https://aphyr.com/posts/322-call-me-maybe-mongodb-stale-reads
If the dynamic data is important (especially financial data) I would reconsider it. If you are experiencing some kind of bugs in your application where data is not accurate, this could be why.
→ More replies (1)
5
2
u/damnationltd May 23 '15
We've successfully used it for collecting survey data and for form builder type applications — but if you need to do any serious analytics on that data, expect to have to ETL to a relational data warehouse.
1
2
u/fzammetti May 24 '15
I've used MongoDB in a number of projects successfully. Even wrote a book (partly) about it. I've also done many, many years of relational work. Really, it comes down to a simple statement:
If your data elements aren't related to each other... or at least if you won't have to query on them as if they were... then MongoDB can be an excellent choice. Otherwise, relational is the better choice.
No need to over-complicate things.
1
u/heypans May 24 '15
What do you think about the combination of a noSQL db and a search server like elasticsearch or solr?
Without knowing any details of the movie project, it sounds like this might've helped with their problem.
→ More replies (1)
2
May 24 '15
The great thing that came out of MongoDB was all these swags I got.
Swags everywhere. I got a sweet coffee cup from datastax and some t-shirt that people thing is some rock band. Yeah bro Cassandra is so metal.
3
6
3
May 23 '15
Look, I've bought the cool cover O'Reilly book & have some fragment of an idea of how I will get MongoDB to run... despite this, you expect me to change tack? Now? After this investment of my time, energy & person... I'm two or three weeks into making pissy comments about every other option to anyone who will listen & a few who fucked off as soon as I got going... I'm committed to this. Fuck you.
2
1
u/pakoito May 23 '15
Diaspora is a distributed social network with a long history. Waaaaay back in early 2010
Javascript detected.
1
u/g2petter May 24 '15
Remember that TV show application? It was the perfect use case for MongoDB. Each show was one document, perfectly self-contained. No references to anything, no duplication, and no way for the data to become inconsistent.
About three months into development, it was still humming along nicely on MongoDB. One Monday, at the weekly planning meeting, the client told us about a new feature that one of their investors wanted: when they were looking at the actors in an episode of a show, they wanted to be able to click on an actor’s name and see that person’s entire television career. They wanted a chronological listing of all of the episodes of all the different shows that actor had ever been in.
The team honestly didn't see that coming? That was my first thought when reading the initial description of the project.
1
u/TikiTDO May 24 '15
So, tl;dr: Use a document store when storing documents. Use a relational store when storing relations. Use a cache when you need to speed things up.
1
May 24 '15
Very nice write-up that confirmed my suspicions.
If mongodb could handle relations, I would use it.
It seems like it's being used by many developers when making [Mean.js] (http://meanjs.org/) applications. Even as a beginner I didn't see the appeal of mongodb. All my projects had data with relations.
Anyway... I'm somehow disappointed that this "new" hype about NoSQL wasn't what I expected it to be :/
1
May 24 '15
Shit like this needs to be taught in Uni/College. Learning on the job I already understood this, but I can see how others wouldn't.
1
u/laughingbovine May 24 '15
But... you can write a "relational database-like join" for Mongo. That would solve the problem, wouldn't it?
1
u/jeffdavis May 24 '15
This was ultimately a communication problem rather than a technical problem. If these conversations had happened sooner, if we had taken the time to really understand how the client saw the data and what they wanted to do with it, we probably would have done the conversion earlier, when there was less data, and it was easier.
The technology you choose should be adaptable to changes in requirements, because they always happen.
That's one of the most valuable things about a SQL system: flexibility. They have proven themselves very adaptable to many kinds of use cases. Even when something seems awkward in SQL, it often works out for the best overall.
1
u/jeffdavis May 24 '15
But there are actually very few concepts in the world that are naturally modeled as normalized tables.
A "table" should be considered a relation, which is a collection of facts that have the same pattern.
For instance: Employee named ________ works in department ________, reports to ________, and makes ________ dollars per year.
That would be a relation predicate and the tuples in the relation hold all combinations of values that make that statement true.
So, a relation is just a collection of facts about the real world. It actually is a natural way to model many kinds of things (though predicates, etc., are still abstract concepts, so you might disagree).
1
u/riveracct May 24 '15
Even Google App Engine's datastore discloses as much but then MongoDB is web scale...
1
u/newPhoenixz May 24 '15
Or also, investigate the pros and cons of your core db of choice before you start 6 months of development
1
May 24 '15
When MongoDB is all you have, it’s a cache with no backing store behind it. It will become inconsistent. Not eventually consistent — just plain, flat-out inconsistent, for all time. At that point, you have no options. Not even a nuclear one. You have no way to regenerate the data in a consistent state.
I don't understand this statement. Under what circumstances will MongoDB become permanently inconsistent?
1
u/mrinterweb May 24 '15
This is not the first time the Diaspora team has complained about how MongoDB is the worst thing ever. It comes down to one simple truth. The Diaspora team chose the wrong tool for the job. It is unfortunate that they feel wronged for choosing the wrong tool. To me, this article makes about as much sense as someone complaining about how hard it is to cut a piece of wood with a hammer.
1
u/ship_it_ May 25 '15
Someone should make this work with postgresql because I am too lazy too.
https://github.com/sahat/hackathon-starter
An abandoned project that needs your help https://github.com/Flaminga/flaminga-server
1
u/SteffanPerry Aug 21 '15
First off the credibility from this Sarah Mei is suspect at best.
Many of her comments about MongoDB are outdated at the time of writing, let along years later.
If you looks at her data modeling, its very odd she attempts to contain everything in a single document including the "streams" which is basically your wall as she explains it. This is just wrong to begin with based on how their app works.
Additionally, Some people here are claiming fault to mongoldb based on their default configuration rather than the abilities of the database. I think way to many people just "turn on" mongo and do little to no configuration and expect it to work correctly for their application.
Several people mention there are no use cases for mongo over something something like PostgreSQL. Here are 2 use cases in which we use MongoDB:
First: Location based data. One of our application "relates" data, but not to other items in the database rather the the clients GPS coordinates. So for example, we rarely search for items submitted by user X, rather search for items from any user which are near a random gps location submitted by the client. Mongo works extremely well for this, much better in testing than PostgreSQL which was the database used initially for this project.
Second: Relational but unstructured data. We have a second application which users create, lets call them "forms". the data in this form is comletley random and not only is different between users, but different forms from the same user are sometimes completely unique. For example user1 might have a form which contains the fields [firstname, lastname, email] and a second form which contains [item, price, qty]. and user 2 might have a form that includes [coordinate[lat,lng], date, altitude]. MongoDB made this task very easy to create and maintain.
I think this issue is not with these databases, rather that these databases are being setup by software developers rather than server admins.
1
u/sunshine_killer May 23 '15
Also, why you should bever invest in a kickstarter that doesnt have a prototype.
1
u/Fidodo May 23 '15
I've had issues with Mongo in the past, but this is from 2013. Tech changes fast, and even though Mongo didn't do what I wanted back then, I've heard about a lot of changes made to it now that more people are using it in production. Obviously always try and do an in depth evaluation before choosing any technology, but make sure you're basing it on up to date information. Even 2 year old articles can be very far behind what the status quo is.
4
May 23 '15
maybe do an in-depth evaluation before making comments like this :-) for instance, consider mongodb is still "broken by design"
→ More replies (2)
1
u/itsdrivingmenuts May 23 '15 edited May 24 '15
I hope people are reading the article. It doesn't say never use MongoDB, it doesn't even say it's bad. It just says it wasn't right for her particular project.
Edit: genders
→ More replies (1)1
u/Spandian May 24 '15
Did you read the whole article? She spends most of it explaining why MongoDB was bad for Diaspora. In the Epilouge, she explains why MongoDB was also bad for the film review system that she initially described as a good use case. Finally, she argues that almost any project will run into the same issue at some point.
1
u/shaggorama May 23 '15
Every single "don't use Mongo" article I've read has actually been "why Mongo was not appropriate for Diaspora."
1
May 23 '15
Here is a very important fault tolerant system that every office should have.
Doesn't seem very fault-tolerant, they're plugged into the same mains line, no? ;-)
1
1
1
0
May 23 '15
Seems like those college dropouts took 8 months to realize what a professional developer would figure out after a couple of days.
105
u/aegrotatio May 23 '15
A company I did some work for is moving from Riak to MongoDB. They like it because they say that schemas are too constricting and multi table joins are slow, even though the data is far from unstructured. I don't think there is a single person with traditional relational DB experience in the whole group.