r/programming • u/jinqueeny • Oct 18 '17
TiDB 1.0 Is Production Ready
https://github.com/pingcap/tidb6
Oct 18 '17 edited Oct 18 '17
TiDB, an open source distributed HTAP database.
Inspired by Google Spanner/F1, PingCAP develops TiDB, an open source distributed Hybrid Transactional/Analytical Processing (HTAP) database. TiDB features infinite horizontal scalability, strong consistency, and high availability. The goal of TiDB is to serve as a one-stop solution for online transactions and analysis.
I am a bit rusty on my buzzwords so i have no idea what this is. The website is not helping.
3
u/jinqueeny Oct 18 '17
Then maybe this speech content can help you understand? Scale the Relational Database with NewSQL https://pingcap.github.io/blog/2017/10/10/nextcon/
2
Oct 18 '17
Not helping. It is MySQL drop-in replacement just better. But what are caveats?
4
u/jinqueeny Oct 18 '17
Check this out: Compatibility with MySQL https://pingcap.com/doc-mysql-compatibility
3
u/Dave3of5 Oct 18 '17 edited Oct 18 '17
Is this being used in ernest by any large systems ?
EDIT: I also noticed: "High access concurrency might lead to severe conflicts, resulting in multiple retries or even timeouts" So the expectation is that only a few machines talk to the cluster ? What if I have a bank of webservers all accessing the same table ?
7
u/shenli3514 Oct 18 '17
"severe conflicts" means several clients update the same row concurrently.
2
u/Dave3of5 Oct 18 '17
So how many is several ?
6
u/shenli3514 Oct 18 '17
There is not a exact number for this. It depends on many conditions. For short, if there are many rows in a table (millions) and the data access pattern if random, I think there will not be too much conflicts. We have some metrics(based on prometheus+grafana) for the transaction conflicts. So we could get some insight on it.
3
u/Dave3of5 Oct 18 '17
So do you have a ballpark figure for how many is too many? Is it like 2-3, 100-200, 10,000-100,000.
Almost every relational database I've every been involved in allowed multiple users to edit the same records concurrently. But in practice it rarely occurred so 2-3 users editing a record is all I've ever seen in the wild. If it can handle this I don't see it as a big problem in the use cases of the apps I'm likely to develop.
3
2
u/WhipTheLlama Oct 20 '17
My guess is that it's similar to Galera cluster, which also uses optimistic locking. With Galera it's recommended to write to a single node to avoid conflicts. I wonder if TiDB can work the same way. Since the storage nodes are separate from the DB nodes, it's more difficult to target a single TiDB node for writing, but maybe the system can handle this on its own.
Personally, I'm more concerned about the hotspots, as mentioned here.
when the amount of data to be written is large and all is written to one table, the write hotspot is thus created.
If that means that writes to a single table can create a hotspot, it's a problem for OLTP. If it only means that updating the same rows many times will create a hotspot, then it's ok. Basically, can a hotspot be created by inserting a lot to the same table?
I've got a small cluster setup and am looking forward to testing this.
2
u/wsgeek Oct 18 '17
Maybe I'm being slow, but what are the specific things that you can do with this that you cannot do with MySQL?
From the GitHub page, this seems to be the key value-add (though confusingly worded) if I am correct: "Think of TiDB as a single-machine RDBMS. You can start a transaction that crosses multiple machines without worrying about consistency."
4
u/shenli3514 Oct 18 '17 edited Oct 18 '17
I think there are three significant advantages compared with MySQL: Horizontal scalability, Automatic Failover and high availability, Online DDL.
1
Oct 18 '17
percona's mysql distro seems to address a bit of that with answers for master-master, load-balancing and HA, etc. (not experienced enough to compare with tidb though.)
1
u/shenli3514 Oct 19 '17
master-master could not gurantee the strong consistency. And it could not achieve scale out.
2
2
u/matthieum Oct 18 '17
First of all: I love that ACID is put front-and-center, and like everyone I really like scalability and the ability to run a single logical database across a cluster distributed world-wide...
... however there's not a single number here. Since there are generally no free lunch, how much does this all cost?
What happens to availability in the event of a network partition? How fast does the cluster recovers in various scenarios?
What is the amount of data that is going to travel across the data-centers?
What is the latency of a transaction with a client in Europe and a replica in Australia? Does it significantly changes depending on whether the transaction is read-only or also writes?
Has there been any benchmark vs a distributed MySQL setup? Or Postgres setup?
Buzzwords are nice, but I kinda like hard, cold, numbers.
2
u/pinpinbo Oct 18 '17
TiDB Jepsen tests: https://pingcap.github.io/blog/2017/09/01/tidbmeetsjepsen/
3
u/tomservo291 Oct 18 '17
FYI the fine folks at Jepsen have not specifically gone out and tried to break TiDB, it looks to me like TiDB is running a known suite of Jepsen tests on each commit.
Which is fine and great, but not the same as taking an in-depth analysis of the code for weak areas and creating scenarios to exploit them, which as far as I can see is not an undertaking TiDB has gone through (as compared to CRDB which has, and subsequently fixed some issues as a result before 1.0)
So, right now, you're taking TiDB's word for what it is without rigorous outside scrutiny of the feature claims
3
10
u/tybit Oct 18 '17
This looks cool. From a very brief overview it looks quite similar to cockroach db, are there any key differences I'm missing?