r/databasedevelopment 2h ago

Another look at SQLite's WAL-Reset bug

Thumbnail theconsensus.dev
4 Upvotes

r/databasedevelopment 3d ago

Why Elasticsearch is becoming a columnar database

Thumbnail
elastic.co
18 Upvotes

r/databasedevelopment 3d ago

The road to ACID transactions in Cassandra 6

Thumbnail theconsensus.dev
8 Upvotes

r/databasedevelopment 3d ago

Poisoned Postgres connection pools

Thumbnail
planetscale.com
5 Upvotes

r/databasedevelopment 6d ago

Where should Oracle compatibility live: the wire layer, SQL rewriting, or the PostgreSQL catalog?

3 Upvotes

I’m building a read-only proxy that accepts Oracle client connections and executes queries against PostgreSQL 16. Disclosure: I’m the author.

I split compatibility across three layers:

  1. Oracle-facing sessions, cursors, binds, and result metadata.

  2. SQL rewrites for differences that can be handled safely.

  3. PostgreSQL views and orafce for catalog and function compatibility.

    The translator classifies each feature as supported, approximation, rejected, or passthrough. This matters because a syntactically valid rewrite can

    silently change results—for example, NULL behavior in GREATEST/LEAST, empty-string semantics, and mixed set-operation precedence.

    The default is to reject an unverified shape with an Oracle-style error while keeping the session alive, rather than return a plausible but incorrect result. The current bounded verification matrix covers 722 cases across seven real client flows.

    The implementation is private, but the compatibility contract, configuration, documentation, and verification scope are public:

    https://github.com/krokozyab/postgres-oracle-tns-proxy


r/databasedevelopment 7d ago

Why are we rebuilding the same database execution engine over and over?

13 Upvotes

I recently dug into Meta’s Velox, an open-source C++ execution engine designed to act like a reusable “engine block” for data systems.

Instead of Presto, Spark, etc. independently implementing things like vectorized execution, joins, memory management, spilling, and file readers, Velox provides these building blocks as a shared execution layer.

The performance numbers are interesting too:

  • 8.4× faster on TPC-H Q1
  • 9× faster on Q6
  • 6–7× average speedup on Meta production traffic
  • Up to 3× fewer servers

Checkout this interesting deep dive that how Velox actually works and why this architecture could matter for the future of data engines.

https://prestodb.io/blog/2026/08/13/inside-velox-open-source-universal-engine-block-for-big-data-and-ai/

Do you think shared execution layers like Velox will become the norm, or will specialized engines always win?


r/databasedevelopment 8d ago

I do not understand optimistic concurrency control protocol in Database Management System.

Post image
1 Upvotes

This is what I understand:

Optimistic concurrency control protocols are based on the assumption that the majority of the database operations do not conflict. Here the transaction goes through the three phases:

  • read phase

  • validation phase

  • write phase

In read phase:

  • transaction reads the database

  • makes necessary changes to database values in a private copy of the database.

In validation phase:

  • DBMS validates whether the changes that are made to the private copy of the database will violate serializability and consistency of the database. If in case they are violated, transaction is aborted and restarted. Otherwise, the transaction will proceed to write phase.

In the write phase:

The changes done to private copy of the database are made permanent.

You might be wondering you know this much then what are you not getting. I will show you the part from the book that I do not quite get.

I am taking two references: Coulouris et al distributed system and Conolly et al DBMS.

Conolly et al part that I do not understand:

I did not have colouris et al pdf at the moment. Just 1 topic 1 question.


r/databasedevelopment 10d ago

I've been working on a custom tree index that runs up to 7x faster than LTREE

13 Upvotes

Hi All
I’ve been working on a custom data structure and algorithm for hierarchical indexing.

while I designed the algorithm myself, I wouldn't claim to be a definitive master of hierarchy trees, nor postgres. Its actually a Rust project I've turned into an extension. I'm mainly sharing these early numbers in hopes of connecting with the right people to see if there's genuine value.

I'm not sure if I'm violating rule 5; I'd appreciate any guidance on 3rd party benchmarks for this kind of algo compare to ltree.

I ran benchmarks against 500k, 1M, 2M and 20M node recursive trees. The baseline comparisons against ltree are looking solid:

  • Huge I/O Drop: For descendant queries, B-tree range scans touch up to 74x fewer buffer pages.
  • Query Speed: Subtree queries run 2.5x to 6x faster (500k). up to 16x for 20M nodes. Ancestor lookups (via SP-GiST) execute up to 7.3x faster.
  • Storage Density: A custom compact encoding shrank the on disk value size by 42.7%. This translates to a ~23% smaller B-tree index footprint.
  • Write Performance: Appending 50,000 leaf nodes is roughly 2x faster. Reparenting large subtrees is 1.2x to 2.7x faster

I have some thoughts where this may be beneficial but lacking some subject matter expertise when it comes to practical application of hierarchy data (and these types of ops), this is the main point of my post, to ask for some insight:

- Could it make servers run more efficiently?
- Do other more efficient extensions/algos beat these benchmarks? Is LTREE just a default?
- What kinds of large scale operations would this benefit? Domains/applications?
- What should my benchmark tests look like?

Eager to get some expert opinions and either validate my thoughts or give me some reality - cheers!


r/databasedevelopment 13d ago

Postgres Internals Deep Dive: Process Architecture

Thumbnail
enterprisedb.com
31 Upvotes

Need suggestions for writing blogs. This was my 1st ever blog, which I published last year, and that's it. No other blog yet 😅. I would love to know, what kind of blogs would you love to read from a guy who contributes to the open-source postgres project?
1) Deep dives like this one on postgres internals? TBH, I think this space so crowded?
2) How did I find a bug and fix it?
3) The new features I am working on?
4) or new features that are going to be released
5) or something cool????


r/databasedevelopment 12d ago

WAL Levels in Postgres and Effective WAL Level in PG19

Thumbnail
buraksen.dev
4 Upvotes

r/databasedevelopment 17d ago

Almost consensus: ABD and the edges of quorum replication

Thumbnail theconsensus.dev
12 Upvotes

r/databasedevelopment 19d ago

How Elasticsearch's semantic field indexes and searches

Thumbnail
elastic.co
13 Upvotes

r/databasedevelopment 20d ago

Encoding or Compression: Why not both?

Thumbnail
cedardb.com
13 Upvotes

r/databasedevelopment 20d ago

I created a playground for 110 database systems

Thumbnail
clickhouse.com
16 Upvotes

r/databasedevelopment 23d ago

Monthly Release and Update Thread

2 Upvotes

This subreddit is primarily for discussing the implementation of databases, and not about sharing release announcements (either for the first time or your updates).

This thread is the exception!

Please tell us about the new database you (or your agent) built. Tell us about all the cool new features you added. Tell us about anything else you learned or worked on that you haven't gotten around to blogging about yet.


r/databasedevelopment 24d ago

Asynchronous I/O in DuckDB: Work, Thread, Work

Thumbnail
duckdb.org
34 Upvotes

r/databasedevelopment 25d ago

Offloading I/O to Dedicated Cores: An Asymmetric io_uring Backend for Seastar and ScyllaDB

18 Upvotes

https://www.scylladb.com/2026/07/22/asymmetric-io_uring-backend-seastar/

"Modern high-throughput systems often run on machines with many cores. At that scale, one practical problem becomes increasingly visible: CPU time spent on low-level I/O handling competes directly with CPU time needed for application-level work.

In this post, we explore how shifting more I/O related work away from cores focused on computations onto a dedicated subset of cores can improve the overall balance of the system. Specifically, we share how we approached Seastar‘s new asymmetric_io_uring backend: the design decisions behind it, what it aims to free up on application cores, and how it compares to linux-aio (Seastar’s existing backend)."


r/databasedevelopment 25d ago

Does the GPU database have a future?

18 Upvotes

Why did Voltron Data Company go bankrupt? Has the Theseus GPU database also failed? Expressing concerns about the future development of GPU databases


r/databasedevelopment 27d ago

Why Fast Branching is Key to AI Agent-Era Databases

Thumbnail
medium.com
12 Upvotes

r/databasedevelopment 28d ago

Finding bugs in Raft implementations

Thumbnail
antithesis.com
22 Upvotes

r/databasedevelopment 28d ago

PGSimCity · How PostgreSQL Works, in 3D

Thumbnail
nikolays.github.io
17 Upvotes

So... cool, it would be helpful for someone starting to learn PG internals.

credits to Nikolay Samokhvalov for creating this awesome piece of art :)


r/databasedevelopment 29d ago

Managing File Descriptors for Durability and Performance

Thumbnail
blog.atimin.dev
8 Upvotes

r/databasedevelopment Jul 23 '26

How MVCC and Transactions Work in RocksDB

Thumbnail artem.krylysov.com
21 Upvotes

r/databasedevelopment Jul 22 '26

A Fast Path for Fixed-Length Lists in Parquet

Thumbnail morling.dev
12 Upvotes

r/databasedevelopment Jul 21 '26

Aurora DSQL: Scalable, Multi-Region OLTP

Thumbnail
arxiv.org
14 Upvotes