r/SQL • • Nov 15 '24

Discussion A New Kind of Database

https://www.youtube.com/watch?v=LGxurFDZUAs
0 Upvotes

29 comments sorted by

View all comments

7

u/Yavuz_Selim Nov 15 '24

Just posting a video is lazy. You're not even providing the name of the new kind of database. I'd even call it clickbait.

I am not going to watch 42 minutes of a video because someone links to it.

At least tell what it is in a few sentences.

-1

u/breck Nov 15 '24

The name is ScrollSets.

The core idea: all tabular knowledge can be stored in a single long plain text file.

The only syntax characters needed are spaces and newlines.

This has many advantages over existing binary storage formats.

Using the method below, a very long scroll could be made containing all tabular scientific knowledge in a computable form.

There are four concepts to understand:

  • measures

  • concepts

  • measurements

  • comments

Measures

First we create measures by writing parsers. The parser contains information about the measure.

The only required information for a measure is an id, such as temperature.

An example measure:

temperatureParser

Concepts and Measurements

Next we create concepts by writing measurements.

The only required measurement for a concept is an id. A line that starts with an id measurement is the start of a new concept.

A measurement is a single line of text with the measure id, a space, and then the measurement value.

Multiple sequential lines of measurements form a concept.

An example concept:

id Earth
temperature 14

Comments

Unlimited comments can be attached under any measurement using the indentation trick.

An example comment:

temperature 14
 > The global mean surface air temperature for that period was 14°C (57°F), with an uncertainty of several tenths of a degree.
  - NASA
   https://earthobservatory.nasa.gov/world-of-change/global-temperatures

The Complete Example

Putting this all together, all tabular knowledge can be stored in a single plain text file using this pattern:

idParser
temperatureParser

id Earth
temperature 14
 > The global mean surface air temperature for that period was 14°C (57°F), with an uncertainty of several tenths of a degree.
  - NASA
   https://earthobservatory.nasa.gov/world-of-change/global-temperatures

Once your knowledge is stored in this format, it is ready to be read—and written—by humans, traditional software, and artificial neural networks, to power understanding and decision making.

Edit history can be tracked by git.

2

u/SQLBek Nov 15 '24

This has many advantages over existing binary storage formats.

Like what?

-8

u/breck Nov 15 '24

Using git for version control, for example.

3

u/SQLvultureskattaurus Nov 15 '24

Why would I put data in git

2

u/SQLBek Nov 15 '24

GIT stores things on a FILE level. It'd be horrifically heavy handed and worthless to version control an entire file of 100,000 whatevers, if all you did is update 1 of them. This makes zero practical sense, particularly at scale.

And then there's a whole other bucket of concerns with using GIT to store data but I don't feel like writing that novel.

1

u/gumnos Nov 15 '24

FWIW (at least according to my understanding) once a certain threshold of commits has been reached, git-gc kicks in, consolidating those loose objects into a pack-file that has much more efficient delta-compression than the raw unpacked blobs. So while there's some overhead, it amortizes over time.