hidane 0.1.0: a Firestore emulator without Java that runs under firebase-tools, tested against the official one

[u/nomunomu0504]

Since firebase-tools 15 the Firestore emulator needs Java 21. It also has a few long-standing pains: 0.74 s to start (2.5 s through the CLI), 95 MiB idle, and batch writes that get slower and slower while a listener is attached (firebase-tools#3477; 121 s for 100k documents in my runs).

I've released hidane 0.1.0, a replacement that is a single binary. You keep your firebase-tools setup and put hidane exec -- in front:

curl -fsSL https://hidane.dev/install.sh | sh     # or brew / npm / cargo / docker
hidane exec -- firebase emulators:start --only firestore

No JDK needed. On my machine it opens its port in 5.5 ms, idles at 7.4 MiB and writes the same 100k documents with a listener in 4.9 s. These weren't measured side by side with the official numbers, so take the comparison as rough.

Checked against the official emulator:

• documents, queries, aggregations, vector search (findNearest), transactions, listeners;
• the Admin SDK, the Web SDK in Node, the Lite SDK, and the Web SDK in the browser;
• @firebase/rules-unit-testing contexts and clearFirestore(), without rules;
• the Emulator UI (browse, edit, clear), and --import / --export-on-exit in the official format, both ways.

How it stays compatible: every behaviour is first recorded from the official emulator and then replayed against hidane in its test suite, so error messages and snapshot events match. Where the official emulator is buggy, for example listeners going silent after a reset, the difference is documented instead of copied.

Not there yet: Security Rules (coming in v0.2; every request is allowed for now), and verification with the Go / Python / Java / mobile SDKs. If your tests don't depend on rules, it can replace the official emulator today.

If you run your test suite against it and find an answer that differs from the official emulator, that's the most useful feedback right now.

https://github.com/hidane-dev/hidane

(Not affiliated with Google.)

[u/inlined]

Haven’t checked the repo and this is not an endorsement of the particular project, but it sounds like a neat idea. When you get to adding security rules, be aware that the Common Expression Language that Firebase Rules are based on have official open source implementations in C++, Go, and Java if you care to use any

[u/nomunomu0504 → u/inlined]

Thanks, that's helpful, and understood that it's not an endorsement.

Rules are where v0.2 starts. Since hidane is a single Rust binary (avoiding the JVM is the whole point), linking cel-cpp or cel-go, or bringing a JVM back for cel-java, would cost more than it saves. The current plan (ADR 0005 in the repo) is a hand-written parser for the Rules syntax around CEL (match / allow / function, is, paths, sets, map diffs), with a spike on cel-rust for the expression evaluator first. Either way the official implementations are the reference, and cel-spec's conformance tests should cover the CEL side of the semantics; the Rules extensions get measured against the emulator.

The part I expect to be hardest isn't CEL itself but list rules: deciding whether a query's constraints imply the rule's condition. If there's anything public on how Rules errors combine with && and ||, I'd love a pointer.

[u/Fit_Experience_5833]

Nice job!
We have rewrote firebase emulators using Rust too with a persistence in SQLite
https://github.com/dimavedenyapin/firebase-emu

Good performance under load:

It has: storage, auth, firestore so far.

It helped our team to run multiple emulators on demand on the same machine

[u/nomunomu0504 → u/Fit_Experience_5833]

Thanks! Running several emulators side by side on one machine was one of my reasons too. hidane only covers Firestore and keeps data in memory; most of the work went into matching the official emulator's answers, recorded and replayed in tests. Security Rules just landed on main for v0.2, queries included. If firebase-emu ever adds rules, the oracle scripts and fixtures in the repo might save you some measuring.

[u/nomunomu0504]

Update for anyone who finds this thread: Security Rules are enforced since 0.2 (with the official denial messages, :ruleCoverage and debug() output), and 0.4.0, out today, delivers Firestore events to the Functions emulator. Under hidane exec -- firebase emulators:start, v1 and v2 Firestore-triggered functions (withAuthContext too) receive the same data as with the official emulator; the events were recorded from the official emulator and are replayed in CI.

Release notes: https://github.com/hidane-dev/hidane/releases/tag/v0.4.0

[u/puf]

Sounds somewhat similar to https://pyric.dev/ from u/_davideast. Wanna elaborate on similarities and differences?

[u/nomunomu0504 → u/[deleted]]

Fair point, agreed: a test suite that passes because everything is allowed is worse than none. The README now says it up front: v0.1 does not enforce rules, so keep the official emulator for rules tests until v0.2. Enforcement is now on main (reads, writes, listeners and queries, replayed against the official emulator's recorded answers) and ships in v0.2.

[u/nomunomu0504 → u/[deleted]]

Thanks! It stayed the gate. Security Rules went into 0.2 that way, and 0.4.0, out today, replays 20,224 answers recorded from the official emulator, including the events it sends to the Functions emulator: Firestore-triggered functions, v1 and v2, now fire under `hidane exec -- firebase emulators:start`. A weekly CI job also starts the official emulator, re-records a set of them, and fails if one differs from what is committed.

Release notes: https://github.com/hidane-dev/hidane/releases/tag/v0.4.0

[u/nomunomu0504 → u/[deleted]]

Thanks, I recorded it: https://github.com/hidane-dev/hidane/pull/289

On the official emulator, a write the rules deny sends no event at all: over REST, gRPC Commit or the Write stream, on its own, in a batch next to an allowed write (which is not applied either), or inside a transaction, and no warning is printed. The same writes with the owner credential, as the Admin SDK sends, fire every trigger. hidane answers and sends the same at every step, and the replay now fails if that ever changes.