The Iron Code

decorative triangles

Cypherlite 1: I Built My Own Graph Database (And Then Migrated This Blog To It)

written on 2026-04-17 14:57 · 23 views · 13 min read

I've been building Cypherlite for a few months now. It's an embedded graph database written in Rust, backed by Sled, with a Cypher-like query language. This blog now runs on it.

The idea started back when I was team lead engineer of the data team at SPREAD.ai. Our CTO Tomas Karlsson (now founder of velr.ai) and we kept ending up in the same conversation: why are all graph databases so bad? Too heavy, too locked-in, query languages that feel like they were designed by committee. We'd brainstorm about what a good one would look like — embedded, simple, multimodal-aware — and then go back to work and use Postgres and Neo4j like everyone else. We even used ArangoDB for some time, what a time to be alive.

Those sessions stuck with me. Not as a product idea, just as an itch. Cypherlite is what happened when I finally scratched it — evenings, weekends, months, beers and a Rust compiler.

This is also the first post in a series. Whenever I have a few spare minutes I'll write about what I'm trying next with this thing. New features, new benchmarks, new bugs. A build log for another database that nobody asked for.

The obvious question is: why? do you even do this???

The less obvious answer: because I wanted to understand databases from the inside. And because use Postgres is usually the right call — which means actually building a database is one of the few places left where you're guaranteed to confront genuinely hard problems.

What Cypherlite is

It's a property graph database. Vertices and edges, each with typed properties. You query it with a Cypher dialect:

MATCH (u:User)-[:WROTE]->(p:Post)
WHERE p.title STARTS WITH 'Rust'
RETURN u.name, p.title, p.views
ORDER BY p.views DESC
LIMIT 10

The storage layer is Sled — an embedded key-value tree based on a lock-free B+ tree implementation. No daemon, no TCP socket, no connection pool. You open a file and you're done. On top of that sits a query executor that parses Cypher, plans the traversal, walks edges, filters properties, aggregates results.

There's also a schema-first GraphQL server: you write an SDL schema, and the server generates a full CRUD API with sorting, pagination, filtering, and relationship traversal at runtime. No codegen, no ORM.

The architecture in one paragraph

Storage lives in SledDatastore — 13 named Sled trees for vertices, edges, properties, indexes, and changeset metadata. The query path is: Pest grammar → AST → optional Op-tree IR → execution. All mutations are staged in a PendingChanges (in-memory) and applied atomically via Sled batch on commit. The database has git-like versioned history: every commit is a revision, you can reconstruct the full graph at any past revision, and snapshots every 50 commits keep time-travel O(commits since snapshot) rather than O(all commits ever).

A property-value index on every property gives O(limit) sorted pagination. No ORDER BY name full-table-scan when the index already gives you pre-sorted UUIDs.

Let's briefly talk about testing: 496 tests for the core logic, but it's not enough now and it probably will never be. The real test suite can only really be written when you go to production and run it against real data.

Why a graph database for a blog

Honestly, a blog doesn't need a graph database. A flat file works. SQLite works. The whole point was dogfooding — running production workload through Cypherlite to find bugs that unit tests don't catch.

The blog has posts, tags, users, and edges between them: (User)-[:WROTE]->(Post), (Post)-[:TAGGED]->(Tag). That's a tiny graph. But the migration script I wrote to move the data over turned up a real bug immediately (despite over 490 unit tests).

The bug the migration found

The migration reads the old binary journal files, then uses Cypherlite to rebuild the graph:

MATCH (u:User {username: $user})
CREATE (u)-[:WROTE]->(p:Post {legacy_id: $id, title: $title, ...})

MATCH finds the user. CREATE should attach the post. Simple.

It created zero tagged edges. The posts got created, but every MATCH (p:Post {legacy_id: $id}) MATCH (t:Tag {legacy_id: $tid}) CREATE (p)-[:TAGGED]->(t) produced nothing.

The bug: the CREATE pre-pass ran before MATCH. All CREATEs executed upfront on an empty binding set, with no access to the matched nodes. A pure CREATE query was fine. MATCH + CREATE silently did nothing useful.

The fix was structural: pure CREATE (no MATCH in the query) runs the pre-pass and returns immediately. MATCH+CREATE accumulates match rows first, then runs CREATE once per row with the matched bindings injected. Now MATCH (u:User) CREATE (u)-[:WROTE]->(p:Post {...}) creates one post per matched user, which is what Cypher has always specified.

Three unit tests added for this behaviour. The migration re-ran and produced the correct graph: 1 User, 4 Tags, 18 Posts, 18 WROTE edges, 32 TAGGED edges.

That's the value of dogfooding. You can have 496 unit tests and still miss a fundamental semantic error in the most common write pattern in the language. Running real code against real data found it in minutes.

What running this blog on Cypherlite looks like

The server is actix-web. Every request handler does something like this:

let ds = data.ds.clone();
let post = web::block(move || db::post_by_id(&ds, id)).await?;

db::post_by_id runs a Cypher query synchronously inside web::block, which moves it off the async executor. SledDatastore is Send + Sync, so cloning an Arc and handing it to a thread pool is safe.

Post IDs in URLs are legacy integers (/view_post?post_num=7). The graph stores these as a legacy_id integer property with a property-value index, so the lookup is:

MATCH (p:Post {legacy_id: $id})
OPTIONAL MATCH (p)-[:TAGGED]->(t:Tag)
RETURN p.title, p.post, p.outline, p.timestamp, p.views, collect(t.legacy_id)

One query. Index-backed lookup. Tags collected in the same pass. No joins in application code.

Admin auth is a single user in the graph:

MERGE (u:User {username: $u}) SET u.password_hash = $hash

Run at startup every time. MERGE is idempotent: creates the node on first boot, updates the hash on subsequent boots. Password rotation is just setting an env var.

The blog is tiny — 1 user, 18 posts, 4 tags — so for analytics you don't need a database at all. But the graph makes queries like this trivially expressible:

MATCH (u:User)-[:WROTE]->(p:Post)-[:TAGGED]->(t:Tag)
RETURN t.name,
       count(p)       AS post_count,
       sum(p.views)   AS total_views,
       max(p.views)   AS top_post_views
ORDER BY total_views DESC

That's: traverse two hops, group by tag, aggregate three things per group, sort by one of them. In a relational database you'd write two joins and a GROUP BY. Here the shape of the query mirrors the shape of the graph. MATCH (u)-[:WROTE]->(p)-[:TAGGED]->(t) is literally how the data is stored — you're not joining on foreign keys, you're walking edges.

On the blog dataset this returns four rows in a few milliseconds. On the Jira benchmark below it runs as a full scan at 72ms over 100k tickets. Same query, same semantics, different scale.

Numbers: what does it actually do?

I wrote a benchmark crate that generates a Jira-like dataset and measures both ingestion and query throughput. Numbers from a release build on my development machine:

Dataset: 100 000 tickets · 500 000 comments · 10 users · 20 labels → 600 000 nodes, 1.35M edges.

Ingestion (bulk insert via Sled batch):

Ingest time: ~56 seconds
Throughput:  ~10 700 nodes/s  ·  ~24 000 edges/s

Queries (7 runs each, median, all paginated LIMIT 20):

Q1   222ms   all tickets, first page
Q2   252ms   all tickets, SKIP 33333
Q3   192ms   status='open', first page
Q4   344ms   status='open', SKIP 6666
Q5   191ms   by user (edge traversal), first page
Q6   392ms   by user (edge traversal), SKIP 3333
Q7   190ms   by label (edge traversal), first page
Q8   552ms   by label (edge traversal), SKIP 3333
Q9   245ms   single ticket + all comments
Q10   72ms   count by status (full scan)

The actual queries, so you can see what the numbers mean:

-- Q1/Q2: paginated ticket list, newest first
MATCH (t:Ticket)
RETURN t.title, t.status, t.created_at
ORDER BY t.created_at DESC LIMIT 20

-- Q3/Q4: filter by status, same sort
MATCH (t:Ticket) WHERE t.status = 'open'
RETURN t.title, t.status, t.created_at
ORDER BY t.created_at DESC SKIP $skip LIMIT 20

-- Q5/Q6: tickets by a specific user — edge traversal
MATCH (u:User)-[:REPORTED]->(t:Ticket) WHERE u.id = $uid
RETURN t.title, t.status, t.created_at
ORDER BY t.created_at DESC SKIP $skip LIMIT 20

-- Q7/Q8: tickets with a specific label — reversed edge index
MATCH (t:Ticket)-[:HAS_LABEL]->(l:Label) WHERE l.id = $lid
RETURN t.title, t.status, t.created_at
ORDER BY t.created_at DESC SKIP $skip LIMIT 20

-- Q9: ticket detail with all comments
MATCH (t:Ticket) WHERE t.title STARTS WITH 'TICKET-000001:'
WITH t
OPTIONAL MATCH (t)-[:HAS_COMMENT]->(c:Comment)
RETURN t.title, t.status, c.body LIMIT 50

-- Q10: aggregate count by status — full scan, no index
MATCH (t:Ticket) RETURN t.status, count(t) AS n LIMIT 10

These numbers came out of actually writing and running the benchmark — and fixing a real bug in the process.

The first-page queries (190–222ms) use the property-value index. Every property is indexed on write, label-scoped, in value order. ORDER BY created_at DESC LIMIT 20 walks the index in reverse and stops after 20 reads. O(limit) — not O(N).

Before I ran the benchmark, DESC queries were falling back to a full O(N log N) sort because the fast path incorrectly excluded the descending direction. The fix: entry_point.rs now handles NULLS FIRST/LAST correctly for both directions, so the sort order from the index is always right and the full post-sort can be skipped. First-page latency dropped from ~700ms to ~190ms.

SKIP is more expensive than the first page. Q2 (SKIP 33333) is 252ms — the index scan must step over 33k entries before returning 20. Q8 (label edge traversal + SKIP 3333) hits 552ms because each advance requires a property read through the edge index. This is the classic problem keyset pagination solves: WHERE created_at < $cursor would jump to the right position in O(log N). Not implemented yet.

Edge traversal (Q7, 190ms) is competitive with a plain label scan. Walking (Ticket)-[:HAS_LABEL]->(Label) uses the reversed edge index — it's one Sled range scan per edge, not a full vertex scan.

Q10 (72ms) is a full label scan with aggregation. No index optimization for GROUP BY yet — all 100k tickets are visited. Still fast because it's a key-only scan with no property reads.

Ingestion at ~10k nodes/s and ~24k edges/s is the Sled batch write throughput. Every property goes into the label-scoped B-tree index on write — no separate index-build step, reads are fast from the first insert.

Pushing further: GROUP BY, EXISTS, and more bugs

After the Jira benchmark I wanted to see how aggregation held up at scale. That required actually implementing GROUP BY first. The query:

MATCH (e:Employee)
RETURN e.level AS level, count(e) AS cnt, avg(e.salary) AS avg_sal
ORDER BY avg_sal DESC

was silently collapsing everything into a single row. All aggregates, no grouping keys — the executor never split by group. Fix was structural: detect when RETURN mixes aggregate and non-aggregate items, accumulate per-group in a hash map, sort post-group. Also fixed a related bug: RETURN t.name AS tag_name was storing the value under key name instead of tag_name. The alias is the output key.

The aggregation benchmark — 100 000 employees, 100 departments, 1 000 projects, ~330 000 edges:

Dataset: 880k BulkInsertItems  ·  ingest ~12s

Query                                                  median
─────────────────────────────────────────────────────────────
count all employees (label-count fast path)              149ms
headcount per level (5 groups)                           199ms
salary stats per level (sum + avg + count)               312ms
headcount per dept + level (500 groups)                  327ms
avg salary per dept via edge traversal                   812ms
employees per project per dept (three-hop)             1 927ms
depts with avg_sal > 110k  (WITH/HAVING)                 833ms

The plain scans (199–330ms) are O(N) reads over 100k vertices with in-memory grouping. The edge traversal queries scale with the number of hops: one hop costs roughly 600ms on top of the base scan because each edge requires an additional Sled lookup to fetch the target's properties.

The count query deserves a note. MATCH (e:Employee) RETURN count(e) AS total previously went through the full executor — load every vertex, count rows — at 628ms. I added a fast path: when the query is a single labeled node with no WHERE, no GROUP BY, exactly one count() aggregate, it calls vertex_count_for_label() directly. That's a key-only Sled scan (no property reads), taking it to 149ms. A stored counter per label would get it to ~0ms but requires careful atomicity.

The diverse pattern benchmark — same dataset, different query types:

Query                                                  median
─────────────────────────────────────────────────────────────
OPTIONAL MATCH + IS NULL (unmanaged employees)          737ms
EXISTS {} — employees who manage someone                497ms
COUNT {} subquery — managers with 2+ reports            505ms
UNION ALL — senior + principal employees                144ms
UNION (with dedup)                                      519ms
variable-length *1..2 from a junior                      29ms
CALL { WITH d } correlated subquery (edge-indexed)      491ms
shortestPath single pair                                 52ms

UNION ALL at 144ms vs UNION at 519ms is just the deduplication cost — two full scans that share no row identity require O(N) set membership checks.

Two bugs that the benchmarks found.

The first one crashed the process immediately. MatchIterator::try_next and try_next_entry_point were mutually tail-recursive: every non-matching entry point added two stack frames. On a query like MATCH (e:Employee) WHERE EXISTS { (sub:Employee)-[:REPORTS_TO]->(e) } the EXISTS check invokes a new MatchIterator for each outer row. Without a match, the iterator scans all 100k employees before returning — 200k stack frames, guaranteed overflow. Fix: convert both functions to an explicit loop {}.

The second bug was subtler. After fixing the stack overflow, EXISTS was still wrong — it took minutes and consumed gigabytes. The root cause: find_entry_point_offset was picking the first labeled node in the path as the start, regardless of whether a later node had a better key. For the EXISTS pattern (sub:Employee)-[:REPORTS_TO]->(e) with e seeded by UUID (injected by the correlated binding logic), the correct entry point is e — a direct O(1) lookup — not sub, which forces a full Employee scan. Adding a priority check for __id__ UUID equality constraints turns each EXISTS check from O(N) to O(1) + O(in-degree). The entire EXISTS filter over 100k employees went from OOM to 497ms.

This is the pattern I keep running into: the planner has heuristics that work for simple cases, and benchmarks find the case that falls outside them.

What's still missing

Cypherlite is not Postgres. It's not Neo4j. It's just me writing a database when I have had a beer and a hard day to come down. List of the known limitations:

  • Unique constraintsCREATE CONSTRAINT UNIQUE ON :Post(slug). The DDL is in the grammar but enforcement at write time isn't wired up.
  • Transactions across requests — the current API commits per-operation. Long-lived transactions would require connection state, which embedded databases make complicated.
  • Aggregation pushdown into the plan layerCOUNT, COLLECT etc. are evaluated post-plan right now. Correct, but not streaming.
  • Ruthless planning optimizations — the current query planner is doing well in simple cases, it even smartly selects the right index scans. But it's not smart enough to elide redundant matches or other operations (yet).
  • Keyset paginationSKIP N forces a full sort. A proper cursor-based paginator needs first-class support in the executor.

For a blog: irrelevant. For a real application at scale: matters. That's why you should use a real database for real workloads.

The actual lesson

Building a database is a good way to understand why databases are hard. The edge cases are in places you don't expect: WHERE semantics in MATCH+CREATE, property index scoping when multiple vertex types share a property name, ordering guarantees across Sled tree iterators.

And the most useful thing I did was not write another unit test — it was delete the old binary journal, point the server at a Sled file, and watch where things broke.

One closing note: if you're reading this and thinking huh, I'd actually like a proper graph database that doesn't suck — you don't need to build your own like I did. velr.ai is Tomas' take on solving exactly the problems we used to rant about. My version is a hobby project; his is the real thing. Keep an eye on it.

Tagged: