A high-performance Rust library for local Apache Cassandra SSTable access
Status: v0.17.0 (2026-09-08) β read correctness and read-path performance against stock Cassandra 5.0. CQLite reads (and writes) Cassandra 5.0 SSTables with no cluster dependency: CLI, Rust library, Python and Node.js bindings, an Arrow Flight server, and a Trino connector are all shipped and published on every tag. v0.17.0 closed 97 issues: a read-correctness campaign pinned against Cassandra-written bytes (silently truncated rows, a misaligned point-read cell offset, BTI row-index framing, empty collection keys,
inet/timeordering), a measured read-path program (a 3.06Γ Arrow Flight single-SSTable fast path; 1.25Γ stock Cassandra'scount(*)and 1.12Γ itsSELECT *on the same core and SSTable), and one value/error contract across Python, Node and the CLI. Release notes Β· CHANGELOG.md Β· all releases.
Upgrading to v0.17? It carries breaking changes with before/after examples and consumer guidance in the
[v0.17.0]CHANGELOG section:cqlite-core'sValuegainsValue::Empty,Config.storageis the single storage config and decorative knobs are gone, CLI--format jsonrendersdecimal/varintunquoted and drops the injected UDT_typekey, and the bindings carry UDT type identity out of band. Earlier migrations: v0.13 Migration Guide.
CQLite provides SQLite-like local access to Apache Cassandra SSTables, enabling developers to read Cassandra 5.0+ data files without cluster dependencies. Built in Rust for performance and safety.
β Find CQLite useful? Star the repo β it is the clearest signal that this work matters and directly drives how much time goes into it. π Hit a bug or need a feature? Open an issue. For questions and ideas, use Discussions. See Known Issues and the Roadmap before filing.
Full documentation is at https://pmcfadin.github.io/cqlite/:
| Section | URL |
|---|---|
| User Docs β install, quick start, CLI, Python, Node.js | /cqlite/user-docs/ |
| SSTable Format Guide β binary format deep-dive | /cqlite/sstable-format/ |
| For Agents: Using CQLite β LLM/agent integration | /cqlite/agents-using/ |
| For Agents: Developing CQLite β contributor doctrine, gate contract | /cqlite/agents-developing/ |
| Releases β what shipped in each version and how to upgrade | /cqlite/releases/ |
| Roadmap Β· Known Issues Β· Limitations | /cqlite/user-docs/roadmap/ Β· /cqlite/user-docs/known-issues/ Β· /cqlite/user-docs/limitations/ |
| Field validation β measured runs against live Cassandra clusters | /cqlite/field-validation/ |
| API docs (rustdoc) | /cqlite/api/latest/ Β· docs.rs/cqlite-core |
In-repo: docs/releases/ (per-release notes), docs/sstables-definitive-guide/ (the SSTable format single source of truth), docs/reports/ (measurement reports), docs/development/ (PRD, methodology, gate ops).
CQLite aims to provide a Cassandra-compatible storage engine that shares reconciliation and materialization across analytics, bulk ingestion, and compaction, with incremental integration into Cassandra. The CLI, bindings, and query interfaces remain the accessible product surfaces. See the Product Requirements for scope, guarantees, and qualification milestones.
CQLite is designed by Patrick McFadin, Apache Cassandra PMC member with over a decade of Cassandra experience. The project embodies Apache Cassandra community values and will be donated to the Apache Cassandra project upon maturity.
The quickest path on macOS (Apple Silicon or Intel) and Linux (x86_64 or arm64). The formula verifies the release checksum before installing:
brew install pmcfadin/cqlite/cqlite
cqlite --helpcargo install cqlite-cli # installs the `cqlite` binary
cqlite --helpEach GitHub release attaches a
prebuilt cqlite CLI binary for the common platforms, each with a .sha256
checksum sidecar:
| Platform | Asset |
|---|---|
| macOS (Apple Silicon) | cqlite-aarch64-apple-darwin.tar.gz |
| macOS (Intel) | cqlite-x86_64-apple-darwin.tar.gz |
| Linux x86_64 (glibc) | cqlite-x86_64-unknown-linux-gnu.tar.gz |
| Linux x86_64 (static musl) | cqlite-x86_64-unknown-linux-musl.tar.gz |
| Linux arm64 (glibc) | cqlite-aarch64-unknown-linux-gnu.tar.gz |
| Windows x86_64 | cqlite-x86_64-pc-windows-gnu.zip |
# Example: macOS Apple Silicon
TARGET=aarch64-apple-darwin
curl -fsSLO https://github.com/pmcfadin/cqlite/releases/latest/download/cqlite-$TARGET.tar.gz
curl -fsSLO https://github.com/pmcfadin/cqlite/releases/latest/download/cqlite-$TARGET.tar.gz.sha256
shasum -a 256 -c cqlite-$TARGET.tar.gz.sha256 # verify (use sha256sum -c on Linux)
tar xzf cqlite-$TARGET.tar.gz
./cqlite --helpcargo add cqlite-core # use cqlite-core as a dependencySee Using cqlite-core as a dependency and the API docs.
pip install cqlite-py # Python
npm install @cqlite/node # Node.jsQuery a Cassandra node's SSTables over Arrow Flight (gRPC) with the
cqlite-flight server, published as a multi-arch image on every release tag.
Mount the data dir read-only and point --data-dir at it:
docker run --rm -p 8815:8815 \
-v /var/lib/cassandra:/var/lib/cassandra:ro \
ghcr.io/pmcfadin/cqlite-flight:latest \
--data-dir /var/lib/cassandra/data --listen 0.0.0.0:8815See cqlite-flight/README.md for image tags, the
ticket/predicate API, and the trino-connector that builds
on it.
# Clone the repository
git clone https://github.com/pmcfadin/cqlite.git
cd cqlite
# Build the project
cargo build --release
# Run the CLI tool
cargo run --package cqlite-cli -- \
--schema test-data/schemas/basic-types.cql \
--data-dir test-data/datasets/sstables \
--query "SELECT * FROM test_basic.simple_table LIMIT 5" \
--out jsonpip install cqlite-pyimport cqlite
with cqlite.open('path/to/sstables', schema='schema.cql') as db:
for row in db.execute('SELECT * FROM keyspace.table LIMIT 5'):
print(row.to_dict())npm install @cqlite/nodeimport { Database } from '@cqlite/node';
const db = await Database.open('path/to/sstables', { schema: 'schema.cql' });
const result = await db.execute('SELECT * FROM keyspace.table LIMIT 5');
for (const row of result.rows) {
console.log(row.name);
}
await db.close();CQLite v0.9.0 (M5) ships write support across all interfaces: Rust core, Python,
Node.js, and CLI. Written data flushes to portable Cassandra 5.0 SSTables that
Cassandra can read directly via nodetool refresh.
The schema file below is included in the repository at
test-data/schemas/write-test.cql.
import cqlite
# Open in writable mode β write_dir stores the WAL and flushed SSTables
with cqlite.open(
'test-data/datasets/sstables',
schema='test-data/schemas/write-test.cql',
writable=True,
write_dir='/tmp/my-writes',
) as db:
db.execute(
"INSERT INTO test_basic.simple_table (id, name, age) "
"VALUES (11111111-1111-1111-1111-111111111111, 'Alice', 30)"
)
path = db.flush_run()
print(f'Flushed SSTable: {path}')const { Database } = require('@cqlite/node');
const db = await Database.open('test-data/datasets/sstables', {
schema: 'test-data/schemas/write-test.cql',
writable: true,
writeDir: '/tmp/my-writes',
});
await db.execute(
"INSERT INTO test_basic.simple_table (id, name, age) " +
"VALUES (22222222-2222-2222-2222-222222222222, 'Bob', 25)"
);
const path = await db.flushRun();
console.log('Flushed SSTable:', path);
await db.close();# Build with write support
cargo build --package cqlite-cli --features write-support
# Write via CQL INSERT
cargo run --package cqlite-cli --features write-support -- \
--writable --write-dir /tmp/my-writes \
--schema test-data/schemas/write-test.cql \
--execute "INSERT INTO test_basic.simple_table (id, name, age) \
VALUES (33333333-3333-3333-3333-333333333333, 'Carol', 28)"
# Flush memtable to SSTable
cargo run --package cqlite-cli --features write-support -- \
--writable --write-dir /tmp/my-writes \
--schema test-data/schemas/write-test.cql \
--flushSee docs/write-support.md for the full write guide,
including the Cassandra export workflow and known limitations. To embed
cqlite-core in your own Rust project (dependency line, feature flags, and a
compiling write example), see
docs/using-cqlite-core-as-a-dependency.md.
cqlite-core gates optional functionality behind Cargo features. The table below
maps the public API you're likely to reach for to the feature that enables it.
| Want⦠| Enable feature | In defaults? |
|---|---|---|
Read / query path (Database::open, execute, scan, get) |
state_machine |
β yes |
| Compression (LZ4 / Snappy / Deflate / Zstd) | all-compression |
β yes |
Write path (WriteEngine, Mutation, WriteEngine::write/flush) |
write-support |
β yes |
Database::flush / Database::compact (high-level convenience) |
experimental |
β opt-in |
CLI ingestion / REPL helpers (cqlite-cli) |
cli-helpers |
β opt-in |
| Performance metrics collection | metrics |
β opt-in |
Default features are ["all-compression", "state_machine", "write-support"]
(see cqlite-core/Cargo.toml). write-support was folded into the defaults in
#558 β it gates only first-party
code and adds no extra dependencies, so read-only consumers pay nothing for it.
flush/compact on the high-level Database type remain behind experimental;
the equivalent engine-level WriteEngine::flush is part of write-support.
# Default build (read + write + compression)
cargo build
# Read-only consumer: drop the write path (still zero-cost to keep it, but explicit)
cargo build -p cqlite-core --no-default-features --features all-compression,state_machine
# Opt into high-level Database::flush / compact
cargo build -p cqlite-core --features experimental
# Minimal build (no compression, no query engine)
cargo build -p cqlite-core --no-default-features- Cassandra 5+ SSTable format parsing (100% of test tables)
- All CQL types including collections and UDTs
- All compression codecs (LZ4, Snappy, Deflate, Zstd)
- CLI tool with REPL and one-shot query modes
- SELECT with WHERE clause (partition/clustering key equality)
- Output formats: Table, JSON, CSV
- Parquet output format with Snappy compression
- Export command (
cqlite export) - Streaming export for large datasets
- Output formats: CSV, JSON, Parquet, CQL
- Python bindings with full CQL type support
- Node.js bindings with TypeScript definitions
- Streaming API for memory-efficient queries
- pip/npm installable packages (5 platform builds each)
- Type stubs for IDE support (Python mypy, TypeScript)
- Write support: WAL + memtable + flush to Cassandra SSTables
- STCS compaction via
maintenance_step() - Write API in Python, Node.js, and CLI
- Full type coverage: Inet, Varint, Duration, Tuple, Frozen
- E2E readback gate: write β flush β Cassandra
nodetool refreshβ verify
- Embeddable Parquet writer in
cqlite-core(behind aparquetfeature) +export_parquetin Python/Node - Version-gated reads for the Cassandra 5.0
oaformat; graceful handling ofda(BTI) - Real BTI trie node-type dispatch and schema-typed query result columns
- Published documentation site at pmcfadin.github.io/cqlite
- Read correctness campaign β row assembly no longer swallows a decode error into a silently truncated row; a P0 misaligned cell offset on point reads is fixed; BTI
Rows.dbrow-index root base matches Cassandra's framing; empty map keys / set members are kept, nested types are bounds-checked at every level,inet/timeorder as Cassandra orders them;frozen<scalar>is refused instead of guessed, which also fixed the writer'sStatistics.dbheader type forfrozen<UDT> - Measured read path β Arrow Flight
do_getsingle-SSTable merge bypass (3.06Γ rows/s per core); admission default derived from core count; byte-bounded Arrow batches; box ceiling established at 2.73M rows/s on 6 physical cores; opt-injemallocfeature forcqlite-flight - One value/error contract across bindings β shared authoritative error table, 3-way golden parity (same
SELECTthrough Python, Node and CLI must agree), UDT identity out of band - JSON/CSV and Parquet output parity lanes against
sstabledumpgoldens; platform-hygiene epics (decorative config knobs deleted, dead read metrics wired, dead code removed) - See
docs/releases/RELEASE_NOTES_v0.17.0.md
β v0.14 β v0.16.1 (Jul 2026) β Flight field-readiness, Trino latency/throughput, CommitLog reader
- v0.14 Arrow Flight + Trino read path validated against a live at-scale Cassandra deployment;
Index.dbparsed once per open (~100Γ faster cold index build) - v0.15 ~15Γ warm through-Trino throughput (p50 2.9 s β 227 ms), admission control, saturation gauges, lazy Summary-guided index (O(summary) open), snapshot reuse
- v0.16 typed collection columns through Trino (
array/row/map), weight-balanced split fan-out,LIMIT-cancellation hang fixed; v0.16.1 Cassandra 5.0 CommitLog segment reader (library API +read-commitlogCLI) - See
docs/releases/for each release's notes
- Read-path constant-factor speedups β query-engine hot-path cleanups (schema
Arc, single projection, cached sort keys, plan cache), read-path idiom bundle, and point-read I/O via a positional-read (ReadAt) trait - Node.js bindings throughput β batch-fetch streaming rows, move (not clone) row values in
executeNative, cachedSet/Mapconstructors - Byte-bounded result budget β
Error::ResultTooLarge+QueryConfig.max_result_bytes(default 64 MiB) - Per-surface SSTable freshness contract + explicit
Database.refresh() - No-heuristics correctness β removed blob-decode byte-pattern guessing; unknown-table reads fail honestly instead of fabricating a default schema
- See CHANGELOG.md for the full per-release detail
- Byte-for-byte compaction parity vs Apache Cassandra β
cqlite compact+ a differential harness in CI, full reconciliation rule set (complex deletions, tombstone tie-breaks,gc_gracepurging, range tombstones, per-cell/dropped-column purging, non-frozen UDT multi-cell) - Arrow Flight server + Trino connector β query SSTables as a federated source with predicate, token-range, and aggregation pushdown
- Canonical BTI (
da) write + end-to-end read β emit Cassandra-format trie-indexed SSTables - CDC-style delta-scan /
delta-exportβ project SSTable generations to Parquet envelopes with full tombstone fidelity -
WRITETIME()/TTL()inSELECTand query-engine completeness (PER PARTITION LIMIT, static columns, clustering order/bounds, partition-targeted lookups) - crates.io OIDC trusted publishing + Homebrew tap
- See CHANGELOG.md for the full per-release detail
See the Roadmap section below for in-flight epics and milestones.
The PRD defines the storage-engine qualification program. M1βM5 remain historical delivery milestones; the new milestones measure supported workflows and operator outcomes.
| Milestone | Outcome |
|---|---|
| SE1 β Engine contract | Explicit scope, authority, capabilities, and workload acceptance criteria |
| SE2 β Shared offline execution | Safe compaction and reusable analytics materialization |
| SE3 β Analytics offload value | Measured benefit within Cassandra foreground-service and freshness budgets |
| SE4 β Bulk writing and policy tools | Qualified native ingestion plus compaction planning/simulation |
| SE5 β Qualified freshness | Proven memtable-tail behavior for opted-in analytical reads |
| SE6 β Bounded Cassandra integration | One reversible native/offload pilot |
| SE7 β Native lifecycle qualification | Supported integrated behavior backed by operational evidence |
Qualification is pending; existing implementation and research contribute evidence but do not automatically complete these milestones. External-engine 1.0 requires SE1βSE4 and the PRD's release gates. WASM and native lifecycle integration are separate tracks. See the user roadmap.
The roadmap follows real-world use. Want something prioritized? Open or π an issue β and β star the repo.
CQLite is honest about its sharp edges. The current release (v0.17.0) has a few
known gaps β none of which block the core read/export workflows. Full, dated list:
pmcfadin.github.io/cqlite β Known Issues.
| Issue | Impact | Tracking |
|---|---|---|
| A scan over an SSTable whose metadata is refused returns empty rows instead of an error | Pre-existing in every release; a malformed file reads as "no data" rather than failing loudly | #4159 (fix in flight, 0.18) |
Unbounded SELECT count(*) through the Trino cqlite catalog can stall |
Use a bounded predicate or an aggregate that reads a column; found on a 3-node field run | #4170 |
Trino scans are served by the sidecar-uri host's Flight pod only |
Single-node throughput on multi-node clusters; other replicas idle | #4175 |
timeuuid maps to varchar in the Trino cqlite catalog (stock connector: uuid) |
"Same SQL" does not hold for timeuuid columns |
#4173 |
Cassandra trunk SSTables (BIG pa, BTI ea) are out of scope |
BTI ea is refused; BIG pa needs a version ceiling |
#4142 |
Pre-5.0 formats (md/mc/la/ma) unsupported |
By design β Cassandra 5.0 only | Limitations |
For what CQLite does not do by design (older formats, network access, query features), see Limitations.
Found something not listed? Open an issue β a good report (Cassandra version, schema, command, output) is the most valuable contribution you can make.
Design Philosophy:
- No cluster dependency - Read and write SSTables directly, with no running Cassandra node
- CQL parser - Native CQL support using an Antlr4 grammar
- Cassandra 5+ focus - Modern 'oa' format with BTI support
- Memory efficient - <128MB usage target for large files
- Self-contained engine - Pure-Rust parsing and writing, including STCS compaction
CQLite is developed in the open as an Apache-licensed project. We welcome contributions from the Cassandra community!
CQLite uses a spec-driven, agent-orchestrated, gate-enforced workflow built on Claude Code. In short:
- Specs are the source of truth. Design-driven work lives in a durable OpenSpec spec under
openspec/specs/; oracle-driven bug fixes are a GitHub issue plus a fixture-pinned parity test. GitHub issues and the project board are the execution ledger, not the contract. - A delivery pipeline of skills (
flow-groomβflow-activateβflow-implementβflow-addressβflow-finalize, plusdrive-issuefor one issue end to end) runs on a fleet of machines, one issue per worktree/branch/PR, with a server-arbitrated claim ref so two machines cannot take the same issue. - Every change passes a deterministic gate (
scripts/agent-gate.sh:cargo fmt,clippy -D warnings, core/integration/write/CLI tests, structural audits, smoke) whoseAGENT-GATE SUMMARYblock is the only run that counts; GitHub branch protection then requires arequiredcheck that aggregates the CI tiers before any merge. - The author is never the reviewer. Work is reviewed in a fresh context by roborev (a second model family) +
rust-reviewer, and audited against the spec (spec-auditor) and for meaningful coverage (coverage-reviewer). - Humans decide product, agents decide implementation. Ambiguous scope and tradeoffs are escalated on a NEEDS YOU list, never guessed.
Definition of done: gate passes Β· spec-auditor confirms acceptance criteria Β· coverage-reviewer confirms tests are meaningful Β· roborev is clean.
π Full workflow, lifecycle, and how to run it yourself: docs/development/METHODOLOGY.md
# Prerequisites
# - Rust 1.95+
# Clone and build
git clone https://github.com/pmcfadin/cqlite.git
cd cqlite
cargo build
# Fetch test data (JSONL reference files are in git, SSTable binaries fetched separately)
bash test-data/scripts/fetch-datasets.sh
# Run tests
env CQLITE_DATASETS_ROOT=$PWD/test-data/datasets cargo test --package cqlite-core- Check Issues: Look for
good-first-issuelabels - Discuss: Join our community discussions
- Code: Follow Rust best practices and include tests
- Test: Ensure compatibility with real Cassandra data
- Document: Update docs for user-facing changes
- All SSTable components parsed (Data.db, Index.db, Summary.db, Statistics.db, TOC)
- 33/33 test tables passing (100% validation)
- All 21 CQL primitive types + collections + UDTs + frozen types
- All compression algorithms working
- Tiered test coverage targets (see historical PRD Section 5.1)
- CLI with one-shot and REPL modes
- SELECT queries with WHERE clause support
- Multiple output formats (Table, JSON, CSV)
- Parquet output format with Snappy compression
- Export command with CSV, JSON, Parquet, CQL formats
- Streaming export for memory-efficient large dataset handling
- Progress bar and statistics for exports
- Python bindings via PyO3 with sync-first API
- Node.js bindings via napi-rs with Promise-based API
- Full CQL type system (20+ types including collections, UDTs)
- Thread-safe database handles
- 500+ tests with 98%+ pass rate across both bindings
- Write support: WAL-backed memtable + flush to portable Cassandra 5.0 SSTables
- STCS compaction (
maintenance_step()) - Write API exposed in Python (
flush_run,maintenance_step,write_stats), Node.js (flushRun,maintenanceStep,writeStats), and CLI (--writable,--write-dir,--flush,maintenance,write-stats,export-sstable) - Type roundtrips verified for all major types including Inet, Varint, Duration, Tuple, Frozen
- E2E validation against live Cassandra 5.0 (write β flush β
nodetool refreshβcqlsh)
Eight releases since M5 β compaction parity, Arrow Flight + Trino, BTI write/read, delta export, the performance
release, Flight field-readiness, Trino latency/throughput, the CommitLog reader, and the 0.17 read-correctness +
read-path program. Per-release notes: docs/releases/ Β· website releases.
See docs/development/PRD.md for the qualification milestones (SE1βSE7) that follow M1βM5.
- Cassandra 5.0: BIG format versions
na/nb/oaand BTI (trie-indexed) formatda; pre-5.0 (maβme) is out of scope by design, and trunk'spa/eaare not supported (#4142) - SSTable components: Data.db, Index.db, Summary.db, Statistics.db, CompressionInfo.db, Filter.db, TOC.txt, Digest.crc32; BTI Partitions.db / Rows.db
- CommitLog: Cassandra 5.0 segment files (version 7), read-only (
read-commitlog) - Compression: LZ4, Snappy, Deflate, Zstd (reads); the write surface emits uncompressed SSTables only
- Format authority: the pinned
cassandra-5.0.8source andsstabledumpoutput β never CQLite's own prior behaviour (SSTable definitive guide)
- Bare scan ceiling: 2.73M rows/s on 6 physical cores at 93.5% marginal efficiency (#3299)
- Same core, same SSTable vs stock Cassandra: 1.25Γ its
count(*)on the read path, 1.12Γ itsSELECT *when shipping every row over Arrow Flight; ~5Γ less memory per row (docs/reports/ws0-3100-report.md) - 3-node Cassandra 5.0.9 cluster, through Trino: a full-table analytic costs the OLTP client 2.21Γ baseline p99 via CQLite vs 5.61Γ via Cassandra's own CQL path; same SQL 1.66Γ faster on a 22M-row key-value table; 1.72M rows/s sustained from one Flight pod (
docs/2024-09-meetup/REPORT.md, single-pod β see #4175) - Memory: <128 MB target for the core library on large SSTables; Flight server RSS under load is an open measurement
- Method and CI gate policy:
docs/performance.md
- Python: Production-ready sync API (see Python README)
- Node.js: Production-ready Promise API (see Node.js README)
- WASM: Deferred optional surface; not a prerequisite for external-engine 1.0
- Documentation site: https://pmcfadin.github.io/cqlite/ β user docs, SSTable format guide, agent integration docs
- API docs (rustdoc): latest tag Β· published per release tag at
https://pmcfadin.github.io/cqlite/api/<tag>/ - Releases:
docs/releases/(per-release notes) Β· website releases page Β· GitHub releases (binaries, wheels, checksums) - Changelog: CHANGELOG.md β what each tagged release contains, with every breaking change
- Performance: Methodology, local repro, and CI gate policy
- CQL Grammar: Patrick's Antlr4 CQL Grammar
- Issues: GitHub Issues
- Discussions: GitHub Discussions
- β Star the project: github.com/pmcfadin/cqlite β the single best way to support it and shape where the time goes
- π Bugs & feature requests: GitHub Issues
- π¬ Questions & ideas: GitHub Discussions
- π Contributing: see CONTRIBUTING.md, the Roadmap, and our Code of Conduct β look for
good-first-issuelabels
CQLite is an independent open-source project, not an Apache Software Foundation project. It is built in the spirit of the Apache Cassandra community, with the goal of contributing it upstream as it matures.
Licensed under the Apache License, Version 2.0. See LICENSE for details.
Special thanks to the Apache Cassandra community and the many contributors who make projects like this possible. CQLite builds on decades of database engineering innovation from the Cassandra project.
Planning note: M1βM5 record the toolkit and execution foundation. The current PRD replaces the remaining M6/M7 sequence with SE1βSE7 qualification milestones. Release-specific support and parity claims must follow the parity release checklist.
