Benchmarking SQLite FTS5 and Turso Native FTS

I recently wrote a blog post comparing SQLite and the filesystem for document search workloads.

One of the most interesting findings was that SQLite with FTS5 outperformed filesystem search using rg.

• blog post

After publishing that article, I came across Turso's blog post:

• Beyond FTS5: Building Transactional Full-Text Search in TursoDB

Turso recently introduced a new full-text search implementation that is built on top of Tantivy, a Rust search engine inspired by Apache Lucene.

Naturally, I became curious:

How does SQLite FTS5 compare with Turso Native FTS?

This article contains the results of a small benchmark I ran to explore that question.

Disclaimer

This is not a production-grade benchmark.

The goal of this experiment was simply to compare the behavior of SQLite FTS5 and Turso Native FTS under a small local workload.

Important caveats:

• This benchmark was performed on a single local machine.
• The dataset is synthetic.
• The workload is intentionally simple (count(*) WHERE body MATCH ?).
• Turso Native FTS is still experimental.
• Turso's FTS implementation is designed to provide capabilities beyond a simple SQLite FTS5 setup, so this benchmark does not evaluate the full feature set or intended use cases.
• This benchmark focuses purely on indexing and term-search performance.

The results should therefore be viewed as observations from a simple experiment rather than definitive conclusions about production workloads.

How SQLite FTS5 Works

According to the SQLite documentation:

FTS5 is an SQLite virtual table module that provides full-text search functionality to database applications.

• SQLite FTS5 Extension

At its core, FTS5 is an inverted index.

Without an inverted index, a query such as:

SELECT *
FROM documents
WHERE body LIKE '%sqlite%';

requires scanning every document and checking whether the term appears.

An inverted index stores the relationship in the opposite direction.

Instead of:

doc1 -> sqlite, fast
doc2 -> markdown, search
doc3 -> sqlite, search

it stores:

sqlite   -> doc1, doc3
search   -> doc2, doc3
markdown -> doc2

When the query is sqlite, SQLite can jump directly to the matching documents instead of scanning the entire dataset.

This is why FTS5 is often dramatically faster than LIKE queries or filesystem scans.

How Turso Implements FTS

SQLite already has FTS5, but Turso's native FTS takes a different route.

Instead of using SQLite's built-in FTS implementation, Turso integrates Tantivy, a Rust search engine library inspired by Apache Lucene. Tantivy gives Turso a search-engine-style foundation: BM25 scoring, phrase queries, boolean queries, field-aware search, and configurable tokenizers.

Normally, Tantivy stores its index as a directory containing many files: metadata, term dictionaries, posting lists, positions, stored fields, and other segment data.

Turso cannot simply let Tantivy write those files to an external directory if the index is expected to behave like part of the database. The search index has to follow database transactions and live with the rest of the database state.

The interesting part of Turso's implementation is that it replaces Tantivy's filesystem directory abstraction with a B-Tree-backed directory.

Turso implements Tantivy's Directory abstraction through a component called HybridBTreeDirectory. From Tantivy's point of view, it is still reading and writing files. Under the hood, those files are stored inside Turso's internal B-Trees.

Conceptually, the internal storage looks like this:

CREATE TABLE __turso_fts_dir_<index_name> (
    path TEXT NOT NULL,
    chunk_no INTEGER NOT NULL,
    bytes BLOB NOT NULL
);

There is also a backing_btree index over (path, chunk_no, bytes), so the implementation can seek directly to a file chunk.

The idea is simple:

• path identifies the Tantivy file.
• chunk_no identifies the chunk within that file.
• bytes stores the chunk contents.

Large Tantivy files are split into chunks and stored as BLOBs. Small or frequently accessed files can be cached in memory.

This lets FTS index updates participate in Turso's transaction machinery instead of being managed as unrelated files next to the database.

Turso also keeps two cache layers inside HybridBTreeDirectory:

• Hot Cache: a whole-file cache keyed by path, used for metadata, term dictionaries, fast fields, and field norms.
• Chunk Cache: a per-chunk cache keyed by (path, chunk_no), used for large segment files such as posting lists and document stores.

Both caches are bounded and LRU-like. The difference is the unit of caching: whole files for hot data, individual chunks for larger segment data.

For queries, Turso's planner recognizes FTS patterns such as fts_match(...) and fts_score(...) and routes them through the custom FTS index method. Tantivy executes the search, returns matching document addresses and scores, and Turso maps those results back to rowids in the original table.

See more:

• Turso document about FTS
• Turso blog
• Tantivy

Benchmark Setup

Dataset:

• 3,000 documents
• 1 KB per document
• Synthetic markdown-like content

Compared approaches:

1. Filesystem + ripgrep
2. SQLite LIKE
3. SQLite FTS5
4. Turso Native FTS

Query shape:

SELECT count(*)
FROM documents
WHERE body MATCH ?

The benchmark measures:

• Index build time
• Database size
• Query latency

Each query was executed:

• 1 warmup run (warmup OS page cache)
• 10 measured runs

Reported metrics:

• min
• p50
• p95

See more:

• benchmark project repository

Results

For the full results, you can check this report.

Query Latency

SQLite FTS5 showed the lowest latency in this benchmark.

In these measured runs, the observed p50 latency for Turso Native FTS was roughly 50–70x higher than SQLite FTS5, while still lower than filesystem scanning.

Index Build Time

SQLite FTS5 completed indexing in a fraction of a second, while Turso Native FTS required more than 90 seconds for the same dataset.

Storage Size

The Turso Native FTS database was about 2.8x larger than the SQLite FTS5 database in this run. This measurement is the allocated size of each store/database, not the isolated size of only the search index.

I also observed that Turso's indexing time increased more sharply than I expected as document counts increased.

However, I have not investigated whether this behavior is caused by:

• the benchmark itself,
• configuration choices,
• OPTIMIZE INDEX,
• implementation trade-offs,
• or behavior specific to the current experimental implementation.

Therefore I do not want to draw strong conclusions from the indexing numbers alone.

Conclusion

In this particular benchmark, SQLite FTS5 showed lower query latency, faster index build time, and smaller measured database/store size.

I would interpret that result narrowly.

This benchmark only measures a small local workload: body-only term search using count(*) WHERE body MATCH ? over synthetic documents. It does not evaluate the broader search-engine features that motivated Turso's Tantivy-backed design, such as richer query syntax, scoring behavior, tokenizer flexibility, or transactional integration of the search index with the database.

SQLite FTS5 and Turso Native FTS are also built under different design constraints. SQLite FTS5 is a mature built-in extension for embedded full-text search, while Turso Native FTS integrates Tantivy into Turso's storage and transaction model.

So I do not read these numbers as a general judgment on Turso FTS. They are simply the results I observed for one small benchmark. For this workload, SQLite FTS5 was the faster and smaller option; for other workloads, especially those that use more search-engine-oriented features, a separate benchmark would be needed.