feat(nbd): add Turso/libsql cache for list performance [833807]
Migrate async runtime from async-std to tokio (required by libsql), then add a mtime-based libsql cache at .nbd/cache.db that accelerates nbd list, nbd ready, and nbd next by avoiding redundant file reads. - Cargo.toml: replace async-std with tokio + libsql = "0.6" - src/store.rs: async_std → tokio fs API; add open_cache() and list_tickets_cached() with fallback to list_tickets on error - src/main.rs: tokio::main, tokio::fs::remove_file; wire cmd_list, cmd_ready, cmd_next to list_tickets_cached - src/tests.rs: async_std::test → tokio::test, async_std::fs → tokio::fs Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>quotesdb
Elijah Voigt
3 months ago
parent
61ac72aee4
commit
5ddf9a56e1
@ -0,0 +1 @@
|
||||
cache.db
|
||||
@ -1,63 +1,67 @@
|
||||
+++
|
||||
title = "SQLite cache for list performance"
|
||||
title = "Turso cache for list performance"
|
||||
priority = 3
|
||||
status = "todo"
|
||||
status = "done"
|
||||
ticket_type = "feature"
|
||||
dependencies = []
|
||||
+++
|
||||
Add an optional SQLite cache in `.nbd/cache.db` to accelerate `nbd list` and `nbd ready` for large ticket stores.
|
||||
Add a Turso (libsql) cache in `.nbd/cache.db` to accelerate `nbd list` and `nbd ready` for large ticket stores.
|
||||
|
||||
## Motivation
|
||||
|
||||
`list_tickets` currently does O(n) file reads on every call. For stores with hundreds of tickets this is measurably slow. A SQLite cache avoids re-reading unchanged files by comparing file modification times (mtimes).
|
||||
`list_tickets` currently does O(n) file reads on every call. For stores with hundreds of tickets this is measurably slow. A Turso/libsql cache avoids re-reading unchanged files by comparing file modification times (mtimes).
|
||||
|
||||
## Approach
|
||||
|
||||
### Runtime migration: async-std → tokio
|
||||
|
||||
libsql requires tokio. Migrate the entire crate:
|
||||
|
||||
- `Cargo.toml`: remove `async-std`, add `tokio = { version = "1", features = ["full"] }` and `libsql = "0.6"`
|
||||
- `src/main.rs`: `#[async_std::main]` → `#[tokio::main]`, `async_std::fs::remove_file` → `tokio::fs::remove_file`
|
||||
- `src/store.rs`: `use async_std::fs` → `use tokio::fs`, `use async_std::prelude::*` removed (tokio ReadDir uses `.next_entry().await` not stream iteration)
|
||||
- `src/tests.rs`: `#[async_std::test]` → `#[tokio::test]`, `async_std::fs` → `tokio::fs`
|
||||
|
||||
### Crate dependency
|
||||
Add `turso` to `Cargo.toml` (file-based SQLite, no `sync` feature needed):
|
||||
|
||||
```toml
|
||||
turso = "0.4.3"
|
||||
tokio = { version = "1", features = ["full"] }
|
||||
libsql = "0.6"
|
||||
```
|
||||
|
||||
> **Note:** Turso requires tokio. The existing `async-std` runtime in `nbd` must be replaced with `tokio` (or a compatibility shim used). Recommendation: switch `#[async_std::main]` to `#[tokio::main]` and update `Cargo.toml` accordingly.
|
||||
|
||||
### store.rs additions
|
||||
New async function `open_cache(root: &Path) -> Result<turso::Connection>`:
|
||||
- Opens (or creates) `.nbd/cache.db` via `turso::Builder::new_local(path).build().await?`.
|
||||
- Runs a migration: `CREATE TABLE IF NOT EXISTS tickets (id TEXT PRIMARY KEY, json TEXT NOT NULL, mtime INTEGER NOT NULL)`.
|
||||
|
||||
New async function `open_cache(root: &Path) -> Result<libsql::Connection>`:
|
||||
- Opens (or creates) `.nbd/cache.db` via `libsql::Builder::new_local(path).build().await?.connect()`.
|
||||
- Runs migration: `CREATE TABLE IF NOT EXISTS tickets (id TEXT PRIMARY KEY, json TEXT NOT NULL, mtime INTEGER NOT NULL)`.
|
||||
|
||||
New function `list_tickets_cached(root: &Path) -> Result<Vec<Ticket>>`:
|
||||
1. Open cache via `open_cache`.
|
||||
2. Read directory listing to get file names and mtimes.
|
||||
3. For each file: query the DB (`connection.query(...).await?`) for a row with matching mtime; if found, use cached JSON; otherwise read file, parse, and upsert with `connection.execute(...).await?`.
|
||||
1. Scan directory for files + mtimes (using tokio::fs::read_dir + metadata).
|
||||
2. Open cache DB.
|
||||
3. For each file: query cache for matching (id, mtime); if hit, deserialise from cached JSON; if miss, read file, parse, upsert to DB.
|
||||
4. Delete DB rows for IDs no longer on disk.
|
||||
5. Return deserialized tickets sorted by priority desc.
|
||||
|
||||
Keep existing `list_tickets` as the non-cached fallback. `cmd_list` and `cmd_ready` use `list_tickets_cached`, falling back to `list_tickets` on error.
|
||||
|
||||
### Turso API reference
|
||||
- Open local file DB: `turso::Builder::new_local("path/to/file.db").build().await?`
|
||||
- Execute (INSERT/UPDATE/DELETE): `conn.execute("SQL", params).await?`
|
||||
- Query rows: `let mut rows = conn.query("SQL", params).await?`
|
||||
- Iterate rows: `while let Some(row) = rows.next().await? { row.get_value(0)? }`
|
||||
|
||||
### Migration strategy
|
||||
- The cache is always optional. If `cache.db` can't be opened or any cache operation fails, fall back to `list_tickets` (log a warning to stderr).
|
||||
- The cache is never the source of truth — the JSON files are. The cache is always reconstructable by deleting `.nbd/cache.db`.
|
||||
|
||||
## Decision point
|
||||
Decide whether to enable the cache unconditionally or gate it behind a flag (`--cache` / `NBD_CACHE=1`). Recommendation: enable by default once the feature is stable.
|
||||
|
||||
## Tests
|
||||
- Unit test: cache hit returns same data as direct file read.
|
||||
- Unit test: cache miss (mtime changed) re-reads the file.
|
||||
- Unit test: deleted ticket is evicted from cache.
|
||||
- Performance test (optional): benchmark 1000-ticket list with and without cache.
|
||||
|
||||
## Files touched
|
||||
- `Cargo.toml` — add `turso`, replace `async-std` with `tokio`
|
||||
- `src/main.rs` — switch to `#[tokio::main]`
|
||||
- `src/store.rs` — `open_cache`, `list_tickets_cached`
|
||||
- `src/tests.rs` — cache unit tests
|
||||
- `docs/ARCHITECTURE.md` — document the cache layer
|
||||
5. Return sorted by priority desc.
|
||||
|
||||
Fall back to `list_tickets` on any cache error.
|
||||
|
||||
### main.rs
|
||||
|
||||
`cmd_list`, `cmd_ready`, `cmd_next` use `list_tickets_cached` instead of `list_tickets`.
|
||||
|
||||
## Files to change
|
||||
|
||||
| File | Change |
|
||||
|---|---|
|
||||
| `Cargo.toml` | Replace async-std with tokio, add libsql |
|
||||
| `src/store.rs` | async_std → tokio fs API, add cache functions |
|
||||
| `src/main.rs` | async_std → tokio main/fs |
|
||||
| `src/tests.rs` | async_std → tokio test/fs |
|
||||
| `tests/integration.rs` | Add cache smoke test |
|
||||
|
||||
## Validation
|
||||
|
||||
```sh
|
||||
cargo fmt && cargo check && cargo clippy && cargo test
|
||||
cargo run -- list # should create .nbd/cache.db on first run
|
||||
cargo run -- list # second run uses cache
|
||||
```
|
||||
@ -0,0 +1,8 @@
|
||||
{
|
||||
"title": "Test archive",
|
||||
"body": "",
|
||||
"priority": 5,
|
||||
"status": "archived",
|
||||
"dependencies": [],
|
||||
"ticket_type": "task"
|
||||
}
|
||||
@ -0,0 +1,8 @@
|
||||
{
|
||||
"title": "Test ticket",
|
||||
"body": "Some body text",
|
||||
"priority": 7,
|
||||
"status": "done",
|
||||
"dependencies": [],
|
||||
"ticket_type": "bug"
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
Loading…
Reference in New Issue