latlng

Durable logs and recovery behavior.

Use AOF for the runnable server when durability matters. Keep the default safe fsync behavior unless measured workload data points to tuning the writer queue or group commit settings.

Storage modes

Memory

Ephemeral, process-local storage for development and disposable workloads.

AOF

Append-only file storage with replay, compaction, integrity verification, backup, and restore tooling.

SQLite

Embedded/native storage backend. The runnable server currently exposes memory and AOF storage modes.

AOF operations

latlng-cli aof-verify ./data/appendonly.aof
latlng-cli aof-backup ./data/appendonly.aof ./backup/appendonly.backup.json
latlng-cli aof-restore ./backup/appendonly.backup.json ./restore/appendonly.aof

aof-backup is an offline tool. Use it against a stopped server or a copied AOF that is no longer being written. For live systems, prefer graceful shutdown or crash-consistent filesystem snapshots followed by verification.

Recovery guarantees

  • Replay keeps the valid prefix when the final frame is truncated.
  • Complete corrupt payloads fail startup with a codec error.
  • Truncated final batches are ignored as one unit, preserving command-plus-webhook atomicity.
  • Startup replay is incremental, so the core applies entries as they are read.
  • Expirations are persisted as absolute deadlines so restart and AOFSHRINK do not extend TTLs.
  • Replay loads existing objects and geofence definitions as they are. Only hooks or channels whose area is a Reference, which could never be evaluated, are skipped with a warning.

Active expiry

  • Objects with a TTL are tracked in a per-collection index ordered by deadline.
  • On the leader, a background sweep runs every expiry_sweep_interval_ms (default 100, 0 disables it) and deletes due objects in batches of up to 10,000.
  • Each expiry goes through the normal delete path: it is appended to the log as a Del, replicated to followers, and fires Del geofence events to hooks and channels whose commands include Del.
  • Followers never expire objects themselves. They apply the leader's replicated deletes and hide expired objects from reads in the meantime.
  • The sweep is skipped in read-only mode. POST /admin/gc still removes expired objects from memory without logging them.
  • latlng_expired_objects_total counts objects deleted by the sweep.

Sweep interval

# config file
expiry_sweep_interval_ms = 100

# environment variable or CLI flag
LATLNG_EXPIRY_SWEEP_INTERVAL_MS=100
latlng-server --expiry-sweep-interval-ms 100

Frequently asked questions

Do expired objects trigger geofence events?

Yes. Since v0.2.0 the leader deletes expired objects with a background sweep, and each expiry goes through the normal delete path. Hooks and channels whose commands include Del receive a Del event, so a tracker that stops reporting can raise an alert.

Which storage modes does the native server support?

The runnable latlng-server currently exposes two storage modes: memory (ephemeral, process-local) and AOF (append-only file with full durability, replay, compaction, verification, backup and restore). The Rust storage layer also includes SQLite for embedded use.

Is it safe to take a live AOF backup?

aof-backup is an offline tool. Use it against a stopped server or a copied AOF that is no longer being written. For live systems, prefer graceful shutdown or crash-consistent filesystem snapshots followed by aof-verify.

What happens to a truncated AOF on restart?

Replay keeps the valid prefix of the AOF and discards a truncated final frame. A truncated final batch is ignored as one unit so command-plus-webhook atomicity is preserved. Complete-but-corrupt payloads fail startup with a codec error instead of silently dropping data.

← Hooks Back to overview →