latlng

Durable hook delivery with explicit retry state.

Hooks turn geofence events into outbound webhook jobs. The native server stores hook definitions, enqueue intents, delivery results, retries, and dead-letter state so operators can recover and inspect delivery behavior.

Delivery lifecycle

Register

Create hooks through the /hooks HTTP routes. A hook stores a name, endpoint, and geofence definition scoped to a collection.

Enqueue

When a mutation produces a matching geofence event, the command and WebhookEnqueue records are persisted in one atomic storage batch before the mutation becomes visible.

Dispatch

The native outbox leases due jobs up to webhook_concurrency_limit and sends each event as an HTTP POST to http:// or https:// endpoints.

Finalize

Successful 2xx responses append WebhookAck. Failed attempts append WebhookRetryScheduled or WebhookDeadLetter after the retry budget is exhausted.

Storage and guarantees

  • Delivery is at-least-once. Consumers should deduplicate with the stable event and job identifiers.
  • Webhook payloads include the geofence event as JSON. HTTP requests include X-LatLng-Event-Id and X-LatLng-Job-Id when those IDs are present.
  • Durable storage keeps enqueue, acknowledgement, retry, and dead-letter state in the primary log. The SQLite queue is rebuilt from that log on startup.
  • AOF-backed servers recover unresolved jobs across restarts. Memory storage keeps the runtime queue file, but durable webhook recovery is not guaranteed because the primary log is ephemeral.
  • A truncated final AOF batch cannot replay only part of a command and its webhook enqueue records.
  • AOF compaction preserves active hooks, channels, and unresolved webhook jobs.
  • FLUSHDB clears collections, geofence definitions, geofence state, and the materialized webhook queue in one coordinated reset.
  • Expired objects are deleted through the normal delete path, so hooks whose commands include Del receive a Del event when an object's TTL runs out.
  • A geofence that fails to evaluate is skipped instead of failing the triggering write, and is counted in latlng_geofence_eval_errors_total.
  • Hook and channel definitions are validated at registration. Areas that reference a stored object are rejected with 400; define the area inline as GeoJSON.

Delivery settings

webhook_queue_path = "./data/webhook-queue.sqlite"
webhook_timeout_ms = 5000
webhook_concurrency_limit = 128
webhook_retry_count = 8
webhook_retry_initial_backoff_ms = 200
webhook_retry_max_backoff_ms = 30000
webhook_lease_ms = 30000
Name Description
webhook_queue_path SQLite queue path. Defaults next to the AOF as *.webhooks.sqlite. In memory mode the queue is kept in memory unless a path is set.
webhook_timeout_ms Per-attempt HTTP delivery timeout.
webhook_concurrency_limit Maximum number of webhook delivery attempts in flight.
webhook_retry_count Number of retries after the first attempt. A value of 8 means up to 9 total attempts.
webhook_retry_initial_backoff_ms Initial retry delay used by the exponential backoff calculation.
webhook_retry_max_backoff_ms Upper bound for retry backoff.
webhook_lease_ms Lease duration for in-flight jobs. Expired leases are returned to pending state for another attempt.

Operations and observability

Signal Description
GET /admin/webhook-queue Admin-only queue summary with pending, leased, dead_letter, and oldest_pending_age_ms fields.
latlng_hook_attempts_total Total webhook delivery attempts.
latlng_hook_success_total Successful webhook deliveries.
latlng_hook_failure_total Failed delivery attempts.
latlng_hook_retry_total Attempts that scheduled a retry.
latlng_hook_dead_letter_total Deliveries moved to dead-letter state.
latlng_geofence_eval_errors_total Geofence evaluations that failed and were skipped without failing the triggering write.
latlng_expired_objects_total Objects deleted by the background expiry sweep (each can fire Del events).
latlng_webhook_jobs_pending Current pending jobs.
latlng_webhook_jobs_leased Current leased jobs.
latlng_webhook_jobs_dead_letter Current dead-letter jobs.

Frequently asked questions

Are webhook deliveries exactly-once or at-least-once?

Webhook delivery in latlng is at-least-once. Consumers should deduplicate using the stable event and job identifiers carried as X-LatLng-Event-Id and X-LatLng-Job-Id HTTP headers and in the JSON payload.

What happens to webhook jobs across a server restart?

AOF-backed servers recover unresolved webhook jobs across restarts. The materialized SQLite queue is rebuilt from the durable primary log on startup. With memory storage the runtime queue file is kept, but durable webhook recovery is not guaranteed because the primary log is ephemeral.

How are retries scheduled?

Retries use exponential backoff bounded by webhook_retry_initial_backoff_ms and webhook_retry_max_backoff_ms. webhook_retry_count caps the number of retries after the first attempt, so a value of 8 means up to 9 total attempts before a delivery is dead-lettered.

Can I inspect the webhook queue at runtime?

Yes. The admin endpoint GET /admin/webhook-queue returns a summary with pending, leased, dead_letter, and oldest_pending_age_ms fields. Detailed per-state counters are also exported as Prometheus metrics (latlng_hook_attempts_total, latlng_webhook_jobs_pending, etc.).

← Transports and auth Next: Persistence →