# Architecture

## Ideas

- SigChains Ed25519-signed, hash-chained append-only logs per node. Trust via local signature verification.
- Log-Based State: KV store derived by replaying entries. Watermarks enable safe log pruning + snapshots.
- Offline-First: Iroh for networking. Vector clocks identify missing entries on reconnect—converges mathematically.
- Full Replication: All nodes keep all logs until watermark consensus, then prune and snapshot.

## Concepts

- Transitive Pairing. Nodes can introduce new nodes to the mesh.

## Stack

- rust
- iroh
- prost protocol buffers

### Bootstrap

- New peers request a full state snapshot from their first connection.
- The snapshot allows them to skip replaying the entire log history.
- After bootstrap, the node receives incremental updates via gossip.

### Networking

- Designed for mobile clients that may only sync a few times per day.
- When peers connect, they exchange vector clocks to identify missing entries.
- Missing entries are fetched via unicast.
- MAX_DRIFT should be generous (e.g., hours) to accommodate sleeping devices.

Networking modes:
- Active (servers/laptops on power): Frequent gossip broadcasts, proactive sync.
- Low-power (mobile/battery): Pull-based sync on wake. Query peers instead of relying on push gossip.

## Parts

### Nodes

- Identified by their Ed25519 public key.
- Private key stored locally in `identity.key` (not replicated).
- Node data stored in KV:
  - `/nodes/{pubkey}/info` = static metadata (name, added_by, added_at)
  - `/nodes/{pubkey}/status` = `active` | `dormant` | `disabled`
  - `/nodes/{pubkey}/role` = `server` | `device` (optional, hints sync priority)
- Inviting a node = writing entries to `/nodes/{pubkey}/...`.
- Liveness: Each node tracks `last_seen` locally (from watermark gossip). UI alerts if a peer hasn't been seen for threshold (e.g., 30 days). User decides to mark dormant/disabled.
- Status effects:
  - `active`: Normal sync participant, blocks watermark until acknowledged.
  - `dormant`: Excluded from watermark consensus, can be reactivated.
  - `disabled`: Permanently removed from mesh.
- Sync priority: Low-power clients prefer peers marked as `server` or recently active.

### Data Model

- Keys are flat strings using path conventions (e.g., `/nodes/{pubkey}`, `/config/sync/interval`).
- Prefix queries via string matching (sorted map enables efficient range scans).
- State is computed by replaying `Put`/`Delete` operations from all authors.
- Entry ordering: by HLC timestamp, then by author ID as tiebreaker.
- Conflicts resolved by last-write-wins (using the ordering above).

### Timestamps (Hybrid Logical Clocks)

Timestamps use HLC `<wall_time, counter>` with Causal Clamping:

- Each entry includes an HLC and a reference to its parent (prev_hash).
- Standard HLC: `new_hlc = max(local_wall_clock, max_seen_hlc + 1)`.
- On receive: if `entry.hlc > local_wall_clock + MAX_DRIFT`, clamp to `parent.hlc + 1`.
- All nodes compute the same clamped time from the parent (deterministic).
- Genesis entries (no parent) with future timestamps are dropped.

Pre-flight check (before signing):
- Compare local_clock to max_peer_hlc (from recent gossip/entries).
- If `local_clock > max_peer_hlc + MAX_DRIFT`, use `max_peer_hlc + 1` instead.
- This catches future-clock nodes before they poison the log.

Authors apply their own entries through the standard receive path to ensure consistent clamping.

### Storage

Each node stores logs as one file per author:

```
data/
├── identity.key               # Local node's Ed25519 private key
├── logs/
│   └── {author_id_hex}.log    # Append-only SignedEntry stream per author
└── state.db                   # redb: KV snapshot + vector clocks + indexes
```

- Logs: Append-only binary files per author, containing serialized `SignedEntry` messages.
- State DB (redb): Combined KV state, vector clocks, and indexes. Updated as entries are applied.

#### state.db Tables (redb)

```
Table           Key                     Value           Purpose
─────────────────────────────────────────────────────────────────────────────
kv              String (path)           Vec<u8>         Replicated key-value data
vector_clocks   [u8; 32] (author_id)    u64 (max seq)   Track sync state per author
entry_index     (author_id, seq)        u64 (offset)    Fast entry lookup by position
meta            String                  Vec<u8>         System metadata (own_seq, watermark, etc.)
```

### Watermarks

- Nodes gossip their watermarks periodically (throttled).
- A watermark is a vector clock: how much of each author's log the node has seen.
- All nodes keep all logs (own + others) for redundancy until watermark consensus.
- Once all peers have acknowledged entries, they can be pruned and replaced by the snapshot.
- If a node is offline too long, it re-bootstraps with a fresh snapshot when it reconnects.