feat: Refactor mesh server to use Arc<Node>, introduce JoinAcceptance for mesh joins, and enhance store listing with prefix filtering and deleted entry inclusion.

This commit is contained in:
2025-12-23 00:31:54 +01:00
parent 76810f8d8e
commit 9d4495b3d7
13 changed files with 341 additions and 206 deletions
+5 -5
View File
@@ -59,14 +59,14 @@ Networking modes:
- Identified by their Ed25519 public key.
- Private key stored locally in `identity.key` (not replicated).
- Node data stored in KV:
- `/nodes/{pubkey}/info` = static metadata (name, added_by, added_at)
- `/nodes/{pubkey}/status` = `active` | `dormant` | `disabled`
- `/nodes/{pubkey}/name` = display name
- `/nodes/{pubkey}/added_at` = timestamp when added
- `/nodes/{pubkey}/status` = `invited` | `active` | `dormant` (removal deletes keys)
- `/nodes/{pubkey}/role` = `server` | `device` (optional, hints sync priority)
- `/nodes/{pubkey}/iroh` = Iroh NodeId for network connection
- Peer invitation flow:
1. Inviter runs `invite <peer_pubkey>` → writes `/nodes/{peer}/info` + `/status`
2. Inviter shares their Iroh NodeId out-of-band (QR code, link, text)
3. Invited peer runs `connect <inviter_nodeid>` → syncs with inviter
3. Invited peer runs `join <inviter_nodeid>` → syncs with inviter
4. Sync pulls `/nodes/{self}/info` + `/status` → peer is authorized
5. `connect` implicitly adds inviter to peer's `/nodes/*` (mutual awareness)
- Accepting = syncing. The invited peer discovers authorization by receiving the entries.
@@ -181,7 +181,7 @@ Each node stores logs as one file per author:
Table Key Value Purpose
─────────────────────────────────────────────────────────────────────────────
kv Vec<u8> (key) Vec<HeadInfo> Current tips for each key
applied_frontiers [u8; 32] (author_id) (u64 seq, [u8; 32] hash) What's applied to this store
AUTHOR_TABLE [u8; 32] (author_id) (u64 seq, [u8; 32] hash) Per-author frontier tracking
meta Vec<u8> Vec<u8> Store metadata (incl. merkle_root)
```
+25 -11
View File
@@ -127,10 +127,7 @@
- [x] Entry ordering: Per-author streaming is correct (hash chain per author, HLC for cross-author).
- [x] Multi-head sync fixed: SyncState now tracks HashSet of head hashes per author.
- [x] Sync entry ordering: Entries sent in HLC order (merge-sort across authors) to ensure causal order.
*Background Sync:*
- [ ] Periodic sync with known peers
- [ ] Track last sync time per peer
- [x] `join_mesh` doesn't populate `node.root_store`: Fixed with `complete_join` method.
### Success Criteria
@@ -138,26 +135,43 @@
- Works offline-first (sync when connected)
**Post-M2 Refactoring:**
- [ ] Unify `node.rs` from `lattice-cli` and `lattice-core`
- [ ] Use prost for node status in store
- [x] Unify `node.rs` from `lattice-cli` and `lattice-core`
- [x] Move network code to `lattice-net`
---
## Milestone 3: Multi-Node Mesh
**Goal:** N nodes form a gossip mesh with watermark consensus.
**Goal:** N nodes form a gossip mesh for real-time sync.
### Deliverables
- [ ] Gossip protocol
- [ ] Watermark tracking & log pruning
- [ ] Node invitation (sigchain membership)
- [ ] Conflict detection (LWW resolution)
**Phase 1: LatticeServer Refactor**
- [ ] `LatticeServer` struct in `lattice-net` wrapping `Arc<Node>` + `Endpoint`
- [ ] Move `join_mesh`, `sync_with_peer`, `sync_all` to `LatticeServer` methods
- [ ] Encapsulate `spawn_accept_loop` inside `LatticeServer`
- [ ] CLI uses `LatticeServer` instead of raw `Node` + `Endpoint`
- [ ] Route sync command through `LatticeServer` (not raw functions)
- [ ] Integration test: invite → join → sync end-to-end
- [ ] Periodic background sync with known peers
- [ ] Track last sync time per peer
**Phase 2: Gossip Protocol**
- [ ] Proto: `GossipAnnounce` message with author + latest seq + HLC
- [ ] `LatticeServer::spawn_gossip_loop` for periodic announcements
- [ ] On receiving announce: detect missing entries, trigger sync
- [ ] Track last-seen per peer for staleness detection
---
## Future
- remove_peer should be a transactional operation on store
- Watermark tracking & log pruning
- Track minimum confirmed seq per author across all peers
- Log pruning: remove entries below watermark
- Multi-KV-Store sync
- Optimized sync on join. Only transfer current watermark state, then sync missing entries. This would allow pruning. Might need snapshot support in KV store.
- Mobile (iOS/Android) clients
- Key rotation
- Secure storage (Keychain, TPM)