architecture: future ideas for rich crtds
This commit is contained in:
@@ -114,3 +114,46 @@ meta String Vec<u8> System metada
|
|||||||
- All nodes keep all logs (own + others) for redundancy until watermark consensus.
|
- All nodes keep all logs (own + others) for redundancy until watermark consensus.
|
||||||
- Once all peers have acknowledged entries, they can be pruned and replaced by the snapshot.
|
- Once all peers have acknowledged entries, they can be pruned and replaced by the snapshot.
|
||||||
- If a node is offline too long, it re-bootstraps with a fresh snapshot when it reconnects.
|
- If a node is offline too long, it re-bootstraps with a fresh snapshot when it reconnects.
|
||||||
|
|
||||||
|
### Rich CRDTs (Future)
|
||||||
|
|
||||||
|
Instead of a generic scripting language, use specific data types that merge better than LWW.
|
||||||
|
|
||||||
|
Extend value types in redb:
|
||||||
|
|
||||||
|
```rust
|
||||||
|
enum ReplicatedValue {
|
||||||
|
LWW(Vec<u8>), // Standard Last-Write-Wins (current model)
|
||||||
|
Counter(i64), // PN-Counter (Increment/Decrement)
|
||||||
|
Set(HashSet<Vec<u8>>), // OR-Set (Observed-Remove Set)
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Counter** (for "storage used" etc.):
|
||||||
|
- State is `{node_id: value}` map. Merge = sum all nodes. No conflicts possible.
|
||||||
|
|
||||||
|
**OR-Set** (for group membership etc.):
|
||||||
|
- Merge = union. Element present if add timestamp > remove timestamp.
|
||||||
|
|
||||||
|
**Op Code Compromise**: Use commutative operations instead of a VM:
|
||||||
|
|
||||||
|
```protobuf
|
||||||
|
message Entry {
|
||||||
|
oneof operation {
|
||||||
|
PutOp put = 1;
|
||||||
|
DeleteOp delete = 2;
|
||||||
|
MergeOp merge = 3;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
message MergeOp {
|
||||||
|
string key = 1;
|
||||||
|
oneof payload {
|
||||||
|
int64 counter_delta = 2;
|
||||||
|
bytes set_add_member = 3;
|
||||||
|
bytes set_remove_member = 4;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Recommendation: Use Put/Delete for 90% of data. Add CRDT primitives only when needed (concurrent counters, lists) rather than a scripting language.
|
||||||
|
|||||||
Reference in New Issue
Block a user