I’ve recently extracted another module out of Grail Diary: KeyValueCRDT.
It turns out designing a file format that works in an era of cloud document storage is hard! Cloud documents and mobile devices make it really easy for people to make conflicting changes to the same document. It’d be nice to provide a better experience for people than a “pick which version of the file to keep” dialog box.
The key to avoiding the “pick the version of the file to keep” dialog is making your file format a Conflict-Free Replicated Data Type (CRDT). With a CRDT, you can reliably merge changes made from multiple devices rather than forcing a person to pick which file version to keep.
My goal with KeyValueCRDT is to provide a CRDT implementation that can work as a file format for a wide range of applications. There are more details about the API the GitHub page, but here’s the bullet-point summary:
KeyValueCRDTuses SQLite for its storage, for all of the reasons listed in SQLite As An Application File Format.- The data model is a key-value store.
- Values can be text, JSON, or arbitrary data blobs. Text values are indexed using FTS5 for fast full-text search.
- At its core,
KeyValueCRDTis an observed-remove set and provides multi-value register semantics. When you read a key from the database, you may get multiple values returned if there were conflicting updates to that key. - In addition to the underlying database operations, the module provides a
UIDocumentsubclass that lets you integrate with the iOS document ecosystem (including iCloud documents). The module also provides a command-line tool (kvcrdt) to allow you to inspect and manipulate the database from scripts.
Currently I use KeyValueCRDT for the document format for Grail Diary, and I hope it will be a useful format for other applications as well.