I have been working with RocksDB for a while, and in my use case, complex data structures are stored as values in the database — for example, a Map that may itself contain nested maps.
1. Update complexity: Updating a specific entry within a map currently requires fetching the entire serialised object from the database, deserializing it, modifying the target field, reconstructing the map, and then writing it back to RocksDB. This represents the worst-case scenario in terms of read–modify–write overhead.
2. Key flattening approach: To optimise fine-grained updates, I experimented with flattening the map into multiple key-value pairs using relational-style keys: key = mapName#keyInMap value = map.get(keyInMap)
This structure allows each field within the map to be represented as a unique entry, enabling partial updates without reading the entire map.
3. Column family distribution: I then explored distributing data across multiple Column Families (CFs), where each CF represents a field of the map. For example, if a map contains the keys name, age, and gender, then three CFs — Name, Age, and Gender — would be created. Each CF would hold values for a single attribute across all maps:
Column Family Name: map1#name, map2#name, map3#name
Column Family Age: map1#age, map2#age, map3#age
Column Family Gender: map1#gender, map2#gender, map3#gender
In this setup, each column family is responsible for storing one logical field from all maps, allowing targeted updates and reads for specific attributes.
At this stage, this is the design approach I have developed. However, I am open to further optimisation ideas or alternative architectural strategies that could improve efficiency, consistency, or scalability in this scenario.
