Timeline for How can users be notified of changes during collaborative editing in a system like Google Docs?
Current License: CC BY-SA 4.0
11 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Sep 15, 2021 at 20:51 | comment | added | JimmyJames | @sdds It appears 200K partitions is the current limit or recommended max. I kind of threw that out of the cuff. It would probably make more sense to divide it up across groups of users. Maybe you mean topic when you said queue but a queue doesn't seem to make sense here at all. | |
| Sep 15, 2021 at 20:19 | comment | added | sdds | @Ewan Modern games have servers dedicated to hosting matches, which is rather different from the dynamically scaling/replaceable nodes patterns often used for scaling web services. Dedicated nodes is what I proposed in my question and what is apparently used at Google docs as per the article in the answer. A shared queue is not going to cut it at large scale since each node would have to process a large number of irrelevant messages to pick the messages that it can send to its connected clients. | |
| Sep 15, 2021 at 20:13 | comment | added | sdds | @JimmyJames As far as I know, Apache Kafka has practical limits on the number of topics/partitions in a cluster, so that would potentially limit scaling (although now that I google it, newer versions of Kafka have significantly increased limits — stackoverflow.com/a/32963227). I'm not familiar enough with other streaming/MQ solutions, maybe some can provide better scaling. | |
| Sep 15, 2021 at 20:02 | vote | accept | sdds | ||
| Sep 14, 2021 at 17:12 | comment | added | Ewan | @sdds when modern games easily handle 64 players all jumping around and shooting each other in real time with millisecond response times, why is have doc edits for 100 people and sub second response times considered a problem? Your naive approach with a queue and web-sockets is pretty much going to work | |
| Sep 14, 2021 at 15:54 | comment | added | JimmyJames | "Running the changes through a single message queue and processing it on each node to notify users connected to node" Why a single queue and not a topic per document? | |
| Sep 14, 2021 at 14:38 | answer | added | Max N | timeline score: 4 | |
| Sep 14, 2021 at 6:27 | comment | added | sdds | @Ewan My question is not whether it's technically possible, how to do it quickly and efficiently at scale, because for only thousands of concurrent users and hundreds of documents this is a no-problem. Could you elaborate on your comment? | |
| Sep 13, 2021 at 20:41 | comment | added | sdds | @DanWilson Could you please elaborate? How would those messages be partitioned, and how would the hub be scaled between multiple nodes? | |
| Sep 13, 2021 at 17:26 | comment | added | Ewan | the reason people like to ask this question is that it breaks the web paradigm of request and response. But really websockets has been a thing for a while now and anyone who has played a multiplayer game in the last 20 years is unsurprised that you can see what others are doing to a document | |
| Sep 13, 2021 at 17:11 | history | asked | sdds | CC BY-SA 4.0 |