+972 52-548-6969
Transitive Replication in RavenDB
TLDR;
Replication topologies make my head hurt.
One of our customers had an interesting requirement, several months ago:
Basically, he wanted to write a document at node #1, and have it replicate, through node #2, to node #3. That was an easy enough change, and we did that. But then we got another issue from a different customer, who had the following topology:
And that client problem is that when making a write to node #1, it would be replicated to nodes 2 – 4, each of which would then try to update the other two with the new replication information (it would skip node #1 because it is the source). That would cause… issues, because they already had that document in place.
In order to resolve that, I added a configuration option, which controls whatever the node that we replicate to should receive only documents that were modified on the current node, or whatever we need to include documents that were replicated to us from other nodes as well.
It is a relatively small change, code wise. Of course, documenting this, and all of the options that follows is going to be a much bigger task, because now you have to make a distinction between replicating nodes, gateway nodes, etc.
Comments
I wonder if these customers REALLY needed bi-directional replication. This is not a very common setup and chances are they could have designed it in a better way - witho only one direction of replication and without cycles.
Maybe this should be handled like a pub/sub message distribution in message bus - each document type has its master/origin database where it can be inserted and updated and all other databases are only allowed to take read-only replicas of that document from the master. And a 'slave' for one document type can be a master of another type.
PS your captchas are barely readable or I'm just a computer programmed to consider itself a human
@Rafal - It's the google one that helps to read old book texts. One of the words is the real captcha the other is a word that you are helping to decode. But you are right they are a pita.
Not related to topic:
You're publishing to RSS only post titles. Is it intentional?
Valeriu, no, it isn't. I just updated the blog software, and it might be related to that, I'll check it next week.
Valeriu, no, it isn't. I just updated the blog software, and it might be related to that, I'll check it next week.
s/whatever/whether/g
where 'g' means in all your other posts, too...
A little additional context, the transitive replication was needed to support blue/green deployment (in my case, at least). While the replication needed to happen across an arbitrary number of servers, only 2 of those servers would ever actually exist at any one point in time.
I showed you Correspondence while you were in Dallas. The way it solves the problem is simple. Every object is immutable. And any two objects with exactly the same values are considered to be the same object. The object can come from multiple directions and it won't be a duplicate.
I agree... I am very surprised this causes "issues"... The document is identical in every respect... Why does it matter if I receive another copy of an identical document? Hmmmmm....
Wow, really brainfucking!
Comment preview