+972 52-548-6969
Voron perf testing, part II
After the previous round of perf testing, I decided to compare just Voron and Esent. The machine is the same as before, and I am testing this on a 300GB HDD. One of the reasons that I choose to use an HDD here is that it is drastically slower, so it shows problematic I/O patterns more easily.
I also decided that it would be interested to see exactly how it scale with regards to concurrency.
And here you can see the latencies involved in each 100 items transaction.
Now, to be fair to the poor thing, it is running on an old HDD. Here are the stats from running this. Notice the actual latencies that are involved:
But interestingly, we can see that the more concurrent writes we use, the slower we become. At a guess, I would assume that this is related to concurrent threads issuing different writes and causing a lot of seeks.
But after investigating a bit deeper, I think that the problem is actually different. Concurrent writes are inherently random. This happens because as we read from the sequential stream in multiple threads, and each thread writing them to Esent, what the db actually gets is a stream of non sequential ranges.
Those can be roughly in the same place, but they aren’t sequential, negating a lot of potential optimizations.
Comments
Just out of curiosity: have you done any testing where the journal (and/or backing memmap files for scratch pages memory) are located on a different disk than the data file?
Alex, No, not yet.
Comment preview