Storing the header information
time to read 1 min | 200 words

In Voron, we use a double buffer approach. We use the first two pages of the file to alternately write the last version of the database info. For example, the last transaction id, among other things.

The problem is that when we make those changes, we have to call fsync on that, and as we have seen, that is something that we would like to avoid if possible. Because of that, we are going to try something different. We are going to extract the header information from the first few pages of the file in favor of holding them as separate files: header.one and header.two

The idea is that they are very small files, and as such, it would be cheap to fsync them independently. Moreover, we can take advantage of the fact that very small files (and in this case, I am not sure we even above 256 bytes) are usually stored in the MFT in NTFS and in the inode in ext4. That means that fsync would get both data and metadata at the same time, hopefully just writing out a single block.

I am not sure how useful that is going to be, but I have hopes.

Tweet Share Share 0 comments
Tags: