+972 52-548-6969
Buffer allocation strategiesA possible solution
time to read 3 min | 427 words
After my recent posts about allocations, I thought that I would present a possible solution for buffer management.
The idea is to have a good way to manage buffers for things like I/O operations, etc. Here is the code:
[ThreadStatic] private static Stack<byte[]>[] _buffersBySize; private static byte[] GetBuffer(int requestedSize) { if(_buffersBySize == null) _buffersBySize = new Stack<byte[]>[32]; var actualSize = PowerOfTwo(requestedSize); var pos = MostSignificantBit(actualSize); if(_buffersBySize[pos] == null) _buffersBySize[pos] = new Stack<byte[]>(); if(_buffersBySize[pos].Count == 0) return new byte[actualSize]; return _buffersBySize[pos].Pop(); } private static void ReturnBuffer(byte[] buffer) { var actualSize = PowerOfTwo(buffer.Length); if(actualSize != buffer.Length) return; // can't put a buffer of strange size here (probably an error) if(_buffersBySize == null) _buffersBySize = new Stack<byte[]>[32]; var pos = MostSignificantBit(actualSize); if(_buffersBySize[pos] == null) _buffersBySize[pos] = new Stack<byte[]>(); _buffersBySize[pos].Push(buffer); }
I’m going to discuss it in my next post, but for now, can you figure out what it is doing, and what are the implications?
More posts in "Buffer allocation strategies" series:
- (09 Sep 2015) Bad usage patterns
- (08 Sep 2015) Explaining the solution
- (07 Sep 2015) A possible solution
Comments
Seems like a simple buffer pool implementation where each buffer is allocated in a power of two. You call GetBuffer with a needed size and you get a buffer that can fit at least that many bytes (but quite possibly more). Once done with the buffer you call ReturnBuffer to return the buffer back into the pool.
The most obvious implications seem to be that there's no way the buffers gets expired, and of course the chance that you request a buffer of a very unfortunate size - which causes you to get a buffer that is very close to twice as big as what was needed.
This looks like a pool of different size byte arrays allocated per thread. This way GC does not have to collect them after every use which minimizes GC time (probably GEN 2 as byte arrays are usually large)
Microsoft has a similar library here: https://github.com/Microsoft/Microsoft.IO.RecyclableMemoryStream
Two implications I see:
- Since a reused buffer isn't cleared, you have to be very careful not to accidentally leak data from an earlier use.
- Manual Memory Management
Unusable in async code with the default SynchronizationContext as you'll allocate new buffers on one thread and then store them on another, re-use would be a lucky coincidence.
@Jahmai Lay: If you assume buffers being used thousands or even millions of times, luck is no longer such a big factor.
Though that does make me think of: if one thread is a 'producer' and another a 'consumer', this solution will be worse than the problem.
@Patrick If you're allocating buffers that many times and spreading them across the entire threadpool, you're probably consuming a lot more memory than you intended. I take your point about luck though.
Patrick, In practice, you usually get the buffer, then immediately write to it / read into it. That is a consideration, though, yes. And yes, this is manual memory management, which is what you want when you know that this will save you a lot of allocations
Jahmai, Sure it is usable in async code. The number of threads you have is roughly the same over time, and assuming a even spread of workload, all threads will have some things in the pool eventually
Jahmai, Assume that we have 10 threads that read something from the network. They each allocate 4KB buffer for reading a message, process it, then do it again. You will have roughly 40Kb of memory allocated. Vs. 4KB times the number of messages
Patrick, That is indeed a weakness of this approach, if you have a thread that just return the buffers, this is a pretty bad impl, because all memory would be held there.
Jahmai, I'm assuming here that all threads are actually doing roughly the same work. If there is a set of threads that just release, that would be a problem in this impl, yes
Damien, In that case, by the way, you can start stealing buffers from other threads, of course, that would balance things out, eventually. It won't give you locality, but it will avoid the memory leak
Comment preview