+972 52-548-6969
Tricks of working with native memory
I want to start by saying that this isn’t my idea, I read about it a few times, and I recently encountered it with sodium_malloc, so I decided to write my own implementation of the world’s most expensive memory allocator.
What this code does is pretty simple, and quite brutal. It allocates memory in such a fashion that absolutely guarantee that you can’t get away with a whole host of memory problems.
For example, if you try overwrite a buffer allocated by this method, you’ll immediately hit the guard page and die horribly (and predictably, in the place where the error actually happened, not a long way off). If you somehow write before the buffer, that will be detected on free if this is a small under write (which tend to be much rarer, by the way), or immediately if this is a big change.
What is more, once the memory is freed, it is poisoned, and can never be used again. This pretty much rely on us running on 64 bits with effectively unlimited virtual memory, and has the nasty side effect of turning a 50 bytes allocation to something requiring 12 KB. Having said that, as a debugging tool, this is invaluable.
And yes, I’m aware that windows already have that with the heap verifier. But as I’m using this in .NET code, I needed to write my own (this also pretty much work the same way with Linux, you just need to switch the API, but the functionality is the same).
This was built because we were chasing a memory corruption error, and I run this, but it pointed me to a totally different location than suspected. So it is either doing a very good job, or it found me another issue.
… investigating …
Or, as the case may be, we found a bug in the actual memory guard (we didn’t handle allocations of exact page size correctly, and they broke), but at least it broke consistently and was pretty easy to find once I looked in the right place 
.
Comments
Your inventive use of unmanaged memory in C# is a piece of brilliant engineering. Let me go on a tangent a bit though: the same trick can be played on the web too.
Of course there isn't any unmanaged memory in JavaScript you'd say! Or is it?
Say you want stuff your bits away from GC pressure. Could potentially use local storage or DB features of HTML5. But they come with quotas and persky compatibility issues. If only we could get lots of cheap memory fast, with low overhead and no strings attached.
Funny enough, there is such a place! It's DOM. One can create hidden DOM nodes, put loads of content there and retrieve later. Browser design is heavily optimised for DOM, including massive chunks of text and quick allocate/deallocate/access. Funny how similar ideas surface in different environments :-)
Comment preview