Optimizing memory usage to improve .NET performance

There is a common misconception about the importance of avoiding memory allocations when it comes to .NET performance tuning. People think that since memory allocations are fast, they rarely have a performance impact.

To understand what leads to this misunderstanding, we have to go back to the days of COM programming as seen in C++ and Visual Basic 4 through 6. With COM, memory was managed using a garbage collector in the form of reference counting. Whenever an object was assigned to a reference variable, a hidden counter was incremented. If the variable was reassigned or exited from scope, the counter was nullified. If the counter reached 0, the object was deleted, freeing the memory for use elsewhere.

[[213309]]

This memory management system is "deterministic". Through careful analysis, you can determine when to delete an object. This means that you can automatically release resources such as database connections. With .NET, you need a separate mechanism (e.g., destroy/enable) to ensure that non-memory resources are released in a timely manner.

Reference counting garbage collectors have three major disadvantages. First, they are susceptible to "circular references." If two objects reference each other, even indirectly, then the reference count cannot drop to 0, and a memory leak will occur. We must be careful to write code to either avoid circular references or provide some kind of destructuring method to break the cycle when the object is no longer needed.

Another major disadvantage is encountered when working in a multithreaded environment. In order to avoid race conditions, some type of locking mechanism (e.g. lock, increment, spinlock, etc.) is needed to ensure that the recount remains correct. These operations are surprisingly expensive.

Finally, the list of available memory locations can become fragmented, with many small, unusable spaces between live objects. Memory allocation typically involves walking a contiguous linked list of free space in order to find a location large enough to accommodate the requested object. (Memory fragmentation also exists in .NET with the "large object heap" or "LOH.")

In contrast, allocating memory with a mark-and-sweep style garbage collector like .NET or Java is a simple pointer increment mechanism. The assignment is no more expensive than allocating an integer. The actual cost is only paid when the GC actually runs, and this is usually mitigated by using a generational collector.

When .NET first came out, many people complained that the non-deterministic behavior of .NET's garbage collector would hurt performance and be hard to explain. Microsoft's rebuttal at the time was that for most use cases, a mark-sweep garbage collector would actually be faster despite the intermittent GC pauses.

Unfortunately, this message has become somewhat muddied over time. Even if we accept the theory that a mark-and-sweep garbage collector is faster than reference counting, it does not mean that it must be in an absolute sense. Memory allocation and the associated memory pressure are often the cause of performance problems that are difficult to detect.

Also, the more memory you use, the less effective your CPU cache becomes. While main RAM is so large that disk-based virtual memory is barely used in most use cases, the cache in the CPU is tiny by comparison. The time it takes to fill a CPU cache from RAM can take tens or even hundreds of CPU cycles.

In a recent article, Frans Bouma identified several techniques for optimizing memory usage. While he focused on improving ORM performance, these suggestions are useful in a variety of situations. His suggestions include:

Avoid parameter arrays

The argument keyword is useful but is expensive compared to a normal function call because it requires memory allocation. The API should provide no-argument overloads for commonly used argument calculations.

An overload for IEnumerable or IList should also be provided so that the collection does not need to be copied into an array before calling the function.

If you add data immediately after definition, you can predefine the size of the data structure.

A List or other collection class can be resized multiple times while being filled. Each resize operation allocates another internal array and fills it with the previous one. You can avoid this overhead by providing a capacity parameter to the collection's constructor.

Lazy initialized members

If you know that a given object will not be needed most of the time, then you should use lazy initialization to avoid allocating memory for it prematurely. Usually this is done manually, because the Lazy class itself needs to allocate memory.

We reported on Microsoft's attempts to reduce the size of tasks using similar techniques back in 2011. They reported a 49% to 55% reduction in the time it took to create a task and a 52% reduction in the space required.