Abstract:Programs developed in C/C++ perform efficiently, but often suffer from memory leaks. This article provides an idea for finding Memory leaks through Wrap Malloc.
Programs developed in C/C++ perform efficiently, but often suffer from memory leaks. This paper provides an idea to find memory leak through wrap Malloc. Relying on this method, the author urgently solved the memory leak problem and avoided the bleeding of the project. This method was brilliant in the future work and found a large number of long-standing memory leak problems in the project.
What is a memory leak?
A memory leak occurs when a memory reference is lost and there is no way to reclaim it.
Programming languages like Java manage memory reclaim automatically, while C/C++ requires explicit release. There are many ways to avoid memory leaks, such as RAII, such as smart Pointers (mostly based on reference counting), such as memory pools.
Theoretically, as long as we are careful enough, at the time of each application, remember to release, that the world is clean, but the reality is often not so good, such as throw exceptions, free memory statement execution, or a novice programmer accidentally buried a ray, so, we must face the real world, that is what we will encounter a memory leak.
How do I check for memory leaks?
We can review code, but looking for hidden problems in a sea of code is like looking for a needle in a haystack.
Therefore, we need to use tools such as ValGrind, but these tools to find memory leaks often have certain expectations or constraints on the way you use dynamic memory. For example, objects residing in memory will be misreported, and the truly useful information will be buried in a sea of false positives. Many times, even ValGrind doesn’t solve problems on everyday projects.
Therefore, in order to use ValGrind to run, many famous open source projects have made great efforts to modify the source code greatly, so that the project can meet the requirements of ValGrind. To meet these requirements, the project running valGrind without any alarm is called ValGrind clean.
Since these things all look useless, so, beg for others rather than yourself, or have to rely on their own.
What is a dynamic memory allocator?
The dynamic memory allocator is a function library between kernel and application programs. Glibc provides a dynamic memory allocator called PTmalloc, which is the most widely used implementation of the dynamic memory allocator.
From the kernel’s perspective, the dynamic memory allocator belongs to the application layer. From an application perspective, the dynamic memory allocator belongs at the system layer.
Applications can apply for dynamic memory directly from the kernel through the MMAP system, or allocate memory through the MALloc interface of the dynamic memory allocator. The dynamic memory allocator allocates memory to the kernel through SBRK and MMAP. Therefore, the memory released by the application through free is not necessarily returned to the system. It may also be cached by a dynamic memory allocator.
Google has its own dynamic memory allocator, TCMALloc, and Jemalloc is also a well-known dynamic memory allocator with different performance and caching and allocation strategies. You can use them to replace the ptmalloc that comes with Linux glibc.
New /delete and malloc/free
New is a c++ use, like Foo *f = new Foo, which actually has 3 steps.
Operator new() allocates memory at sizeof(Foo), and eventually malloc.
(2) Build Foo on newly allocated memory.
(3) Return the address of the newly constructed object.
New = allocate memory + construct + return, while delete equals destructor +free.
So fixing malloc and free is basically fixing dynamic memory allocation.
chunk
Each chunk of memory returned through malloc is called a chunk, as the dynamic memory allocator is defined and will be called later.
wrap malloc
GCC supports wrap, which changes the behavior of calls to malloc by passing -wl,–wrap,malloc, and linking calls to a custom __wrap_malloc(size_t) function. In the __wrap_malloc(size_t) implementation, we can actually allocate memory by __real_malloc(size_t), and then we can do a little tinker.
Similarly, we can wrap free. Malloc is paired with free, and there are other related apis like Calloc, Realloc, and Valloc, but it’s basically malloc+ Free. Realloc is Malloc + Free.
How do YOU locate a memory leak?
We malloc various chunks of different size, that is, there are different numbers of each chunk of different size. If we can track the number of chunks of each size, we can know which chunk size is leaking. Quite simply, if the number of chunks of that size keeps growing, it’s likely to leak.
It is not enough to know that a chunk of a certain size leaked. We need to know on which calling path the chunk of that size was allocated to check that it was released correctly.
How do I track the number of chunks for each size?
We can maintain a global unsigned int malloc_map[1024 * 1024] array whose subscript is the size of the chunk and whose value malloc_map[size] corresponds to the chunk allocation of that size.
This maintains a chunk size to chunk count mapping table. It’s fast enough, and it can cover chunks from 0 to 1M in size. It’s big enough that a Megabyte at a time is scary enough to cover most scenarios.
What about the ones that are bigger than 1M? We can log it.
In __wrap_malloc, ++malloc_map[size]
In __wrap_free, –malloc_map[size]
Quite simply, we record the allocation of chunks of different sizes through malloc_map.
How do I know the size of the chunk released?
Free (void *p) has only one argument. How do I know the size of the chunk to be released? How to do?
__wrap_malloc(size_t) allocates 8+size chunks. The first 8 bytes store the size of the chunk and return (char*)chunk + 8. That is, an offset of 8 bytes is returned to the application calling malloc.
Void * p = void* p; void* p = void* p; void* p = void* p; void* p = void* p;
It exists in the array malloc_map[1M]. Assuming that 64 bytes of chunk are allocated all the time and the number keeps growing, we think that the size of chunk is likely to leak. How can we locate where the chunk was called?
How do I record the call chain?
We can maintain a toplist array of 10 elements that holds the 10 largest chunk sizes. This is easily done by taking the top 10 from malloc_map.
Then we test __wrap_malloc(size_t) to see if the size is one of the toplists. If so, we dump the call stack into a log file via glibc backtrace.
Note: there is no memory allocation, so you can only use backtrace, not backtrace_symbols, so you can only get the symbolic address of the call stack, not the symbol name.
How do you convert a symbolic address into a symbolic name, that is, a line of code?
addr2line
The Addr2line tool can do this, and you can trace the call chain to locate the memory leak.
At this point, you’ve got the whole core idea.
Of course, in the actual project, we did more. We not only recorded the toplist size, but also recorded the incremental Toplist of each size chunk, recorded the malloc/free of large chunks, and wrapped more APIS.
To recap: You can now locate a memory leak with wrap Malloc/Free + Backtrace + addr2line. Congratulations.
Click to follow, the first time to learn about Huawei cloud fresh technology ~