Current implementation - syscall abstractions
I refactored my approach to test my malloc by just mocking syscalls under testing conditions to get rid of dynamic linking altogether.
This post shows how we used gdb to confirm that calls originating inside shared libraries (e.g. libstdc++) were reaching our malloc implementation, and how the PLT/GOT mechanism participates in the control flow. This is a part of dynamic linking investigation series.
The goal is not to teach gdb, but to make the investigation reproducible.
Note: If you want, you can follow along by running make investigation-container to exec into the container into a direct gdb session.
We run the test binary under gdb and break on malloc:
(gdb) break malloc
(gdb) run test_align
The first breakpoint hit is typically not a test-authored allocation. It often comes from runtime startup, shared libraries, or the test harness.
At the first breakpoint inside our malloc, we take a backtrace:
(gdb) bt
#0 malloc (size=73728) at src/malloc.c:276
#1 0x0000fffff72a4f2c in ?? () from /lib/aarch64-linux-gnu/libstdc++.so.6
#2 0x0000fffff7fc1b4c [PAC] in ?? () from /lib/ld-linux-aarch64.so.1
#3 0x0000fffff7fc1c60 [PAC] in ?? () from /lib/ld-linux-aarch64.so.1
#4 0x0000fffff7fd8360 [PAC] in ?? () from /lib/ld-linux-aarch64.so.1
Here we see the stack frames until this point i.e. the call to malloc.
If libstdc++ had debug symbols installed, frame #1 would have a proper name and file:line; but right now it’s stripped, so we see ??, we cannot know for sure what is calling and for what specifically.
A characteristic result:
malloc (in the test executable)Interpretation:
This confirms that foreign library code is entering our allocator.
A quick mental model:
malloc and will resume after it returnsSo malloc returns to Frame #1.
(gdb) info locals
ret_addr = 0xfffff7461530 <<malloc@got[plt]>>
tail = 0x0
aligned_size = 0
blk = 0xfffff72a4f2c
We also observe the variable and its value when we record the return address with __builtin_return_address(0).
Note that we are at the beginning of malloc, nothing executed yet BUT ret_addr is populated.
Why do we see this address even though the
ret_addrassignment is not done yet?
We stopped just before the assignment and gdb is showing us whatever garbage was on the stack.
That garbage happens to be a tunnel into the GOT (probably because the caller (libstdc+++) just did a GOT load for malloc and stored something in that stack slot). This is important for other reasons which we will discuss in detail in the next section,
but for now we have to:
mallocWe know that we are coming from 0x0000fffff72a4f2c (from /lib/aarch64-linux-gnu/libstdc++.so.6), gdb shows us the frame information.
As ret_addr we have 0xfffff7461530, remember it is garbage.
To validate that suspicious addresses fall inside GOT/PLT segments of the shared library:
(gdb) info files
Symbols from "/app/build/tests/test_malloc".
Native process:
Using the running image of child process 54.
While running this, GDB does not access memory from...
Local exec file:
`/app/build/tests/test_malloc', file type elf64-littleaarch64.
...
0x0000fffff745ffe8 - 0x0000fffff74621d0 is .got.plt in /lib/aarch64-linux-gnu/libstdc++.so.6
...
What we have here is the range of memory addresses for ‘s GOT/PLT entries.
The garbage address we had earlier ret_addr (0xfffff7461530),
falls in between the address range. So we proved that, libstdc++ jumped in our mallocthrough it’s dynamic loading mechanism. It makes sense, every code dynamically linked to our test binary should use our implementations and
we achieved it by exposing our implementations as strong symbols.
On AArch64, the return address is in lr:
(gdb) info registers lr
lr 0xfffff72a4f2c 281474828488492
We see that lr matches the address shown in frame #1, the callsite is exactly as expected: the bl instruction in the caller set lr to the next instruction.
To confirm that malloc is called via the shared library’s PLT stub, let’s disassemble around lr:
(gdb) disas $lr-16, $lr+16
Dump of assembler code from 0xfffff72a4f1c to 0xfffff72a4f3c:
0x0000fffff72a4f1c: str x19, [x21, #64]
0x0000fffff72a4f20: add x20, x20, #0x480
0x0000fffff72a4f24: mov x0, x19
0x0000fffff72a4f28: bl 0xfffff729ef80 <malloc@plt>*
0x0000fffff72a4f2c: str x0, [x20, #56]
0x0000fffff72a4f30: cbz x0, 0xfffff72a4fd4
0x0000fffff72a4f34: stp x19, xzr, [x0]
0x0000fffff72a4f38: str x0, [x20, #48]
We see the instruction where the branching is done with *.
We should see a bl instruction that targets something like malloc@plt inside the shared object:
malloc@pltThe important point:
Even though symbol resolution uses PLT/GOT, the return address recorded in lr still points into the caller’s .text.
So return address tracing reflects the true callsite.
When we take a step, and let the garbage address we had earlier ret_addr (0xfffff7461530),
to be set, we now see that it is set to 0x0000fffff72a4f2c, the address of /lib/aarch64-linux-gnu/libstdc++.so.6. It now correctly points to where it is coming from. We double checked it by looking at the value stored in lr register, and it holds as well.
(gdb) s
debug_write_str("[mm_malloc] size=");
(gdb) info locals
ret_addr = 0xfffff72a4f2c
tail = 0x0
aligned_size = 0
blk = 0xfffff72a4f2c
(gdb) info registers lr
lr 0xfffff72a4f2c 281474828488492
After continuing, the next malloc hit is often called from inside our own implementation (e.g., calloc). We also see our log coming directly from malloc which is good.
(gdb) continue
Continuing.
[mm_malloc] size=73728
Breakpoint 1.1, malloc (size=240) at src/malloc.c:276
276 void *ret_addr = MM_RET_ADDR();
A backtrace for an internal call typically looks like:
(gdb) bt
#0 malloc (size=240) at src/malloc.c:276
#1 0x0000aaaaaaab5890 in calloc (len=15, size_of=16) at src/malloc.c:212
#2 0x0000aaaaaaaadcdc in main (argc=2, argv=0xfffffffffd28) at tests/acutest.h:1831
malloc called by calloc (both in the executable)main/test harnessIn this case, addresses tend to fall in the executable’s PIE range (often the 0xaaaa...-like region on AArch64 Linux).
We take one step and again see the address is set to the same address of calloc’s frame shows 0x0000aaaaaaab5890. It is written right next to it… We also show that our counter is set as well. We check the array of recorded call sites, and see the previous execution of malloc is recorded. (0x0000fffff72a4f2c, the address of /lib/aarch64-linux-gnu/libstdc++.so.6.
(gdb) s
278 debug_write_str("[mm_malloc] size=");
(gdb) info locals
ret_addr = 0xaaaaaaab5890 <calloc+176>
tail = 0xfffffffffb10
aligned_size = 18858823296101900
blk = 0x500000500
(gdb) print malloc_called
$1 = 1
(gdb) print mm_callsites
$2 = {{blk = 0xaaaaaaad9000, ret_addr = 0xfffff72a4f2c}, {blk = 0x0, ret_addr = 0x0} <repeats 1023 times>}
This provides a clean A/B comparison:
Using gdb, we confirmed:
malloc through their PLT stubs.malloc symbol in the executable..text, enabling callsite classification.Next, the ELF: GOT, PLT & Lazy Binding page explains the dynamic linking mechanisms behind what we observed here.