Clang 12 documentation
Clang 12 Documentation includes a range of tools such as AddressSanitizer, ThreadSanitizer, LeakSanitizer, LibTooling, and more.
- The clang AddressSanitizer
- The clang MemorySanitizer
- The clang LeakSanitizer
- The clang UndefinedBehaviorSanitizer
- The clang of the Hardware – assisted – AddressSanitizer
- The clang SafeStack
- The clang ShadowCallStack
- The clang ThreadSanitizer
- The clang of the Thread – Safety – Analysis
- The clang DataFlowSanitizer
This section is to document clang clang 12 documentation HardwareAssistedAddressSanitizerDesign translation. For reference only.
AddressSanitizer design documents for hardware support
This article is a design document for hardware-enabled AddressSanitizer (HWASAN), a tool similar to AddressSanitizer but based on partial hardware support.
introduce
AddressSanitizer tags one byte for every eight bytes of memory (using shadow memory), uses redzones to detect buffer overflow problems, and quarantines to detect use-after-free problems. Redzones, isolation, and, to a very small extent, shadow memory is the source of AddressSanitizer memory consumption. See AddressSanitizer’s article for more details.
The AArch64 architecture has Address Tagging, or top-byte-Ignore (TBI), which is a hardware feature that allows software to use the eight most meaningful bits of a 64-bit pointer as a tag. HWASAN uses address tags to implement a memory-safe tool, similar to AddressSanitizer, but with a smaller memory footprint, slightly different accuracy, and more accurate in most cases.
algorithm
- For each heap, stack, or global memory object, memory alignment is enforced by TG bytes (for example, TG is 16 or 64). Here TG is referred to as the labeling interval.
- For each of these objects, a random TS bit token is selected. (TS is the tag size, such as 4 or 8).
- Pointers to objects are marked T.
- The memory of the object is also marked T (use oneTG=>1Shadow memory).
- Each memory load and storage is staked by the compiler to read memory markers and compare them to pointer markers. If the tags do not match, an exception is thrown.
For a more detailed discussion of this implementation, see arxiv.org/pdf/1802.09…
Short Granules
Short particles are small particles with sizes between 1 and TG-1 bytes. The size of the short particle is stored in shadow memory where the mark of the short particle is normally stored, and the actual mark of the short particle is stored in its last byte. This means that in order to verify that a pointer’s tag matches the memory tag, HWASAN must check for two possibilities:
- Pointer markers are equal to memory markers in shadow memory, or
- The shadow memory mark is actually the size of a short particle, the loaded value is in the range of the short particle, and the pointer mark is equal to the last byte of the short particle.
Pointer markers between 1 and TG-1 are possible, just like any other marker. This means that there are two interpretations of these markers in memory: full tag interpretation (the pointer marker is between 1 and TG-1, and the last byte of the short grain is raw data) and short tag interpretation (the pointer marker is stored in the short grain).
When HWASAN detects an error with a memory marker between 1 and TG-1, it displays the memory marker and the last byte of the short grain. At this point, it’s up to the user to decide which approach to adopt.
Compiler staking
Memory access
In most usage scenarios, an invocation of an external sequence of instructions is performed to validate these tokens before memory access. The code size and performance cost of this calling operation has been reduced by custom calling conventions:
- Keep most of the registers, and
- Registers contain the address, type, and size of memory access.
Currently, the following sequence of instructions is used:
// int foo(int *a) { return *a; }
// clang -O2 --target=aarch64-linux-android30 -fsanitize=hwaddress -S -o - load.c[...]. foo: str x30, [sp, #- 16]!
adrp x9, :got:__hwasan_shadow // load shadow address from GOT into x9
ldr x9, [x9, :got_lo12:__hwasan_shadow]
bl __hwasan_check_x0_2_short // call outlined tag check
// (arguments: x0 = address, x9 = shadow base;
// "2" encodes the access type and size)
ldr w0, [x0] // inline load
ldr x30, [sp], #16ret [...] __hwasan_check_x0_2_short: ubfx x16, x0, #4#,52 // shadow offset
ldrb w16, [x9, x16] // load shadow tag
cmp x16, x0, lsr #56 // extract address tag, compare with shadow tag
b.ne .Ltmp0 // jump to short tag handler on mismatch
.Ltmp1:
ret
.Ltmp0:
cmp w16, #15 // is this a short tag?
b.hi .Ltmp2 // if not, error
and x17, x0, #0xf // find the address's position in the short granule
add x17, x17, #3 // adjust to the position of the last byte loaded
cmp w16, w17 // check that position is in bounds
b.ls .Ltmp2 // if not, error
orr x16, x0, #0xf // compute address of last byte of granule
ldrb w16, [x16] // load tag from it
cmp x16, x0, lsr #56 // compare with pointer tag
b.eq .Ltmp1 // if matches, continue
.Ltmp2:
stp x0, x1, [sp, #- 256.]! // save original x0, x1 on stack (they will be overwritten)
stp x29, x30, [sp, #232] // create frame record
mov x1, #2 // set x1 to a constant indicating the type of failure
adrp x16, :got:__hwasan_tag_mismatch_v2 // call runtime function to save remaining registers and report error
ldr x16, [x16, :got_lo12:__hwasan_tag_mismatch_v2] // (load address from GOT to avoid potential register clobbers in delay load handler)
br x16
Copy the codeThe heap
Use the malloc operation to mark heap memory or Pointers. This is true of any MALloc that forces all objects to comply with TG’s memory alignment. The free operation marks memory differently.
The stack
The frame is staked by the compiler to TG align memory for all non-promotable allocAs. The stack memory is marked at the beginning and end of the function.
The labels for different assignments (allocAs) in a function are not generated independently. Marking in a function with M allocas would require maintaining M live stack Pointers, which would significantly increase register stress. Instead, we generate a single BaseTag value at the beginning of the function, which is built to allocate the number M as a ReTag(BaseTag, M), where retags can simply be mutually exclusive. M could be a constant here.
Tags for different allocas in one function are not generated independently; doing that in a function with M allocas would require maintaining M live stack pointers, significantly increasing register pressure. Instead we generate a single base tag value in the prologue, and build the tag for alloca number M as ReTag(BaseTag, M), where ReTag can be as simple as exclusive-or with constant M.
The piling operation of the stack has a high resource consumption, but it is optional.
The global variable
In HWASAN’s staked code, most of the global variables are marked. It is possible to use the following mechanisms:
- The address of each global variable has a static tag associated with it. The first global variable defined in the transformation unit is associated with a pseudo-random tag calculated based on the hash value of the file path. Subsequent global variable tags are incremented over the previous tags.
- The symbol of a global variable is appended to the symbol address of the variable in the symbol table of the object file. As a result, global variables need to be marked when their addresses are retrieved.
- When the address of a global variable is fetched directly (for example, not through a GLOBAL offset), a special sequence of instructions is required to append the token to the address, because otherwise the token takes the external address of the small code model (4GB on AArch64). When fetching the address from the GLOBAL offset, no changes are needed because the address stored in the global offset already contains the token.
- For each flagged global variable, there is an associatedhwasan_globalsSection (hwasan_globals section), which shows the address, size, and tag of the global variable. When the binary is loaded, the linker will string the segments together to get a singlehwasan_globalsSegments, which are enumerated through the Runtime (ELF) when memory is marked in turn. Emitted by An associated hwasan_globals section is emitted for each tagged global, which indicates the address of the global, its size and its tag. These sections are concatenated by the linker into a single hwasan_globals section that is enumerated by the Runtime (via an ELF note) when a binary is loaded and the memory is tagged.)
A complete example is as follows:
// int x = 1; int *f() { return &x; }
// clang -O2 --target=aarch64-linux-android30 -fsanitize=hwaddress -S -o - global.c[...]. f: adrp x0, :pg_hi21_nc:x// set bits 12-63 to upper bits of untagged address
movk x0, #:prel_g3:x+0x100000000 // set bits 48-63 to tag
add x0, x0, :lo12:x // set bits 0-11 to lower bits of addressret [...] .data .Lx.hwasan: .word1
.globl x
.set x, .Lx.hwasan+0x2d00000000000000[...]. .section .note.hwasan.globals,"aG",@note,hwasan.module_ctor,comdat
.Lhwasan.note:
.word 8 // namesz
.word 8 // descsz
.word 3 // NT_LLVM_HWASAN_GLOBALS
.asciz "LLVM\000\000\000".word __start_hwasan_globals-.Lhwasan.note .word __stop_hwasan_globals-.Lhwasan.note [...] .section hwasan_globals,"ao",@progbits,.Lx.hwasan,unique,2
.Lx.hwasan.descriptor:
.word .Lx.hwasan-.Lx.hwasan.descriptor
.word 0x2d000004 // tag = 0x2d, size = 4
Copy the codeThe bug report
Errors are generated by HLT instructions and are handled by signal handlers.
attribute
HWASAN uses its own LLVM intermediate code (IR) property, Sanitize_hwAddress, and a property that matches the C function. An alternative is to reuse the ASAN property sanitize_address. The reason for using separate compiler attributes is:
- Users may need to disable ASAN instead of HWASAN. And vice versa, because these tools all have different trade-offs and compatibility issues.
- Ideally, LLVM does not use tags to decide which pass to use. Whether you use ASAN or HWASAN depends on the compiler properties of the function.
This means that users of HWASAN may need to add new compiler properties to code that already uses them.
Compared with AddressSanitizer
HWASAN
- Compared with theAddressSanitizerDifficult to port because it relies on hardware address markers (AArch64). Address markers can be modeled using compiler staking, but would require staking to remove the markers before each load or store operation, which would not be feasible in any real-world environment containing unstaked code.
- If the object code uses a high pointer bit for some other purpose, there may be compatibility issues.
- Some changes to the operating system kernel may be required. (such as Linux doesn’t seem to accept from the address space is marked pointer passed) see www.kernel.org/doc/Documen… .
- No need to use Redzones to detect buffer overflows, but buffer overflow detection is probabilistic, approximately1/(2**TS)The probability of missing bugs (). (6.25% and 0.39% for 4-bit and 8-bit TS, respectively).
- No need to use quarantine to detect using heap-use-after-free or stack-use-after-return. The test appears to be probabilistic.
HWASAN’s memory consumption will be much smaller than AddressSanitizer’s: 1/TG’s extra memory for shadow memory and some of the loss associated with using TG memory to align all objects.
Supported architectures
HWASAN, which relies on address tags, is currently only available in the AArch64 architecture. For other 64-bit architectures, it may be possible to remove address markers before each load and store operation through compiler pegs. But that’s not always the case, because not all code gets staked.
HWASAN’s scheme does not work with 32-bit architectures.
Related work
- SPARC ADI implements a similar tool mostly in hardware.
- Effective and Efficient Memory Protection Using Dynamic Tainting discusses similar Approaches (” Lock & Key “).
- Watchdog discussed a heavier, but still somewhat similar “lock & key” approach.
- TODO: Add more “related work” links.