2025-12-05¶

1. What is binary exploitation?¶

Binary Exploitation is about abusing bugs in compiled programs to make them do things they were not meant to do.

Most of the time, this happens in low-level languages like C or C++, where the programmer has to manage memory manually.

Common problems include:

Writing past the end of a buffer,
Using memory that has already been freed,
Or leaving memory uninitialized. Attackers use these bugs to:
Crash programs,
Read sensitive data,
Or, most importantly, change the program’s control flow so they can run their own code or gain higher privileges.

Even today, a huge chunk of serious vulnerabilities are memory safety bugs in C/C++ code, especially in low-level or high-performance systems (OS kernels, browsers, network daemons, etc.).

Mitigations (like ASLR, stack canaries, NX, etc.) exist, but they’re not perfect, so exploitation techniques keep evolving (ROP, ret2libc, heap exploitation, etc.).

2. How memory and the stack work¶

To understand exploitation, you need a picture of how a program’s memory is laid out.

2.1 Process Memory Layout (simplified)¶

A simple view looks like this (from top to bottom):

Kernel Space (not directly accessible)
Stack: At the top, you have the stack, which grows downwards↓.
Heap: A region called the heap, used for dynamically allocated memory like malloc.
.bss / .data: Here we have global variables and static data.
.text: At the bottom, you have the program’s code section, where the machine instructions live.

We care most about the stack.

Whenever a function is called, some important things are placed on the stack:

The return address – this is the address of the instruction where the program should continue after the function finishes.
The saved base pointer (the saved RBP), which helps the function keep track of its local variables.
The local variables themselves, like small arrays or integers.

2.2 The stack & function calls (x86-64 SysV)¶

When a function foo() is called, roughly this happens:

Caller pushes arguments (in registers like RDI, RSI, RDX… first, but also stack for extra).
call foo pushes the return address (address of instruction after call) onto the stack and jumps to foo.

Inside foo, the prologue usually:

push rbp       ; save old base pointer
mov rbp, rsp   ; set new base pointer
sub rsp, X     ; reserve space for local variables

Local variables are stored below rbp on the stack.

When foo returns, epilogue:

leave          ; mov rsp, rbp ; pop rbp
ret            ; pops return address into RIP

On x86–64, there are a few key registers:

RIP: the Register Instruction Pointer – it tells the CPU which instruction to execute next.
RSP: the Register Stack Pointer – it points to the top of the stack.
RBP: the Register Base Pointer – it’s a fixed reference point used to access local variables.

The crucial idea is this:

[!important] If you can overwrite the saved return address on the stack, you can control where the ret instruction sends the program next. That means you control RIP, the instruction pointer.

This is the heart of many classic exploits.

3. The classic Stack buffer overflow ¶

Imagine a simple C function like this:

#include <stdio.h>
#include <string.h>

void reachMe() {
    printf("You reached reachMe()! Here's the flag.\n");
    // print flag, or something important
}

void vuln() {
    char buf[32]; // fixed-size buffer

    printf("Enter your input:\n");
    gets(buf);    // SUPER unsafe: no bounds checking
    printf("You entered: %s\n", buf);
}

int main() {
    vuln();
    return 0;
}

It declares a small character array as a buffer, for example 32 bytes (char buf[32]).
It then reads user input into that buffer without checking the length, using something unsafe like gets().

If you type in more than 32 characters, the extra characters don’t just disappear. They keep being written into memory past the buffer:

First, they overwrite other local variables,
Then they overwrite the saved base pointer (the saved RBP),
And finally, they overwrite the saved return address. If you carefully control those extra characters, you can:
Leave the first part as random junk or padding,
And then place a fake return address – the address of some function you want to run.

For example, the binary might contain a function named “reachMe()” that is never called normally. The challenge is: “make the program call reachMe() by exploiting the overflow.”

So your plan is:

Fill the buffer until you reach the return address.
Overwrite the return address with the address of reachMe().

When the vulnerable function finishes and executes ret, instead of going back to its caller, it jumps straight into reachMe().

4. Using GDB to find the offset¶

To make this precise, you need to know how many bytes it takes to reach the return address. This number is called the offset.

The typical workflow looks like this:

Run the program with a long input, such as a hundred characters of “A”.
- If it crashes with a segmentation fault, that’s a good sign that you might have overwritten something important.
Open the program in a debugger like GDB.
- Run it again with a patterned input or a long string.
- When it crashes, inspect the registers.
If you see that the instruction pointer, RIP, is filled with a value like 0x41414141, which is the hexadecimal representation of repeated “A” characters, then you know your input has overwritten the return address.
Then, you adjust your input length to find the exact number of bytes needed before you start overwriting RIP.

That means your exploit input will be:

40 bytes of padding,
then the new return address.

4.1 Discovery phase¶

You run the binary:

$ ./vuln Enter your input: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA Segmentation fault (core dumped)

If we give it a big input and the program crashes, that’s suspicious.

You then:

Use different lengths of input.
Confirm it crashes in a way that suggests stack smashing (e.g., SIGSEGV at weird address).

4.2 Analyze with GDB¶

You open it in gdb:

gdb ./vuln
(gdb) run
Enter your input:
AAAA... (a pattern)

When it crashes:

Check registers:
```
(gdb) info registers
```
Look at the backtrace:
```
(gdb) bt
```

f RIP = 0x4141414141414141 (0x41 = ‘A’), you know you’ve overwritten the return address with your input.

Goal now: find how many bytes until RIP is overwritten → the offset.

4.3 Finding the offset to RIP¶

Instead of just 'A' * 100, we often use a cyclic pattern (unique non-repeating bytes) so we can identify the exact offset.

With pwntools (more on that later), you can generate a pattern. But conceptually:

Send a pattern of length, say, 200.
Crash the program.
Check RIP; find where those bytes occur in the pattern → this’ll give you the offset.

The slides’ example:

Offset to overwrite RIP was 40 bytes (so the 41st–48th bytes overwrite the return address).

5. Overwriting the return address with a target function¶

Once you have the offset, you need the address of the function you want to jump to, such as reachMe().

You can find this using tools like:

nm to list function symbols and their addresses, or
objdump -d to disassemble the binary, or
In a debugger, by asking for the address of the symbol. Say you learn that reachMe() is at address 0x40116c. Then your malicious input looks like:
First: 40 bytes of arbitrary padding, for example 40 ‘A’s,
Then: the address 0x40116c, written in the correct 64-bit little-endian format.

When the vulnerable function returns, it pops this 64-bit value from the stack into the instruction pointer, and execution continues at reachMe().

You’ve just done a simple “ret to win()” exploit: you return directly into a “win()” function that was not reachable before.

5.1 Crafting the exploit: ret to `reachMe()`¶

So we now know:

Offset to RIP = 40 bytes.
Address of reachMe = e.g. 0x40116c (from objdump -d or nm ./vuln etc.).

Idea:

Payload = "A" * 40 + address_of_reachMe (in little-endian)

In Python (basic):

offset = 40
reachMe = 0x40116c

payload = b"A" * offset
payload += reachMe.to_bytes(8, "little")  # 64-bit address

You feed this as input to the program. When vuln() returns, it ret’s to reachMe.

6. Why the flag sometimes doesn’t show: buffering¶

In the lecture example, there’s a detail about output buffering.

Standard output, such as printf, is often buffered. That means:

Text you print is first stored in a buffer,
It only appears on the terminal when certain conditions happen, such as:
- A newline character (\n) is printed,
- The buffer is full,
- The program ends normally,
- Or the buffer is explicitly flushed (fflush(stdout)) If your exploit:
Jumps into reachMe(),
reachMe() prints the flag without a newline, or something crashes immediately after the print,

then the program might exit abruptly before the buffer flushes, meaning you don’t actually see the flag on screen.

To fix that, you can chain a second function call that prints a newline or calls fflush().

For example, you might:

First overwrite the return address with the address of reachMe(),
Then place another address after that on the stack, for a function that prints a newline or otherwise flushes the output.

So the sequence of returns becomes:

Vulnerable function returns into reachMe(),
reachMe() runs, prints the flag,
reachMe() returns, and control flows into the second function that prints a newline and flushes the buffer,
Only then does the program crash or exit.

That’s why sometimes the first exploit “works” technically, but doesn’t visibly show the flag until you add this extra step.

7. Using Pwntools to make life easy¶

Pwntools is a Python library for CTF-style binary exploitation. It helps with: - Packing/unpacking integers (addresses) in little-endian - Spawning processes or remote connections - Sending/receiving data - Automating exploit steps Example of the same exploit in pwntools-style:

from pwn import *

# Adjust context (arch, OS)
context.binary = "./vuln"
elf = context.binary

# Addresses from analysis:
offset = 40
reachMe = elf.symbols["reachMe"]    # nice shortcut if symbols present
what    = elf.symbols["what"]       # newline/flush function

p = process(elf.path)

payload  = b"A" * offset
payload += p64(reachMe)
payload += p64(what)

p.sendline(payload)
print(p.recvall().decode(errors="ignore"))

p64() packs a 64-bit address into the right little-endian bytes.

8. Return Oriented Programming (ROP) and “ret to libc” – high level overview¶

Modern systems often mark the stack as non-executable(NX/DEP). This means you can’t just put shellcode on the stack and jump to it.

To get around that, attackers use Return Oriented Programming, or ROP.

The idea behind Return Oriented Programming (ROP) is:

Instead of injecting new code, you reuse small instruction sequences that already exist in the program’s code or in shared libraries.
Each of these small sequences ends with a ret instruction and is called a gadget.
By carefully overwriting the stack with a sequence of gadget addresses, you can build a chain of tiny operations that collectively do something powerful. One common pattern is a ret to libc attack:
Libc is the standard C library, which contains useful functions such as system.
You find gadgets that let you set up arguments, like a gadget that does “pop rdi, ret”, which lets you put an address into the RDI register.
Then you return into system with RDI set to the address of the string “/bin/sh”.
The result: the program calls system("/bin/sh"), giving you a shell.

You don’t need to be able to write new code; you just reuse existing code in clever ways by chaining return addresses on the stack.

Example gadgets you often look for:

pop rdi; ret → set up the first function argument (on x86-64 ABI).
pop rsi; ret → second argument.
pop rdx; ret → third argument. Tool from slides: ROPGadget:
```
ROPgadget --binary ./vuln | grep "pop rdi; ret"
```

Then a simple ROP chain to call:

system("/bin/sh");

would be:

pop rdi; ret → put address of "/bin/sh" into rdi.

ret to system. So payload stack could look like:

"A" * offset
+ address_of_pop_rdi_ret
+ address_of_bin_sh_string
+ address_of_system

When the function returns, it:

Goes to pop rdi; ret (sets rdi = "/bin/sh").
Returns to system → system("/bin/sh"), giving you a shell.

9. ret2libc ¶

ret2libc = “return to libc”: instead of jumping to your own shellcode, you reuse functions in the C standard library (libc), like system, printf, execve, etc. Basic steps in a typical ret2libc exploit: 1. Find or leak a libc address at runtime (e.g., via puts a GOT entry). 2. Use that to determine the base address of libc in memory. 3. Compute addresses of system and "/bin/sh" inside that libc. 4. Build a ROP chain:

"A" * offset
+ pop_rdi_ret
+ address_of_string_bin_sh
+ address_of_system

The lecture just introduces ret2libc + ROP as an exploit chain you can use to get remote code execution by sending a single malicious payload.

10. What you should take away from this lecture¶

By the end of this lecture and explanation, you should be able to:

Explain, in your own words, what a stack buffer overflow is and why it’s dangerous.
Describe the basic stack frame layout:
- Local variables,
- Saved base pointer,
- Saved return address.
Understand that if you control an overflow into the stack, you may be able to overwrite the return address, and therefore control RIP, the instruction pointer.
For a simple vulnerable program, follow this workflow:
- Give it long input until it crashes.
- Use a debugger to confirm that the crash is caused by your data overwriting RIP.
- Find the exact offset to the return address.
- Find the address of a target function, like reachMe().
- Craft an input that is: padding up to the offset, then that function’s address, and, if needed, an additional address to flush output.
Have a rough, high-level understanding of:
- What ROP is,
- What “ret to libc” means,
- And why they’re used to bypass modern protections like non-executable stacks.

Never do this:

There are so many compilers
char name[12], the compiler says “screw you, I’m putting 16” because the compiler always wants to align things to 4 or 8 bytes. So always make it a multiple of 4 or 8.

Ret2Libc¶

Ret2Libc is really useful technique that allows us to call functions we use when programming in C such as printf, system, fflush,...
We can even use pwntools to look for libc functions available in binary

elf = ELF(BINARY)
if 'system' not in elf.plt:
    log.error("system@plt not found")
    exit(1)

ROP (Return Oriented Programming) is a technique that allows us to execute code in a program by chaining together small snippets of existing code, called “gadgets”, that end with a return instruction. This is often used in exploitation to bypass security mechanisms like non-executable memory. A ROP gadged is a small sequence of instructions that ends with a return instruction. By chaining together multiple gadgets, an attacker can create a ROP chain that performs arbitrary operations.

ret2libc + ROP (exploit chain)¶

Find the address of the system function in libc.
Find the address of the /bin/sh string in libc.
Find a gadget that allows us to set up the arguments for the system function call (usually by setting the appropriate registers).
Create a ROP chain that sets up the arguments and calls the system function.
Overwrite the return address of a vulnerable function to point to the start of our ROP chain.

Useful things to look into for the lab:¶

Pwntools docs
- The Pwntools docs are a great resource for learning how to use the library effectively.
- They provide detailed explanations of the various functions and classes available in Pwntools, along with examples of how to use them.
ROPGadget docs
- The ROPGadget docs provide information on how to use the ROPGadget tool to find ROP gadgets in binaries.
- This can be useful for creating ROP chains for exploitation.
Ret2Win
- Ret2Win is a common exploitation technique that involves redirecting the program’s execution flow to a specific function (often called “win”) that grants the attacker control or access to sensitive information.
- This technique is often used in Capture The Flag (CTF) challenges and other security competitions.
ret2libc
- ret2libc is an exploitation technique that involves redirecting the program’s execution flow to the system function in the C standard library (libc) to execute arbitrary commands.
- This technique is often used in buffer overflow attacks to gain control of a vulnerable program.
Write-what-where attacks
- Write-what-where attacks are a type of exploitation technique where an attacker can write arbitrary data to an arbitrary memory location.
- This can be used to overwrite function pointers, return addresses, or other critical data structures to gain control of a program’s execution flow.

Final Words¶

If you would like to learn more about binary exploitation, best resource is just trying it for yourself and debugging there is an entire world I did not cover in this presentation for example:
- heap-based buffer overflows,
- JOP(Jump Oriented Programming),
- SROP(Stack Return Oriented Programming),
- JIT (Just In Time) exploitation,
- Micro-architecture/Hardware exploits,
- fd (file descriptor) hijacking,
- etc.
An okay place to start is playlist on YouTube → binary exploitation by CryptoCat.
You could also practice on pwn college website and later on when you get enough practice look into old exploits and try to reproduce them.
Full honesty there is not as good resources available for binary exploitation as for other things in Cyber Security but still this is a nice start ☺