A binary of any size is hundreds or thousands of functions, and you will read very few of them. The skill is choosing which. That takes three things: finding where the program's own code starts, understanding how values get into and out of a function so that a call makes sense without opening it, and telling the author's functions apart from the library code around them.
This lesson's chall asks for a name and prints a few things about it:
echo ada | ./challName: Hello, ada.checksum: 294total: 36The download holds chall, its source, and chall32, the same program built for 32-bit x86, which we use for a comparison later.
mainSo far every binary has kept its symbols, so Ghidra's Symbol Tree had an entry called main and we started there. Take the symbols away and that entry is gone:
strip -o stripped challnm strippednm: stripped: no symbolsImport stripped into Ghidra and the function list is a column of names like FUN_00400486. One of them is main. Reading them all to find out which is not a plan that works on a large binary.
There is a reliable way in. Every program has an entry point, the address where the system starts running it, and that address is written in the file's header, so it survives stripping. Ghidra labels it entry. It is not main. It is a short piece of startup code, the same in nearly every C program on Linux, and this is all of it:
1void processEntry entry(undefined8 param_1,undefined8 param_2,undefined1 param_3)
2{
3 undefined1 auStack_8 [8];
4
It makes one call, to __libc_start_main. That is a C library function, so its name survives as an import. Its job is to set the program up and then call main, and to do that it has to be told where main is. That address is its first argument.
So the first argument to __libc_start_main is main. Here it is FUN_00400588. Double-click it, press L, and name it main. This works on any dynamically linked C program, and it is usually the first thing to do with a stripped binary.
When one function calls another, both sides have to agree on where the arguments are and where the result comes back. That agreement is a calling convention. It is not part of the processor. It is a rule that compilers follow, and knowing the rule for your platform lets you read a call straight from the assembly.
On 64-bit Linux, the rule is the System V convention:
rdi, rsi, rdx, rcx, r8, r9.rax.The lower halves of those registers have their own names, so a 32-bit argument in rdi shows up as edi. Function Calls in Assembly 101 covers the convention in full.
Our program calls a function with eight arguments, which is enough to see both halves of the rule. In the source, the call is total(1, 2, 3, 4, 5, 6, 7, 8). In the binary:
objdump -d -M intel --no-show-raw-insn --disassemble=main chall | grep -B8 '<total>' 4005ea: push 0x8 4005ec: push 0x7 4005ee: mov r9d,0x6 4005f4: mov r8d,0x5 4005fa: mov ecx,0x4 4005ff: mov edx,0x3 400604: mov esi,0x2 400609: mov edi,0x1 40060e: call 4004b2 <total>Read it from the bottom up, since the compiler loaded the arguments in reverse. edi gets 1, esi gets 2, edx gets 3, ecx gets 4, r8d gets 5 and r9d gets 6. That is the six registers used up, in order. Arguments 7 and 8 are pushed onto the stack, the last one first.
Ghidra applies the same rule to rebuild the call. It looks at which of these registers are loaded before a call and which the called function reads, and writes out an argument list:
1 param1_00 = total(1,2,3,4,5,6,7,8);
2 printf(This is why decompiled calls are usually so readable. It is also why they are sometimes wrong: Ghidra infers the argument count from register use, and a register that happens to hold a leftover value can look like an argument. If a call shows an argument that makes no sense, check the assembly just before the call for the instruction that sets that register. If there is none, the argument is probably not real.
The return value works the same way in reverse. Whatever is in rax, or eax for a 32-bit value, when a function returns is its result. Here Ghidra gets one wrong:
1void checksum(char *param_1)
2{
3 size_t sVar1;
4 ulong local_18;
Ghidra says this function returns nothing: it is void, it calls clamp and throws the result away. That would make the function pointless, since it adds up the characters of a string and then does nothing with the sum.
The source ends with return clamp(sum, 0, 999);. In the binary, clamp leaves its result in eax, and checksum returns straight away without touching it, so the value is still in eax for the caller. No instruction in checksum writes to eax, and Ghidra concluded that nothing is returned.
The caller tells the truth:
objdump -d -M intel --no-show-raw-insn --disassemble=main chall | grep -A1 'call.*<checksum>' 4005d4: call 400505 <checksum> 4005d9: mov esi,eaxRight after the call, main copies eax into esi, the second argument register, for printf. It is using a return value. So checksum returns an int. Click the function name in the Decompiler, press F, and change the signature to int checksum(char * text):
1int checksum(char *text)
2{
3 int iVar1;
4 size_t sVar2Now the function makes sense. The general lesson: a function's signature is a guess like everything else, and the way callers use the function is evidence about it. A function that computes something and returns void deserves a second look.
Older and embedded targets are often 32-bit x86, and many CTF binaries still are. The convention there, called cdecl, is simpler: every argument goes on the stack, and the result comes back in eax. Here is the same call in chall32:
objdump -d -M intel --no-show-raw-insn --disassemble=main chall32 | grep -B8 '<total>' 8048494: push 0x8 8048496: push 0x7 8048498: push 0x6 804849a: push 0x5 804849c: push 0x4 804849e: push 0x3 80484a0: push 0x2 80484a2: push 0x1 80484a4: call 8048388 <total>Eight arguments, eight push instructions, last argument first, so that the first argument ends up on top of the stack. No registers are involved. If you open a binary and see a row of push instructions before each call, you are looking at 32-bit code. If you see edi and esi being loaded, it is 64-bit. file will tell you too, but it is worth recognizing on sight.
Ghidra handles both, and its output for the two binaries looks nearly the same. The convention only matters when you drop down to the assembly to check something.
chall calls printf, scanf and strlen. Nobody reverse engineers printf: its behavior is documented, and reading its code would take days. The work is in the author's functions. In a dynamically linked binary, the two kinds are easy to tell apart, because library functions are not in the file.
What is in the file is a stub for each one, in a section called the PLT, the procedure linkage table:
objdump -d -M intel --no-show-raw-insn -j .plt chall | grep -A3 '<printf@plt>:'0000000000400380 <printf@plt>: 400380: jmp QWORD PTR [rip+0x2c82] # 403008 <printf@GLIBC_2.2.5> 400386: push 0x1 40038b: jmp 400360 <_init+0x24>A call to printf in our code goes to this stub, and the stub's first instruction jumps to an address stored elsewhere in the file, which the system fills in with the location of the real printf when the program starts. The other two lines only run the first time, to look that address up. The stack exploitation course has a lot more to say about this mechanism, because it can be abused.
For reading a binary, the point is simpler. Any call that lands in the PLT is a library call. Ghidra marks these stubs as thunks, functions that only jump somewhere else, and lists the real targets under Imports in the Symbol Tree. When a decompiled function calls something from Imports, look up its manual page and move on.
printf is a function like any other, often with no name. Statically-Linked Binaries covers how to cope.Put those ideas together and you can map a program before reading any of it. Start at main, list what it calls, list what those call, and stop at anything from a library:
main
printf (library)
__isoc99_scanf (library)
greet
printf (library)
checksum
strlen (library)
clamp
totalGhidra builds this for you. Put the cursor in a function and open Window > Function Call Trees: one side shows everything the function calls, and the other shows everything that calls it.
The tree tells you where to spend time. Four functions here belong to the author: greet, checksum, clamp and total. greet only calls printf, so it prints something. checksum calls strlen, so it works on a string. clamp and total call nothing at all, so they are pure arithmetic and can be read in isolation. We have a rough idea of each function from its position alone.
In a real binary, read the tree the same way. A function that calls strcmp or memcmp is comparing something, and in a crackme that is likely the check. A function that calls fopen and fread handles a file. A function called from fifty places is a utility, and one called from a single place sits close to the logic that matters.
__libc_start_main, and the first argument of that call is main.rdi, rsi, rdx, rcx, r8, r9, then on the stack, and the result comes back in rax. On 32-bit x86, everything goes on the stack.The last lesson in this section, Strings & Constants, covers the fastest way to find the function that matters without walking the tree at all.