Every check so far has compared our input with something we could read: a number in an instruction, a PIN in .data, a string in .rodata. Authors who do not want their check read that easily take one more step. They scramble the input first and compare the scrambled result against scrambled data. The answer is no longer in the file. What is in the file is the answer after a transformation, and the code that performs it.
That turns out to be enough. If we can read the transformation, we can run it backwards. This lesson reverses one such check from start to finish and ends with a script that prints the password.
echo hunter2 | ./challPassword: Wrong length.It tells us the length is wrong, which is already more than it should. The download holds chall and its source. As before, we work from the binary and keep the source for checking afterwards.
strings finds the messages and nothing that looks like a password, so we go to main in Ghidra:
1undefined8 main(void)
2{
3 int iVar1;
4 charMost of this is familiar. The input is read and trimmed, and its length must be 0x13, which is 19. Then come the two lines that matter:
1 scramble(local_48,0x13);
2 iVar1 = memcmp(local_48,target,0x13);Our input is compared with 19 bytes at target, but only after a function has been run over it. That ordering is the signature of an encoded check: the input is modified, and then compared. Whatever target holds, it is not the password. It is what the password turns into.
You will not always get a function named scramble. The thing to look for is a loop that rewrites the input buffer, sitting between the read and the compare.
1void scramble(long param_1,ulong param_2)
2{
3 char cVar1;
4 ulong local_10;It is dense, so take it apart. param_1 is the buffer and param_2 its length, and local_10 counts from 0 up to the length, so call it i. Inside the loop there are three steps, and it helps to write each as its own line:
*(byte *)(local_10 + param_1) is buf[i], and *(byte *)(&key + (i & 3)) is key[i & 3]. The two are combined with ^, which is XOR. i & 3 keeps the lowest two bits of i, which is the same as i % 4: it goes 0, 1, 2, 3, 0, 1, 2, 3. So each byte is XORed with one of four key bytes, in rotation.rotate_left with an argument of 3.i is added, and the sum is stored back into buf[i].In short: buf[i] = rotate_left(buf[i] ^ key[i % 4], 3) + i.
The helper is one line:
1uint rotate_left(byte param_1,byte param_2)
2{
3 return (int)(uint)param_1 >> (8 Two shifts joined with OR, where one shifts left by n and the other shifts right by 8 - n. This pattern is worth memorizing, because C has no rotate operator and this is how everyone writes one. A plain left shift pushes the top bits off the end and loses them. A rotation brings them back in at the bottom, so nothing is lost. Rotating 10110001 left by 3 gives 10001101. The name in this binary gave it away, but in a stripped binary the two-shifts-and-an-OR shape is how you would recognize it.
Before undoing this one, it is worth knowing the usual building blocks, because obfuscated checks are nearly always a few of these stacked up:
| Transformation | Looks like | Undone by |
|---|---|---|
| XOR with a key | x ^ k | XOR with the same key |
| Addition or subtraction | x + k, x - k | The opposite operation |
| Rotation | Two shifts and an OR | Rotating the other way by the same amount |
| Substitution | table[x], with a 256-byte table | A table that maps each value back |
| Reordering | out[i] = in[perm[i]] | Putting each byte back where it came from |
| Multiplication by an odd number | x * k | Multiplying by the modular inverse |
Every row can be reversed exactly, because no information is lost: each possible input gives a different output. That is the property to check for. A plain shift, a division, or an AND with a mask throws bits away and cannot be undone, and neither can a hash like the CRC-32 in Strings & Constants. When a check uses one of those, you need a different approach, such as trying candidates until one matches.
XOR deserves a special mention, because it is the most common of all. XOR with a value is its own inverse: (x ^ k) ^ k is x again. If a program XORs data with a key, running the same XOR over the result gives the original back.
To reverse a sequence of steps, undo them in the opposite order, last step first. Getting dressed, you put on socks and then shoes; getting undressed, the shoes come off first.
The forward direction, for the byte at position i:
key[i % 4]iSo the reverse is:
ikey[i % 4]One detail matters throughout. These are single bytes, so every result wraps around at 256. Adding i to 250 does not give 260. In C that happens automatically because the buffer holds char values. In a script we have to do it ourselves, by applying & 0xff after each step.
We need the two pieces of data, key and target. Both are constants, so they are in .rodata:
objdump -s -j .rodata chall | head -n 8Contents of section .rodata: 4012d0 01000200 00000000 00000000 00000000 ................ 4012e0 1337c0de 00000000 00000000 00000000 .7.............. 4012f0 d28c1e30 37cf2b7e 6a2917f8 e750ab64 ...07.+~j)...P.d 401300 114bff50 61737377 6f72643a 20000a00 .K.Password: ...Ghidra shows key at 0x4012e0 and target at 0x4012f0. So the key is the four bytes 13 37 c0 de, and the target is the 19 bytes that start d2 8c 1e 30. In Ghidra you can also select the bytes in the Listing, right-click, and use Copy Special to copy them in a format ready to paste into a script.
Try the first byte by hand before writing any code. target[0] is 0xd2, and i is 0:
0xd2, which is 11010010 in binary.010, move to the top, giving 01011010, which is 0x5a.key[0], 0x13: 0x5a ^ 0x13 is 0x49.0x49 is the letter I. That is a plausible first letter for a flag, which is good evidence that we read the function correctly. Checking one value by hand like this catches most mistakes before they cost time.
Now the same thing for all 19 bytes:
1key = bytes.fromhex("1337c0de")
2target = bytes.fromhex("d28c1e3037cf2b7e6a2917f8e750ab64114bff")
3
4
The loop body is the three reverse steps, in order, and rotate_right is the same two-shifts-and-an-OR idiom pointing the other way.
python3 solve.pyIFC{und0_3ach_st3p}python3 solve.py | ./challPassword: Correct.We never guessed a password and never ran the program with one until we had the answer. All the information came from reading.
There is a shortcut that works on many checks like this one. Look at what each output byte depends on: only the input byte at the same position, and the position itself. No byte affects any other. That means each position can be solved alone, and a single byte has only 256 possible values.
So you can skip the inversion: copy the forward transformation into a script exactly as the decompiler shows it, and for each position try all 256 values until the output matches target[i]. That is at most 19 times 256 attempts, which takes no time at all, and it needs no understanding of how to reverse a rotation.
This works whenever bytes are transformed independently, and it is often the fastest route in a timed competition. It stops working when the transformation mixes positions together, for example when each byte is combined with the previous output. Then the reverse-order method above is the one to use.
i & 3 is i % 4, and two shifts joined by an OR are a rotation.Next is Other Languages: what changes when the program was not written in C.