This part of the programming section covers serialized data. Serialized data is stored in binary, meaning it is not readily human-readable. Since many of the bytes stored will not fall in the printable ASCII range, opening the file in a text editor does nothing for us — we have to read it as raw bytes and interpret it ourselves.
Each challenge will instruct us on how the data is formatted. We will typically be given a layout for the records that make up the file. Each field in a record has a specific size — either fixed or variable — and we use that layout to pull the bytes back out in the right order.
Once we have read the structure, we process it to recover the flag. This step is different for each challenge, but we will walk through each one. It is typically not the hard part, especially if you are comfortable with the ASCII half of the programming section.
When we read binary data, we pass the binary flag to fopen so the platform does not try to translate newlines or stop early on a stray byte:
1FILE *fp = fopen("data.bin", "rb"); // "rb" = read, binary
2if (fp == NULL)
3To pull the bytes out we use fread, which lets us say exactly how many bytes we want and where to put them. It takes a destination pointer, the size of one element, how many elements to read, and the file pointer. It returns the number of elements actually read, which we check to know when we have hit the end of the file:
1uint8_t tag;
2while (fread(&tag, sizeof(tag), 1, fp) == 1)Because the layout is fixed, we can often read a whole record straight into a struct:
1typedef struct
2{
3 uint16_t index;
4 uint8_t letter;
5 uint16_t leftstruct with a uint16_t, a uint8_t, then two more uint16_ts may be 8 bytes in memory but only 7 bytes on disk. When the on-disk size and the in-memory size differ, read each field individually instead of reading the whole struct at once — or pack the struct. Never assume sizeof(struct) == bytes on disk.A multi-byte integer can be stored with its least-significant byte first (little-endian, what x86 uses) or its most-significant byte first (big-endian, the traditional "network byte order"). The two orderings store the same number as different byte sequences, so reading with the wrong one gives you garbage.
The challenge instructions always tell you which ordering the file uses. If it matches the machine you are running on, fread into an integer just works. If it does not, read the raw bytes and assemble the value yourself so your code is correct regardless of the host:
1uint8_t b[2];
2fread(b, 1, 2, fp);
3uint16_tAssembling values by hand like this is the safest habit: it never depends on the host's own byte order or on struct padding.
Serialized files come in two broad shapes, and almost every challenge is one of them:
fread runs out. A node that is always { index, letter, left, right } is the classic case — read one record, repeat.length more bytes:1uint8_t tag, length;
2while (fread(&tag, 1, 1, fp) == 1 An opcode stream is a close cousin: a one-byte opcode decides how many bytes follow it, so you branch on the opcode and read the operands it implies.
As you read records, you need somewhere to keep them. We use two primary approaches, each with trade-offs:
Nearly all of the challenges follow the same shape, so once you have a template for reading records and storing them, you can reuse it across the section. The same is largely true of the ASCII section.
Let's get solving!