This article was originally presented as a lightning talk during ElixirConf 2025.

Axis was a challenge deployed during DEF CON Capture the Flag finals on August 9, 2025. It’s an Elixir Phoenix app, mostly using Phoenix LiveView as the user interface layer, with an intentional CWE-94 Improper Control of Generation of Code flaw that allows flag disclosure.

Getting it deployed was a gigantic pain. I spent quite a bit of time fighting Puppeteer and then Playwright for browser automation, Docker’s x86-64 emulator for arm64, but the most interesting issue was the Erlang Port Mapper Daemon, or “EPMD”.

Erlang Distribution

In languages running on the Erlang runtime system (“ERTS”) like Erlang and Elixir, applications are generally a collection of small processes running in a single Erlang node. This Erlang node is normally a single beam.smp (or just “BEAM,” for “Bogdan’s Erlang Abstract Machine”) host process. You’re encouraged to use high level abstractions like “generic servers,” “agents,” or “finite state machines;” those abstractions are built on asynchronous messaging betwen those small processes. In Elixir, this is done with the send(destination, message) function, which uses the Erlang Destination ! Message syntax. The message can be any Erlang term, and the destination is an atom representing a named process on the current node, an Erlang Pid (which could be local or remote), an Erlang Ref that might be a process alias, or a pair of {ProcessName, NodeName}.

What goes into a node name? Since they can be on different physical machines, a network address is going to be part of it. However, given that a single machine may have multiple Erlang nodes on it, instead of letting them fight over well-known ports, what you really want is for nodes to be named, take random ports, and have an independent host process map those names to ports.

CTF Infrastructure

Nautilus Institute ran our CTF finals on containers, but with krun instead of runc like normal Docker to support challenges that bring their own pwnable kernel. While runc does some network and process namespacing tricks to let container guest processes coast off the host’s kernel for the sake of performance (Legitimate Business Syndicate had per-connection containers for quals starting up in under a second a decade ago), krun uses full on virtualization to let a container guest bring and run in their own kernel. We originally looked into this for the opportunity to run kernel challenges a few years back, and kept it for the better process isolation.

It also lets you do some tricky stuff with networking. We had a proxy setup that let us restrict VMs to a single listen socket and no outbound sockets (to keep teams from replacing a challenge with a proxy that lets them sidestep our container patch restrictions).

How Axis Starts

Axis was distributed as a release image generated through the normal Phoenix mix rel process. Assuming you don’t run into build issues with prim-tty while trying to build an x86-64 image on an arm64 machine, you get an OCI image that starts a BEAM process via a bunch of shell scripts.

1. Shell scripts grab information from environment variables to set arguments to the erl command that kicks off BEAM, which on a modern machine names itself beam.smp
2. The beam.smp process starts with a file of arguments for the VM available, traditionally in a file called vm.args
3. The BEAM process loads VM arguments.
4. BEAM kicks off an EPMD process.
5. The EPMD process tries to bind port 4369 on the zero IP address (i.e. 0:4369, every interface). If that fails, it dies, since that probably means an EPMD is running and it doesn’t need a second.
6. BEAM tries to bind port 0 on the zero IP (0:0). This tells the kernel “give me any port that’s available on every interface.”
7. BEAM tries to connect to EPMD to register its name and the port from the previous step.
8. BEAM starts running the application.

How to Deploy a CTF Challenge

I started working on Axis in September 2024. In March 2025 we switched it from a quals to a finals challenge, and in June 2025 I really started finding a route to finish it. Because I’ve had a lot of experience with running Elixir Phoenix apps in containers, I felt like I could focus on getting it done with a poller and known proof-of-vulnerability and not spend a lot of time on integration. There’s usually time the week of DEF CON for that.

And then I spent two weeks fighting (first) Puppeteer and (then) Playwright. By the time I had a poller I was happy with, it was Friday afternoon and our infrastructure team was busy with a bunch of other services. Some of the issues that popped up in this last sprint were:

• my computer is arm64 and we deploy on x86-64
• this compose.yml doesn’t do what you want it to on arm64:

  services:
    web-prod:
      build:
        context: .
        dockerfile: Dockerfile
      platform: linux/amd64
  

• the mix local.hex --force step to install the normal Elixir package manager doesn’t like the x86-64 emulator my Docker desktop app runs
• if you make an arm64 machine on EC2 out of habit it can’t build x86-64 images
• once we got a built x86-64 image that the infrastructure could actually start, it would open ports we weren’t expecting and immediately die

We spent a few minutes with strace trying to solve this mystery. The tell-tale sign to put us on the right track was the 4369 bind call. This introduced me to EPMD, which I’d previously been fuzzy on, and helped get to the issue that was killing BEAM before it could open the expected port 4001.

The rub comes when BEAM tries to bind 0:0. Because of a bug in our krun setup, it would, instead of returning 0 for success, it returned a failure condition (that we didn’t bother triaging at the time).

I spent a minute trying to coax a couple commercial LLMs to help, but then I remembered the VM arguments concept from a previous job.

After some research on the flags and arguments for erl (the command line tool that kicks off BEAM), I tried a few different combinations of arguments.

--no_epmd on its own didn’t help. It looks like that requires more configuration than I was willing to learn about in the rush to deploy.

--dist_listen false did work! It tells BEAM to just not open a listen port for distributed Erlang, and lets the app continue to boot and work.

Once that was in, we got the poller running (it was pretty unreliable, since I’m either not experienced with browser automation or browser animation is simply terrible) at a reliability we were comfortable with, and decided to enable it in the scoreboard and post about it in #ctf-announcements-text

Conclusion

Shout out to the teams that spent time hacking and patching Axis! I hope you had fun, and I’m glad I was able to finally bring a challenge to a CTF at DEF CON, even if it was web sqli.

Incredible thanks to itszn for your patience with me getting this disaster out the door.

Finally, huge thanks to Josef! Our time riffing on how this would work and your time helping develop it are why it happened at all.

Vito

Vito is a world-recognized CTF thought leader with over ten succesful CTF games run, speeches about CTF on multiple continents, and a pretty good showing at the 35C3 Junior CTF.

« January 2025 Qualifiers Update