Image: HFCTF.
Original article on MediumA Look Back on HFCTF 2026 and Where Are We Headedmedium.com ↗
Hi everyone! I’m dax, the current team lead of the HFCTF which is the main CTF event of the Hackfest, one of the biggest cybersecurity events in Canada, of which I am also a committee member. I’ve been a volunteer in the team for 11 years (slowly taking a leadership role for the past 3 years), and so I have seen this event evolve and adapt to the world in which it is set.
This year’s CTF was a weird one. Lower attendance (under 250 attendees, when a few years back we were actually considering applying for the biggest on-site CTF Guinness World Record), official change in leadership (I also led last year’s CTF and part of the one before, but my predecessor and mentor PhenixCorp was still on standby in case I needed him), more challenges and designers than we’ve ever had (139 challenges from more than 20 designers) and a new approach to try to give everyone an enjoyable CTF experience: different scoreboards for players using AI, and for those who don’t. This led to a lot of uncertainty in our team and forced us to take chances, some of which we weren’t sure how they would be accepted by our community.
Like every year, we send a survey to all contestants so that we can adjust our approach for the year to come. So here are the takeaways, and what we plan to do about them.
Was splitting the scoreboard between contestants using AI agents from those who don’t a good idea?
I would tend to say yes. We had twice as many players that signed up for the No-AI CTF compared to the AI one. Contestants in the No-AI category did not feel the obligation to use AI agents to have a competitive edge, which led to a more enjoyable experience for everyone. It does however place the load on the CTF team, since we need to act as the “AI police”. Even if we did not have to do it often this year, not every challenge designer is comfortable asking questions to potentially cheating teams, but this is a unfortunate side effect that cannot be avoided. In any case, the feedback from the contestants was very positive, and we are planning a similar approach for next year, with potential changes that will allow us to ensure that contestants are playing fair.
Are you satisfied with the No-AI ruleset?
For the most part, yes. Most criticism we received was regarding one of those three points:
- Not being clear enough regarding what “giving artifacts of a challenge to any AI tooling” meant.
- Not making it clear enough that the rules were applicable only to the No-AI CTF.
- Having too many rules (which is arguably harder to work around).
Our current stance is that the base rule set is good, and most teams enjoyed it. We will however make it clearer, and more concise.
Should the AI CTF have different challenges?
This one is not as simple as it seems. I had 18 designers in my team who created around 100 challenges. Creating different challenges for the AI CTF requires not only to have twice as many designers (or have them work twice as hard), but also to encourage them to use generative AI more, which I do not plan to do. Even producing variant of existing challenges brings an additional complexity that falls on volunteer designers. It is my opinion that doing so would reduce the general quality of the challenges produced.
Also, after speaking with AI teams and reading the feedback survey, opinions about the topic seem to be very divided. Some teams complained that there was not enough “AI traps”, wishing to push their agentic setup to the limit. Others were annoyed by the only two tracks that had these types of traps.
Our current takeaway is that given the divergence of opinions, we will favour unified challenges for both CTF categories so that quality doesn’t suffer, and will leave the decision to individual designers to add variations for the AI version of their challenges.
Why the deviation from the standard Jeopardy style scoring?
We’ve introduced two new scoring systems this year (King of the Hill and First Bloods). Before the event, many options have been proposed to deal with agentic challenge solving. One that a lot of people thought could solve the issue was to move away from the Jeopardy format, and push towards more diversity on ways to make points to make the CTF less linear (open CTFd, choose a challenge, solve it, repeat). The few deviations we tried were to “test the waters”. Turns out, most people actually prefer the classical format. King of the Hill forced teams to “allocate a team member” to this specific challenge for the whole event. First Bloods seemed unfair to some when there were so many challenges, since it will mostly depend on which challenge was seen first.
The other things that caused some friction was the lack of uniformity between the scoring systems of different challenges (some had First Bloods, some didn’t) and the fact that the value of some flags changed during the event (points were increased when there were less solves than expected for example). The former made it hard to know where to put efforts to maximize points, and the latter could cause some unexpected changes in the scoreboard (since increasing the points of a challenge also changes the value of previously submitted flags).
We had couple more ideas of novel ways to make points (black markets, HFCTF crypto, inter-team collaboration, bug bounties, awards, etc.), but in my opinion they will all have the same effect: make the CTF experience less predictable and less dependent on pentest and cybersecurity skills. While I haven’t decided just yet if this is the way to go, my current takeaway is that most players are here for a classical CTF experience, and so your feedback might sway us in that direction. We will also try to make it more uniform between challenges, and to be careful when adjusting the value of certain challenges mid-event (as it is a way for us to bring more traction to challenges that aren’t getting enough love from the players). In any case, we plan to be more upfront about such decision during the year through announcements, so keep an eye on our Discord!
What do you think about the CTF quality and diversity?
On diversity, the feedback is similar each year. Why so many X, not enough Y. The truth is, this stems from a decision that the HFCTF team leads have been trying to stand by for as long as I can remember. Designers are volunteers, and I will not force them to create challenges for categories that do not interest them. While this leads to the “best possible scenario” for designers who get to spend their free time on creating quality challenges on subjects that they love, it does have the drawback of causing an imbalance in the types of tracks we offer. This year, we had more Windows related challenges than usual, and less pwn. As we do each year, we will take this feedback into consideration if some of our designers do not have a specific subject in mind, but in general, if they join the team next year, it’s because they already know what they want to work on.
On quality, AI hit us hard. Suffice it to say that while some challenges were designed without the use of generative AI, others were heavily created by it. I personally think that this takes away from the amount of effort that some designers put in their tracks, and so our guidelines will be stricter on our end next year to provide a more organic experience, as opposed to the generic imagery that AI produces. It is hard to step away from tools like these, especially when some contestants will actively use them to get the flags, but my opinion is that challenges should be designed with human players in mind, not agents.
What did people think about the music?
Ah the age-old HFCTF question. Truth is, feedback shows that the majority of contestants do not care. They put on their headphones either way, and hope that our music is not louder than theirs. All I can say on this one is that while it is not possible to please everyone, we will try to get a DJ for at least the opening night of the CTF next year. If you know someone that might be interested to take the job, hit me up on the Hackfest Discord! Oh and also, we’ll see that the acoustics are better next year so everyone can actually hear the announcements!
Do you think the way you give hints is fair?
While many challenges have paid hints in the CTFd and we drop some occasional hints in the Discord when we see tracks getting less solves than expected, we also give hints from time to time to players that show high levels of commitments towards a specific challenge and come to speak with us at the admin table (and sometimes through Discord, but taking the time to come to see us in person will play in your favour). Hackfest has always been an event centred on learning. In this spirit, while we will not give flat-out solutions, we will gently steer players in the right direction or vaguely confirm hypotheses when it is our opinion that they will not influence the outcome of the competition in a significant way and when the contestants show genuine interest in the learning experience the CTF provides.
This can be seen as unfair to some, but on the other hand not giving any hints can generate frustration that will make the general experience less enjoyable. Given the fact that some of our designers absolutely LOVE when contestants come to see them to talk about their work and that feedback shows that players like these nudges, we consider that these slight advantages are acceptable.
Why did you choose the weekend of Halloween for the event?
For those who were not at the closing ceremony, we absolutely did not, and were forced to move to the only remaining option which was from the 29th to the 31st. The only other option was changing venue, and with the size of the event, there was no good compromise. Next year will be one week before Halloween, if we don’t get “kicked” again.
What did people think about the pizza?
Might seem like an obvious question right? Free pizza is a no brainer. The biggest problem is that people don’t seem to follow the “one pizza box per table” rule, and so with 65 boxes for ~40 teams, some teams didn’t get any. My math isn’t mathing. We’ll see what we can do next year that is not a nightmare to handle for our team (like asking each table what type of pizza they want and going around giving it away ourselves), but that will give everyone a chance to have a slice. Yes we know that there are 8 slices per pizza, for teams of 6. This is not meant to be a replacement for your dinner. Just a small midnight snack to give you a bit of gas for the rest of the night.
What did people think about not giving away cash prizes?
Feedback shows a slight preference for it. The idea behind this was to align with our view of the CTF, which is more academic than competitive. Giving away hacking tools and goodies seemed to be more in tune with this view. We also decided to give the same prize in both CTFs (except for the NorthSec invitations that were given only to the No-AI winners, as per the decision of the NorthSec committee). We felt that this would strengthen our stance that both AI and No-AI teams are welcome to our competition and that it would encourage fair play by not having any incentive to sign up in the other category with the intention of breaking the rules.
Why won’t you pre-provision infrastructure for the teams?
We’ve used the challenge provisioning service for the past four years, and have improved it every year to a point where designers can effortlessly integrate with it, scaling is not an issue anymore, and stability is good enough that we pretty much do not even look at it during the event and let it do its thing. A lot of time and effort went into it (pre-ai era, so like a lot a lot, and I’m continuing to trad code it), and it was originally developed to solve four problems:
- Lowering costs
- Distributing per-team access information
- Taking care of post-event cleanup
- Allowing team segmentation
Pre-provisioning has to do of course with point #1. On average, the majority of challenges will be spawned by around 75% of the teams. Also, a good amount of teams will only spawn what they actually want to target, and some teams will also be so kind as to shut down their instances when they’re done (if that’s you, thanks!). Pair that with a library of templates that are optimized for cost, and per challenge TTLs that will automatically deprovision instances and you get a 24h CTF for under 1k USD$ in cloud computing.
That being said, while most challenges can be spawned in around five minutes, some beefier challenges can take almost an hour. Even if we advertise the average spawn time in the platform, this can be frustrating. We currently have the option to pre-spawn infrastructure (although it was never used at CTF scale), so we will look into pre-spawning challenges that are too long to spin up next year. We’ll also look into integrating the provisioning directly into CTFd so you have access to everything in one place!
Was there too much AI in the CTF tracks?
I would say it depends on whom you ask. For those who know me, you know that my answer would be yes. I am a strong advocate of the coding craft. This also seems to be the opinion of the people answering the survey, and it was not only regarding the CTF, but the whole Hackfest event. I find it hard to read terms like “AI slop”, “low effort”, and “cheap out” when I see all the work that goes into an event like this. But I unfortunately must agree that this year certainly had a different feel about it.
For the CTF, the problem before us is simple: if the designer team do not use AI, we will produce fewer challenges. Higher quality for sure, but less nonetheless. Contestants will use AI to solve them (either in tokenmaxxing fashion in the AI CTF, or in a more conservative way in the No-AI CTF, but AI is there in both cases) and thus will “burn” through more challenges. This gives us three options:
- Get more designers. We are looking for new team members this year if you’re interested! This seems like the obvious choice, just get twice as many designers and problem solved. But finding people that actually want to create challenges with limited AI usage (which means they’re going to spend a lot more of their free time on Hackfest) is pretty hard. We also have a small selection process which filters out a few potential recruits. A bigger team also costs us more. We’re all volunteers, but we still pay for hotels, food, swag, coins and a few other things. And finally, managing a team of volunteers is a job on its own. The more you have, the more time I (yes me) need to invest in weekly meetings, follow ups, and keeping people aligned and focused.
- Start creating challenges earlier. I have a terrible secret. The official team kickoff for HFCTF is always in August, only three months before the CTF. Of course some designers start working on their stuff as early as December, but the actual grind starts only in August. This is a combination of the fact that we manage a second event, the iHack, in June, so our focus is mostly on that in the first half of the year, and also because we don’t want to force our volunteers to work on the CTF all year long.
- Accept that some teams will finish the CTF early. Personally I would hate that to happen. People sign up for a 24h CTF, not a 16h one, no matter if they use AI or not.
We haven’t decided yet what exactly we will do to address this. But what is sure is that we will not simply “do the same thing again”. I want the HFCTF to be a staple of quality in terms of CTF, and I have the privilege of working alongside very talented designers that are entirely capable of delivering such quality.
In conclusion
If you took the time to fill the feedback form, thank you! The HFCTF is an event that is shaped by the community, and your comments are taken very seriously for this reason. If you thought that your experience this year was not on par with what you expected, your feedback will help us adjust the formula for next year. We also know that in general, people that have more positive feedback tend not to answer surveys as much. So if you left us comments about what you loved, also thank you! And if there is a specific challenge that you particularly enjoyed, take a minute to tell it to the friendly-neighborhood challenge designer that crafted it! These comments are what keeps us going year after year. In any case, I hope that you enjoyed the CTF this year, and we hope to see you in the trenches next year!
That’s all folks, stay safe out there, and hack the planet!
Also, I know you know, but this article was written without the use of generative AI.
Sources
- A Look Back on HFCTF 2026 and Where Are We Headed medium.com ↗