Volunteer OSINT Coaches


January 14, 2024

We've been running our CTFs for over 5 years now. The competition and the community have continued to change, grow and evolve and with them so has the CTF. We wanted to take a few paragraphs to touch on the back end volunteers that make our CTFs possible. The role has changed a lot over the years and is worth discussing for both newcomers and community veterans alike.

What happened to CTF Judges? 

Yep. Historically, like right up until the time we're writing this, the volunteers working behind the scenes at a CTF were referred to as judges. At the beginning of our CTF journey that name made sense. It was their responsibility to vet the intel submitted during an event, ensuring that only correct information was marked for inclusion in the final reporting. In effect, they were expected to "judge" each submission that came across their feed. The expectation for the role was to check each submission for accuracy and that it lined up with one of the previously established flags and categories from the flag list.

Over time the role has evolved and we felt the changing expectations for the role warranted a name change.

So what's an OSINT Coach? What changed?

The original responsibilities of our back end volunteers are still in place. They're still a layer of filtering in the intel collection process and the ones that make the CTF an actual competition.

The current role is so much more than a "judge" though. Our coaches want their teams to succeed, get points and even "win" the CTF. While they still have the responsibility to filter the intel submitted they're also expected to give quality feedback to the submissions they don't accept. It's completely ok for a coach to talk their team through how to craft a submission that they would accept. Essentially, our coaches are explaining to their teams how to be successful in the competition and that is so much more than "judging".

They want their teams to submit as much quality intelligence as possible. The key word in that sentence being "quality". Coaches have the ability, and responsibility, to turn down submissions that appear "spammy". That is to say, if a team appears to be submitting intel simply because it fits a flag description with no supporting evidence on how it would further an investigation or no explanation of its relevance then the coach will deny those submissions. Our coaches are active participants in the process not bots checking a set of criteria and awarding points.

That doesn't seem like a fair way to run a CTF?

No, it isn't. However, it can be an effective way to crowd source quality intel collection. Please remember that we've tried to gamify the collection process but this isn't a game. The intent of our CTFs isn't to see who can get the most points but instead to collect quality intelligence in an efficient and safe way. Both our teams competing and our coaches working behind the scenes are all human. Our CTFs have a subjective component and this is why communication is key. We encourage our teams to connect with their judges during an event to streamline this process.

I'm a contestant. What can I expect?

  • Your coach should be providing timely feedback. "Timely" can mean different things to different people though. If you feel like your judge has dropped off please message one of the admins.
  • Your coach should be providing clear feedback on the submissions they do not accept.
  • Your coach should demonstrate consistency in their decision making.
  • Your coach's ultimate goal is to help you provide quality intel to support the investigation not to let you get points on a scoreboard. Please keep that in mind.

I'm a coach. What can I expect?

  • You will be provided with a written guide and a video briefing to review. It is expected you will have reviewed these before you begin receiving submissions during the CTF.
  • It is expected that you will have no pre-existing conflicts with the team you are assigned. These conflicts could be either personal or professional. If there is anything that would prevent you from objectively evaluating your team's submissions please reach out to the admins for a team reassignment.
  • Your teams should be clearly articulating the submissions they are making.
  • Your teams should be staying true to the spirit of the competition and not be "spamming" submissions.

If you have any questions or concerns we're always available at contact@tracelabs.org


Tom Hocker

More Posts

RSS Feed