In late 2017, a good friend named Eric Byres came to me and said I know you love packets - I have something that would be great exposure for you and your company (which at the time, Atlants Embedded was still consulting). As with everything that my former mentor touches, I was compelled to say yes knowing it would be copious of work, but quite frankly, cutting edge, challenging and interesting. To quote Dale - "it was a high-wire act, I wasn't sure if you could pull it off, but you did." The challenge was far from perfect, and there was flak, but it was a success in the fact we carried out something never before done on that scale publicly. After all, it's just a test, and the horrible truth is: the industry doesn't really have any definitive tests/challenges that demonstrate security claims or independant benchmarks.
It turns out, my intuition was correct regarding the amount of work required: over three months of work to prepare, which I volunteered my time and one of my developers (Nazmul Alam - great guy by the way), and all supporting costs to attend minus the cost of the ticket. All out of my company's pocket and immeasurable support from only support from Eric, John Cusimano, a few contacts in the industry sneaking test bed packet captures, and of course Dale Peterson. There were countless hours spent surveying the packets for a number of sensitive items that ranged far beyond the Company Name, and IP/MAC addresses (and who cares on the latter). And to make things worse, understanding what needed to be changed and approved was only the start of the adventure... For 2019, at least Deloitte Canada is supporting my independent efforts financially this year, & again Digital Bond with donated tickets for myself and a colleague.
When people think anonymizing PCAPs, they think that all it takes is to look for a few NETBIOS names, DNS requests, IP address ranges and MAC address, but to perform a deep clean of these real-world PCAPS so much more is required, and the tooling does NOT exist publically. To find the strings beyond simple string searches, Atlants Embedded had to write a tool to parse large packet captures, but in doing so, we had to quickly recreate a number of protocol dissectors that could handle all of the real-worlds idiosyncrasies.
A simple string replacement exercise quickly became a full development project, complete with a rule-based engine to replace (and re-encode/checksum) particular content in a particular protocol. Tools then, and tools now still do not offer the capabilities needed to perform mass edits, re-encoding, and string searching while maintaining integrity of the original pcap. Nor do they support mass packet weaving and normalisation to hide the butt end of captures being adjoined.
Once Eric and I managed to discover the list of sensitive strings, we set out to replace the sensitive data and stitch together massive packet captures. Given the requirements and limitations of a single data source, we set to work to build a story fit for kings. Every string and IP had an equivalent that matched the story of a fictitious oil company in California; whose true identity was not determined as far as I know. Then we tied together a series of attacks and flags within the traffic - there was an entry point into a privileged domain, traffic that should set off flags, and changes on the OT side of things including anomalous packet behaviours... again this took hours to make coherent sense to some degree. Last minute changes, and merges the night before... all adrenaline!
Then came the actual contest - it was show time! For those who have ran a competition with a whole industry watching, mismatched with skepticism, privacy/NDA nightmares and other shenanigans; they will understand. But to those who have only competing in sports events or events such as hackathons, this event had everything from media coverage, participating vendors, big names and lights - with multiple presentations. And quite frankly, from a technical perspective without a hitch - except a small one on day two were we crashed tcpreplay with a small timestamp bug that was quickly corrected. It's not your standard event.
Unfortunately, with any competition, there are marked winners, losers, competitors better in different avenues, politics, and scoring challenges. Combining this event with the NDA/legal issues, scoring issues reared their heads and some groups thought it wasn't as subjective as it could have been. Perhaps they were right, but I think the key message was is that all competitors had areas where they had strength over each other, but also it was a volunteer event done on a level never done before (by fact, or by the actual asset owner releasing pcaps to us). And of course, we did validate some of the vendors claims, just as I was planning to for 2019 & that takes a strong product (or team) - all were great products and they all had strengths favoring their approach/and founder.
From a product/vendor perspective, the competition ranking could be deemed a "risk" (and I'm sure it is classified as such by some executives), and if I was CTO, I'd still compete and push the envelope of what is possible because in the FOSS/or non-patented world, I'd rather spend my fiscal budget on delivering great features at a pace that exceeds my competition without copying, or needlessly referring to a legal team as a business plan. Entering the competition is a brave move that should be commended, particularly as its an opportunity for product testing and demonstrates commitment to customers/industry; it is literally putting your money where your mouth is.
Now, at S4x19, things are being changed up a bit and many of the lessons learned were applied. The approach and presentations will be different this year, among a few other things (hint, the challenge wasn't related to oil & gas). As I end this article, again, I offer my thanks to the packet owners, event organisers, competitors, community and friends who enable me to chase my passions. I hope S4x2019 to be an even better success and look forward to a follow up article.