Threat Hunting Team Maturity Model

After talking about scaling a hunt team with the concept of "Pack Hunting" in my last post, now I want to approach a topic that has been weighing on my mind as the year comes to a close.

How do I build a kickass hunt program? And how would I know if I did?

Admittedly, this is on my mind because next year my program will have been around long enough to be audited, which means I need to be able to measure my program in an audit-able way. I am not talking about the outcome of the program here, the hunting itself, but rather the processes and procedures that lead to the outcomes. The outcomes are currently measured using KPIs, but I haven't stress tested those enough to share yet.

Developing a Threat Hunting & Research  Team Maturity Model

Why a maturity model?

As I looked into how to approach this question I came across the idea of using a maturity model. According to the Institute of Internal Auditors (IIA), a maturity model describes process components that are believed to lead to better outputs and better outcomes. Maturity models are perfect for highlighting continuous process improvement, which makes them well suited for assessing a program that is currently still under development. It also provides an easy starting place for auditors of a new and developing field like threat hunting.

How do you design a maturity model?

To learn more about the necessary components of a maturity model, I read "What makes a useful maturity model? A framework of general design principles for maturity models and its demonstration in business process management" by Jens Pöppelbuß and Maximilian Röglinger.

The authors describe a framework of three design principles that build upon one another based on the purpose of the maturity model.

Relying on their framework, I've decided to try my maturity model at the basic level for now and seek to advance it in the future to descriptive and later prescriptive. I am trying to make sure that the final product can indeed hold up to an audit, so I am purposefully limiting this first attempt.

What goes into a threat hunting maturity model?

The dimensions that make up a model are usually levels, likely 0-5, and attributes or components.

I started with the attributes first. What was needed for a world class hunt program? The answer came pretty close to the cliche - "People, Technology, Process"

1. Talent - I am going to need to be able to recruit, retain and train hunters and huntresses.
2. Data - Not quite technology, but close. The idea here, I don't want my team tied to specific technology, but we will need specific data to accomplish our mission.
3. Methodology - Our process and procedures used to complete hunts.
4. Metrics - Still have to measure outcome of a program consistently and accurately, so can't leave this one out.

Then I rummaged the internet for some good names for my levels, which for me had to start with 0. For labels like these, I suggest just finding words you believe will resonate with your leadership.

• Level 0 - Initial
• Level 1 - Reactive
• Level 2 - Defined
• Level 3 - Repeatable
• Level 4 - Integrated
• Level 5 - Optimized

I purposefully tried to make my model from 0-5. I felt it would help make the leap from one level to the next more manageable, but if you can't come up with enough in your model to fill it in then just reduce the levels.

Next came the hard part, filling in the model. Full disclosure - I spent days staring at an empty table then switching tasks when I would come up empty handed, this part was a struggle for me.

To move forward, I started out adding characteristics of the attributes I thought were indicators of a kickass program.

Talent

For talent, I decided on a few areas I thought showed robust talent management.

• Recruting
• Training
• Career Development Paths
• Roles and Responsibilities
• Succession Planning

Data

Data needed completely different characteristics.

• Visibility
• Quality
• Automation
• Tools
• Data Science Integration

I want to briefly explain what I mean by visibility and quality as for my team they have specific meanings.

• Visibility is a combination of coverage, retention, and reliability of the data sources. 
• Quality includes the usability, normalization and standardization of the data. 

Right now the team is testing some equations for these characteristics, but that wouldn't be needed until the descriptive use phase so I will circle back to those another time.

Methodology

This one was a little easier to poach ideas from pre-existing process maturity models.

• Consistency
• Documentation
• Communication
• Relevancy

Metrics

Plenty of models incorporate metrics, so find what includes the characteristics your management and auditors care about and tailor it.

• Consistent reporting
• Accuracy

https://github.com/Kathayra/threathuntingmaturitymodel/blob/master/HuntProgramMaturityModel.pdf

Building your own Threat Hunting & Research Team Maturity Model

Chances are this model isn't going to be perfect for your team. So I have uploaded the slide for you to easily edit it.

I also created a worksheet of the questions I used when creating the example above. - 6 Questions to Guide your Maturity Model Development.

One last note, if you are considering building your own model to assist in future audits be sure to document your model development process, your references, and resources. This documentation will be instrumental in showing auditors the thought process behind your model in the absence of an industry recognized one.

Happy Hunting,
K