Embedding Threat Modelling in the DevOps Lifecycle (Part 1: Backlog Management)

If you’re part of this community, I’m sure you don’t have to be convinced of the huge value that threat modelling can bring to teams and how it helps create better and more secure software.

However, I’ve often seen through my employee and consultant career that HOW threat modelling outputs are managed in the organisation is often at odds with the organisation dynamics, making it hard for threat modelling to “stick” as a repeatable organisational practice.

This will be a two-part series, in which we’ll first talk about backlog management practices and how they can make or break threat modelling, and in the second part, we’ll talk about the often problematic relationship with risk management practices which already exist.

In this blog, I’ll talk about a few patterns I’ve seen that can often have a negative impact on your threat modelling activities, even though they sit outside of it, in how they become integrated, or not, with backlog management practices by the engineering teams.

• Failure to build relationships and negotiate success with Product owners
• Inability to aggregate results across the organisation and prove its value
• Failure to connect to the compliance framework and objectives

Failure to build relationships and negotiate success with Product owners

I’ve seen many programmes where in the haste to prove its value, threat modelling practices are overly focused on the Engineering teams, starting with great and comprehensive training programmes to teach how to perform a particular framework. Having experienced it, it tends to be short-lived or become something done with little engagement. And often a big  contributing factor to that lack of engagement, is that not enough effort was put in understanding the organisational dynamics around what gets done or how prioritisations or re-prioritisations are managed. I’d highly advise interviewing and building great relationships with Product Owners to understand how bugs are managed, previous instances of re-prioritisations and how they came to be and how the teams handled them. This should give you some insight into how security will (likely) either succeed or fail.

This often means you need to pay attention to Agile / Scrum practices and if they consider that threat modelling is something they should pay attention to. What gets done when using these practices tends to be defined in “Definition of Ready” and “Definition of Done”.

I often propose that “Definition of Ready” should establish the criteria by when threat modelling is actually required, and it should be negotiated, not imposed. For instance, only new types of data processing, or significant changes to architecture should warrant a threat modelling session to be required, so the normal activities performed by teams which don’t introduce any new patterns aren't unburdened by the extra effort. If you expect threat modelling to happen all the time, pretty quickly it tends not to be done at all.

Inability to aggregate results across the organisation and prove its value

If we bring it up a level from Product teams, another challenge I’ve seen which contributes to the lack of success of threat modelling in organisations is the security team not focusing on acquiring the ability to have a 10.000 foot view of how threat modelling is happening across the organisation. To do this, and if using the existing tooling that teams tend to use for backlog management, like JIRA. It means you should find a way to use metadata that you can use to query and make sense of how much threat modelling is actually happening, and what tickets are being identified that mitigate identified threats. I like to say that if the output of a threat modelling session wasn’t tickets in a team backlog, then what you did was a nice chat, not a security practice.

There are 3 ways in which I’ve seen this attempted, both with pros and cons

• New issue type or project
• Using labels
• Using fields

Approach

Pros

Cons

New issue type or project

Better flexibility and be more prescriptive in categorisation for vulnerability management or risk management

Easier to aggregate reporting

It’s imposing on Engineering workflows

May reduce sense of ownership for those items by Engineering

Using labels

Very easy to start

Allows for most flexibility and evolution of categorisation as practice evolves

Bulk retrospective changes are easy, as you evolve your threat modelling practice

Very easy to mislabel, skewing results

Harder to ensure consistency

High dependency on team members remembering to do it

Using fields (within existing projects)

Using fields with drop-downs makes for an easier user experience*

Easier to get the categorisations maintained centrally

You need to convince teams to change their projects

* for instance, if you decide to create drop-downs relating to areas of ASVS

I would generally advise against creating new projects, even issue  types, as they tend to reduce the sense of ownership by Engineering. Using fields is generally my preferred option.

This will then allow to aggregate reporting easily, using for instance dynamic Confluence pages querying the JIRA data, that you can use to aggregate results and not only see the status of threat modelling activities across the organisation in real time, but be able to show Senior Management that it’s an effective practice which is identifying and mitigating threats.

Failure to connect to the compliance framework and objectives

Another challenge I’ve seen, is the dark-side of bottom-up adoption of threat modelling, which is that if there aren’t good relationships between Compliance and Engineering, the practices will be done in isolation without real benefit or integration to the ISMS (Information Security Management system). This often leads to the security engineering benefits brought by threat modelling, to be disconnected from the Compliance programme and as such pose no actual benefit to Compliance, which is crazy, but seen it more than once. Sometimes, yes, I agree with what you’re thinking, if you bring Compliance to co-design they’ll just overcomplicate it. But it’s a relationship worth putting some effort in, for the mutual benefits it can bring to the organisation as a whole. Ideas such as risk-informed threat modelling, where teams perform dedicated threat modelling sessions to explore mitigations for identified risks, are a good way to start bringing the teams together. This bringing together can also be thought about from a categorisation perspective, for instance, with drop-downs to ISO 27001 mappings or any other relevant frameworks. You’ll certainly need to discuss who’s accountable to do what and when, but that’s your contextual challenge that you need to figure out what works, maybe even experiment.

Did you ever see these effects in your own organisation? Being able to spot a pattern or organisation dynamics and how they interplay to make work successful, is something that those wishing to embed threat modelling shouldn’t overlook if we’re aiming for success.

If you’ve tried implementing threat modelling in the past and couldn’t, and before thinking that threat modelling failed you, consider first that you may have failed threat modelling too.

‍