Q&A with Dave Soldera on High Assurance Threat Modeling

A round of applause for Dave Soldera, who delivered an exceptional presentation at our Community Meetup last month! We'd also like to extend a heartfelt shoutout to Robin Ninan for his engaging facilitation, ensuring the conversation flowed seamlessly.

We received an abundance of questions during the Q&A. While we couldn't address all of them due to time constraints, fear not! Get ready as Dave takes on these unanswered questions right here:

• Q: How much time do you typically spend for threat modeling an average-size app/service?
• Q: At what point in time during threat modeling process, would you recommend reviewing the effectiveness and value of the “consistency” and modify it for better results?
• Q: How do you avoid duplicating efforts between the data privacy impact assessments and threat modelling not looking for similar information at different stages SSDLC?
• Q: In your journey towards how you ask for information and attempting to have a repeatable process, how have you evolved how information be formatted and contain specific detail?
• Q: Beyond your Confluence page, do you have a TM report template that you follow?

Q: How much time do you typically spend for threat modeling an average-size app/service?

This is a great question, to me it’s really asking ‘how much time should you spend on a threat model?’, and that’s really tricky to answer.  All things being equal, I think you should get a dev team to allocate about 2 weeks for a single person to complete a threat model.  The allocated dev doesn’t need to spend 100% of their time during that 2 weeks threat modelling, but that should be enough time to have a couple of meetings to kick-off the process, with a few check-in meetings, and time enough for them to gather any information they need.  It’s nice that making it about 2 weeks is also about the unit of time (for some a sprint) that dev teams like to break-up their work into, so it will hopefully fit their scheduling as well.

Naturally you’d think the answer would depend on the size of the system being threat modelled, but I would try to (and it’s not always possible) to scope the threat model so it could be done in 2 weeks.  If you need to threat model other things, bring those into scope (or create new threat models) 2 weeks at a time.

Of course this is just guidance and it certainly isn’t always possible, but I think this is the ideal.  But remember, if you have to spend longer threat modelling because the system can’t easily be broken into smaller pieces, then give yourself credit for more than 1 threat model!  It’s misleading to complete a huge threat model and have your metrics only count it as one!

Q: At what point in time during threat modeling process, would you recommend reviewing the effectiveness and value of the “consistency” and modify it for better results?

It’s certainly something you do towards the end of the process of creating a threat model.  During the initial and middle parts of the process a lot of information will be continually added as knowledge pours out of someone’s head into the threat model, and that’s a good thing that we don’t want to inhibit by demanding consistency.  But as the rate of information being added to the threat model slows, towards the end, then that’s the best time to start ensuring the information that’s provided is consistent.  Moreover, I would incorporate a consistency check into your ‘definition of done’ for a threat model, so it’s not complete until it’s been reviewed for any inconsistencies and those have been fixed.

Q: How do you avoid duplicating efforts between the data privacy impact assessments and threat modelling not looking for similar information at different stages SSDLC?

I think threat modelling and a PIA can be done at similar stages of the SSDLC, usually after some first version of the system has been complete (I’m really talking about a small part of the total system that’s a good size to threat model).  Early enough that you can influence the design but not so early that the devs can’t provide the required detail about the design.

The template I use for threat modelling captures information about the assets flowing through the system and that would always specifically call out any personal data being processed.  That’s often very useful information for the PIA.  I would suggest looking into changing your threat model template to gather information in such a way that any assets captured that are personal data are captured in a format where they can easily be copied (i.e. cut & paste) into your PIA.  Either that, or alter the PIA template to suit the threat model format.

I would be careful to not get into the detail of the PIA in the threat model itself, as that could be a burden on those teams that might not have personal data.  Although an optional section in the threat model that only needs to be populated by those systems processing personal data may also be an option.

Lastly, if it’s possible to benefit from any automation in moving data from a threat model to a PIA, then that would be a great solution.

Q: In your journey towards how you ask for information and attempting to have a repeatable process, how have you evolved how information be formatted and contain specific detail?

This is a great question because I’m a big fan of extracting out the information captured in a threat model so it can be used to make better decisions and improve the process.  That only works if you can capture data in a similar way across numerous threat models.

I can’t say I’ve solved this problem, but I use these approaches:

• Allowing others to read and copy the contents of other threat models helps content stay in a similar format - just make sure they are copying from something that’s formatted correctly!
• Pre-populate content in your threat model template so people have an example of what it should look like.
• Have lots of help documentation on how to populate the threat model, both as an external docs site, and inline help tips (also linking to the external docs site).
• Use tooling where possible to verify that certain answers in the template are correctly formatted.

If all else fails, sometimes you just have to go through a threat model and correct the formatting so you can have clean data - it’s worth it when you know others will copy the content and the alternative is folks copying badly formatted data.

Q: Beyond your Confluence page, do you have a TM report template that you follow?

I linked to this in the slides, you can find a Google Doc version of my latest template here http://tinyurl.com/threat-model-template.  A full description of the template contents can be found starting here https://threatware.readthedocs.io/en/main/create/template.html.  Happy to answer any follow up questions you have about the template.