Measurement is key

Setting up a security awareness program is a regulatory requirement for many organizations. But as it is in many areas of cyber security, being compliant does not necessarily mean that you also reduce existing risks. And using compliance focused metrics like completion rates and training attendance does not necessarily tell you anything about the success of your security awareness program.

The National Institute of Standards and Technology (NIST) published a study on challenges and approaches of measuring the success of awareness programs. Forty-four percent of survey participants responded that “what to measure and how to measure program effectiveness” is very or moderately challenging. Although the study focuses on US Government organizations, the authors stated that their results may be transferable to other sectors and countries. This CISO READ whitepaper contains recommendations about how to set up and manage a modern security awareness program that focuses on reducing risks and how to prove the program’s value to gain full management support.

‍

Security Behaviour Affects Human Cyber Risk

Forrester stated in their most recent report on Security Awareness and Training Solutions: “You need a different way to manage human risk, not better ways to train people.”. Training was and will continue to be a key element of every security awareness program, but in order to effectively manage an organization’s risk, more is necessary. Trainings focus on imparting knowledge. But what your employees know and what they do in a critical situation does not have to be the same thing.

Consider the following non-security example. Everyone knows that smoking or eating too much fast food is not very healthy. But there are still many tobacco companies and fast-food chains on the market. The same is true when it comes to cyber security. Trainings tell people to use a password safe, to keep their desks clean and to report any suspicious email, but we keep seeing major security breaches resulting from for example the usage of simple passwords or very basic phishing attacks. Things that – in theory - should not happen to an employee that took part in a security awareness training. Just as knowing about the consequences of tobacco and fast food does not convince everyone to live healthier, it’s the employees’ behaviour and the organisation’s culture that is the main contributor of human cyber risk.

Trainings alone do not reduce existing risks. Measuring completion rates, training attendance or quiz results do not tell you anything about the success of an awareness program. As part of the NIST study2, managers reported, from their perspective, what was data demonstrating a security awareness program’s value. Not even half of the managers reported that completion rates or training participation do. It is necessary to look beyond training metrics to prove that cyber security budget spending actually affects human cyber risk. And, by the way, these metrics also help managing and improving your program.

Let’s have a look at a simple example; Assume you have a clean desk policy stating that every employee has to remove and securely store all work-related paper before they leave the office in the evening. You create a fancy online training and roll it out to every employee in your organization. After a while your corporate audit team adds this aspect to their onsite checklist and finds out that many employees do not follow the policy. What can you do? Creating just another training or force everyone to re-take the training will probably not change the situation significantly.

To change behaviour, you have to find out WHY people do not follow the policy. There might be many reasons. To name a few; there might be too few cabinets available, the cabinets might be at the other end of the building, or managers do not set a good example and leave their desk untidy themselves. Eliminating the cause of the behaviour will be much more effective to prevent data breaches or other security incidents than forcing people to conduct just another training.

You also need to support the people. They don’t do this on purpose. They are just human. Empathy in each and every component of the program, little nudges from time to time, celebrating the people who follow the policy, all this helps building a positive cyber security culture in the organisation. This is much more effective than trying to threaten or frighten people.

‍

Metrics: What, How and Why?

Metrics are the basis for managing a security awareness program. And the basis for proving its value to the management. If security behaviour and culture is the main contributor to human risk, you must to measure security behaviour and culture. Unfortunately, this is more difficult than just measuring training attendance. So where should you start?

First, we need to understand how exactly human behaviour affects cyber risks. What behaviours contribute to which risk and what interventions have an impact on these behaviours to reduce the risk. SebDB  is an Open-Source project which provides a database of security behaviours maintained by security professionals. It maps security risks like “Account Compromise” or “Malware Infection” with security behaviours like “Using a password manager” or “Reporting security incidents”. Pick one or two risks, check what security behaviours affect these risks and pick 3 or 4 of them as a first step to address. This can already be your first touchpoint with your management (“We want to reduce risk X and Y, that’s why we want to change our people’s behaviours A and B”).

Next step is to measure. If available, you can use data you already have (like a database containing incidents or audit results). In most cases, you will have to ask people directly using a survey or interview approach. You should measure the behaviour itself and, if possible, the aspects that cause the behaviour. This will help you to define the most effective interventions afterwards.

After defining and rolling out your interventions, measure again. This will (hopefully) prove the value of the overall program by making transparent how it reduces the risks for the organisation (“We changed behaviours A and B by … and could therefore reduce risk A and B”).

The following aspects are important to roll out a modern security awareness program.

• Be empathic. People follow recommendations more likely if they are presented without fear or pressure. This would end up in a culture of hiding, which is a bad thing in cyber security.
• Keep your end users engaged continuously. Little nudges from time to time are far more effective than another 2-hour training once a year.
• For end users, a one-stop-shop for everything related to cyber security is the best approach. Don’t confuse your end users with too many tools. People want help, lower the burdens as much as possible.
• Automate routine tasks. For the security teams, life should also be as easy as possible. Data should be available for reporting as you need them. So, they can focus on the complex aspects and the strategy.
• Keep improving. As people have to learn about cyber security, the security team has to learn about the people. Keep asking “what was helpful, what did you like”.

‍

Key Takeaways

The right metrics are the basis for a successful security awareness program. Since security behaviour and culture are the main contributors to human risk, the metrics have to measure security behaviours and culture. If you only measure training participation and quiz results, all your interventions are like shooting into the dark.

Follow these steps to take your awareness program to the next level:

• Familiarize yourself with the basics. Which behaviours affect which risks. SebDB4 can be a starting point here.
• Pick one or two risks to start with. Pick 3 or 4 behaviours contributing to these risks and understand how people in your organisation behave. Use existing data or conduct surveys or interviews.
• If you want to change a particular behaviour, first find out WHY people behave like they do. This helps you to find the most effective interventions.
• Measure behaviour again to prove your interventions’ effect.
• During all these steps, keep in touch with your management team. If you can prove the value of their budget spent, it’s much easier to get their support. Keep in mind that the management teams’ good example is key for changing all people’s behaviour.
• To best support the program, check what technical support you need. Life must be as easy as possible for both the people in your organization and you as part of the security team.
• Keep improving every day.

If you want to have a chat with us about the topic, reach out to us. We are happy to help improving your program and come up with ideas.