Privacy Risk Quantification: How to Develop the Right Scoring Methodology

In the article “Privacy Risk Quantification: How and Why to Do It Effectively,” we went over the basics and best practices of privacy risk quantification and a few relevant industry benchmark frameworks. If you are implementing privacy risk quantification at your organization, one of the most important steps is tailoring a scoring methodology to align with your business.  

Tailoring what you track and score provides a more accurate depiction of risk at your organization and can help align the privacy team's goals with broader business objectives.  

Scoring: Keep it Simple  

You may be tempted to get granular in your methodology. Resist that urge. While complex statistics and graphical representations are more impressive, they are far less easily understood by a large audience.  

Simple math means easier explanations. A range between whole numbers is a common method because they are easy to compute and understand. Also consider the width of that range. When tailoring your scoring, consider this: Will a Customer Impact category with a 1-to-10 range convey the point any more effectively than a category with a 1-to-4 range? Consider whether 10 unique levels provide the right amount of granularity or too much.  

What to Consider in Your Scoring Methodology 

Variable selection should be grounded in the most important impacts for your organization and the person whose data is being processed. Businesses view PII as an asset while consumers perceive it as a cost. This process is a balancing act. We want to align with business considerations, but don’t want to lose sight of the personal impact and vice versa.  

Extent of Processing 

Extent of processing relates to the quantity and sensitivity of the personal information you are processing about a data subject. For example, basic contact details would fall into a lower risk category, while personal health information would be one of the highest risk categories. It’s also a highly subjective variable and is a great opportunity to show that your methodology is specifically designed for your organization. This value is created only when it is closely aligned with how your organization processes personal information. Without this alignment, your risk scoring could be construed as too generic and be dismissed by your organization’s stakeholders.  

The quantity component of this variable speaks to the number of data types you process about a data subject. Mirroring the previous example, basic contact information that is limited to an email address is less risky than having multiple data points, like their name, physical address, and phone number.  

Customer Expectations  

Consumer privacy awareness is growing, but it isn’t ubiquitous. Two underlying barriers to increased awareness are knowledge and time. We don’t analyze every facet of how each company we interact with processes our data. (Nor should we.)  

So, when tailoring your methodology, it is helpful to incorporate the reasonable expectations a consumer has about how your organization processes their data. For example, a consumer using the services of a bank would expect them to process their financial information, but less so their location.  

Customer Impact  

The goal of calculating customer impact is to determine the personal cost of an infringement of confidentiality, integrity, and availability of the personal information your organization is processing. This is closely tied to the sensitivity and quantity of data being processed. No one wants their contact information to end up on the dark web, but this consequence would not have the same impact as their social security number or personal health information being leaked.  

A helpful way to think about this variable is to consider what you would have to do in response to your data being breached. For example, if your email is leaked, you may have to be more vigilant of incoming emails, while having your social security number exposed would require you to sign up for an identity theft prevention service.  

It may be helpful to review your organization's previous responses to PII breaches. If none have occurred, other companies that experienced a major breach often have their response to consumers publicly available.  

Probability  

In privacy risk quantification, we need to define the likelihood that a negative event will occur. Incorporating a probability estimation helps an organization allocate resources to the likeliest risks.  

There are two common ways to estimate the probability of privacy risk having a negative outcome. The first uses historical data from previous projects, or publicly available data to calculate: 

𝑃𝑟𝑜𝑏𝑎𝑏𝑖𝑙𝑖𝑡𝑦(𝐸𝑣𝑒𝑛𝑡) = 𝑈𝑛𝑓𝑎𝑣𝑜𝑟𝑎𝑏𝑙𝑒 𝑂𝑢𝑡𝑐𝑜𝑚𝑒𝑠/𝑇𝑜𝑡𝑎𝑙 𝑂𝑢𝑡𝑐𝑜𝑚𝑒𝑠 = 𝑥/𝑛

The second method is a qualitative approach relying on the privacy professionals’ judgment to select a likelihood from several options:  

• Will almost never occur  

• Will rarely occur  

• Will sometimes occur  

• Will frequently occur  

• Certain or almost certain to occur  

Timeline  

Providing an actionable timeline not only helps the audience understand the severity of the risk but also sets an accountable roadmap for mitigation. When considering what timetables should be used, you need to account for how fast your organization moves, and its agility.  

A smaller company can more likely implement changes faster than a larger organization that involves more people, more processes, and so on. An important note: Slower organizational movement should not be used to discount cases where quick remediation is essential. This can occur when bureaucratic safeguards are used to deflect risk mitigation efforts.   

For example, a significant privacy risk is identified in your organization’s product that requires engineering to fix within the month. However, the product manager is pushing back on addressing this issue because product planning is done on a quarterly cadence and does not allow for ad hoc changes. As a privacy professional, you will need to advocate for timely risk mitigation.  

Your Risk Severity Matrix 

Once you assess all these factors in the context of your own organization, you will be able to define a risk severity matrix that looks something like this: 

Example: Risk Scoring Matrix 

 A tailored matrix with clearly defined risk levels will become invaluable to your team and your organization in quantifying privacy risk quickly and consistently. Without it, privacy risk scores can all too quickly become an abstract theory. But looking at risk in a concrete way with quantification is not necessarily impersonal: The best risk quantification happens when you put yourself in the shoes of the person whose personal information is in the hands of your organization.  

When creating your quantification scale, remember: Perfection is the enemy of good. Any new process will have its own quirks and issues that can only be addressed through ongoing improvement. Our best advice is to get started, get something out there, then iterate along the way as you road test the system. Good luck! 

Learn More About Privacy and Risk Management 

To learn more about how to assess and quantify privacy risk at your organization, schedule a one-hour consultation with our experts at Osano.