Data Privacy Metrics: Questions From Our Webinar

In our webinar, "Data Privacy Metrics: Are You Measuring Your Privacy Program's Success," we hosted a Q&A session for our attendees. These questions were valuable insights into some of the struggles that privacy professionals are running into, and we thought some of the answers might be helpful for you on your own privacy journey! 

If you haven't checked it out yet, download the webinar at the link above and then come back to see if any of your own questions were answered! 

What kind of tools have you seen customers use to get privacy metrics? How can Osano help?

Skye McCullough: Yeah, of course. I'll take some of that and then lead it over to Rachel. We've seen a good handful of tools in use, right? We generally see customers who necessitate some sort of tooling to help with automating DPIAs, RoPAs, internal assessments, assessments that they are sending or receiving from vendors that they're working with if they are doing their due diligence for when vendors come on.

So, we see that all the time. That is something Osano can help with. On the RoPA, DPIA and vendor assessment front. But we see that a lot. We see a lot of external evaluation tools being utilized.  There's obviously consent management that allows you to manage browser-based consent as well as universal consent, marketing-based consent, consent to be communicated with, consent to SMS messages, consent to be sent emails in the future, right?

We see a lot on the consent management side and getting data on those opt-in and opt-out rates. We see, as Rachel mentioned before, monitoring DSARs or data subject rights requests as they come in is a big need for tooling. We see a big need for tooling there to help at least manage or report on those requests and kind of keep them managed in an appropriate workflow. Customers get a lot of them through email or through various emails to different people. And there is a legal obligation to respond to those within a period of time. So having those kind of all together and being able to report on those as they come in. How long it takes you to complete them, not just for regulatory purposes, but also for internal headcount and planning purposes, is going to be really important. So having somebody to manage those as well.

A lot of those that I mentioned Osano can support, but I'll let Rachel touch on any others that she's seen in the market.

Rachael Ormiston: Yeah, I was going to say, so we use the Osano tooling as well internally for the reasons that Skye's mentioned. It gives us those insights, and I think just outside of the privacy sphere where you're thinking about other departments that you can potentially get metrics from.

I think there's definitely a great number of tools out there when you think about how to be creative and get those insights you need. So maybe you're working with your security team and they've got a phishing program in place. Maybe you can get those phishing metrics. So not necessarily a direct privacy tool, but just understanding how you can bring your privacy program to other parts of the business by looking to see what data they have there. 

I think something like phishing metrics is great because it maybe indicates some training that you can do. It maybe indicates where you need to refine your RoPAs or your data mapping because maybe there's a gap in understanding. So, some really great tools out there by connecting with other parts of the business for sure.

For companies in the B2B space that don't interact with customers, are there different priorities or recommendations vs. B2C companies? 

Rachael Ormiston: Sure, so I can take this, and Skye, feel free to chime in with anything you've got, but I think just knowing that you're B2B, probably the type of communication is going to be slightly different.

I think in a B2C context, you're probably thinking about how you can make things a little bit easier to understand, a little bit more accessible in a B2B context, or maybe not looking at it in quite the same way. So, the priorities might be different. I think ultimately it probably comes down to, again, understanding what your priorities are and what your customer expectations are, and making sure that people are getting the information they need and are understood.

But I think, yes, there will be nuances, but maybe the goals might be the same. It's just probably how you're executing them might be slightly different.

Do you recommend surveying customers about their direct impressions regarding privacy and trustworthiness, or should you simply monitor metrics related to customer behavior?

Rachael Ormiston: Oh, that's a good one. I think it's important to find out how customers perceive you. I think it's really good to do that market analysis around what signals trust within your industry. So, I think, even if you're not wanting to get insights from your customers, I think you definitely want to make sure you're tuned into the market that your customers are in.

I think that can certainly help you bolster your program and mature it by actually delivering and meeting their expectations. I think we've spoken a lot about compliance today, but actually, I think what customers are seeing is not just about being compliant. It's about delivering to your customer's expectations and that will certainly vary in the industry.

But trust will be a huge part of that and making sure you can address that is going to be key for some of those relationships. Skye, what do you think?

Skye McCullough: Yeah, I think you and I were having this conversation about trustworthiness when we were exploring trustworthy brands from the B2C side and trustworthy brands from the B2B side and what is included in that trust.

What customers perceive as included in trustworthiness for a brand and some of it is going to be public perception. So, keeping a monitor on your TrustRadius and your G2 reviews and general customer sentiment is going to give you a good insight into that. Do I think that actually surveying our customers on just our privacy standpoint is going to be a useful data point?

I don't know if I would agree with that entirely. But I think, to Rachael's point, using it as a piece of the puzzle rather than the entirety of the puzzle. Like when we're talking about trustworthiness, we're talking about a lot more than just your privacy stance. I think that's probably more of an accurate representation of customer perception.

Are there buckets of metrics that are common across most privacy programs?

Rachael Ormiston: I'm not sure if there are necessarily common buckets. I know that there are various reports out there. I think the Future of Privacy Forum has a metrics report that's got some sample metrics.

I think probably the common ones I would see are just getting started on how many. I think that's probably the volume-based metrics. How many times am I seeing this come up? How many requests am I receiving? How many impact assessments am I doing? I think, right now, there's a lot of privacy pros that are trying to make the case for additional support as their privacy programs are growing.

And I think that's probably where the matrix for volume is certainly very important. But yeah, I'm curious, Skye, what do you see on the customer side?

Skye McCullough: Yeah, I think that's a really good, good take. The first thing customers start with us or mention that they are tracking with us is a count — a count of how many DPIAs, a count of how many times they have been asked for a SOC2 when they are putting a bid out for a customer account, of how many people have opted into the category of marketing. And that might be from an overall privacy perspective or from a marketing-specific perspective. But to Rachael's point, it really starts out as simply as let's count what we're getting, and then we can associate this count with load and effort, and then we can actually start being critical thinkers about the data that we're getting back and expounding upon that.

What are some meaningful Privacy by Design metrics?

Rachael Ormiston: I think it could be as simple as how many, again, like the volume-based ones, how many impact assessments are you doing?

It could be around how many checklists are you completing? Maybe if you're building a product, you have a checklist before you go to market and you're getting ready to launch. It could also be looking at the number of controls or features that you have. So, I think it depends on how you're using privacy by design. Is it product-based? Is it internal for process-driven and design? But I think it could probably be volume-based to start with, and then it could get more mature to think about more granularity of the controls. But that's a great question.

Skye McCullough: Yeah, we do see some LMS monitoring too, Rachael. I think that talking about privacy by design around, "How many courses that are security- or privacy-focused are our new employees required to take on a yearly basis?"

What are the outcomes of those? And are we making adjustments to solve for gaps in those trainings? So, to Rachael's point, it really depends on where the focus is. Is it on the product that you're offering? Is it on a feature set? Is it on a service? Is it on the people that are providing that service?

How can a company ensure the privacy metrics established are the right ones that accurately reflect the state of its privacy program?

Rachael Ormiston: It's probably going to be a bit of trial and error, to be honest. I think as you're starting out, you're going to find that some resonate more than others.

It's not an exact science. You're probably going to have to try some and see where you get the most support and try to deliver that way. I think the most important thing is that you're really trying to give an accurate representation of your privacy program.

The idea of metrics is to help you articulate where you need support, where you need buy-in, where you're doing well, where you're not doing well, etc. So as your program changes and as your program evolves, you're probably going to see that your metrics will change over time as well.

At least, that's what I would have been imagining, but Skye, what are your thoughts from the customer side as well?

Skye McCullough: Yeah, I would say it's very similar to a lot of other metrics that we track internally, right? There are two times that I'm tracking metrics: One is to monitor the state of my current organization to make sure it's operating at a particular level, and the other is when I want something.

My boss is always very much a "don't come to me and ask for something if you can't tell me why you want it." And he is also very much, "I would also like some numbers or at least some quantitative evidence to back that up." It can't just be qualitative. So, we can't just talk about it. I need to know why you want it.

So, I think that a really good point is: We found that certain metrics don't matter. Certain metrics that we were tracking for two years didn't actually matter. It didn't actually affect the customer experience in any way. So, we sunset them after two years and replaced them with something that we think would matter more as our organization grew and as we became solidified in that organization.

So, to Rachael's point, it is going to be a lot of trial and error, and you're going to measure the wrong ones first, and then you'll adjust and grow and pivot. But when you're asking for something like headcount, like budget, like new tools, that's when metrics really, really shine in an organization alongside the monitoring our current posture to make sure we're executing at a particular level.

If you're early on in establishing a privacy program, do you recommend focusing first on metrics for internal use/buy in?

Rachael Ormiston: That's a great question, and it's probably going to be an "it depends" answer for me because I think it's probably going to be dependent on what the challenges you're experiencing are. I think the preference would probably be for those internal metrics, probably for the reasons around volume and getting your program started and making sure that you're addressing compliance concerns and you're able to facilitate that organic growth. But it could be that, depending on where you are, some of those risks that you're seeing and you're wanting to highlight are externally driven as well.

So, it's going to depend on your stakeholders and what your risks are, and ultimately, what your goals are, but I think starting off with internal metrics from a compliance perspective and to help build up that. That foundational element of volume is very important, but it could certainly be a combination of both for some organizations and still be the right approach to take.

Privacy metrics are an important piece of the puzzle when it comes to ensuring the success of your organization's privacy program, but it's not the only piece. If you're interested in learning more about how privacy metrics can help your organization, check out the Osano Privacy Program Maturity Model to dive into how to measure and assess your organization's privacy maturity!