6 Principles for Cyber Risk Scores — and Why We Need Them
Blog: Enterprise Decision Management Blog
The use of scores that rate a firm’s cybersecurity risk — such as the FICO® Enterprise Security Score — is picking up momentum. In an effort to ensure that these scores consistently add value, and to ensure that they help rather than harm businesses, a group of firms recently convened to develop industry standards for cybersecurity ratings. FICO joined this group, along with several Fortune 500 companies and a number of the country’s biggest banks, and I am proud of the principles we developed.
By creating these principles, we sought to:
- Promote quality and accuracy in the production of security ratings
- Promote fairness in reporting
- Inject best-practice decision management governance standards into a new domain
- Include a coordinated process for adjudicating errors or inaccuracies in reported content
- Establish guidelines for appropriate use and disclosure of the scores and ratings
Why were principles needed? One reason is that there is a potential for cybersecurity ratings to be used in ways that create damage rather than value.
We’ve seen a bit of this in the market already, where firms’ scores were publicly disclosed or compared to advance the marketing goals of the provider. This certainly seems like a bad idea, and one of the key goals of the principles is to ensure that the scores are distributed and used for the right purposes. The right purposes are those that advance actual security and encourage improvement in commercial infrastructure, both within individual firms, and collectively.
Another reason the principles are important is that they encourage quality in the ratings, and an understanding of quality by the users of the ratings. The ability to know which ratings are empirically derived and which are based on judgmental, subjective criteria is critical to knowing what you’re getting, and how you can put it to most effective use. Judgmental rankings may certainly have utility for specific use cases, but they should not pose as empirical scores derived from supervised modeling techniques that are statistically aligned with real, measured outcomes.
As more entities rely on these scores and ratings, their governing bodies and relevant regulatory agencies will care more about how these tools are used to drive decisions to mitigate risk. Establishing appropriate levels of transparency and responsible practices for model governance are equally important. These standards and practices are very well-developed in banking, for instance, but are not yet well understood across other vertical markets. Whether or not these kinds of decision management governance practices are part of the regulatory backdrop for a given user of the scores, establishing them as a best practice now will serve everyone well.
6 Principles for Security Ratings
Here are the principles we adopted as an industry group, and which the US Chamber of Commerce has now published:
- Transparency: Rating companies shall provide sufficient transparency into the methodologies and types of data used to determine their ratings, including information on data origination as requested and when feasible, for customers and rated organizations to understand how ratings are derived. Any rated organization shall be allowed access to their individual rating and the data that impacts a change in their rating.
- Dispute, Correction and Appeal: Rated organizations shall have the right to challenge their rating and provide corrected or clarifying data. Rating companies should have an appeal and dispute resolution process. Disputed ratings should be notated as such until resolved.
- Accuracy and Validation: Ratings should be empirical, data-driven, or notated as expert opinion. Rating companies should provide validation of their rating methodologies and historical performance of their models. Ratings shall promptly reflect the inclusion of corrected information upon validation.
- Model Governance: Prior to making changes to their methodologies and/or data sets, rating companies shall provide reasonable notice to their customers and clearly communicate how announced changes may impact existing ratings.
- Independence: Commercial agreements, or the lack thereof, with rating companies shall not have direct impact on an organization’s rating; any rated organization will be able to see and challenge their rating irrespective of whether they are a customer of the rating company.
- Confidentiality: Information disclosed by a rated organization during the course of a challenged rating or dispute shall be appropriately protected. Rating companies should not publicize an individual organization’s rating. Rating companies shall not provide third parties with sensitive or confidential information on rated organizations that could lead directly to system compromise.
These principles are no stranger to FICO — in fact, we follow the same principles with our FICO® Score, the industry standard for consumer credit risk assessment, as well as other analytics-forward scoring systems and software solutions that drive decisions for thousands of enterprises in banking, insurance, government, retail, telecommunications, logistics and government. They’re the principles that a score needs to follow if it’s going to serve as an industry standard serving all players in the ecosystem — not just a point solution serving its buyers.
Any business that buys the FICO Enterprise Security Score to rate its own operation, or its vendors and partners, can have full confidence that we stand by these principles. We see a big future for cybersecurity scores, and these industry-developed principles are an important step forward.
The post 6 Principles for Cyber Risk Scores — and Why We Need Them appeared first on FICO.
Leave a Comment
You must be logged in to post a comment.
