AIS-03: Are technical and operational metrics defined and implemented according to
business objectives, security requirements, and compliance obligations?
Actionable metrics should be defined with consideration to business goals, the criticality of service, security requirements, and compliance obligations.
Example technical metrics include: • Count or percentage of vulnerabilities by weakness. • Count or percentage of vulnerabilities by severity. • Count or percentage of vulnerabilities by detection source (design review, code review, SAST, DAST, penetration test, VDP, or bug bounty). • Count or percentage of vulnerabilities by environment detected (pre-production vs. production). • Average time to resolution. • Count exceeding remediation service level objectives (SLOs).
Example operational metrics include: • Count or percentage of applications using automated security testing by test type (SAST, DAST, SCA). • Count or percentage of applications have completed penetration testing in the last “n” months. • Count or percentage of development teams or individuals who have completed application security training in the last “n” months. • Count of proactive engagements by development and business teams. • Results from surveys delivered to application security customers, such as business and development teams.
Reporting: Reporting should be designed with various users in mind. For example, security professionals, engineering teams, business stakeholders, and executives will often have different interests requiring specialized views, filtering, and delivery mechanisms.
a. The collection, visualization, and distribution of reporting data should be automated. b. Data may be further analyzed using application criticality, business units, platforms, languages, and other factors relevant to the viewer. c. Compare actual metrics to standards to evaluate performance. d. Enable comparisons over time to identify trends. e. Enable correlations, such as relating a reduction in vulnerabilities of a specific type after new tools or training.
Control implemented
Control ownership
Description
Define and implement technical and operational metrics in alignment
with business objectives, security requirements, and compliance obligations.