Skip to main content

Criticality Score

This model is based on the implementation of the OpenSSF Criticality Score project to generate a criticality score for each open source project, create a list of critical projects, and use this data to proactively improve the security of these critical projects.

Metrics in the Metrics Model

Creation Time​

  • Definition: The time since the project was created (in months).
  • Weight: 9.523%
  • Threshold: 120

Older projects have a higher chance of being widely used or depended upon by other projects.

Update Time​

  • Definition: The time since the project was last updated (in months).
  • Weight: -9.523%
  • Threshold: 120

Unmaintained projects with no recent commits are less likely to be depended upon.

Contributor Count​

  • Definition: The number of contributors to the project (with commit history).
  • Weight: 19.047%
  • Threshold: 5000

Participation from different contributors indicates the importance of the project.

Organization Count​

  • Definition: The number of different organizations to which the contributors belong.
  • Weight: 9.523%
  • Threshold: 10

Indicates cross-organizational dependencies.

Commit Frequency​

  • Definition: The average number of commits per week over the past year.
  • Weight: 9.523%
  • Threshold: 1000

A higher rate of code changes slightly indicates the importance of the project. It is also more susceptible to vulnerabilities.

Recent Release Count​

  • Definition: The number of releases in the past year.
  • Weight: 4.761%
  • Threshold: 26

Frequent releases indicate user dependency. The weight is lower because this is not always used.

Closed Issue Count​

  • Definition: The number of closed issues in the past 90 days.
  • Weight: 4.761%
  • Threshold: 5000

Indicates high contributor involvement and a focus on closing user issues. The weight is lower because it depends on project contributors.

Updated Issue Count​

  • Definition: The number of updated issues in the past 90 days.
  • Weight: 4.761%
  • Threshold: 5000

Indicates high contributor involvement. The weight is lower because it depends on project contributors.

Comment Frequency​

  • Definition: The average number of comments per issue in the past 90 days.
  • Weight: 9.523%
  • Threshold: 15

Indicates user activity and dependencies.

Dependent Count​

  • Definition: The number of times the project is mentioned by other projects in commit messages.
  • Weight: 19.047%
  • Threshold: 50

Indicates repository usage, often in version updates. This parameter applies to all languages.

Metrics Model Algorithm

Model Weights and Thresholds​

We use the following parameters to calculate the criticality score of an open source project:

Metric Name (Si)Weight (αi)Threshold (Ti)Definition
Creation Since9.523%120The time since the project was created (in months)
Update Since-9.523%120The time since the project was last updated (in months)
Contributor Count19.047%5000The number of contributors to the project (with commit history)
Org Count9.523%10The number of different organizations to which the contributors belong
Commit Frequency9.523%1000The average number of commits per week over the past year
Recent Releases Count4.761%26The number of releases in the past year
Closed Issues Count4.761%5000The number of closed issues in the past 90 days
Updated Issues Count4.761%5000The number of updated issues in the past 90 days
Comment Frequency9.523%15The average number of comments per issue in the past 90 days
Dependents Count19.047%50The number of times the project is mentioned by other projects in commit messages

Score Calculation​

The criticality score of an open source project defines its influence and importance. It is a number between 0 (least critical) and 1 (most critical). The score is based on an algorithm proposed by Rob Pike:

Where:

  • αi is the weight of each metric
  • Si is the actual value of each metric
  • Ti is the maximum threshold for each metric

References

Contributors

Backend​

  • Yehui Wang
  • Shengbao Li

Metrics Model​

  • Yehui Wang
  • Guoqiang Qi
  • Shengbao Li

Copyright © 2023 OSS compass. All Rights Reserved.