How a score is made
Method
Every number on a KwalitAI page comes from a rule stated here.
Score out of 100
The six axes are graded from 1 to 10 and weigh the same. Axes marked N/A are left out. The score, formula k1, is:
score_100 = min(round_half_down(10 × mean of rated axes), 10 × (lowest axis + 2))
The mean is capped by the weakest axis. With five axes at 9 and one at 4, the mean gives 82 but the cap is 10 × (4 + 2) = 60, so the score is 60.
Six grades out of 10 cannot carry 100 levels of precision: one axis moving one step moves the score by about 1.7 points (10 divided by 6). Read small moves against the noise floor below.
When two scores can be compared
- Signed number
- Both evaluations cover the same repository and were made with the same method, identified by its fingerprint.
- Signed number, method approximated
- Both carry the same method label but the fingerprint is unknown, as for historical reports. The number is shown with the words "method approximated".
- method changed
- Any other case. No number is shown, since it would compare unlike things.
Noise floor
Two runs of the same review on the same commit may not give the same grades. A move within the noise floor reads "stable" instead of a signed number.
The noise floor has not been measured yet. A repeatability study, several runs of the same repositories at a fixed commit, will measure it. Until then only an identical score reads stable, and any other move reads "delta withheld", never a signed number.
Finding detectability
Each defect is classed by how it can be caught, so what a tool can catch is never mixed with what only review finds. The review model assigns the class.
- tool-detectable
- A static tool catches this defect, or a rule the review names would. It does not mean the project runs that tool.
- found by review
- A logic defect no static rule catches; tests or a careful reader find it.
- check gaps
- A quality check that is missing, not run, failing or set up too loosely. Counted per check, never added to the defects.
- unclassified
- From a report that predates this classification. Shown only when there are some.
Rows the review marks as not defects (coverage gaps, process notes, resolved leads) are not counted; the page says how many were left out. Public pages show counts only. Finding details stay in the private report.