What happened
Mariia Bakhtina, a research specialist at Masaryk University, explained to Help Net Security what a Common Criteria or FIPS 140 certificate actually promises, and where procurement teams read more into the document than it supports.
The central finding is about time. In her team's 2024 analysis, roughly six in ten vulnerabilities matched to certified products, smartcards aside, were disclosed after the certification date. A certificate records an evaluation that happened once, against a fixed configuration.
Bakhtina is precise about the claim. A Common Criteria certificate means an accredited lab evaluated a defined product, in a defined configuration, against security claims the vendor wrote in a Security Target, often based on a Protection Profile. The evaluation followed a process at a stated assurance level. The promise covers the claims and the method, not the product being secure in general. FIPS 140 validates a cryptographic module within its boundary and says nothing about the system around it.
Four assumptions fail on that reading. "No vulnerabilities" is not what the document says, and ROCA, Minerva and TPM-Fail were all found in certified products, some at high assurance levels. Full coverage is not guaranteed: her team found a certified network camera whose Security Target assumed the network interface was trusted, so it was likely never tested there. Nor is the version bought necessarily the version tested, since a certificate applies to a fixed configuration and usually expires after about five years. A higher assurance level means more thorough examination, and her data showed less severe vulnerabilities, but not noticeably fewer.
Her lab's response is sec-certs, an open source tool that collects public certification documents weekly, extracts the key data, links certificates to known vulnerabilities, and records which certificates reference each other. She is candid about the limits: the matching runs at roughly 89% precision, so the results are indicative and need verification.
Why this is a GRC story
Certification is evidence, not assurance. A badge answers whether something was evaluated. It leaves open whether the control you need is inside that evaluation. The gap between them is where procurement risk lives.
Third party risk starts with the scope clause. Assumptions and exclusions in a Security Target read like fine print and behave like the boundary of the guarantee. If the function you are buying sits outside the evaluated configuration, the certificate describes a different product.
Certificates age silently. Five year validity and no automatic alert means a folder of valid looking documents can describe a product three versions old. Keeping the certificate, its version and its expiry in the vendor record makes that a reportable fact.
Matching a certificate to a CVE is a control. That public linkage is only about 89% accurate is an argument for doing the reconciliation yourself. Someone has to own what has been published against the products on the register.
What to watch
Watch how searchable certification data changes vendor behaviour. Bakhtina notes vendors welcome a page supporting their trust claim and dislike the same page listing potential vulnerabilities.
The action is small. Before signing, read the Security Target scope for the exact version delivered, then ask the vendor where the security function you rely on is covered. If the answer is that it sits outside scope, the assessment has done its job before the contract did.
Attribution: Analysis based on Help Net Security and related public reporting. This article is original commentary, not a repost of the source material.