1. Messages
  2. Proposal - OpenChain Quality of Conformance Assessment Levels (including a sub-proposal for tooling quality assessment levels)

Re: Proposal - OpenChain Quality of Conformance Assessment Levels (including a sub-proposal for tooling quality assessment levels)

Shane Coughlan
 

All, context slides at link:

During our last call we had some strong push-back in terms of the project itself defining levels of competence. However, we may have an elegant way forward.

If the project defines the context (continual improvement), we can leave the specific implementation to the market with a natural dynamic addressing the “are people just making stuff up?”

For fields like defense, automotive and aerospace, some companies are likely to prefer third-party certification rather than self-certification in the context of procurement. This creates a market opportunity for companies like PwC or Hitachi Solutions to consider the product they offer. Currently it runs along the lines of “OpenChain certified and occasionally audited as per the standard”, but with our signaling for evolution, it makes sense for PwC and Hitachi Solutions to diversify and offered stepped products. This is to their benefit (more product) and to the community benefit (third-party grading if desired). 

While not being intrusive, the project and broader community can signal through official examples and case studies that provide a mental model for where grading may land. For example, this is slide 10 of the current deck:

The most important feedback we received is that no one should feel undervalued for having reached OpenChain ISO 5230 conformance. Adoption of the standard itself is transformative for the ecosystem, and we do not want to dilute that. That said, it is useful to offer inspiration when the question of “what next?” is raised.

On Aug 26, 2021, at 0:52, Jacob Wilson via lists.openchainproject.org <Jacob.Wilson=synopsys.com@...> wrote:

I agree with Steve’s sentiment of wandering tokens, in my opinion levels of assurance with progressive technology independent criteria are required.
Borrowing from other ISO standards, I propose these "levels" outline a 1) goal 2) business case 3) metric. Goal criteria should increment as assurance or level of trust is achieved, while the case and metric move as the technology and supplier landscape change around the business.
 
In the below table I outline one view of this concept:
• Let’s say for example SBoM incorporates distributed ledger technology as discussed in other conversation threads. First the organization would build the business case around adapting SBoM technology to include blockchain, then they would establish a viable metric to measure this business justification. Ultimately the goal would remain intact because it was forward thinking and progressive in level of assurance.
• Now let’s say that an organization is considering the move from internal assessment of OpenChain ISO 5230 to an external 3rd party assessment. They may revise their business case from “Prevent uncertainty through alignment with OpenChain ISO 5230” to “Prevent bias and uncertainty through independent confirmation of OpenChain ISO 5230 compliance”. The Metric remains intact to facilitate the audit process and the goal of dedicating assurance resources remains intact as well.
 
These are a couple of ways we can move along the field while keeping the goal posts intact. Of course I’m always open to feedback, and I realize this doesn’t prevent an organization from collecting artifacts of conformance (silver) before establishing a management framework (bronze). But we can see the crawl, walk, run analogy in the progression in levels of assurance (goals).. Perhaps platinum level is hiring Toni Kroos for 3rd party assessment.
Join main@lists.openchainproject.org to automatically receive all group messages.