Re: Proposal - OpenChain Quality of Conformance Assessment Levels (including a sub-proposal for tooling quality assessment levels)


Christopher Wood
 

Hello all
This is a great discussion.  I do like Jacob's suggestion with some simple notation that indicates that the "levels" or whatever you call them are cumulative going from the base (Level 1 up through Level 4).  I also believe that formal adoption of each level's requirements will become necessary for any "vendor or supplier" due to market demands and agree with Shane that they should probably not be dictated by the OpenChain Project.  To show that the additive features of the Chart 10 could be added to an inverted stack of the levels. (Level 1 is foundational to the concept and building on Level 1 gives you advanced capabilities as a 4th column.



On Wednesday, August 25, 2021, 10:52:57 AM CDT, Jacob Wilson via lists.openchainproject.org <jacob.wilson=synopsys.com@...> wrote:


I agree with Steve’s sentiment of wandering tokens, in my opinion levels of assurance with progressive technology independent criteria are required.

Borrowing from other ISO standards, I propose these "levels" outline a 1) goal 2) business case 3) metric. Goal criteria should increment as assurance or level of trust is achieved, while the case and metric move as the technology and supplier landscape change around the business.

 

In the below table I outline one view of this concept:

  • Let’s say for example SBoM incorporates distributed ledger technology as discussed in other conversation threads. First the organization would build the business case around adapting SBoM technology to include blockchain, then they would establish a viable metric to measure this business justification. Ultimately the goal would remain intact because it was forward thinking and progressive in level of assurance.
  • Now let’s say that an organization is considering the move from internal assessment of OpenChain ISO 5230 to an external 3rd party assessment. They may revise their business case from “Prevent uncertainty through alignment with OpenChain ISO 5230” to “Prevent bias and uncertainty through independent confirmation of OpenChain ISO 5230 compliance”. The Metric remains intact to facilitate the audit process and the goal of dedicating assurance resources remains intact as well.

 

These are a couple of ways we can move along the field while keeping the goal posts intact. Of course I’m always open to feedback, and I realize this doesn’t prevent an organization from collecting artifacts of conformance (silver) before establishing a management framework (bronze). But we can see the crawl, walk, run analogy in the progression in levels of assurance (goals).. Perhaps platinum level is hiring Toni Kroos for 3rd party assessment.

 

 

 

Jacob Wilson

Senior Security Consultant
M: 773-372-8868 | jacob.wilson@...

Synopsys Software Integrity Group (SIG)

http://www.synopsys.com/software

 

From: main@... <main@...> On Behalf Of Jari Koivisto
Sent: Wednesday, August 25, 2021 4:36 AM
To: OpenChain Main <main@...>
Subject: Re: [openchain] Proposal - OpenChain Quality of Conformance Assessment Levels (including a sub-proposal for tooling quality assessment levels)

 

Hi All,

 

I think that this new slide 10 is better now. It might add unnecessary complexity, but maybe each of the 4 arrows could have a small loop arrow on top depicting the nature of iterative work involved at each state, e.g. a company may try several types of automation to create the artefacts. 

 

   Jari

 


---
Jari Koivisto
E-mail: jari.p.koivisto@...
Mobile: +41 78 7479791
Skype: jari.p.koivisto
LinkedIn: http://www.linkedin.com/in/jarikoivisto

 

 

On Wed, 25 Aug 2021 at 07:16, Shane Coughlan <scoughlan@...> wrote:

Thanks Steve. Slide 10 has been refined to place emphasis on the actions being non-normative evolution on the part of a single example company:

 

Regarding SPDX or other SBOM badges, I am sure we will see movement in this direction in the coming weeks.

 

Shane



On Aug 24, 2021, at 16:35, Steve Kilbane <stephen.kilbane@...> wrote:

Hi Shane,

As a comment on the revised slides, slide 10 doesn't work for me, since it shows a cycle, but you wouldn't go from your intended end-point (audit done) back to "just" OpenChain compliance without SBOM or automation.

As noted on the call, these aren't linear journeys, and an organisation could collect these plot tokens by wandering the map in whichever order they choose. They may add external audit first, then automation, then use that automation to produce their SBOMs.

I did like the idea of identifying them as best practices rather than "levels", because they can be adopted in any order. I did wonder whether it made sense to have a receiver award the SBOM badge to the supplier, but that might be problematic with respect to confidential business arrangements.

It would be nice if there were an SBOM badge that could be awarded to open source projects that are producing suitable output - uses SPDX Ids, includes an SBOM - but that's probably more down to the SPDX project than the OpenChain project.

steve

Join main@lists.openchainproject.org to automatically receive all group messages.