There’s been quite a lot of traffic on the list about how audits and certification are going to work, and I’d like to open up how we are approaching this. It’s important that people understand what an OpenChain certification means, and how it can be arrived at. To that end, we’ve spent a lot of time working with an audit specialist with experience in a broad range of fields from process to financial services to quality assurance to come up with our own audit processes and procedures. We’ve come to the conclusion that we need at (at minimum) two levels of certification.
One reason for this is that an organisation may seek third party certification directly after implementing an OpenChain compliance program. There should be a place in the marketplace for such a certification, but, again, such a certification will not be as in-depth as a certification which has tested the operations of the practices and procedures. This second level of certification will necessarily mean that the program has been in operation for a period of time (a year, for example), so that its outputs can be tested against the expectations of the program. Naturally, to receive this certification, the organisation will have to have been operating for sufficient time for the data to be available, so this level of certification cannot be offered immediately after implementation of the program.
In brief, we make a distinction between “Program Certification” and “Systems Certification”.
Program Certification confirms that OpenChain policies, practices and procedures are in place which are compliant. This is similar to verifying that the answers to the self-certification questionnaire are independently verified, and, in addition, confirming that, if operated properly (including build systems etc.) those policies, practices and procedures are capable of producing the required outputs (so in this respect it goes somewhat further than verifying self-certification on its own).
Systems Certification requires Program Certification, but it also requires that the organisation has been operating an OpenChain program for a period of time sufficient for us to certify both that the organisation’s program meets Program Certification standards, and also that those policies, practices and procedures verifiably meet the outcomes required by the OpenChain Specification. The former is, in effect, a readiness and capability certification, and the latter is a more comprehensive certification to audit standards, so it will require sample-testing that compliance artifacts for components are correctly generated, that licence choices have been correctly assigned, that training records have been correctly met, and so on.
Establishing the criteria for Program Certification is fairly straightforward, as it is based very much on the structure and content of the self-certification questionnaire, together with some additional checks to confirm the plausibility of the processes to be employed to implement the program. Systems Certification is somewhat more complex, in that it requires that audit processes are established to check that the expected outputs are in line with the actual outputs. It is not appropriate to do a complete code analysis here, for example (in the same way that a financial auditor undertaking a business’s annual audit will not check to ensure that every expenses receipt submitted to an organisation has been correctly entered into its accounting system), but a structured set of control-based checks will be used to provide the relevant confidence level.
We are putting together an internal specification for how the Systems Certification would work, and we are basing it on existing certification standards (such as Management Systems Certification ISO/IEC 17021 1). Clearly, it is in the interest of OpenChain that the levels of certification to be adopted are agreed on by the OpenChain Project, and, ultimately, we would consider that the certification programs themselves (and their operators) are independently verified by organisations such as UKAS in the UK, and DAkkS in Germany.
We’re very happy to discuss this further.
All the best
+44 1628 470003
+44 7970 835001
Jubilee House, 213 Oxford Street, London, W1D 2LF
Thames House, Mere Park, Dedmere Road, Marlow, Bucks SL7 1PB (registered office)
Orcro Limited is a limited company registered in England and Wales under Number 11173406. VAT number: GB 289 7831 32. Orcro Limited is not regulated as a law firm and does not provide legal advice, but has a relationship with Moorcrofts LLP. We are happy to work with either Moorcrofts LLP or your own chosen legal advisers. Individuals’ qualifications are as set out in their bio page. Reference to an individual as a lawyer, solicitor or paralegal does not mean that they are acting in that capacity as an Orcro staff member.
Data protection: we process your personal data to keep in touch with you, to carry out work for you or your organisation, for internal administration (including employment) for regulatory purposes and for limited marketing purposes (for which you can require us to stop at any time). For more information see https://orcro.co.uk/privacy-summary/ or contact team@...
Marcel (PwC DE)
This is an interesting idea; however, creating different variations of certifications might dilute the credibility of an ISO/IEC 5230 certificate. Also, it would be my understanding that it is not in line with audit and ISO practices and might result in confusion.
With ISO/IEC 5230, we have an official ISO framework which specifies the requirements of a quality open source license compliance program – for which a certification body should follow ISO certification requirements, the ISO/IEC 17021 for audit and certification of management systems.
ISO 17021 - audit and certification of management systems- defines that such an audit is conducted in two stages; stage 1 being a review of documentation of the management system (I would call it “design effectiveness audit”) and stage 2 to evaluate the implementation, including the effectiveness of the management system (I would call it “operating effectiveness audit”). So within stage 2, an ISO 17021 conformant certification body needs to obtain sufficient evidence of the effectiveness of the management measures, which includes sample testing where applicable. This does not require the management system to be in operation for a long period of time, such as a year, but it must be in operation so that the effectiveness of the measures can be demonstrated through evidence or reperformance of measures. Certifications as such (unlike some other audit reports such as ISAE 3000 or ISAE 3402) are per se related to a point in time, so it does not give an indication of the past or a specific period of time; e.g. it does not say that the management system has been effective for the last 12 months etc., but it gives an indication of the date of certification and, if applicable, for the subsequent period through surveillance audits.
According to ISO 17021, there can be no certification of only design effectiveness / stage 1 - operational effectiveness must also be assessed. I assume an accreditation of such a certification procedure by the state authorities (in Germany DAkkS) is not possible. To provide a certificate only on stage 1 would bring a lot of confusion to the market.
There is still the self-certification around, customer specific audits, and audit reports e.g. as per ISAE 3000 can be performed. To support companies even further a “certification readiness audit” can be performed before certification.
PwC issues ISO/IEC 5230 certificates only on the basis of ISO 17021 compliant audits, including e.g. IAF MD #1, #4, #5, ensuring that anyone holding a PwC certificate can rely on full compliance with ISO 5230 and surrounding ISO certification requirements.
Marcel Scholze (DE)
PwC | Director | Open Source Software Services & IT-Sourcing
Phone: +49 69 95851746 | Mobile: +49 151 161 57 049
PricewaterhouseCoopers GmbH Wirtschaftsprüfungsgesellschaft
Friedrich-Ebert-Anlage 35-37 | 60327 | Frankfurt a. M. | Germany
Find out about Open Source Software Management: https://www.pwc.de/opensource
Vorsitzender des Aufsichtsrates: WP StB Dr. Norbert Vogelpoth
Geschäftsführer: WP StB Dr. Ulrich Störk, WP StB Dr. Peter Bartels, Dr. Joachim Englert, WP StB Petra Justenhoven, WP Clemens Koch, StB Marius Möller, WP StB Uwe Rittmann, StB RA Klaus Schmidt, StB CPA Mark Smith
Sitz der Gesellschaft: Frankfurt am Main, Amtsgericht Frankfurt am Main HRB 107858
PricewaterhouseCoopers GmbH Wirtschaftsprüfungsgesellschaft ist Mitglied von PricewaterhouseCoopers International, einer Company limited by guarantee registriert in England und Wales
Datenschutz: Hinweise zur Datenverarbeitung bei PricewaterhouseCoopers GmbH WPG finden Sie unter Datenschutzhinweise PricewaterhouseCoopers GmbH WPG
On Tue, 2 Mar 2021 at 18:15, Andrew K <andrew.katz@...> wrote:
Diese Information ist ausschliesslich fuer den Adressaten bestimmt und kann vertrauliche oder gesetzlich geschuetzte Informationen enthalten. Wenn Sie nicht der bestimmungsgemaesse Adressat sind, unterrichten Sie bitte den Absender und vernichten Sie diese Mail. Anderen als dem bestimmungsgemaessen Adressaten ist es untersagt, diese E-Mail zu lesen, zu speichern, weiterzuleiten oder ihren Inhalt auf welche Weise auch immer zu verwenden. Wir verwenden aktuelle Virenschutzprogramme. Fuer Schaeden, die dem Empfaenger gleichwohl durch von uns zugesandte mit Viren befallene E-Mails entstehen, schliessen wir jede Haftung aus.
* * * * *
The information contained in this email is intended only for its addressee and may contain confidential and/or privileged information. If the reader of this email is not the intended recipient, you are hereby notified that reading, saving, distribution or use of the content of this email in any way is prohibited. If you have received this email in error, please notify the sender and delete the email. We use updated antivirus protection software. We do not accept any responsibility for damages caused anyhow by viruses transmitted via email.
Open Chain has built into it that people are aware of the policy and their role in the open source arena, and evidence needs to be provided that the process is followed (22.214.171.124, 126.96.36.199) so they can't just build the thing, then get certified. They need to show that it is being followed, too.
toggle quoted messageShow quoted text
Adding the relevant text from the specification below for everyone (minus rationale expansion). Full specification here:
Each process identified by OpenChain has verification materials that must be output. This allows the company and (their customers) to check to ensure a software package has been correctly passed through the process at any time during production or later.
This means a company can implement the processes and self-certify or get independent assessment or third-party certification, and thereafter there is a permanent sanity check (or as long as required by licensed and law).
A review, audit or remediation procedure can ask for the verification artifacts. Indeed, the second to final section of OpenChain addresses this specifically with the verification artifact for the verification artifacts:
“3.6.1 In order for a program to be deemed OpenChain conformant, the organization shall [have] A document affirming the program [...] satisfies all the requirements of this [specification].”
Thus any customer/supplier question flow could look like this:
Are you OpenChain Conformant?
Can you provide the verification artifact for section 3.6.1?
As part of our final review, can you provide the verification artifacts for sections X, Y and Z?
Each industry sector will decide what level of fidelity is necessary to proceed. We expect variance to be relatively wide by sector, but less so inside each sector.
(e.g. I would be surprised if a defense company didn’t say “give me all the artifacts” but I would be less surprised if a consumer electronics company focused on specific artifacts to satisfy their assessment, up to and including being satisfied if supplierX simply provided the 3.6.1 + 3.6.2 (confirmation of being current)).
== specification text ==
3.3.1 Bill of materials
A process shall exist for creating and managing a bill of materials that includes each open source component (and its identified licenses) from which the supplied software is comprised.
188.8.131.52 A documented procedure for identifying, tracking, reviewing, approving, and archiving information about the collection of open source components from which the supplied software is comprised.
184.108.40.206 Open source component records for the supplied software that demonstrates the documented procedure was properly followed.
3.4.1 Compliance artifacts
A process shall exist for creating the set of compliance artifacts for the supplied software.
220.127.116.11 A documented procedure that describes the process under which the compliance artifacts are prepared and distributed with the supplied software as required by the identified licenses.
18.104.22.168 A documented procedure for archiving copies of the compliance artifacts of the supplied software - where the archive is planned to exist for a reasonable period of time1 since the last offer of the supplied software; or as required by the identified licenses (whichever is longer). Records exist that demonstrate the procedure has been properly followed.
On Mar 3, 2021, at 21:54, Mattran, Mary <mary.mattran@...> wrote: