Functional Safety Audit & Assessment

Don’t Confuse a Functional Safety "Audit & Assessment"

UNDERSTAND THE CRITICAL DIFFERENCES BETWEEN THE TWO ESSENTIAL EVALUATIONS

Many people working in safety instrumented system (SIS) project development, execution, operation and maintenance treat a functional safety audit (FS Audit) and a functional safety assessment (FSA) as one and the same. So, based on this assumption, they simply ensure that such an activity is undertaken and perhaps signify the need to perform this evaluation at some point when it appears as a milestone on the project schedule. Moreover, often they call upon someone working on the project,
who may or may not have had some previous experience in quality auditing, to deliver this audit/assessment. However, this is not a reasonable approach because the concepts for the audit and assessment markedly differ.

An FS Audit provides a systematic and independent examination of the particular safety lifecycle phase activities under review. It determines whether the “procedures” specific to the functional safety requirements comply with the planned arrangements, are implemented effectively, and are suitable to achieve the specified objectives.

Industry good practice is encapsulated in the IEC 61511 standard. Its clause 5.2.6.2.1 notes: “The purpose of the audit is to review information documents and records to determine whether the functional safety management system (FSMS) is in place, up to date, and being followed. Where gaps are identified, recommendations for improvements are made.”

This review of the FSMS process essentially focuses on the procedures that shall be defined and executed at the time of the project schedule/associated execution activities and, as a result, the following management activities should be in place:

• FS Audit strategy;
• FS Audit program; and
• FS Audit plan, reporting process and follow-up mechanism.

So, in essence, the process and expectations of an FS Audit resemble those of a normal project quality management system (QMS) ISO 9001 audit regarding a “systematic review” of the execution strategy being applied. [For details on the latest edition of ISO 9001.

This usually means the QMS department (with support from the project safety team) performs the FS Audit. People in that department have the relevant audit skills to verify that procedures, forms and templates that constitute the contents and requirements of the FSMS are being correctly implemented. Functional safety competency is not a primary skill-set requirement for them.

An FS Audit is undertaken to ensure compliance with procedures. Auditors do not assess the adequacy of the work they are auditing and do not make specific judgments about functional safety and integrity.

In contrast, an FSA is an independent in-depth investigation into the previous and current lifecycle phase activities based on evidence, aimed at evaluating whether functional safety has been achieved. FSAs rely heavily on assessor judgements and competency. One of the inputs to the FSA process is the FS Audit processes and findings.

As with the FS Audit, there are requirements to formalize a procedure for how this activity shall be defined, executed and planned into the project schedule. However, that’s where the similarity in approach and delivery ends. For an FSA, the focus is on “judgement” about the functional safety and safety integrity achieved by the safety-related project activities under assessment. Its goal is to ensure that functional safety has been achieved within the specific scope of supply for the organization(s) under assessment and in the context of the safety lifecycle.

The safety-related-systems project team implementing one or more phases of the functional safety lifecycle should plan FSA activities, but independent resources with the necessary competencies and SIS application skill set should execute the activities. Note that the FSA team undertaking the assessment must include at least one “senior competent person.” Often, two assessors form the assessment team to ensure the necessary depth and rigor for subject matter coverage.

The two key international safety standards — IEC 61508 [2] and IEC 61511 — cite requirements on how and when to execute one or more FSAs. For IEC 61508, this is Part 1 clause 8, and for IEC 61511 Part 1 clause 5.2.6.

Performing FSAs requires staff with a high level of competency and more often than not relies heavily on subjectivity, particularly when applied to earlier phases of the safety lifecycle.

The FSA activity is a mandatory (“shall”) requirement for claiming compliance to either of the safety standards; justifying such a claim requires documented evidence of an adequate FSA.

Besides helping to satisfy the standards, an FSA usually provides tangible benefits in terms of functional safety assurance and avoidance of costs and resource issues regarding potential rework at later lifecycle phases.

Planning Your FSA Requirements

Two points in the standards bear stressing: FSA requirements apply to all phases throughout the overall safety lifecycle; and the organization performing the FSA (and by implication its assessors) must meet a defined level of independence.

Keeping those points in mind, before embarking on developing an FSA methodology, you must consider:

• which IEC safety standard is being used for the development of the FSA process;
• the organizational and management models operating within the company and how these impact the levels of independence;
• the availability of “competent” resources and the necessary documented
evidence to support the standard’s requirement regarding competency
assurance;
• the role of the FSA requirements within the supply chain
and who is managing the overall activity across the various
organizations;
• the level of planning required, which depends upon
the size of the project, e.g., whether it involves a large capital
expenditure (capex) or a small modification to an existing operational
SIS; and
• optimizing the number of FSA stages and individual FSA phases within each stage regarding the overall cost of safety

A typical capex safety project likely will require more than one FSA. This will depend upon:

• the specific safety lifecycle phase(s) under assessment;
• the duration of the project and operation-and-maintenance lifetime;
• the number and type of safety systems implemented within the project;
• the degree of commonality across the technology solution; and
• the requirements for SIS management of change/modification covering the initial project and the entire SIS mission time.

Therefore, the person with lead responsibility for FSA planning and
execution within the organization that will manage the FSA requirements
must prepare a “functional safety assessment plan” for the safety
project and ensure this appears as a featured “milestone” on the overall
SIS project schedule/plan.

The FSA plan must be written to enable performing a systematic and comprehensive FSA (or a number of FSAs). It must specify:

• the stage(s) within the safety lifecycle when the FSA(s) will occur;
• the schedule and estimated duration of the assessment(s);
• the scope of the FSA(s) to be planned;
• the membership of the assessment team at each FSA stage;
• the degree of independence in accordance with IEC 61508/IEC 61511;
• the skills, responsibilities and authorities of the assessment team;
• the information that will be generated as a result of the FSA;
• the identity of any other safety bodies and departments involved in the assessment;
• the documents referenced at each FSA stage;
• the findings and recommendations from each FSA stage;
• follow-up and corrective action resolution; and
• FSA closure and management of continuous improvement/learning.

At some point in the planning process, the FSA plan will need to be approved by the responsible manager and issued to all parties prior to the assessment. Typically, only one plan is developed for the specific project FSA stages and phases. The individual phase reports effectively become a “living document.” After completion of each phase, evidence is reviewed, and findings, conclusions and recommendations are added to the FSA report to provide the necessary forward/backwards traceability for the assessment process.

Ongoing operational modifications of a smaller nature associated with an installed SIS may not need such regimented formal planning. However, IEC 61511 clause 17.2 requires implementation of some level of planning and verification for any such modifications. More importantly, the proposed changes shall not take place until completion of an appropriate FSA and receipt of proper authorization.

The Essentials Of Performing An FSA

The FSA must address the appropriate part(s) of the safety lifecycle in accordance with the recommended stages in IEC 61511 (see Part 1, Figure 7 — SIS safety life-cycle phases and FSA stages). Essentially, the FSA will review within the lifecycle activities under assessment if
appropriate methods, techniques, competencies, results and processes have been used to achieve functional safety.

The FSA, dependent on the applicable scope and the necessary backwards traceability at the time of the assessment, should check among other things that:

• The SIS has a defined and well-documented concept, hazard and risk identification, and risk reduction allocation to allow it to be designed, constructed, modified, verified and tested in accordance with the hazard and risk assessment, safety requirements specification, functional design specification, installation and commissioning safety acceptance test and eventual operation and maintenance of the SIS (not forgetting that the FSA also applies to part or full decommissioning of
any installed SIS).
• Regulations, mandatory standards and any stated codes of practice have been met and evidence of the requirements is available as part of the safety manual for the project/modification.
• The safety lifecycle activities under assessment have appropriate validation planning in place and the validation activities have beencompleted.
• Adequate and complete documentation is provided throughout and, in particular, the necessary independence is evident between authors, reviewers and approvers.
• Project change-management procedures are in place and have been applied throughout the lifecycle
phases. (There should be evidence of impact assessments, technical project queries, approved solutions and verification specifications, test planning and test records inclusive of document/records analysis and final approvals.)
• The safety integrity level (SIL) for each defined safety instrumented function (SIF) “achieves” and continues to “maintain” the SIL target requirements from design into operation and
maintenance.
• Any support, calculation, development and production tools used have been included in the FSA and have been assessed as being fit for purpose, e.g., “T classification” for support tools in
accordance with IEC 61508.
• Disparities within any of the lifecycle activities have been identified and resolved to ensure functional safety has not been compromised.

Use of specific checklists usually can assist the assessment team in focusing on the key areas to be covered during the required FSA(s). Such checklists are geared towards achieving the necessary functional safety requirements linked to the specific clauses and requirements of the IEC standards. This provides the basis for a robust assessment structure and enables the assessment to build upon a common format, e.g. structured observation recording, and by association, to develop the necessary traceability.

Such checklists:

• provide assessment enquiry consistency regarding project documentation to be presented that is necessary for the safety system being produced;
• support the focus on any shortcomings in requirements, design, implementation or procedure identified by the assessment process; and
• act as an aide memoire to ensure critical appraisal of all aspects of the project. This would be based on the assessment team judgment regarding the questions being raised and their relationship to the particular safety lifecycle activities under assessment.

An important underlying question is who in the organization manages the overall requirements for FSA deliverables and assigning the lead FSA role to a “competent” person? Is there evidence available to support any specific FSMS FSA training and mentoring processes applied for those
“approved” to conduct such FSAs?

What Is The Benefit?

Experience teaches that FSAs can reveal real errors and deficiencies in processes, technical capabilities and alignment with the safety requirements for either the new build or installed operational SIS. These are lapses and omissions that almost certainly would go undetected
in an FS Audit.

Here are only a few examples as found on a number of end-user delivered FSA assignments:

• Insufficient independence between protection layers that is not revealed and not acknowledged during the process of safety function allocation to protection layers, thus leading to inappropriate SIF
requirements and the wrong target SIL.
• Management-of-change issues caused by a loss of system “freeze’ for SIS modifications, resulting in different teams working on differing versions of SIS documentation and associated common SIF modification requirements.
• Lack of substance in change management “impact assessments,” leading to changes being
approved that potentially compromise both safety functionality and safety integrity.
• SIS corrective maintenance that has evolved to a “modification” without supporting impact assessment and document revision controls.
• Real discrepancies and misunderstanding between SIF device response times (DRTs) and overall process safety time (PST), resulting in non-compliant PST claims.
• Deviations in device safety manuals and, by detailed review of supporting device certification
reports, identification that purchased devices do not meet the application and operating environment requirements for use.
•Inadequate hardware reliability calculation where the use of too low failure rates results in too low average probability of failure on demand achieved and omission of compliance with systematic capability requirements, both leading to higher claimed SILs than in reality
• Conflicting specification requirements for both application program
“destruct” and “construct” using the same field devices and input/out for different SIF requirements.


And just for good measure, let’s not forget the FS Audit and FSA time-honored systematic capability chestnut:
• Identification of document and test “authors,” “reviewers” and “approvers” being one and the same person.

Perform A Proper FSA

Shortcomings in planning and executing the FSA process during different stages of the safety lifecycle can contribute significantly to potential SIS failures during the operational lifecycle phase. So, organizations involved in and responsible for the management of any stage of the safety lifecycle of the SIS must ensure the execution of such FSAs rests with assured competent resources. This will form part of the company FSMS and will support the systematic capability for the specification, design, engineering, operation and maintenance of a SIS.

In some cases, FSAs can span several organizations and the FSA activities will require overall management control because they can drill down to specifics, technicalities and results of any verification and validation. Therefore, they should have the relevant senior management support across the supply chain involved for reserving the right to re-do activities where functional safety may be compromised.

In considering industry good-practice expectations, performance of such FSAs should comply with the IEC 61508/IEC 61511 safety standards, which demand prescriptive independence and a high level of competency assurance. For more on FSAs, see Reference 3.

REFERENCES
1. “Functional Safety — Safety Instrumented Systems for the Process Industries Sector,” IEC 61511, 2nd ed., Intl. Electrotechnical Commission, Geneva, Switz. (2016).
2. “Functional Safety of Electrical/Electronic/Programmable Electronic Safety-Related Systems,” IEC 61508, 2nd ed., Intl. Electrotechnical Commission, Geneva, Switz. (2010).
3. Nunns, Stuart R., “Functional Safety Assessment: Setting the Boundaries of the FSA, Defining the Scope and Planning the FSA,” ABB, St. Neots, U.K. (2009),