OECD Nuclear Energy Agency Committee on Nuclear Regulatory Activities (CNRA)
Working Group on Operating Experience (WGOE)

International Workshop on Operating Experience
Programme Effectiveness Measures

8-10 September 2014
Garching, Germany

Hosted by the Gesellschaft für Anlagen- und Reaktorsicherheit (GRS)

General information

The OECD Nuclear Energy Agency (NEA) Committee on Nuclear Regulatory Activities (CNRA) Working Group on Operating Experience (WGOE) sponsored the International Workshop on Operating Experience (OPEX) Programme Effectiveness Measures. The workshop was hosted by the Gesellschaft für Anlagen- und Reaktorsicherheit (GRS). It took place in Garching, Germany from 8-10 September 2014.

Objective

Evaluation of safety significant events including the Fukushima Daiichi nuclear power plant accident indicated that lessons to be learnt from OPEX could provide an important prevention tool if implemented in a timely manner. The main objectives of the workshop were to provide a forum for the exchange of information on the licensees' and regulators' programmes on OPEX and approaches to assess or even measure the effectiveness of these programmes. Participants had the opportunity to meet with their counterparts from other countries and organisations to discuss current and future issues on the selected topic. They developed conclusions and commendable practices regarding the issues and identified methods to help improve their own OPEX feedback and assessment methodologies.

Background

Regulatory bodies, and potentially commercial organisations, could benefit from a generally accepted methodology of assessing the effectiveness of their OPEX programmes. Effective application of OPEX is consistently recognised as a fundamental pillar of nuclear safety. An organisation's ability to effectively consider OPEX information and determine the appropriate application of the lessons learnt and best practices identified through consideration of that information should be measurable to foster continuous improvement to OPEX processes and programmes. A set of effectiveness measures and performance assessment tools – heretofore referred to as measures and tools – applicable to OPEX processes and programmes, would provide useful information for the management of these processes and programmes. Ideally, these measures and tools would be structured such that they would be applicable to all entities responsible for nuclear safety (e.g. reactor, materials, radiation protection) across the international community.

Goal of the workshop

The goal of the workshop was to identify effectiveness measures and performance assessment tools applicable to OPEX programmes. These measures and tools were focused on improving the adoption of OPEX-derived safety improvements and overall OPEX programme effectiveness.

Workshop plan and structure

The workshop spanned two and a half days and included breakout sessions and presentations. The breakout sessions allowed groups of nuclear safety experts to collaborate on the development of measures and tools applicable to general OPEX programme areas. The presentations were used to set the foundation for the breakout session discussions.

A complete list of the presentations and their topics is presented in Table 1.

Table 1 – Workshop Presentation Topics
Presentation 1	The importance of OPEX programmes.
Presentation 2	How to develop and manage effective lessons learnt and continuous improvement programmes.
Presentation 3	How to describe workshop structure and expected outcomes, and place guest speaker speech in the context of the expected goals of this workshop.
Presentation 4	How (and why) to assess for OPEX programme effectiveness. How to communicate and track the results of assessments.

The breakout group discussions were the primary focus of this workshop. The final workshop deliverables, specifically, measures and tools for assessing the effectiveness of OPEX programmes and commendable practices in this area originated from these groups.

There were three breakout groups responsible for developing measures and tools in the following areas:

identifying/gathering OPEX;
processing OPEX information;
assessing outcomes of OPEX programmes.

The objectives of each group are described in Table 2. The objectives included in Table 2 provided a guide for the development of measures and tools applicable to each area. However, the final product was up to each breakout group. Significant deviations from the objectives included in Table 2 should have been justified in the final presentation.

Table 2 – Breakout Group Objectives
Group	Objectives
Group 1 – Identifying/gathering OPEX	This group should have focused on developing measures and tools that apply to the area of identifying/collecting OPEX. The proposed measures and tools should have been effective in measuring aspects that affected timeliness, value and quality. The following was provided solely to stimulate conversation on what attributes could have been measured to ensure: Timeliness That OPEX was considered within an appropriate amount of time. Value That the threshold for information that was considered in the OPEX programme was adequate. That the sources thatwere being considered within an OPEX programme was adequate. That OPEX was being stored appropriately. Quality That the appropriate amount of information was being provided in an OPEX report.
Group 2 – Processing OPEX information	This group should have focused on developing measures and tools that applied to the processing of OPEX information once it had been collected. The processing of OPEX information included the storing and evaluation of OPEX information. The proposed measures and tools should have been effective in measuring aspects that affect timeliness, value and quality. The following was provided solely to stimulate conversation on what attributes could have been measured to ensure: Timeliness That OPEX was being evaluated in a timely manner. Value That the criteria for screening an item for further evaluation were appropriate. Quality That OPEX was stored in a retrievable manner. That effective evaluation of an OPEX issue had taken place. That an effective outcome/application from the evaluation of an OPEX issue had been proposed.
Group 3 – Assessing outcomes of OPEX programmes	This group should have focused on developing measures and tools that assess an organisation's effectiveness in implementing OPEX initiated activities. Within this area, the measures and tools should have specifically reviewed the effectiveness of: The communication of OPEX. The effectiveness of OPEX on regulatory oversight and requirements. The effectiveness of OPEX on NPP design. The proposed measures and tools should have been effective in measuring aspects that affect timeliness, value and quality. The following was provided solely to stimulate conversation on what attributes could have been measured to ensure: Timeliness That the outcomes from OPEX initiatives are assessed in a timely manner. Value That the value added from OPEX initiatives is worth the cost. Quality That the OPEX initiatives accomplish their purpose.

Participants

It was expected that participation in the workshop would be mainly from regulatory organisations familiar with regulatory body's OPEX feedback programmes and activities. The intrinsic value of these workshops has been primarily for people who face these topics in practical situations on a day-to-day basis.

Language

All presentations, discussions and meeting documents were in English.

Workshop proceedings

Proceedings from the workshop will be published and will include presentations from the opening session, conclusions and recommendations from the discussion group, a summary of panel discussions and information papers submitted by participants to the workshop.

Informal sessions

In order for participants to be able to communicate and exchange information on other topics of interest, informal sessions are available during lunch periods and following daily sessions.

Workshop venue

The workshop was hosted at the GRS office in Garching near Munich.

Schedule of workshop activities

The following schedule had been developed for the workshop. As noted in the table, GRS hosted an evening reception on 9 September, Tuesday evening.

Day 1
Topic	Time (location)	Notes
Opening remarks by Prof. Dr. Weiss and Mr. Benoit Poulet, WGOE Chair	10:00-1015 (Main Conference Room)	Welcome to all attendees
Presentation 1 by Kai Weidenbrück	10:15-11:00 (Main Conference Room)	Importance of OPEX programmes
Break	11:00-11:15
Presentation 2 by Riccardo Chiarelli, WANO	11:15-12:15 (Main Conference Room)	Developing and managing effective lessons learnt and continuous improvement programmes
Lunch	12:15-13:45
Presentation 3 by Dr. Michael Maqua, GRS	13:45-14:30 (Main Conference Room)	Workshop organisation and expected outcomes
Break	14:30-14:45
Breakout session guidance	14:45-15:00 (Main Conference Room)	For day one, groups will meet in separate rooms for introductions; receive initial guidance from facilitators; review expected outcomes in more detail.
Breakout session	15:00-16:45 (Room 1)	Group 1 - Identifying/gathering OPEX
Breakout session	15:00-16:45 (Room 2)	Group 2 - Processing OPEX information
Breakout session	15:00-16:45 (Room 3)	Group 3 - Assessing outcomes of OPEX programmes
Day one wrap-up	16:45-17:00 (Main Conference Room)
Day 2
Opening remarks	09:00-09:15 (Main Conference Room)	Welcome toattendees and review of expected outcomes
Presentation 4 by Prof. Dr. Holzapfel	09:15-10:00	Specific aspects of OEF in aviation
Group 1 - Breakout session	10:00-12:00 (Room 1)
Group 2 - Breakout session	10:00-12:00 (Room 2)
Group 3 - Breakout session	10:00-12:00 (Room 3)
Lunch	12:00-13:30
Group 1- Breakout session	13:30-16:00 (Room 1)
Group 2 - Breakout session	13:30-16:00 (Room 2)
Group 3 - Breakout session	13:30-16:00 (Room 3)
Break	16:00-16:30
Facility tour	Institute of Flight System Dynamics Max Planck Institute for Plasma Physics (fusion reactor research)
Social	Evening reception in a typical Munich restaurant.
Day 3
Welcome remarks	09:00-09:15 (Main Conference Room)
Group 1 – Breakout session	09:15-10:45 (Room 1)
Group 2 – Breakout session	09:15-10:45 (Room 2)
Group 3 – Breakout session	09:15-10:45 (Room 3)
Break	10:45-11:15

Group 1 – presentation/discussion	11:15-12:30	Each group will prepare a presentation describing their recommendations for proposed measures and tools in their assigned area of responsibility.
Group 2 – presentation/discussion	11:15-12:30
Group 3 – presentation/discussion	11:15-12:30
Lunch	12:30-13:30
Presentation 5 by Wolfgang Preischl	13:30-14:00	Research work in GRS
Presentation and discussion of working group results	14:00-15:00
Closing remarks by Mr. Benoit Poulet, WGOE Chair	15:00-15:30	Summary of workshop findings

Workshop committee

Michael Maqua	GRS	Germany
Benoit Poulet	CNSC	Canada
Seija Suksi	STUK	Finland
Harold Chernoff	USNRC	United States of America
Yves Van Den Berghe	BEL V	Belgium
Nancy Salgado	NEA	International