Earlier this year, we relayed an update that PHMSA Enforcement Actions increased significantly in 2019 compared to 2018. Our team dug into the 223 reported enforcement cases and found 14 control room-specific cases.
These “inadequacies” were related to several categories of the PHMSA Control Room Management Rule (CRM Rule). In some cases, there were multiple instances of inadequacies, creating 20 total instances:
- CRM Rule Section A: General Requirements (3 instances)
- CRM Rule Section B: Roles & Responsibilities (1 instance)
- CRM Rule Section C: Adequate Information (3 instances)
- CRM Rule Section D: Fatigue Mitigation (4 instances)
- CRM Rule Section E: Alarm Management (3 instances)
- CRM Rule Section H: Training (4 instances)
- CRM Rule Section J: Compliance & Deviations (2 instances)
These instances serve as valuable lessons for pipeline operators to continue striving for compliance with the CRM Rule if your assets are subject to federal oversight.
Compliance can be achieved naturally by understanding the inadequacies, sharing these lessons learned with controllers and support personnel, and continuing to provide training to support safety objectives.
Control Room Inadequacies in PHMSA Enforcement Actions
We have identified the key findings and inadequacies for each section of the CRM Rule that PHMSA reported for the 2019 calendar year.
Section A: General Requirements
We identified three instances of inadequacies not adhering to the general requirements of the CRM Rule to provide controllers and control room support personnel with adequate resources to control and monitor the safety of the pipeline control room.
– One instance noted the operator’s written control room management procedures were inadequate. Specifically, the procedures did not require review and approval of deviations from control room management procedures prior to at the time when the deviations could impact safe operation of the pipeline system.
– For another operator, there was an instance of the operator not meeting the PHMSA requirement for controllers monitoring and controlling all or part of a pipeline facility through a SCADA system. PHMSA determined that the operator’s control room management procedure on backup SCADA systems was inadequate and needed to be amended to provide necessary information, tools, processes, and procedures to controllers to complete their assigned tasks. PHMSA also requested that the operator clarify roles and responsibilities for all controllers.
– For another operator, PHMSA determined there was a general violation of the requirement for a transmission operator without a compressor station to have and follow written procedures. PHMSA determined that the operator’s control room management program was inadequate and needed to be amended to provide adequate guidance on documentation to demonstrate the reason for any deviation from procedures. PHMSA also recommended that the operator update their procedures to include a time frame for when deviations should be documented after the deviation occurred.
Section B: Roles & Responsibilities
We identified one area where an operator did not adequately define the roles and responsibilities of a controller during normal, abnormal, and emergency operating conditions.
The specific notice focused on the operator’s method for recording controller shift changes and the handover of responsibility between controllers. PHMSA determined that the operator failed to provide documentation for shift change in their shift change process over a period of time.
The operator responded by providing “shift change responsibility handover” training to controllers to help close gaps in shift change responsibilities to achieve compliance with regulatory requirements.
Section C: Adequate Information
We identified three instances where pipeline operators did not provide adequate information to controllers and support personnel that would be necessary for control room personnel to complete their defined tasks.
– One instance focused on point-to-point verification. Specifically, PHMSA determined that an operator failed to provide information to controllers on how to conduct an adequate point-to-point verification between the SCADA system and field equipment when new field equipment was added to the pipeline segment.
– For another operator, PHMSA determined that the operator failed to test or verify their documented internal communication plan for manual operation of the pipeline. Additionally, the operator was unable to provide testing or verification records. This violated the “adequate information” requirement to test and verify internal communication for safe, manual operation of the pipeline at least once each calendar year.
– For another operator, PHMSA focused on the operator’s Risk Analysis in high consequence areas (HCAs). PHMSA determined that the operator did not provide adequate information regarding the integrity of the entire pipeline.
Section D: Fatigue Mitigation
Section D contained the most instances in the 2019 PHMSA Enforcement Actions, along with Training (Section H).
– One instance occurred when the PHMSA inspection team reviewed the operator’s records of deviations from Hours of Service (HOS) requirements. The records indicated that the operator did not complete documentation to demonstrate that deviations from HOS requirements noted in their CRM Plan were necessary for the safe operation of the pipeline system.
Additionally, the form that the operator used to document HOS deviations did not note approval from a superior. This called into question the operator’s procedure recording approval after the fact. PHMSA noted that it is inadequate for a review or acknowledgement to take place after the deviation has occurred and the impact on safe operation has passed.
– For another operator, PHMSA noted that the operator’s procedure for fatigue mitigation was inadequate and did not meet the specific guidelines of the CRM Rule to reduce risk associated with controller fatigue.
Specifically, the operator’s procedure allowed controllers to exceed the stated limitations for hours that can be worked during a given period of time. The operator responded by revising this particular area of their CRM plan to align with the CRM Rule requirements.
– For another operator, PHMSA determined that the operator’s control room management procedures did not provide adequate instruction to controllers on how to use appropriate fatigue measures when working during certain high-risk hours. In response, the operator modified this procedure to provide additional information to controllers on how to utilize fatigue mitigation tactics.
– For another operator, PHMSA determined that the operator did not meet the requirement to set a maximum limit on controller HOS. Specifically, the operator’s control room management procedures were deemed inadequate because it did not establish a maximum limit on controller HOS, which may provide for an emergency deviation from the maximum limit if necessary for the safe operation of the pipeline.
Section E: Alarm Management
We found three instances of PHMSA reporting inadequacies related to Alarm Management. The specific language of Section E states that “each operator using a SCADA system must have a written alarm management plan to provide for effective controller response to alarms.”
– One operator was cited by PHMSA because the inspector determined that the operator’s written control room management procedure did not provide parameters or thresholds for determining whether controllers had sufficient time to analyze and react to income alarms. The operator was required to amend their procedure to include thresholds or parameters to determine whether the controller workloads were adequate.
– For another operator, PHMSA determined the operator did not perform analysis of whether controllers’ performance was adequate. This was in relation to the CRM Rule requirement to monitor each controller’s general activity at least once every year to ensure that controllers have sufficient time to analyze and react to incoming alarms. The operator was unable to provide records demonstrating that the annual compliance of analysis was adequate.
– For another operator, there were multiple violations of Section E. In summary, PHMSA determined that the operator’s control room management plan was missing sections on maintenance, alarm philosophy, detailed design, and other topical sections. There was also no procedure for developing alarm set points. There was also no procedure to address safety-related alarm set-point values and alarm descriptions when field instruments are calibrated or changed at least once each year. PHMSA recommended that the operator update their alarm management plan and add necessary sections to support controller response.
Section H: Training
Section H had the most instances in the 2019 PHMSA Enforcement Actions, along with Fatigue Mitigation (Section D).
– One instance focused on the pipeline operator’s controller training program. PHMSA determined that the operator’s program did not provide controllers with working knowledge of the pipeline system, especially as it relates to abnormal operating conditions. This included a lack of safety-related data points to support understanding.
– For another operator, there was an instance of not providing adequate training on a calendar-year basis. PHMSA determined that the operator failed to review the content of its controller training program to identify potential improvements at least once each calendar year, in accordance with its control room management plan.
– For another operator, there was an instance of not providing training on how to respond to abnormal operating conditions likely to occur simultaneously or in sequence. PHMSA determined that the control room management procedures were inadequate because they lacked adequate information to help controllers understand how to respond to each operating condition in the correct order.
– The same operator had a separate inadequacy related to how often they reviewed the controller training program. The operator was unable to provide records that demonstrated that the review occurred annually. It’s a lesson of it not being enough to have the training; the training must also be documented.
Section J: Compliance & Deviations
We found two instances of pipeline operators not adhering to the Section J requirements to maintain adequate records documenting deviations from control room management plans to maintain safe operations.
– One instance focused on the operator not maintaining records that demonstrated compliance. Specifically, the operator could not produce records for point-to-point verification. The operator claimed to have performed the verification, but they could not provide documentation validating the activity.
– For another operator, there was an instance of the operator lacking records that demonstrated compliance with the CRM Rule requirements for deviations. Specifically, PHMSA determined that the operator’s records were inadequate because they were incomplete and lacked sufficient information to validate that the backup SCADA systems were tested once per year and the alarm management plan had been reviewed.
Find Support Assessing Your Pipeline Control Room
We provided this information to help pipeline operators take steps toward improvement. We want operators to understand what PHMSA is looking for during an inspection, and we also encourage operators to review control room management plans to find gaps in comparison to the PHMSA CRM Rule requirements.
Our team is available to review the PHMSA Enforcement Actions specific to the pipeline control room to facilitate the proper response for your operation. We are also capable of providing training and consultation to address specific areas of concern.