Pipeline operators whose assets are subject to federal oversight are required to comply with a series of highly-detailed regulations captured in the PHMSA Control Room Management Rule (49 CFR 192.631 for the transport of natural gas and 49 CFR Part 195.446 for the transport of hazardous liquids).
Our focus at EnerSys Corporation is to maximize the effectiveness of each pipeline operator’s Control Room Management safety program so that you are in Natural Compliance with regulations as part of the regular course of operations.
As such, when there is a trend in audit findings pertaining to Control Room Management, we share lessons learned to help each operator understand where they need to examine their own policies, procedures, and implementation to identify gaps that need to be closed.
One such trend that we have noticed is the presence of gaps in operators’ internal communication plans (ICPs), which is internal communication between the control room and field when performing manual operations. Gaps in your internal communication could lead to audit findings, safety issues, incidents, and fines.
Additionally, this trend has risen to the top of PHMSA’s focus because cyber attacks pose a great risk to communication and control system failures in pipeline operations. During cyber events, the problem becomes larger than just a control room issue; it could shut down the entire operation. That’s why PHMSA is squarely focused on evaluating each operator’s readiness.
We’d like to share insights to help your operation strengthen internal communication and support operational effectiveness in the control room to protect your operation in the event of a cyber attack.
Pipeline Communications: Recent Control Room Findings
There is a recent example of an operator being subjected to tens of thousands of dollars worth of fines due to internal communication gaps that were tied to an incident. The most significant issue that PHMSA identified was that control room communication gaps prevented the operator from being able to continue manual operations of the pipeline during a cyber incident.
In our research, we identified several other examples of ICP findings in the 2020 and 2021 PHMSA CRM Enforcement Actions. These findings were related to Section C (Adequate Information) of the CRM Rule where an operator’s internal communication plans did not provide adequate means for the safe manual operation of pipeline assets.
Our research indicates that these findings, on their own, typically carry a nominal fine or no proposed penalty. However, when these types of findings are linked to an incident or another area of concern (such as poor recordkeeping), they can carry a hefty price tag.
That’s why operators need to place a high value on their internal communication plans. This is an issue that many operators are working through right now. Let’s examine recent examples that could help you identify areas of improvement in your operation.
Recent Trends in Internal Communication Plan Findings
In our research, we identified two PHMSA findings related to ICPs in the 2020 Enforcement Actions. In 2021, the number of findings jumped to five. Consider these lessons learned.
2020 PHMSA CRM Enforcement Actions
– Example #1: An operator was subject to a Notice of Proposed Violation (NOPV) / Proposed Civil Penalty for multiple Control Room Management violations. One of the findings was a Section C finding because the operator “failed to test and verify internal communication plan to provide adequate means for manual operation of the pipeline safely, at least once each calendar year.” No specific fine was attached to this finding.
– Example #2: A different operator was subject to a NOPV/Proposed Civil Penalty for the exact same Section C finding. However, this operator was subject to a proposed fine of $19,000. Why did this operator face a potential fine, but the other operator in Example #1 did not?
PHMSA noted that the operator was unable to demonstrate that they tested and verified their internal communication plan. The finding points to the need for sufficient recordkeeping to prove that you did what you said you were going to do. It’s not enough to simply have a plan.
2021 PHMSA CRM Enforcement Actions
– Example #3: An operator received a Notice of Amendment for multiple CRM findings. PHMSA cited the operator for a Section C violation for the following reason:
“The Control Room Management (CRM) Plan was inadequate because it did not provide enough detail in the internal communications plan for the safe manual operation of the pipeline.”
There was no proposed fine attached to this finding. However, the lesson here is that records need to be thorough and complete. Using a digital recordkeeping system can help your operation comply with this PHMSA requirement.
– Example #4: An operator received a Warning Letter for a Section C violation related to ICPs. According to PHMSA, the operator could not provide consistent documentation proving that they tested their internal communication plans.
Additionally, PHMSA stated that the records for their SCADA testing drills were “confusing” and failed to provide information about how the drills were conducted and what scenarios were included in the training. There was no proposed fine, but the finding reinforces the need for adequate recordkeeping.
– Example #5: An operator received a Notice of Amendment related to a lack of information provided to the control room to support manual operations.
Specifically, PHMSA identified a Section C finding because the operator’s CRM Plan “was inadequate to provide guidance to the controller when performing manual operations through actual or test conditions as required.”
While no fine was attached to this finding, the violation points to a Team Training issue of not providing controllers with sufficient training to perform their duties during manual operations of the pipeline.
– Example 6: There was another Team Training-related finding for a separate operator. This operator also received a Notice of Amendment for a Section C finding related to inadequate training on the internal communication plans.
PHMSA cited the operator for not providing its controllers “with the information, tools, processes, and procedures necessary for the controllers to carry out the roles and responsibilities the operator has defined.” Additionally, the operator “did not verify that the internal communication plan provides adequate means for manual operation of the pipeline safety.”
No fine was given, but you can see how the combination of inadequate training, a lack of clarity around roles and responsibilities, and insufficient ICPs can increase an operator’s vulnerability to a cyber attack and create a greater risk of an incident.
– Example 7: An operator was subject to a Proposed Compliance Order for multiple CRM findings. The Section C finding was related to the operator failing to test their ICPs and lacking sufficient documentation.
Specifically, PHMSA said the operator “failed to test and verify internal communications plans to provide adequate means for manual operation of the pipeline safely, at least once each calendar year, but at intervals not to exceed 15 months.” Additionally, the operator “could not produce documentation that the internal communication plan had been tested at the requisite intervals, nor did its procedures even require such testing.”
No fine was attached, but the finding points to the need for adequate recordkeeping, documentation, and plans. Otherwise, controllers will be limited in their ability to achieve situational awareness in the event of a cyber event.
What to Learn From Control Room Management Findings
Operators should take notice that now is the time to act on CRM Rule compliance. For operators that put off compliance or allow weak areas to remain unchecked, there is an element of risk that needs to be considered. Operators should perform a risk analysis to understand the ramifications of what could result from a PHMSA inspection versus an incident review:
- A significant fine – if attached to an incident.
- Damage to reputation.
- Removal of permission to operate.
Keep in mind that compliance has social and reputation impacts. It’s not just financial, as an incident affects the public trust in pipelines and how operators are viewed by their peers.
Plus, because of the increased risk of cyber attacks on pipeline operators, regulatory agencies are stepping up inspection efforts and sending a message that non-compliance will not be tolerated. We have seen this trend in our experience supporting multiple operators through the PHMSA audit experience.
How EnerSys Can Support CRM in Your Pipeline Operation
We can help pipeline operators advance their CRM program, ensure compliance with the CRM Rule, and validate the integrity of their systems.
– Perform a gap analysis of CRMP vs. CRM Rule requirements. Using our POEMS® ComplyMgr software tool, we can line up the policies and procedures in your Control Room Management Plan (CRMP) against the language of the CRM Rule. We can also evaluate where you are doing what you said you were going to do (implementation). Consider our solution:
- Utilize the Plan-Do-Check-Act methodology to assess your current approach to Control Room Management.
- Close gaps in policies, procedures, and implementation.
- Look for gaps in your Internal Communication Plans (ICPs).
- Support continuous improvement by continuing to utilize the PDCA cycle.
- Set you on the path to Natural Compliance to operate in a compliant, safe, and efficient manner.
- Help you form a strong defense against cyber vulnerabilities while maintaining a posture of compliance.
Additionally, we anticipate greater emphasis on cybersecurity during audits. Moving forward, we expect PHMSA inspections to speak to cybersecurity threats when reviewing an operator’s internal communication plan for manual control, SCADA security and redundancy, AOC/EOC response, and team training. We can provide support by reviewing your compliance posture.
– To schedule a consultation, call us at 281-598-7100 to speak with EnerSys GM Ross Adams or VP Business Development Dale Schafer. You can also send us an email at sales@enersyscorp.com or complete our website contact form to get started.