Application of Big Data Analytics to Energy Pipeline Integrity Management

The energy industry is undergoing a transformative shift towards more efficient and reliable infrastructure management, driven by technological advancements and the increasing complexity of energy pipeline systems. This PhD thesis investigates the application of Big Data Analytics (BDA) to energy pipeline integrity management, aiming to enhance the overall pipeline system safety, reliability, and integrity by predicting defects, and enhance the performance of these critical assets.

A comprehensive understanding of the challenges faced by energy pipeline operators in ensuring the integrity of these pipelines is first established. These challenges encompass a wide range of factors, including corrosion, material degradation, external threats, environmental changes, third party damages, and operational anomalies. Traditional methods of pipeline integrity management often fall short in handling the large volume and variety of data generated by these factors, especially real time, or process data. This requires investigation of the application of big data analytics to predict defects and its ability to instruct pipeline operators to take proactive action to avoid any breakdowns or emergency shutdowns.

The role of data quality may have a significant impact on the use of big data techniques. Quality in data driven decision making is a big concern for most of the oil and gas companies. The data collected from various sensors has been found to be insufficiently reliable for modelling purposes. This is primarily due to incomplete, missing, and incorrect data, as well as the presence of outliers. Most datasets used for analyzing the current condition of pipelines and for asset decision analysis, particularly those including In-Line Inspection (ILI) data, contain missing values. Inappropriate treatment of such data can cause large errors in the classification of data patterns, leading to inaccurate anomaly detection and predictions. A case study using ILI data shows that outliers in particular impact on overall decision making relating to pipeline condition assessment. Five outlier detection methods have been applied to the testing of ILI datasets selected to make sure the robust outlier detection results to be achieved. The contribution of this study is to present an approach for outlier detection of ILI data, which is robust and makes the detection results accurate.

To utilize real-time data collected from the sensors installed on pipelines, a novel framework is proposed for the integration of Big Data Analytics into the pipeline integrity management process. Leveraging current data collection technologies, including Internet of Things (IoT) devices and sensor networks, the research focuses on the use of large volumes of real-time data from pipeline assets. This data was processed and analyzed using advanced machine learning algorithms and data mining techniques to identify patterns, anomalies, and potential threats and anomalies to pipeline integrity. It was found that the application of BDA enables predictive maintenance strategies, allowing operators to proactively address potential issues before they escalate, thereby minimizing downtime and reducing the risk of catastrophic failures.

Most of the existing machine learning models to detect faults in oil and gas pipelines are not tested on real field inspection, real-time data. Using ILI data, this study explored fitting models and proved how these fitting model (s) can assist pipeline operators to predict growth of pipeline anomalies using simple models with 2~3 parameters, achieving a higher prediction accuracy.