ArcGIS Pipeline Referencing: Choose the best data model - Geocortex

ArcGIS Pipeline Referencing: Choose the best data model

Sam Acheson
Sam Acheson
Energy Solutions Manager
October 20, 2017

ArcGIS Pipeline Referencing: Choose the best data model

Over the past few weeks, I’ve shared foundational knowledge about how data is stored and managed in the pipeline industry. My first post introduced ArcGIS Pipeline Referencing (APR) and explained some options operators have in adopting it. My second post worked to demystify the confusing world of pipeline data models (there’s a lot to consider).

In this post, I will outline important information you need to consider when choosing a data model for your organization, including:

  • Limitations of current data models;
  • How APR is addressing these limitations; and
  • Questions you should ask yourself to help assess the best data model for your organization (should you choose to move to APR).

Limitations of Existing Models

A data model is defined as: “An abstract model that organizes elements of data, and standardizes how they relate to one another and to properties of real world entities.

In the pipeline space, real world entities include not only the pipe and valves that make up the pipeline system, but all of the things that happen to and around the pipe. Things such as repairs & replacements, surveys & patrols, one-call, cathodic protection, ILI & CIS assessments, HCA & class location, land ownership & right-of-ways, and crossing information all have components that, in one form or another, need to be captured in your GIS.

The differing needs of these complex data representations expose limitations in legacy systems. And in a world where critical business decisions must be made from this data, identifying limitations and addressing them is an important step as we move to next-generation solutions.

Limitation #1: Data volume

As the years have progressed, the operational and regulatory needs surrounding pipelines have increased. These needs are driving new levels of inspections and analyses on pipeline systems – resulting in more data, both in terms of volume and complexity. The legacy systems were simply not designed to handle the volume of data current programs produce.

An example is the case of Inline Inspection (ILI) and Close Interval Survey (CIS) data. A single ILI or CIS inspection results in hundreds of thousands of records. With assessment intervals recurring every 1-7 years — and operators performing dozens of inspections each year — the resulting records from these inspections alone add millions of records to the database. This doesn’t include follow-up inspections, digs, and run comparison activities.

When you couple the sheer volume of records with complexities surrounding data management and the need to provide a high-performance environment, limitations in the system are quickly exposed. These limitations force operators to make difficult data storage decisions, often choosing to remove subsets of data from the system of record. This is sub-optimal to say the least; it significantly impacts your ability to view and analyze important trends in the data.

Limitation #2: Engineering Stationing

Engineering stationing is important, complex, and rooted in pipeline data management. Before modern GIS, operators kept track of the location of pipelines and associated appurtenances using engineering stationing on paper or mylar sheets. With a vast majority of pipelines that are in use being constructed before the existence of modern GIS technology, large volumes of data were referenced with this approach.

Engineering stationing doesn’t benefit all operators; however, companies that manage gathering and distribution assets find this method burdensome … and dare I say unnecessary?

When traditional data models were developed, the need to adhere to legacy engineering stationing outweighed the need to redesign the entire system to favor a spatial-first approach. But as technology has improved, and more users have embraced spatial data, new methods to blend modern (spatial-first) and legacy (stationing-first) models have emerged. Operators need this flexibility when managing their assets.

Limitation #3: Native support for the Esri Platform

The emergence of the Pipeline Open Data Standard (PODS) represents the last major shift in data storage solutions for pipelines, and it happened nearly 20 years ago. At that time, the GIS landscape was both immature and fragmented. As a by-product, PODS was designed specifically to be GIS-agnostic. In the nearly two decades since, Esri has emerged as the predominant provider of spatial data management, and they have developed a suite of solutions that enable stronger collection, management, and analysis of data.

Chances are your organization embraces Esri for spatial data management and content dissemination, which begs the question: “If your organization has standardized on Esri technology, does it make sense to employ a data structure that does not natively support the environment?” (Hint: probably not.)

Addressing and Improving Limitations

The core of APR has been engineered to address important limitations currently felt due to the existing designs of PODS and APDM. APR directly addresses the three limitations described above.

Improvement #1: Data volume

Understanding the need to support large datasets, time-aware data, and the ability to offload the storage of data to other systems, APR has been engineered to handle the high volume of data more efficiently, with a focus on scalability. To achieve this, available rules can be configured to allow a more fine-grained approach to managing data during routine line maintenance. No longer are implementations limited to keeping the data referenced to the LRS or detaching it.

Changes like these allow operators to keep more data in the system, providing a baseline for more powerful analysis and decision making.

Improvement #2: Engineering Stationing

As explained above, engineering stationing is firmly rooted in pipeline data management, but it’s not required for all operators. New construction, gathering systems, and vertically-integrated distribution companies are finding the rigorous application of stationing to be unnecessary overhead. If your current database repository requires it, and your organization doesn’t rely on it, you are taking on unnecessary data management cycles – costing valuable time and money.

APR not only provides the ability to manage data in stationed and non-stationed methods: its flexibility allows for both stationed and non-stationed lines to exist in the same model. Let that sink in for a bit: Operators that have deployed two separate data management systems can now consolidate the management of these assets! This functionality benefits a majority of the clients I’ve worked with over the years.

Improvement #3: Native support for the Esri Platform

As I stated in my previous post, APR is (possibly most importantly) integrated with the ArcGIS platform. You can perform complex long transactions on your data, analyze it in ways that have not been possible before, keep track of product flow using the Facility Network, and get the data in the hands of your organization with methods that are integrated, fluid, and connected.

Considerations for Implementation

If you’re considering implementing ArcGIS Pipeline Referencing (APR), knowing why, and which data model to use with it is has more to do with your business than with IT — success can be achieved with either model.

But how do you decide which one is best for your organization? Here are some questions to consider as you’re laying the foundation for your next-generation GIS implementation.

1) Business focus: What segment of the vertical are you in?
If you are a distribution company with transmission assets, the decision is pretty clear: you should choose Utility and Pipeline Data Model (UPDM). It’s designed as a distribution-first model, allowing you to integrate the management of distribution and transmission assets in a single model.

If your company is ‘upstream’ of distribution, the answer gets a bit trickier. Both models are adequate, but my vote tends to lean towards PODS for a few reasons:

  • Out-of-the-box PODS supports APR slightly more natively for operators without distribution assets than UPDM.
  • Are you a liquids operator? As UPDM is focused on gas utility and transmission, the PODS model will provide a better solution for those moving liquid products.
  • As an organization delivering a comprehensive model to the industry, PODS is a thriving community of operators and vendors working together to design a comprehensive model for the industry. This collection of subject matter expertise is invaluable to operators – and provides an opportunity to share your experience with like-minded individuals.

2) Existing model: What are you using now?
As you consider moving to APR, understand that it’s going to require a data migration. The existing system will need to be mapped and loaded into the new solution. If you are currently using PODS and are a gathering, midstream, or transmission company, APR with PODS is probably the best solution to implement. It’s likely that your existing data will migrate more seamlessly, and the model will make more sense to those that manage and interact with the data.

If your organization is primarily gas distribution, and you’ve implemented a PODS model for a small subset of high-pressure assets in the system you manage, consider UPDM. You can take advantage of the intended benefits and consolidate those assets into a common platform.

3) Other programs: ILI, CIS, other survey, Cathodic Protection
If your company has a strong investment in recurring inspections, PODS again rises as the preferred model, especially considering the efforts of the PODS Next Generation initiative around how to efficiently store and process this data moving forward.

4) Interoperability
With the growing importance of importing and exporting data (due to acquisitions, divestitures, etc.), analysis, and reporting, a system that promotes standard mechanisms to exchange data becomes increasingly more important. With the work the PODS organization is putting into a data interchange standard, it again rises as the preferred model.

There isn’t just one approach, but there is a best approach for your organization

While this change is beneficial for operators, many things need to be considered before you commit to an approach. I hope my series of posts provides some clarity for you. To stay up-to-date on the data model landscape and the tools surrounding it, I encourage you to follow the PODS association and Esri. The work of these two organizations in the pipeline space is a great thing for our industry.

If you’d like to discuss any of these concepts further, or would like to have a conversation about which model is best for your implementation, please get in touch with me here. I, and the rest of the Energy team at Latitude, are eager to offer our years of expertise to help you.


Categories:
Archive
Open: 2019
2019
December 5, 2019

Four key benefits to adopting a mobile GIS solution

November 29, 2019

Using Geocortex Mobile to collect survey data [Geocortex Tech Tip]

November 27, 2019

Dive into Geocortex Analytics [Webinar]

November 22, 2019

How to input parameters on prints [Geocortex Tech Tip]

November 15, 2019

How to include feature attachments in reports [Geocortex Tech Tip]

November 14, 2019

Understanding application management in the Designer [Geocortex Tech Tip]

November 14, 2019

How to reference symbology from web maps [Geocortex Tech Tip]

November 14, 2019

How Geocortex Mobile can integrate with other apps [Geocortex Tech Tip]

November 14, 2019

Implementing Arcade Expressions in Geocortex Essentials [Geocortex Tech Tip]

November 14, 2019

Customer Spotlight: Geocortex Mobile

November 14, 2019

How to Format Your Reports [Geocortex Tech Tip]

November 14, 2019

Showing an image in a form after a file picker [Geocortex Tech Tip]

November 14, 2019

Richard Wiegmann Joins VertiGIS as President and CEO

September 27, 2019

Understanding the toolbar in Geocortex Web [Geocortex Tech Tip]

September 20, 2019

Refining Results in Geocortex Web [Geocortex Tech Tip]

September 19, 2019

Charting data using Geocortex Reporting: Honing your charting skills

September 13, 2019

Utilizing the flexibility of layout in Geocortex Web [Geocortex Tech Tip]

September 6, 2019

Creating Geocortex Web applications that move seamlessly between 2D and 3D [Geocortex Tech Tip]

August 28, 2019

Geocortex adds support for Arcade scripting and ArcGIS Online smart mapping

August 23, 2019

How to create sections and groups for your reports [Geocortex Tech Tip]

August 16, 2019

Displaying SQL Server Data in Geocortex Workflow [Geocortex Tech Tip]

August 14, 2019

Customer Spotlight: Geocortex Essentials

August 8, 2019

City of Austin: Spreading floodplain awareness with Geocortex

August 1, 2019

Geocortex Essentials 4.12: Improved integration with ArcGIS® Online

July 25, 2019

Introduction to Geocortex Printing: Learn to build stunning print templates

July 17, 2019

Updates to the Geocortex Product Life Cycle

July 8, 2019

Extending Web AppBuilder for ArcGIS® with Geocortex Essentials [Webinar]

July 4, 2019

City of Bellingham: Using Geocortex to take the stress out of water shut-off

June 27, 2019

Geocortex at the 2019 Esri User Conference

June 14, 2019

Customer Spotlight: Geocortex Workflow

May 17, 2019

Important Notice - Outage Issues & New Support Cases

May 17, 2019

How to build a summary report with aggregations of feature data [Geocortex Tech Tip]

May 16, 2019

Technology Q&A: Geocortex Mobile

May 10, 2019

How to create print templates for Web AppBuilder for ArcGIS®[Geocortex Tech Tip]

May 8, 2019

Dive into Geocortex Printing [Webinar]

May 3, 2019

Using Geocortex Workflow and Geocortex Reporting to produce a multi-layer report [Geocortex Tech Tip]

May 1, 2019

Market-Leading GIS Software and Services Companies Join to Form VertiGIS

May 1, 2019

Introducing VertiGIS

April 18, 2019

Integrating Workflows into Geocortex Mobile [Geocortex Tech Tip]

April 17, 2019

Customer Spotlight: Geocortex Reporting

April 12, 2019

Using Geocortex Go to preview app configurations [Geocortex Tech Tip]

April 10, 2019

How to Address Complex Form Navigation Using Geocortex Workflow

April 5, 2019

Building native, offline IOS, Android and Windows apps with Geocortex Mobile Viewer [Geocortex Tech Tip]

April 3, 2019

Streamlining mobile operations with Geocortex Mobile Viewer [Webinar]

March 29, 2019

A better way to run geoprocessing tasks in Web AppBuilder for ArcGIS [Geocortex Tech Tip]

March 28, 2019

Extend the reach and capabilities of your applications with integrations [eBook]

March 22, 2019

Displaying data from SQL Database when you select features [Geocortex Tech Tip]

March 20, 2019

How Geocortex users are enriching the communities they serve

March 15, 2019

How to Configure a Geocortex Essentials layer report with Geocortex Reporting 5 [Geocortex Tech Tip]

March 14, 2019

Region of Waterloo: How GIS & Geocortex technology helped meet local Source Protection Plan requirements

March 6, 2019

Integrating Business Intelligence and Data Sources with Geocortex Essentials [Webinar]

March 1, 2019

How to quickly add attachments to new features when editing [Geocortex Tech Tip]

February 22, 2019

Generating a report with sub reports from external data sources [Geocortex Tech Tip]

February 20, 2019

Geocortex Essentials 4.11: Saving Time & Increasing Efficiency

February 15, 2019

Accessing Samples and Building your First Workflow [Geocortex Tech Tip]

February 12, 2019

Open Beta of Geocortex Printing 5 Has Begun

February 8, 2019

Different Ways of Creating Selections of Features in Geocortex Viewer for HTML 5 [Geocortex Tech Tip]

February 6, 2019

4 Best Practices to Follow When Using Geocortex Workflow 5

February 1, 2019

Creating a Report that Displays a Screenshot of the Map [Geocortex Tech Tip]

January 28, 2019

Diving Into Geocortex Reporting 5 [Geocortex Tech Tip]

January 24, 2019

How Geocortex is Advancing Water Management Solutions

January 16, 2019

Creating Custom User Experiences with Geocortex Workflow 5 [Webinar]

January 11, 2019

Showing the search results using the item picker in Geocortex Workflow 5 [Geocortex Tech Tip]

January 9, 2019

City of Troy: Using GIS and asset management technologies to manage legislative requirements

January 4, 2019

Using scripting to put attachments in reports in Geocortex Reporting 5 [Geocortex Tech Tip]

Open: 2018
2018
December 21, 2018

Integrating Pictometry, Bing and other 3rd party maps within your Geocortex applications

December 19, 2018

Recapping the Texas Geocortex Regional User Group

December 14, 2018

How to use Geocortex Workflow 5 to populate the Attribute Table in Web AppBuilder for ArcGIS [Geocortex Tech Tip]

December 12, 2018

Finding yourself: Using geolocation in mobile and web applications [Webinar]

December 7, 2018

Enabling real-time user-to-user map collaboration within Geocortex Essentials

December 5, 2018

Reflecting on 2018

November 30, 2018

Creating mailing labels with reports with Geocortex Reporting 5 [Geocortex Tech Tip]

November 29, 2018

Alberta Energy Regulator: Supporting safe and responsible energy resource operations

November 26, 2018

Important update for ArcGIS and Transport Layer Security (TLS) Protocol Support!

November 23, 2018

Using fine-grained security to control access to layers, features, attributes and application functionality [Geocortex Tech Tip]

November 21, 2018

Enhancing Web AppBuilder for ArcGIS® with Geocortex Reporting [Webinar]

November 16, 2018

Using Geocortex Workflow 5 to automatically display a form for editing layer attributes [Geocortex Tech Tip]

November 15, 2018

Geocortex Achieves Esri’s “Release Ready Specialty” Designation

November 14, 2018

Celebrating GIS Day with York Regional Police: How GIS Data Has Mitigated Risk & Increased Efficiency in Crime Prevention

November 9, 2018

How to access a SQL database from a workflow using Geocortex Workflow 5 [Geocortex Tech Tip]

November 2, 2018

How to send an email from a workflow using Geocortex Workflow 5 [Geocortex Tech Tip]

October 26, 2018

How to add and configure charts inside reports with Geocortex Reporting 5 [Geocortex Tech Tip]

October 24, 2018

Geocortex Water Webinar Series

October 19, 2018

Integrating Geocortex Essentials with ArcGIS Online and ArcGIS Enterprise portal [Geocortex Tech Tip]

October 17, 2018

Cross-Platform Development with Xamarin [Webinar]

October 12, 2018

City of Fort Collins: Dynamic flood maps for public awareness, and flood insurance rate saving

October 2, 2018

Technology Q&A: Geocortex Workflow 5 “Behind the Firewall”

June 27, 2018

Geocortex Essentials 4.10 is here!

June 12, 2018

Geocortex and the GDPR

June 6, 2018

Geocortex at the 2018 Esri User Conference

May 15, 2018

Technology Q&A: The evolution of Geocortex and Web AppBuilder for ArcGIS

April 25, 2018

How to manage data collected from Geocortex Workflow 5 forms [Geocortex Tech Tip]

April 18, 2018

Delivering accessible mapping applications for everyone [Geocortex Tech Tip]

April 11, 2018

How to search for data in a non-spatial database [Geocortex Tech Tip]

April 5, 2018

Configuring Geocortex Analytics to monitor a new Portal for ArcGIS instance [Geocortex Tech Tip]

March 27, 2018

Getting started with forms in Geocortex Workflow 5 [Geocortex Tech Tip]

March 21, 2018

Using Geocortex Workflow with Web AppBuilder for ArcGIS [Geocortex Tech Tip]

March 15, 2018

Understanding tool usage in your GIS applications [Geocortex Tech Tip]

March 7, 2018

Running Geocortex Essentials workflows from an identify operation [Geocortex Tech Tip]

March 2, 2018

GIS Health Assessment: A new way to think about your system

February 28, 2018

Using the in-app help system in Geocortex Workflow 5 [Geocortex Tech Tip]

February 21, 2018

How to configure a personalized dashboard in Geocortex Analytics [Geocortex Tech Tip]

January 25, 2018

GIS is shifting to SaaS, and it’s a win for everyone


Categories

Popular Tags
Accessibility Active Operating Picture ArcGIS ArcGIS Online customer spotlight Customer Story Data Models Energy Esri Geocortex Geocortex 5-Series Geocortex Analytics Geocortex Essentials geocortex integrations geocortex mobile Geocortex Mobile Viewer geocortex printing Geocortex Printing 5 Geocortex Reporting Geocortex Reporting 5 geocortex tech tip Geocortex Tech Tips geocortex training Geocortex Viewer for HTML5 geocortex web Geocortex Web Viewer Geocortex Workflow Geocortex Workflow 5 GIS GIS Day GIS Health Integrations mobile gis Oil & Gas Pipelines Portal for ArcGIS SaaS Tech Tip Tech Tips Transportation vertigis Water Industry Web AppBuilder Web GIS Xamarin

Contributors