masthead-background-img

System Engineering Case Study

Design, development and implementation of a system-of-systems routing framework to support efficient planning, implementation and sustainment of data flows in support of organizational goals.

Key activities in this data-centric organization are data collection, processing, and distribution of large volumes of data to broadly distributed endpoints for data analysis and use. Data routing operations are key, with route planning and implementation being high value, high visibility activities.  Due to the complexity of the data flows, the organization was experiencing increasing delays in flow implementation based on the long planning cycle and complex flows were often not fully documented as the flows were implemented, making post-implementation troubleshooting difficult. Due to the large number of systems involved, flow disruptions based on planned or unplanned system updates were increasing, and the work backlog was increasing beyond acceptable limits.

A framework was designed composed of three key components – system nodes, data sets and routing templates.  The components were then implemented in software to facilitate the planning, processing and implementation of data flows.  Individual systems were defined as functional nodes, with attributes specifying types of data that were accepted and actions that could be performed on the data at each node.  Data types and routes were then analyzed and normalized to yield data flow templates that represented the patterns for approximately 80% of the data flows the organization uses.  Interfaces between the systems were characterized and encoded within each template, streamlining the planning process and making data flow plans more uniform across the organization.  Lastly, visual representations of the components were implemented in software, providing visualization of the flows and illustrating attributes of the data flowing between each pair of nodes.

The organization has realized significant improvements in the efficiency and consistency of producing route plans based on a template-based routing framework approach engineered by Business Transformation Institute, Inc.  The templates provide easy-to-use detailed planning guidance for the community of route planners for the majority of the flows, and the framework supports customization for the 20% that require non-standard planning. The framework formalizes basic system functions and interface elements, reducing errors and allowing consideration of complex flows visually, which helps in determining resources needed to implement the data flows.  Use of the templates also helps the teams identify potential problem areas and communicate efficiently both at planning and after implementation.

Data collection and distribution are governed by complex policies and operational procedures, and implementation of data flows involves diverse service organizations in multiple divisions of the enterprise.  Receiving, processing and routing data to the points where it is needed, as well as ensuring that only authorized personnel have access to data is key, but the systems and personnel involved in various steps of the processes have little insight into the end-to-end flows and the accompanying constraints, making it almost impossible to provide all of the information needed to optimize individual tasks for each flow at the time of planning.

Initially, route planning for data was performed manually, based on SME knowledge in a very small group of people.  Expanding the numbers of planners was an obvious choice, but this led to significant inconsistencies in route implementation, and made capacity extrapolations more difficult.  Frequently, the ‘as-built’ flows were not well documented, resulting in high diagnostic costs when a flow was not behaving normally.

Numerous services are involved in the end-to-end flow of the data; interfaces with many of the intermediary systems were not completely described, and unannounced changes often resulted in broad flow disruption.

Progression of the implementation tasks was tracked via a simple in-line comment service, making it necessary to read the entire list of comments in order to determine work status for an individual flow.

Goals of the project were to increase the accuracy, speed and completeness of dataflow planning, to improve the consistency of planning practices across planners, to characterize the interfaces between systems so that more automation could be used, and to increase the status visibility of the data flows to all stakeholders.

Approximately 80% of flows used ‘standard’ core paths, while 20% are completely custom.  Based on information, a series of 28 templates was planned with the help of the SMEs. These templates are based on key characteristics of specific classes of data, and on the functional profiles of each system along the path. The templates contained information about types of data the system handles, the actions that system can take on data, and the outputs that could be expected from each system. Each template was constructed for use with one class of data, with add-on templates to handle variations.  Use of the templates should provide an efficient and visual method for socializing information about each step in the data flow path, and would clearly identify interface points and dependencies.

For 80% of the flows, planning will involve instantiating the template for a particular data flow, and then completing additional steps to ensure the appropriateness and integrity of the flow; similarly, for the 20% of custom flows, system information is added as the route is planned.  The resulting plans should not only result in more consistent data flow plans, but should also allow construction of a data flow inventory of the organization’s’ data assets.

The process of templates construction yielded benefits beyond those that were planned; group discussions of the paths with the technical leadership resulted in greater consistency and agreement about the approach that should be used for each data group, as well as about the paths appropriate for the myriad of types of data that would use the templates.  Subcomponents of the templates representing individual system functions were useful in talking with system owners, in uncovering undocumented features of some interfaces and in improving ongoing communications between different divisions.

As planning personnel have started to instantiate templates for route planning for each dataset, they are finding a large increase in efficiency – estimates are above 50%, although the planning users are still learning the new system, so additional efficiency improvements are expected.  Use of visual data routes has improved the ability of the service providers to understand the specific needs for each dataflow, and to easily locate and review route planning information. Relationships with system owners and service provider groups has been strengthened by their participation in the planning processes, and change coordination has improved significantly. Sustainment personnel have noted that the ‘as-built’ flow plans dramatically reduce the time required to identify breakages in the path, primarily because they don’t have to spend time determining where they should look for the flow.

Based on initial experiences, the organization is planning on expanding their use of the framework components for additional classes of data flows within the enterprise, with the hope that they can reduce the backlog of routing requests and improve their service delivery times.  Automated integration with several reference systems is underway, further supporting efficiency in planning. The goal longer term is to provide proactive planning based on information delivered automatically from the system; for instance, understanding which flows need to be rerouted when an intermediary system is decommissioned, and performing the flow re-planning and implementation proactively.  Increased flow resilience, based on an understanding of the entire flow inventory and an ability to move high-value flows off of struggling processors or to dynamically scale those processors, is also a goal longer term.

Note:  Confidentiality is maintained for competitive reasons.  Examples of environments with similar data flow challenges include organizations that provide networking services, health care provision and insurance services, communications providers, banking and insurance industries, and any organization that conducts large volume data analysis.

Cta Bg

Start Transforming Your Business

Get Started Today
Testimonial Bg

Here’s What Our Clients Are Saying About Us

BTI succeeds in its mission when an organization is measurably better in a way that makes a real difference, is able to sustain the change for the better, knows that it is better, and is satisfied with the result.

Business Transformation Institute, Inc. (BTI) joined in partnership with Raytheon SAS to help us bring together many diverse business units and to stabilize and improve our performance.  Throughout the years we have worked together and achieved CMMI Maturity Level 5 while bringing together geographically and process diverse units. This has allowed us to have a much higher fluidity in our staff and contract execution locations without suffering performance degradation.  Also, BTI has led the way in choreographing novel appraisal methodologies that have significantly reduced the cost of maintaining CMMI certification and ensuring that the programs do not suffer “set-back” during the gap between certification renewals.

Alan Perkowski 1 E1695234482260

Alan Perkowski

Process Maturity Technical Director from Major Aerospace Company

Having BTI’s LSS black belts onboard means that I can now safely retire.  Thank you!

Lean Six Sigma Master Black Belt

Lean Six Sigma Master Black Belt from National Security Agency

CNI was going to graduate from the federal government’s 8(a) program.  We had to be prepared to compete. Part of this is having good processes.  BTI helped us to do it.

Phil Ricks 1 E1695234421930

Phil Ricks

Corporate Quality Manager from Chickasaw Nation Industries

I was fortunate to work with the BTI team on deploying the CMMI Level 2 practice at Western Union. There were a number of obstacles in garnering top-down changes and support everyone that the team was able to work through and make it successful.

Implementing CMMI practices and procedures in an unstructured environment is surely difficult, but the BTI team was able to see the big picture and also make sure that the details were correctly addressed and implemented.

I highly recommend the BTI team!

John Oyhagary 1 E1695234675445

John Oyhagaray

Western Union Systems Development