Part 4: Process Mining to Implement Better DevOps

In the previous articles, we introduced process mining, reviewed a case study, and examined the benefits realized by organizations, projects, and teams using process mining tools to understand their process mining tools.

Here’s another real-life example from a successful project:

The Scenario:  A large Department of Defense (DoD) program is creating products for secure communications.  The program has a dozen software development teams from multiple contractors spread throughout the U.S.  The Executive Program Manager (EPM) owes a progress report to the DoD Chief Information Officer (DoD CIO).

Although each software team is using its own locally-defined agile process, all teams use Atlassian’s Jira tool environment to provide status updates.  The teams also use locally-installed versions of GitHub for configuration management and deployment and SonarQube for software quality.  The individual reports from Jira are summarized to give daily status on the completion percentage and estimated-time-to-complete for each epic and its constituent stories.  The EPM is often told that any given epic is “90% complete” (or some variation thereof).  The EPM is skeptical, since most of the teams have regularly missed significant story completions at the end of each sprint, missed quarterly delivery deadlines, or shipped software that fails “black box” testing.

Figure 1 shows the program’s ability to keep its promises at the story level.  The x-axis represents the sprint when a particular user story was started.  The y-axis represents the sprint when a user story is completed.  Each black dot represents a story that was completed without any changes in story content.  A blue dot represents a story that is split after work on the story has started; that is, the story broken into two or more stories from the original planned story.  Any dot along the diagonal represents a story started and finished in the same sprint, which is the desired agile behavior.  A dot above the diagonal represents a story that was started in one sprint and finished later (sometimes much later) in another sprint.

Figure 1 1024x824

Process Mining Analysis:

  • The Nymbul Mineral© process mining tool automatically collects the following data from the toolchain. In this case, the information comes from Jira:
    • The frequency with which each individual developer is interacting with Jira and number of actions taken by each developer.
    • A comparison of the frequency of developer interactions with Jira to the report of the summarized completion data.
  • Nymbul Mineral’s analysis function produces an Analysis of Variance (ANOVA) to determine the probability that the accuracy of the completion data is related to the frequency of developer status updates in Jira.

The Results:

Based on the ANOVA prepared by staff using the process mining tool, the EPM determines that the developers were not doing daily status updates—despite their claimed adherence to agile practices with daily Scrum meetings.  This disconnect between information in daily Scrums and reporting has led to inaccurate rollup reports.  The EPM subsequently required all development teams and the developers to do daily status updates and direct program staff to monitor the updates for any discrepancies.

A graphical depiction of the analysis is shown in Figure 2.  Each point for each date (coded in days since the project’s in the year 2011 this graph) represents the number of actions recorded in Jira by the individuals on a team over the course of work week, with a different color representing each person.  Although the actual ANOVA did not use this graphical representation of team activity, this plot provides an example of the details available from process mining.

Figure 2

Based on the analysis and the subsequent direction given to the teams, the accuracy of the reports increased.  Additionally, even though the program’s status was worse than previously reported, the EPM now had accurate data to support taking the right corrective action.

How to Implement Process Mining

Receiving the benefits of process mining depends on the availability and analysis of data generated from projects’ toolchain(s).  Most software development projects use a collection of tools in order to:

  • Capture and manage requirements
  • Manage and track activities and report status
  • Update and maintain the integrity of software source code in multi-developer projects
  • Identify software issues and defects
  • Perform testing
  • Deliver software products to customers and track change requests and defect reports.

As shown in Figure 3, BTI has created the Nymbul Mineral© process mining tool to assist in process mining and analysis.  The tool is based on the popular and powerful open source “R” statistical analysis language.  Nymbul Mineral runs on Windows or Linux systems.

To help you realize the benefits of process mining, BTI will:

  • Install Nymbul Mineral on your system.
  • Work with your project analyst in understanding your organization’s goals, selecting a project to analyze with respect to those goals, collect data, analyze the data, and produce a report.

Once the reporting products are providing the information you need in support of your organization’s goals, BTI will work with your analyst to automate process mining capabilities in your organization.

Figure 3

To implement process mining in your organization, contact us today!

Previous ArticlePart 3: Process Mining to Implement Better DevOps Next ArticleProcess Mining and Essential Data Science Tips