Constructing your DevOps toolchain is one of the major tasks in implementing DevOps. While it is not as difficult as potentially changing the culture of the organization to allow DevOps to work, it can be a contentious activity since your organization will want to consolidate the implementation/maintenance of your DevOps toolchain under a single infrastructure team as well as settle on an 80% solution as to the tools that will make up the DevOps toolchain. An 80% solution is realistic because it is difficult, if not impossible, to build a toolchain that will have all the tools necessary to satisfy all teams implementing DevOps in an organization.
When choosing your DevOps tools, you can take one of two approaches — an end-to-end solution from one vendor or piecing together your toolchain based on tools you already use, best of breed tools and tools that integrate with the tools you select.
First, we will address an end-to-end solution from one vendor. This choice is easier from an integration standpoint because all the tools should integrate seamlessly. If not, what’s the point of buying a vendor’s solution? Integrating different vendors’ tools into an organization’s DevOps toolchain can be a project in itself. So, using a predefined toolchain from one vendor can kick start an organization’s DevOps effort. The end-to-end solution also provides you with availability to all the tools immediately. In many current tools/best-of-breed implementations, tools are brought onboard as time and availability allow. While this incremental implementation often makes sense for an organization, it can require teams to follow a pre-DevOps implementation process or utilize a temporary tool implementation until the organization is ready to integrate a tool into the organizational toolchain.
So, why not go out and buy a toolchain from the vendor of your choice? Here are a few considerations:
Cost – While building your own toolchain can be assembled with free or low-cost tools, most, if not all end-to-end solutions are not free. You need to consider that along with the culture change that your organization is going to go through with a DevOps implementation and the associated costs of the culture change. Can your organization also absorb the costs of new tools? However, the cost of building a custom toolchain is not small, especially when you consider the time to integrate the tools, so this is a good place to perform a cost/benefit analysis of the two different approaches if your organization has the option to pick an approach.
Legacy Data – If you have projects that have been correctly using your existing tools for months/years, there is a wealth of historical data in those tools. So, there are a few things to consider. Can your legacy data be imported into the end-to-end toolchain? Can using an archive of your old tools data provide you value you need? Can you live with the loss of your historical data?
Project’s Lifecycle – Where in the system/software development lifecycle are the projects that you want to use the DevOps tools? Transitioning to a new toolset in the middle of a development effort can be extremely difficult. That requires you to get the current data and code into the new toolchain and to train the team on the tools in the new toolchain. Can the project absorb these potential delays in their current development cycle? It is much easier to introduce new projects into a new end-to-end development solution and at least cutting over existing projects at strategic times in their development cycle rather than just when the toolchain becomes available. If this can be done, the end-to-end solution could be a better choice.
Technologies – Does the proposed end-to-end solution support all the technologies your organization uses/may want to use? Building your own toolchain allows you to bring in the tools you need to support the technologies used by your organization. There is the possibility that an end-to-end solution may not. This could either preclude your organization from using certain end-to-end toolchains that do not support the required technologies or will require you to integrate other tools into your end-to-end toolchain that will support the required technologies. This will increase your costs and require you to perform the tool integration that selecting an end-to-end solution is supposed to prevent.
While we pointed out a number of pitfalls regarding end-to-end/one provider toolchains, please do not think that we are against them. They can be valuable to the right organization and can eliminate costly and lengthy tool integration tasks. All we are trying to do is enlighten to possible considerations.
For organizations that already have tools in place that can serve in their DevOps toolchain, as long as these tools satisfy an 80% solution, these can be the starting point for building an already in place/ best of breed DevOps toolchain implementation. Another requirement for these tools is that they can integrate with other tools.
The place to start in building your toolchain is with an Agile management tool. Examples of Agile Management tools include Jira, Version One, Redmine and Jazz. BTI does not recommend one tool over another; your organization needs to determine which one works best for you. The four mentioned above, along with a host of others all perform the same essential functions. Hopefully, your organization is already using one, if not, this is the place to start your DevOps implementation. Agile development is the basis of DevOps. If you are not using Agile, you need to get it up and running successfully before implementing DevOps.
Agile implementation is much easier when using a tool. It gives you a basis for both managing your Agile development and to begin gathering some of the measures that are required for a successful DevOps implementation. While from a pure Agile perspective a tool is not required, a tool makes it much easier and is required for a DevOps implementation. Another consideration for your Agile Management tool, as well as all tools as you select to build your custom DevOps toolchain, is that it will integrate with the other tools you select. A tool that will not integrate with your other tools is pretty much a non-starter unless you are mandated to use a tool by a customer and then you will need to build an interface so that it will work with the rest of your toolchain. Your other, less than ideal options, is to only pick tools that will work with the required tool or build a separate toolchain from the primary organizational DevOps toolchain that will work the required tool. This can be quite expensive, so keep that in mind and be sure to raise this issue with whoever is requiring the use of the tool in question.
The next tool to consider is what you will use for continuous integration (i.e. one of the GIT family of tools). This is another type of tool that is commonly used in the industry, even for those organizations that are not using DevOps (or even Agile). If you are one of those organizations, hopefully, the CI tool you are already using will integrate with the rest of your toolchain and will satisfy your 80% solution. If you do not have one in place, research on this tool can happen in parallel with your search for an Agile Management tool or soon after your Agile Management tool implementation. Even if you go no further in implementing DevOps, if you started without an Agile Management tool nor a CI tool, once you get these implemented, you should have a much better controlled and stable development environment. Hopefully, you had configuration management control over your code before, but if not, your CI tool is now taking care of that.
Next, look into a unit testing tool (i.e. JUNIT if you are developing in Java). Your unit testing tool will need to work with your CI tool. As you build out your automated unit tests, you will hopefully start to see failed builds in your CI tool due to the failed automated unit tests. Automated testing is another basic tenet for DevOps. As much testing as possible needs to be automated to be truly DevOps. When we say as much as possible, we mean everything automated unless there are certain tests that are impossible to automate. Saying you are doing DevOps and not using automated testing, or just using a small set of automated tests, is not really doing DevOps from a textbook definition. It will take time to get all of your testing automated, especially for large applications where huge amounts of regression testing need to be automated. In our opinion, assuming everything else you need to “do” DevOps is in place, the time between starting to automate tests to when you are fully automated you are still “implementing” DevOps.
There is an entirely separate discussion as to what DevOps means to your organization, that we will not address in this article. For instance, we have taught DevOps Foundations classes that have banking, medical and DoD customers where security or regulation testing is handled by a different organization that is not playing in the DevOps arena. Is it enough for the organization to claim they are doing DevOps if they are “DevOpsing” up to where the application is staged for the other organization to start their testing? Or does an organization have to push their application into production for them to claim they are doing DevOps? Again, another topic for another day.
While we are discussing testing, an automated testing tool will need to be incorporated into your DevOps toolchain as well. This tool will allow you to automate your system and regression testing (i.e. Selenium). You will need to ensure that your automated testing tool works with the other tools in your toolchain and works with your development language. Many automated testing tools require “hooks” to be inserted into the developed source code to work. Remember to factor this into your implementation time for implementing the tool and for getting automated testing up and running. Just getting the tool up and running will not allow for the implementation of automated testing, additional steps are required as well.
These additional steps are twofold, the first being the refactoring of the code to allow it to work with the automated testing tool and new code to be written to work with the tool. Your developers will need to be trained in what is now expected in the developed source code and coding standards and associated documentation will need to be updated. Secondly, your current testers will need to have the coding skill set to use the automated testing tool. Your current testers may have this skillset already or they will need to be trained if they are coming from a manual testing environment. This is part of the culture change we talk about when implementing DevOps. If your current testers do not have the coding skillset and refuse to learn it, new testers will need to be brought in and your old testers will need to be reassigned (another topic for another day).
Once your developers have refactored the code to work with the automated testing tool, new code is written to work with the tool and your testers are ready to use the tool, they can start automating their tests. This is a time-consuming effort, especially for large, legacy applications. What many organizations do is either automate for the new/updated functionality developed in the current sprint or automate for the new/updated functionality and automate however much additional high-value regression tests can be done in the current sprint. While this will not get you fully automated regression testing for a large application quickly, it will at least begin to automate some of the necessary tests. Your organization will need to evaluate the value of automating all of your regression testing for large applications to determine the return on investment of fully automated testing.
Next, you will need to automate your deployment and delivery processes. We are not going to discuss this in great detail since the required tools are very environment-dependent. Depending on your environment, you could explore containerization with Docker and/or Kubernetes. As a quick summary, you will need a tool or tools that allow you to deploy your code to the applicable environments and deliver to the target environments/production.
Finally, you will need to bring monitoring and operations tools into your DevOps toolchain. These tools will allow you to monitor the health of the production environment and log issues as they are discovered in the production environment and reported to the help desk/support staff. A goal of the monitoring and operations tools is that as issues are documented and logged, that the issues can kick off a ticket back in the Agile Management tool to task the development team to begin work on addressing the issue. If like in many organizations, the development team is leading the DevOps push, you will now need to work with your Operations team to see what tools they are currently using and how those will fit into your DevOps toolchain (hopefully this is not the case, in DevOps we are one big happy family). If Operations is not currently using tools that are expected in a DevOps toolchain, these tools will need to be identified and implemented as any other tool.
Hopefully, these thoughts and recommendations can help you choose your DevOps tools and implement them. Remember, it’s not about one specific tool that is the latest and greatest, it’s about choosing the best DevOps tools for your organization. Forethought into how you build your toolchain and what tools go into it will potentially save on later pain. Finally, remember the 80% solution — you’re probably not going to satisfy all the programs and project needs in an organization. Aim for 80% and work on the other 20% after the 80% is implemented.
If you need assistance creating a DevOps toolchain or implementing DevOps, contact the DevOps experts at BTI.
