Saturday, April 13, 2024

For DevOps Professionals: Automatically Generating Terraform Documentation

Today, I'm furthering my war on DevOps busy work. This is a continuation of a previous post: For DevOps Professionals: Analyzing and Eliminating Busy Work. For this post, I want to remove the tediousness of properly documenting Terraform code.

Terraform code documentation should detail the following to make it easy for other developers to consume and maintain a Terraform configuration or project. The bottom line is high quality and accurate documentation for Terraform code makes change easier and saves people time. The relevant quote from my previous post is:

Automatically generate Terraform documentation. Like many,  we use the open source product Terraform-docs to generate Terraform documentation.  We seem to be executing that generation locally. When pull requests are created or updated,  a GitHub Workflow that automatically executes the documentation generation and pushes any changes would eliminate the manual step.

Terraform code documentation should have the following components. Without this information, anyone consuming or changing the project would be forced to audit the code line by line.  Here's an example from one of my Terraform modules.

  1. A concise description of what the Terraform project supports
  2. For reusable modules, a usage example
  3. Details about all inputs, including any defaults
  4. Details for all outputs
  5. Identification of all Terraform resources maintained
  6. A listing of any modules used by the Terraform project
  7. Provider requirements, including versions

Keeping these documentation items current is tedious and boring. Consequently, it usually becomes out of date quickly. Fortunately, the open-source product Terraform Docs can generate this information. My team uses this product religiously. 

We go further and have Terraform documentation updated automatically for every pull request update. As we're largely using GitHub, our Terraform projects update documentation automatically when our developers create or update a pull request. We've encapsulated the regeneration logic into GitHub action terraform-docs, making it easy to reuse for all Terraform projects in all our repositories. This concept is usable for most CI/CD toolsets.

Using the terraform-docs action is relatively straightforward. That said, I might take the step of creating a reusable GitHub workflow to further streamline usage. Here's an example of workflow usage.

The terraform-docs action is publicly available under the MIT open-source license. You're free to use it directly or fork it if you prefer.

If you use it directly, we recommend explicitly referencing a release tag. That way, you're not accidentally impacted by changes to the action going forward.

I'm always interested in your views on the topic. Especially if you've discovered additional ways to make documenting Terraform code higher quality and easier to maintain.

What I'm Currently Reading:

Accelerate DevOps with GitHub: Enhance software delivery performance with GitHub Issues, Projects, Actions, and Advanced Security. Kaufmann, Michael: Packt Publishing; 1st edition (September 9, 2022)


Sunday, April 7, 2024

For DevOps Professionals: Analyzing and Eliminating Busy Work

I periodically go through the exercise of identifying areas of busy work my team and I experience as DevOps professionals concentrating in infrastructure code development and maintenance.  While I do have a specific area of focus for my introspection today,  this is a good exercise for anyone in almost any vertical. 

I seem to be concentrating on Terraform IaC these days and migrating to GitHub workflows for CiCd capabilities.  We're entirely GitOps and 100% Infrastructure as Code. The opportunities I identify center on my current areas of focus.  Your areas of focus,  should you do a similar introspection,  might identify different opportunities. 

I've identified several opportunities for reducing and eliminating busy work for me and my team. I expect this list to evolve as some of these items are implemented.  This is an evolution for us. 
 
Automatically format Terraform code. There's no real need for people to manually format Terraform code.  Formatting aids and speeds your ability to read and understand code.  It's reading speed that provides the productivity benefit.  Terraform has a format utility that does a reasonable job (I realize that it's not perfect). When pull requests are created or updated,  a GitHub Workflow that automatically executes the formatting and pushes any changes would eliminate manual formatting. 

Automatically generate Terraform documentation. Like many,  we use the open source product Terraform-docs to generate Terraform documentation.  We seem to be executing that generation locally. When pull requests are created or updated,  a GitHub Workflow that automatically executes the documentation generation and pushes any changes would eliminate the manual step. Update: This topic is addressed in more depth in the post "For DevOps Professionals: Automatically Generating Terraform Documentation."

Automatically tag Terraform applies. Terraform's backend state paradigm pushes most into trunk-based development. Yes,  I realize that's where all people should be,  but that topic is a discussion for another day. Trunk-based development for Terraform introduces a hassle managing change as they are promoted through various environments.  We've settled on tagging to track which version is applied to which environment.  That tracking has been manual,  but there's really no need for that.  It's much better and more reliable to automatically tag the version of code that has been successfully applied. Not only would manual work be eliminated,  but the quality of the tagging would improve. 

Make Terraform state manipulation easy. We've been incorporating Terraform state manipulation operations into Terraform plan pipelines for years.  Now that we're transitioning to GitHub, those capabilities should transition too. The need for Terraform state operations is relatively rare,  but is critical when the need arises. 

Automate Terraform drift detection. Terraform introduced a capability of reporting "drift" in Terraform plan output. This capability has incrementally improved over time and become more useful.  I expect the quality of that drift output to continue to improve.  It's now possible to schedule workflows that perform Terraform plans and alert any drift detected. This saves unplanned work investigating drift encountered when making configuration changes. 

Automate Terraform module testing. We've encountered shifting sand with Terraform modules originating with cloud technology and security team policy changes.  While automated testing on a scheduled basis doesn't eliminate the problem,  it alerts platform engineering personnel to the issue.  That alert means that its possible to fix the issue before individual application teams are negatively impacted. 

I need an easily customizable templating assistant. The amount of time I copy/paste/change GitHub workflows and Terraform projects disturbs me and other members of my team.  I really need a personal Backstage.io implementation that is easy to implement and maintain.  Backstage implementations are complex and require more labor than most teams can afford.  Additionally, templating is a small percentage of Backstage capabilities. A Visual Studio Code extension called Components Boilerplate seems close to this idea,  but isn't quite there. 

This list will be my areas of focus over the coming weeks.  I'd like to know your thoughts on these items and other opportunities for productivity improvement you see in your teams.

What I'm Currently Reading:
Accelerate DevOps with GitHub: Enhance software delivery performance with GitHub Issues, Projects, Actions, and Advanced Security. Kaufmann, Michael: Packt Publishing; 1st edition (September 9, 2022)