Industrialize Infrastructure as Code in Government and Industry

Terraform Agents’ Best Practices for Regulated Industries

From IC Insider HashiCorp

By: Joshua Tracey, Resident Engineer, HashiCorp Federal, & Greg Thomas, Sr Solutions Engineer, HashiCorp Federal

YouTube: @The Sudo

Medium: @cobra16

Modern information technology (IT) abstracts infrastructure from physical hardware using logical constructs programmed in software. The challenge is having a unified way to provision infrastructure and take advantage of multi-cloud despite the skills gap, cost, compliance security, and operational overhead.

Most major Cloud Service Providers (CSPs) have integrated IaC tools within their vertical platform, such as Amazon Web Services Cloud Formation or Microsoft Azure Resource Manager; however, these tools do not scale horizontally across multiple CSPs, support traditional data centers or bare metal. Terraform provides IT administrators with the horizontal, multi-cloud workflow they need.

In our last article, infrastructure as code in government and regulated industries, we learned how to use a platform approach versus DIY scaffolding to obtain value from IaC in regulated industries and government. Recently, an interesting post by Defense Scoop announced that “Intelligence agencies (are) confronting challenges with multi-cloud environments.”  To effectively take advantage of multi-cloud in regulated industries and government, a platform approach is needed to close the skills gap and tighten security. This white paper reveals solutions that did not exist until recently. We aim to explain the often-overlooked benefits of Terraform Agents and HashiCorp integration with ServiceNow, taking a platform approach to IaC for multi-cloud environments.

Challenge: Behind the “push button automation,” there is always code. Even though we can abstract it away, it has to be codified. Terraform Cloud or Terraform Enterprise TFC/TFE are not designed to supplement CI/CD platforms. In most cases, work must be done before and after a Terraform plan and apply, or sometimes between a plan and apply. This leads to creating complex pipelines that, at some point, kick off a plan and apply against a TFC/TFE workspace and then have a mechanism to report back to the pipeline when the plan or apply is finished.

Solution: The hooks for agents help alleviate this by allowing actions pre-plan, post-plan, pre-apply, and post-apply. Imagine you wanted to run a pre-plan action on your Terraform code, then a post-plan action to validate the plan will apply, rather than waiting for a Terraform apply to fail 15 minutes in. Imagine you wanted to send an API call somewhere to notify someone that an apply has finished. You can take advantage of countless options and use cases like this with Terraform agents. You can also increase security by having a clear delineation point between agent pools assigned to respective workspaces.

Examples: Imagine an agency has a custom API to determine if resources can be provisioned. When everything is ok, the API returns “200”, meaning everything is ok, and resources are permitted to be deployed. In an emergency, audit, or security issue, the agency can push a button and force this API to return a 403 not-permitted. This gives complete control over all the infrastructure resources or assets deployed at a very granular level.

Below is an example of a custom Terraform Cloud Agent configured with a pre-plan hook that checks the API. If the API returns status 200, the pre-plan hook exits with a 0 exit code, allowing the plan to continue.

Next, a post-plan hook runs. In this particular deployment, an Azure Storage Account is being created. The name is always dynamically generated, so the team wants to ensure the generated name is globally unique before moving to the apply phase, waiting a few minutes, and then finding out the name is already taken. In this case, it saves time by allowing us to fail fast, iterate on the code and deliver the mission faster. Over the hundreds of thousands of resources deployed, this can save upwards of 10K hours a year.

Because all workspace variables are passed to the Terraform Cloud Agent as environment variables, we can securely interpolate variables into the script that checks the Azure API for the storage account name. The $AZURE_ACCESS_TOKEN variable in the script is taken from the Terraform Cloud Workspace variable below. The same applies to the $ARM_SUBSCRIPTION_ID and $STORAGE_ACCOUNT_NAME variables.

Finally, a post-apply hook is run. In this example, an internal system needs to be updated for audit purposes, a notification is sent via webhook to the workspace owner, and the Ansible automation platform API is called to trigger additional configurations.

Challenge: Resources often must be deployed into multiple secure and isolated environments. These environments usually cannot allow inbound connections from Terraform Cloud or Terraform Enterprise. Some of these environments only allow communication to cloud services via private endpoints. Other environments, such as air-gapped, only allow communication to internal applications such as vSphere from within the network.

Solution: Terraform Cloud Agents can be deployed into any isolated network to help solve this problem. Agents only require outbound connectivity to Terraform Cloud or Terraform Enterprise. The agent architecture is pull-based, so no inbound connectivity is required. Any agent you provision will poll Terraform Cloud or Terraform Enterprise for work and carry out the execution of that work locally. You can leverage ServiceNow and Terraform Enterprise for “push button automation” to get the most value out of multi-cloud environments. This approach allows you to follow the best practices for enterprise use of infrastructure as code for compliance and security, cost savings, and isolation for billing and programmatic purposes. By connecting ServiceNow to Terraform Cloud or Terraform Enterprise, the Terraform Service Catalog integration enables platform teams to provision self-serve infrastructure by creating workspaces and performing Terraform runs from the ServiceNow interface.

Thank you for reading and learning more about HashiCorp’s infrastructure automation solutions.

About HashiCorp

HashiCorp is the leader in multi-cloud infrastructure automation software. The HashiCorp software suite enables organizations to adopt consistent workflows to provision, secure, connect, and run any infrastructure for any application. HashiCorp open source tools Vagrant, Packer, Terraform, Vault, Consul, and Nomad are downloaded tens of millions of times each year and are broadly adopted by the Global 2000. Enterprise versions of these products enhance the open source tools with features that promote collaboration, operations, governance, and multi-data center functionality. The company is headquartered in San Francisco and backed by Mayfield, GGV Capital, Redpoint Ventures, True Ventures, IVP, and Bessemer Venture Partners. For more information, visit www.hashicorp.com or follow HashiCorp on Twitter @HashiCorp.

About IC Insiders

IC Insiders is a special sponsored feature that provides deep-dive analysis, interviews with IC leaders, perspective from industry experts, and more. Learn how your company can become an IC Insider.