Terraform YAML: A Practical, Human-Friendly Guide

Terraform is among the most capable infrastructure-as-code (IaC) tools in the DevOps universe. Historically, it employs its own configuration language named HCL (HashiCorp Configuration Language). In contrast, YAML (Yet Another Markup Language) has emerged as the de facto format for specification of configuration in most tools such as Kubernetes, Ansible, GitHub Actions, and many others. But did you know you can actually integrate Terraform YAML seamlessly to get flexibility as well as maintainability in your infrastructure processes?

This book delves into how exactly you can employ YAML with Terraform. From unpacking YAML files to input data into your Terraform resources or transforming YAML manifests into Terraform-native HCL, there are a host of use cases that this pairing can handle. You’ll not only learn the “how,” but the “why,” through hands-on examples, best practices, and real-world examples.

Why Use YAML with Terraform

Terraform YAML excels at describing infrastructure declaratively and predictably. But HCL is verbose and redundant when describing structured data, particularly when handling complex setups or long lists. YAML, on the other hand, excels in keeping structured data both readable and compact.

With Terraform YAML, you have the best of both worlds. YAML enables you to declare environment-specific variables, lists, maps, and so on in a tidy, concise syntax. Terraform can then import that YAML data and use it dynamically in resource declarations. Separation of concerns also improves maintainability you can keep the static infrastructure code in HCL and dynamic config in YAML. This comes in really handy in use cases such as CI/CD pipelines, where you need to specify deployments or resources from external configuration files.

Reading YAML Files in Terraform Using yamldecode

You can read YAML files directly in Terraform using the yamldecode() function. The yamldecode() function reads a YAML file and returns a Terraform-native data model such as a map or list. For instance, assume that you have a YAML file config.yaml specifying a virtual machine:

name: my-vm<br>size: large

You can load this file in Terraform with:

<code>locals {<br>config = yamldecode(file("${path.module}/config.yaml"))<br>}<br><br>resource "my_cloud_vm" "example" {<br>name = local.config.name<br>size = local.config.size<br>}</code>

This method enables you to separate variable values from your HCL configuration. You can use it to specify everything from VM specs, Kubernetes service names, up to multi-cloud credentials.

Working with Lists and Maps from YAML Files

YAML is a very structured format, so it is ideal for specifying lists and nested maps. As an example, suppose you have a YAML file containing a list of VMs:

vms:<br>- name: vm1<br>size: small<br>- name: vm2<br>size: medium

In Terraform, you can read this file and iterate through the list to dynamically create resources:

locals {<br>vm_list = yamldecode(file("${path.module}/vm_list.yaml"))<br>}<br><br>resource "my_vm" "instance" {<br>count = length(local.vm_list.vms)<br>name = local.vm_list.vms[count.index].name<br>size = local.vm_list.vms[count.index].size<br>}

This allows you to have all the infrastructure data in a YAML file and have Terraform YAML create the logic. It greatly minimizes code duplication and is an excellent practice for complex deployments.

Applying Conditional Logic in YAML-Based Terraform Code

One of the helpful ways to use Terraform YAML is to make use of conditional logic. As an example, you may wish to have different configurations in production and development environments. You can have a YAML file that includes environment flags:

env: production<br>tags:<br>production: "true"<br>development: "false"

Then you can utilize this configuration in Terraform to dynamically create resource tags:<br><br>locals {<br>cfg = yamldecode(file("${path.module}/env_config.yaml"))<br>}<br><br>resource "some_resource" "example" {<br>tags = {<br>Environment = local.cfg.env<br>Production = local.cfg.tags.production<br>Development = local.cfg.tags.development<br>}<br>}

With this approach, you avoid hardcoding environment-specific values in Terraform and make it easy to switch configurations just by modifying the YAML file.

Writing YAML from Terraform Using yamlencode

Although the most typical example of usage is reading YAML, Terraform can also produce YAML through the yamlencode() function. This is particularly helpful when Terraform must output configuration files to be used by other tools like Ansible or Kubernetes.

Here’s an example:

locals {<br>config_data = {<br>name = "app"<br>image = "nginx"<br>}<br>yaml_output = yamlencode(local.config_data)<br>}<br><br>output "yaml_config" {<br>value = local.yaml_output<br>}

This output can be piped into a file and consumed as a configuration file within downstream tools or CI/CD processes.

Type Mapping Between YAML and HCL

Terraform does well in mapping YAML and HCL types. YAML’s strings, integers, booleans, lists, and dictionaries (maps) all map cleanly to Terraform’s internal types. For instance, a YAML list is mapped as a tuple or list in Terraform, and a YAML dictionary as a map or object. Even null values are translated correctly.

This implicit type mapping is something that you generally don’t need to concern yourself with while casting or converting types by hand while using yamldecode() or yamlencode(). Terraform does this for you.

Converting from YAML to HCL

Often, you may already have YAML files particularly Kubernetes manifests and you’d like to convert them to the HCL format used by Terraform. There are a number of tools and techniques that can assist with this.

One choice is to convert the YAML into Terraform directly via the console feature:

terraform console<br>> yamldecode(file("deployment.yaml"))

This will display the parsed YAML as a Terraform object. You can simply copy and modify it into an actual HCL resource definition.

For Kubernetes, there are tools such as tfk8s and k2tf that enable you to translate entire YAML manifests into HCL. They are very handy if you are transitioning from kubectl-based deployments to Terraform-managed infrastructure.

With Tools Like tfk8s and k2tf

Two of the most widely used tools to migrate Kubernetes YAML to Terraform are tfk8s and k2tf.

tfk8s is an open-source tool that reads Kubernetes YAML manifests and translates them into Terraform HCL format. It has support for various types of resources and can handle comments and annotations beautifully. You simply execute:

Enhance Your CyerPanel Experience Today!

Discover a world of enhanced features and show your support for our ongoing development with CyberPanel add-ons. Elevate your experience today!

Learn More

<code>tfk8s -f deployment.yaml > deployment.tf</code>

Likewise, k2tf can accept YAML input and generate HCL output. Both packages are community-developed and well-known in the DevOps sphere.

Both packages are especially useful when importing legacy workloads from Helm or kubectl into Terraform, or when you’d like the same infrastructure definition language throughout the entire stack.

Encoding/Decoding YAML using the Terraform Console

Terraform YAML’s interactive console is an underutilized but very useful tool for experimenting with YAML conversions.

You can do:

terrafrom console<br>> yamlencode({ name = "test", version = "1.0.0" })<br>Or:

> yamldecode(file("some.yaml"))

This is useful for debugging or when attempting to get a better idea of how Terraform interprets your YAML files.

Kubernetes Native Support for YAML in Terraform

As of Terraform 1.x, the Kubernetes provider has been enhanced to natively support YAML manifests through the kubernetes_manifest resource. You can inject an entire Terraform YAML manifest directly into this resource using yamldecode().

Here’s an example:

resource "kubernetes_manifest" "example" {<br>manifest = yamldecode(file("${path.module}/deployment.yaml"))<br>}

This feature enables you to work with Kubernetes resources natively in Terraform, yet continue to store your manifest files in the popular YAML format.

Provider Configuration When Using YAML

When you are using YAML for configuration, Terraform providers remain configured in HCL. But you can refer the values from the YAML file within the provider block. For instance, if your YAML file has AWS region and profile details:

aws_region: us-east-1<br>aws_profile: dev

You can configure the provider as follows:

locals {<br>config = yamldecode(file("config.yaml"))<br>}<br>provider "aws" {<br>region = local.config.aws_region<br>profile = local.config.aws_profile<br>}

This design enables you to store sensitive or environment-dependent configuration outside the HCL files, enhancing security and maintainability.

Real-World Example: YAML-Driven Kubernetes Microservices

Assume you have a YAML file that specifies a microservices architecture consisting of frontend and backend services:

services:<br>- name: frontend<br>replicas: 3<br>image: nginx:latest<br>- name: backend<br>replicas: 2<br>image: myapp/backend:1.2.0<br>env: production

You can use Terraform YAML to loop through these services and create Kubernetes deployments for each:

locals {<br>cfg = yamldecode(file("${path.module}/services.yaml"))<br>}<br>resource "kubernetes_deployment" "services" {<br>for_each = { for svc in local.cfg.services : svc.name => svc }<br>metadata {<br>name = each.key<br>labels = {<br>app = each.key<br>environment = local.cfg.env<br>}<br>}<br>spec {<br>replicas = each.value.replicas<br>selector {<br>match_labels = {<br>app = each.key<br>}<br>}<br>template {<br>metadata {<br>labels = {<br>app = each.key<br>}<br>spec {<br>container {<br>name = each.key<br>image = each.value.image<br>}<br>}<br>}<br>}<br>}

This method allows you to define your architecture in a YAML file and apply changes with Terraform without writing separate resource blocks for each microservice.

Best Practices for Terraform + YAML Integration

When used together with YAML, some best practices can go a long way toward ensuring stability and readability. First, never submit YAML files without using tools such as yamllint to validate them against indentation and syntax issues. Second, confirm that you are familiar with what yamldecode() returns as data by trying it in the Terraform console. Third, place YAML and HCL together in your version control to ensure consistency and traceability. Avoid hardcoding secrets in YAML files tools utilize Terraform variables or a secret manager instead. Finally, make your Terraform YAML files organized and modularize whenever possible. To make everything more manageable, split large YAML files into smaller ones per resource type or environment.

Final Thoughts

Terraform YAML are from different universes, but together they are an awesome, flexible, and scalable infrastructure-as-code combination. YAML provides simplicity and structure for input definition, while Terraform gives you solid logic and resource orchestration. From handling Kubernetes deployments to cloud resource provisioning and crafting advanced pipelines, using both will enable you to write cleaner, more maintainable, and automation-friendly code. With yamldecode and yamlencode as built-in functions and tfk8s and k2tf tools, the integration has never been simpler.

FAQs