thothctl

Platform Engineering: Template Creation & Publishing

Overview

Platform engineers can create reusable infrastructure templates from existing projects (blueprints, POCs, or reference architectures) and publish them for consumption through Internal Developer Platforms like Backstage.

Template Format: ThothCTL uses #{ }# placeholder expressions for parameterization, not Jinja2 syntax.

Example Blueprint: terragrunt_ecs_blueprint - Production-ready ECS Fargate architecture with ALB, RDS, and multi-AZ deployment.

Use Case: From Project to Template

Scenario

A platform engineering team has developed a production-ready reference architecture for a microservices platform. They want to:

Convert the working project into a reusable template
Parameterize environment-specific values using #{ }# expressions
Publish to a Git repository
Make it available in Backstage for self-service

Workflow

%%{init: {'theme':'base', 'themeVariables': {
  'primaryColor':'#3b82f6',
  'primaryTextColor':'#ffffff',
  'primaryBorderColor':'#2563eb',
  'lineColor':'#94a3b8',
  'secondaryColor':'#10b981',
  'tertiaryColor':'#8b5cf6',
  'background':'transparent',
  'mainBkg':'#3b82f6',
  'secondBkg':'#10b981',
  'tertiaryBkg':'#8b5cf6',
  'clusterBkg':'rgba(241, 245, 249, 0.05)',
  'clusterBorder':'#475569',
  'titleColor':'currentColor',
  'edgeLabelBackground':'transparent',
  'nodeTextColor':'#ffffff',
  'textColor':'currentColor',
  'nodeBorder':'#1e293b',
  'fontSize':'14px'
}}}%%
graph LR
    A["Working<br/>Project"] --> B["Convert to<br/>Template"]
    B --> C["Parameterize<br/>Values"]
    C --> D["Test<br/>Template"]
    D --> E["Publish to<br/>Git"]
    E --> F["Register in<br/>Backstage"]
    F --> G["Self-Service<br/>Consumption"]
    
    classDef projectStyle fill:#3b82f6,stroke:#60a5fa,stroke-width:2px,color:#fff
    classDef processStyle fill:#10b981,stroke:#34d399,stroke-width:2px,color:#fff
    classDef publishStyle fill:#8b5cf6,stroke:#a78bfa,stroke-width:2px,color:#fff
    
    class A projectStyle
    class B,C,D processStyle
    class E,F,G publishStyle

Step-by-Step Guide

Step 1: Prepare Your Reference Architecture

Ensure your project is production-ready:

# Navigate to your reference architecture
cd /path/to/microservices-platform

# Verify structure
tree -L 2

Example Project Structure:

microservices-platform/
├── common/
│   ├── backend.tf
│   └── variables.tf
├── environments/
│   ├── dev/
│   ├── staging/
│   └── prod/
├── stacks/
│   ├── networking/
│   ├── compute/
│   ├── database/
│   └── monitoring/
├── docs/
│   └── README.md
└── root.hcl

Step 2: Convert Project to Template

Use ThothCTL to convert your project into a reusable template:

# Convert project to template
thothctl project convert --make-template --template-project-type terraform

# Or for Terraform with Terragrunt
thothctl project convert --make-template --template-project-type terraform-terragrunt

# Or for standalone Terragrunt
thothctl project convert --make-template --template-project-type terragrunt

# Or for OpenTofu
thothctl project convert --make-template --template-project-type tofu

# Or for CDK v2
thothctl project convert --make-template --template-project-type cdkv2

# Or for Terraform modules
thothctl project convert --make-template --template-project-type terraform_module

# Or for custom project types
thothctl project convert --make-template --template-project-type custom

What This Does:

Identifies hardcoded values from [project_properties] in .thothcf.toml
Creates #{placeholder}# expressions in project files
Saves the template to ~/.thothcf/<project_name>/
Preserves project structure
Creates parameterized files

Step 3: Review Generated Template

The conversion creates template files with #{ }# placeholder expressions:

Before (Project):

# environments/prod/main.tf
resource "aws_vpc" "main" {
  cidr_block = "10.0.0.0/16"
  
  tags = {
    Name        = "microservices-vpc"
    Environment = "production"
    Team        = "platform-engineering"
  }
}

After (Template):

# environments/#{environment}#/main.tf
resource "aws_vpc" "main" {
  cidr_block = "#{vpc_cidr}#"
  
  tags = {
    Name        = "#{project_name}#-vpc"
    Environment = "#{environment}#"
    Team        = "#{team_name}#"
  }
}

Example from Blueprint:

# common/common.tfvars
project_name = "#{project}#"
environment  = "#{environment}#"
region       = "#{backend_region}#"
dynamodb_table = "#{dynamodb_backend}#"
backend_bucket = "#{backend_bucket}#"

Step 3.1: Manual Replacement (Alternative)

You can also manually replace values using your IDE or shell script:

# Example replacements from terragrunt_ecs_blueprint
project_name = "#{project}#"
environment  = "#{environment}#"
region       = "#{backend_region}#"
dynamodb_table = "#{dynamodb_backend}#"
backend_bucket = "#{backend_bucket}#"

Using find and replace in your IDE:

Find: "my-project-name" → Replace: "#{project}#"
Find: "production" → Replace: "#{environment}#"
Find: "us-east-1" → Replace: "#{backend_region}#"

Step 3.5: Configure .thothcf.toml

The .thothcf.toml file is critical for template functionality. It maps #{ }# placeholders to parameters with validation rules and defines project structure requirements.

Create .thothcf.toml in your template root:

[thothcf]
project_id = "microservices-platform"
project_type = "terraform"

# Define template parameters with validation
[template_input_parameters.project_name]
template_value = "#{project}#"
condition = "^[a-zA-Z0-9_-]+$"
description = "Project Name"

[template_input_parameters.environment]
template_value = "#{environment}#"
condition = "(dev|qa|stg|staging|test|prod)"
description = "Environment name (dev|qa|stg|staging|test|prod)"

[template_input_parameters.deployment_region]
template_value = "#{backend_region}#"
condition = "^[a-z]{2}-[a-z]{4,10}-\\d$"
description = "AWS Region for deployment"

[template_input_parameters.backend_bucket]
template_value = "#{backend_bucket}#"
condition = "^[a-z0-9][a-z0-9.-]{1,61}[a-z0-9]$"
description = "S3 bucket for Terraform state"

[template_input_parameters.backend_region]
template_value = "#{backend_region}#"
condition = "^[a-z]{2}-[a-z]{4,10}-\\d$"
description = "Backend AWS Region"

[template_input_parameters.backend_dynamodb]
template_value = "#{dynamodb_backend}#"
condition = "^[a-zA-Z0-9_.-]{3,255}$"
description = "DynamoDB table for state locking"

[template_input_parameters.owner]
template_value = "#{team_name}#"
condition = "^[a-zA-Z0-9_-]+$"
description = "Team or role owner for this deployment"

[template_input_parameters.vpc_cidr]
template_value = "#{vpc_cidr}#"
condition = "^\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}/\\d{1,2}$"
description = "VPC CIDR block"

[template_input_parameters.cloud_provider]
template_value = "aws"
condition = "(aws|azure|oci|gcp)"
description = "Cloud provider (aws|azure|oci|gcp)"

[template_input_parameters.deployment_profile]
template_value = "default"
condition = "^[a-zA-Z0-9_.-]{3,255}$"
description = "AWS CLI deployment profile"

# Define project structure requirements
[project_structure]
root_files = [
   ".gitignore",
   "README.md",
   ".thothcf.toml"
]
ignore_folders = [
    ".git",
    ".terraform",
    "Reports"
]

[[project_structure.folders]]
name = "common"
mandatory = true
type = "root"
content = ["backend.tf", "variables.tf"]

[[project_structure.folders]]
name = "environments"
mandatory = true
type = "root"

[[project_structure.folders]]
name = "stacks"
mandatory = true
type = "root"

Key Components:

Project Metadata: Identifies template type and ID
Template Parameters: Maps each #{ }# placeholder to a parameter with:
- template_value: The placeholder used in files
- condition: Regex validation pattern
- description: Human-readable description
Project Structure: Defines required files and folders for validation

Optional: Backstage template.yaml

If you plan to integrate with Backstage for self-service, you can create a separate template.yaml in Backstage scaffolder format (see Step 6 below). The .thothcf.toml is the primary configuration for ThothCTL’s template engine.

Step 4: Test Template Locally

Before publishing, test the template by converting it back to a project:

# Create a test project from the template
thothctl project convert --make-project --template-project-type terraform

# Verify generated project
cd test-microservices
terraform init
terraform plan

Step 5: Publish Template to Git

Push your template to a Git repository:

# Initialize Git repository
cd microservices-platform
git init
git add .
git commit -m "feat: create microservices platform template"

# Push to remote (GitHub, GitLab, Azure DevOps)
git remote add origin https://github.com/your-org/microservices-platform-template.git
git push -u origin main

# Tag the release
git tag -a v1.0.0 -m "Release v1.0.0"
git push origin v1.0.0

Step 6: Register in Backstage

Create a Backstage template definition:

template.yaml (Backstage format):

apiVersion: scaffolder.backstage.io/v1beta3
kind: Template
metadata:
  name: microservices-platform
  title: Microservices Platform on AWS
  description: Production-ready microservices infrastructure with networking, compute, database, and monitoring
  tags:
    - terraform
    - aws
    - microservices
    - platform-engineering
spec:
  owner: platform-team
  type: infrastructure
  
  parameters:
    - title: Project Information
      required:
        - project_name
        - team_name
      properties:
        project_name:
          title: Project Name
          type: string
          description: Name for your microservices platform
        team_name:
          title: Team Name
          type: string
          description: Team responsible for this infrastructure
        environment:
          title: Environment
          type: string
          description: Target environment
          enum:
            - dev
            - staging
            - prod
          default: dev
    
    - title: Network Configuration
      properties:
        vpc_cidr:
          title: VPC CIDR Block
          type: string
          description: CIDR block for the VPC
          default: "10.0.0.0/16"
        aws_region:
          title: AWS Region
          type: string
          description: AWS region for deployment
          default: us-east-1
          enum:
            - us-east-1
            - us-west-2
            - eu-west-1
  
  steps:
    - id: fetch-template
      name: Fetch Template
      action: fetch:template
      input:
        url: https://github.com/your-org/microservices-platform-template
        values:
          project_name: $
          team_name: $
          environment: $
          vpc_cidr: $
          aws_region: $
    
    - id: publish
      name: Publish to GitHub
      action: publish:github
      input:
        allowedHosts: ['github.com']
        description: $ - Microservices Platform
        repoUrl: github.com?owner=your-org&repo=$
    
    - id: register
      name: Register Component
      action: catalog:register
      input:
        repoContentsUrl: $
        catalogInfoPath: '/catalog-info.yaml'
  
  output:
    links:
      - title: Repository
        url: $
      - title: Open in Backstage
        icon: catalog
        entityRef: $

Step 7: Self-Service Consumption

Developers can now use the template through Backstage:

Navigate to Backstage Software Catalog
Click “Create Component”
Select “Microservices Platform on AWS”
Fill in parameters
Click “Create”

Result: A new repository with fully configured infrastructure ready for deployment.

Advanced: Converting Between Frameworks

Terragrunt to Terramate

For advanced deployment patterns:

# Convert to Terramate stacks
thothctl project convert \
  --make-terramate-stacks \
  --branch-name main

# This creates Terramate stack structure
# with proper dependencies and ordering

Template to Project

Convert a template back to a working project:

# Create project from template
thothctl project convert \
  --make-project \
  --template-project-type terraform

Use Cases

1. Blueprint Templates

Scenario: Create standardized blueprints for common architectures

Examples:

Three-tier web application
Serverless API platform
Data lake infrastructure
ML/AI platform

Command:

thothctl project convert --make-template --template-project-type terraform

2. POC to Production Template

Scenario: Convert successful POC into production-ready template

Workflow:

Develop and validate POC
Harden security and compliance
Convert to template
Publish for team use

Command:

thothctl project convert --make-template --template-project-type terraform

3. Reference Architecture Library

Scenario: Build a library of reference architectures

Structure:

reference-architectures/
├── microservices-platform/
├── data-analytics-platform/
├── ml-training-platform/
└── serverless-api-platform/

Each template includes:

Infrastructure code
Documentation
Best practices
Cost estimates
Security configurations

4. Multi-Cloud Templates

Scenario: Create templates for different cloud providers

Examples:

AWS microservices template
Azure microservices template
GCP microservices template

Benefit: Consistent architecture across clouds

Integration with CI/CD

GitHub Actions

name: Validate Template

on:
  push:
    branches: [main]
  pull_request:

jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      
      - name: Install ThothCTL
        run: pip install thothctl
      
      - name: Test Template
        run: |
          cd templates/source-project
          thothctl project convert --make-project --template-project-type terraform
          terraform init
          terraform validate

GitLab CI

validate-template:
  stage: test
  script:
    - pip install thothctl
    - thothctl project convert --make-project --template-project-type terraform
    - terraform init && terraform validate

Best Practices

1. Template Design

✅ Do:

Use meaningful parameter names
Provide sensible defaults
Include comprehensive documentation
Add validation rules
Include examples

❌ Don’t:

Hardcode sensitive values
Over-parameterize (keep it simple)
Skip documentation
Ignore security best practices

2. Version Control

✅ Do:

Use semantic versioning (v1.0.0)
Tag releases
Maintain CHANGELOG.md
Document breaking changes

3. Documentation

Include in your template:

README.md - Overview and quick start
USAGE.md - Detailed usage guide
ARCHITECTURE.md - Architecture decisions
PARAMETERS.md - Parameter reference
EXAMPLES.md - Usage examples

4. Testing

Test your templates:

Unit tests (Terraform validate)
Integration tests (actual deployment)
Security scans (Checkov, Trivy)
Cost analysis (ThothCTL cost analysis)

Template Repository Structure

microservices-platform-template/
├── .github/
│   └── workflows/
│       └── validate.yml
├── common/
│   ├── backend.tf
│   └── variables.tf
├── environments/
│   └── #{environment}#/
├── stacks/
│   ├── networking/
│   ├── compute/
│   ├── database/
│   └── monitoring/
├── docs/
│   ├── README.md
│   ├── ARCHITECTURE.md
│   └── PARAMETERS.md
├── examples/
│   ├── dev-example.yaml
│   ├── staging-example.yaml
│   └── prod-example.yaml
├── .thothcf.toml           # ThothCTL template config (parameters + structure)
├── backstage-template.yaml  # Backstage template config
├── catalog-info.yaml        # Backstage catalog entry
├── CHANGELOG.md
└── LICENSE

Commands Reference

Command	Purpose
`thothctl project convert --make-template`	Convert project to template
`thothctl project convert --make-project`	Convert template to project
`thothctl project convert --make-terramate-stacks`	Create Terramate stacks
`thothctl init project --reuse --space <name>`	Create project from VCS template
`thothctl project bootstrap`	Add ThothCTL support to existing project
`thothctl project upgrade`	Upgrade project from template

Benefits

For Platform Engineers

✅ Standardization - Consistent infrastructure across teams
✅ Reusability - Write once, use many times
✅ Governance - Enforce best practices and compliance
✅ Efficiency - Reduce time from idea to deployment
✅ Quality - Battle-tested, production-ready templates

For Development Teams

✅ Self-Service - Deploy infrastructure without platform team
✅ Speed - Minutes instead of weeks
✅ Confidence - Pre-validated, secure configurations
✅ Focus - Spend time on features, not infrastructure
✅ Consistency - Same patterns across projects

Next Steps

Identify your most common infrastructure patterns
Create reference implementations
Convert to templates using ThothCTL
Publish to Git repositories
Register in Backstage or your IDP
Enable self-service for development teams

Platform Engineering Team
Enabling developer velocity through self-service infrastructure

This site is open source. Improve this page.

thothctl

Platform Engineering: Template Creation & Publishing

Overview

Use Case: From Project to Template

Scenario

Workflow

Step-by-Step Guide

Step 1: Prepare Your Reference Architecture

Step 2: Convert Project to Template

Step 3: Review Generated Template

Step 3.1: Manual Replacement (Alternative)

Step 3.5: Configure .thothcf.toml

Step 4: Test Template Locally

Step 5: Publish Template to Git

Step 6: Register in Backstage

Step 7: Self-Service Consumption

Advanced: Converting Between Frameworks

Terragrunt to Terramate

Template to Project

Use Cases

1. Blueprint Templates

2. POC to Production Template

3. Reference Architecture Library

4. Multi-Cloud Templates

Integration with CI/CD

GitHub Actions

GitLab CI

Best Practices

1. Template Design

2. Version Control

3. Documentation

4. Testing

Template Repository Structure

Commands Reference

Benefits

For Platform Engineers

For Development Teams

Next Steps

Related Documentation