Designing an Internal Developer Platform (IDP) on AWS Using Backstage, EKS & Terraform

Introduction — Why IDPs Are Becoming Mandatory

If you've worked in engineering long enough, you already know the truth nobody wants to say out loud:

Most developers don’t want to touch YAML, Terraform, Kubernetes manifests, or IaC pipelines. They just want their service to run.

And honestly… can we blame them?
Between debugging a failing pod, hunting for IAM permissions, or figuring out why Terraform refuses to plan, developer energy gets drained on infrastructure instead of innovation.

That’s where Internal Developer Platforms (IDPs) come in.

IDPs are the “self-service layer” that removes friction.
The best IDPs allow developers to:

• scaffold services
• spin up infrastructure
• deploy to multiple environments
• observe their workloads
• and do all this without knowing how the underlying platform works

Today, you’ll learn how to build such a platform using:

✔ Backstage (developer portal)
✔ Terraform (self-service IaC)
✔ ArgoCD (GitOps)
✔ Amazon EKS (runtime platform)
✔ AWS native services like IAM, KMS, Secrets Manager

This is the same pattern used at modern engineering orgs and unicorns.

So let’s build a real one, the right way.

The Four Layers of a Modern IDP

Every successful platform has the same 4 layers:

1. Developer Experience Layer → Backstage 2. Platform Orchestration → Terraform + GitOps (ArgoCD) 3. Runtime Infrastructure → Amazon EKS + networking 4. Shared Services → Observability, security, secrets, CI 

Let’s break them down.

1. Developer Experience Layer — Backstage

Backstage gives you:

💠 Software Catalog

A single source of truth for all services:

• Ownership
• Deployments
• Documentation
• Tech stack
• Status

💠 Golden Paths (Software Templates)

This is the real magic.

Instead of telling developers:

“Hey, here’s the Confluence page with our recommended architecture. Follow it.”

You give them a self-service template that does everything:

• Create a Git repo
• Generate CI/CD pipelines
• Create Terraform infrastructure
• Register service in Backstage
• Deploy to EKS
• Wire up monitoring & alerts

With one click.

2. Platform Orchestration — Terraform + GitOps

This is where your platform’s power comes from.

🟦 Terraform modules

These represent reusable “building blocks”:

• microservice module
• VPC module
• RDS module
• DynamoDB table module
• S3 bucket module
• EKS namespace module

Developers never write Terraform.
They fill in a simple form in Backstage → Backstage generates Terraform config → GitOps takes over → ArgoCD deploys it.

🟥 GitOps (ArgoCD)

This ensures:

• everything is deployed from Git
• automatic rollbacks
• drift detection
• clear audit trails
• separation between build (CI) and deploy (CD)

Git is the source of truth.
ArgoCD is the engine.

3. Runtime Infrastructure — Amazon EKS

This is where workloads actually run.

Your EKS architecture includes:

• Multi-tenant namespaces
• IRSA (IAM Roles for Service Accounts)
• Karpenter for fast autoscaling
• Network Policies
• Pod Security Standards
• Ingress controller (ALB/Nginx/Ambassador)
• Internal service mesh (optional)

EKS is your “runtime engine”, Backstage is the steering wheel, and GitOps is the autopilot.

4. Shared Services Layer

This includes all the “invisible” but necessary pieces:

✔ Secrets

AWS Secrets Manager + IRSA.

✔ Observability

• Prometheus
• Grafana
• Tempo / Loki / OTel Collector

✔ Security

• KMS encryption
• Audit logs
• IAM boundaries
• Registry scanning

✔ CI Pipelines

GitHub Actions or GitLab CI.
CI builds → GitOps deploys.

Full Architecture flow diagram

🧬 Golden Path Template Example (Backstage)

Here’s a simplified template:

apiVersion: scaffolder.backstage.io/v1beta3 kind: Template metadata: name: aws-eks-service title: "Create a Service on EKS" description: "Scaffold a microservice with Terraform + GitOps" spec: owner: platform-team parameters: - title: "Service Info" properties: name: type: string description: Name of the service owner: type: string description: Team name steps: - id: template name: Create Repo from Template action: fetch:template - id: terraform name: Terraform Module action: publish:github input: repoUrl: "{{ parameters.name }}" files: - terraform/main.tf - id: register name: Register in Backstage Catalog action: catalog:register 

📦 Terraform Module Example (EKS Namespace)

module "namespace" { source = "git::https://github.com/org/platform-modules.git//eks-namespace" name = var.service_name team = var.team_name iam_roles = var.iam_roles } 

🔁 GitOps Flow (ArgoCD App)

apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: myservice spec: source: repoURL: https://github.com/org/myservice path: deploy/ destination: namespace: myservice server: https://kubernetes.default.svc syncPolicy: automated: prune: true selfHeal: true 

Real-World Challenges & Lessons Learned

No one tells you these things until you experience them:

• Templates need constant updating
• Teams will bypass your platform unless you make it genuinely delightful
• Backstage plugin versioning can be chaotic
• Keep the IDP simple: don’t over-engineer
• EKS upgrades require strict compatibility planning
• Terraform module contracts must be stable and versioned

But once it works, developer onboarding drops from weeks to minutes.

Final Thoughts — The Future of IDPs

The next evolution of IDPs will include:

• AI-powered troubleshooting
• Auto-generated infrastructure from natural language
• Predictive autoscaling
• Auto-remediation
• Learning-based deployment risk scoring
• AI-assisted documentation & code generation

And you’ll be ahead of the curve because you’re already building the foundation today.