Kubernetes vs Docker Compose for Internal Developer Platform Services
Run internal services on Compose when a single host and modest traffic suffice.
Kubernetes runs most production cloud-native workloads in 2025, and that fact alone has convinced a lot of platform teams that every service in an internal developer platform belongs on a cluster. That instinct is wrong more often than it's right, and it's worth saying plainly: most internal services do not need Kubernetes, and putting them there is a cost with no offsetting benefit. An IDP is a stack of layers with wildly different reliability, scale, and audience demands, and the platforms that stay operable are the ones that match orchestration complexity to what each layer actually needs.
Adoption numbers explain the default, but they don't justify it. Industry data puts overall Kubernetes usage, piloting, or evaluation at 93% of organizations, concentrated heavily in companies with 1,000-plus employees, which is exactly the population building internal platforms. That's a strong signal about where the industry landed, but it says nothing about which services inside a given platform actually need that complexity. Running Kubernetes well costs real money before a product engineer writes a line of application code: a security-hardened cluster runs close to six figures in first-year labor, with tens of thousands more in annual overhead, and mid-sized deployments can burn well over $150,000 a year in Kubernetes-specific engineering time. Kubernetes earns that cost when the workload calls for it, but most platform teams never ask whether the internal wiki, the CI runner sidecar, or the local dev environment actually needs any of what a cluster provides. They just assume it does, and each one ends up paying cluster tax for capabilities it will never touch.
What each tool actually does, and where each one stops
Docker Compose orchestrates services on a single host. A compose.yaml file lays out services, networks, and volumes in one place, and any developer who already knows Docker can read it, change it, and run docker compose up without learning a second system. There's no scheduler moving workloads across machines, no distributed state to reconcile, no self-healing when a node dies, because there's only one host to begin with. That's the ceiling, and for a large share of internal services, it's a ceiling that never gets tested.
Kubernetes solves a different problem: coordinating workloads across many machines, with a scheduler deciding placement, health checks driving restarts, rolling updates replacing containers without downtime, and an API surface (Pods, Deployments, Services, Ingress, ConfigMaps, Namespaces) built to model distributed systems formally rather than by convention. Canary releases, blue-green deployments, and zero-downtime rollbacks are native patterns here. That expressive power costs configuration surface, though, and there's a real crossover point where the trade tips. At high container counts, in the thousands, Kubernetes delivers multiples of the cluster-wide throughput a single host can produce. Below roughly 20 containers, Compose matches that throughput with a fraction of the operational complexity. That crossover is a fact about the workload, not a matter of taste, and most teams place it wrong by defaulting to Kubernetes before they've counted containers at all.
Docker Swarm is worth a mention only to close the door on it: it offered multi-node orchestration with Compose-like syntax, but feature development stopped years ago and the ecosystem moved on. It isn't a viable third option in 2025. What's worth tracking instead is how far Compose and Kubernetes have converged as toolchains. Docker's "Compose for Kubernetes" support lets a team take a compose.yaml straight to a cluster without a rewrite, and lightweight distributions like K3s and MicroK8s give a single node full Kubernetes API compatibility without the overhead of running a multi-node cluster. The real choice has more gradations than a binary switch suggests.
How an IDP is actually layered, and why the layers have different orchestration needs
An internal developer platform is a composition, not a monolith. A rough map that holds across most organizations looks like this: a control plane (service catalog, self-service provisioning, a developer portal such as Backstage), a delivery layer (CI runners, build caches, artifact registries), an observability stack (metrics, logs, traces, alerting), developer environment services (local dev, cloud development environments, preview environments), and a layer of internal tooling: code search, documentation, internal APIs used as much by support and product staff as by engineers.
Most organizations now build an IDP specifically to abstract Kubernetes complexity away from product engineers, a share that's grown sharply over the past two years. That's the tell: the platform's entire reason for existing is to hide cluster complexity from the people writing application code. Using that same complexity to run every internal platform service defeats the purpose, and it's a contradiction most teams don't notice until the bill arrives. The portal a developer opens each morning has a different uptime and concurrency profile than the temporary test database a CI job spins up and tears down in minutes. A framework applied at the whole-IDP level, rather than layer by layer, misses that distinction entirely. A blanket rule at either extreme runs into the same wall: it skips the question of what the specific layer actually does.
Services where Docker Compose is the right call
The pattern is consistent: single host is enough, concurrency is modest, a restart is an inconvenience rather than an incident, the audience is internal, and no external SLA rides on uptime. Most platform teams overshoot this tier by instinct, not by analysis, and end up running a documentation search tool on the same cluster infrastructure built for customer traffic. That instinct carries a real cost, dressed up as caution.
Local and cloud development environments fit this closely. A platform team maintains one compose.yaml that defines a reproducible dev container, developers get environment parity without needing cluster credentials, and the old "works on my machine" complaint mostly disappears. Internal observability for the platform team's own use, Prometheus, Grafana, and Alertmanager running on a single node, belongs here too: cardinality is low, the audience is the platform team itself, and a five-minute restart costs nothing meaningful. CI runner sidecars, build caches, and stub services that live for the length of one pipeline run are Compose's home turf almost by definition; they don't outlive the job that spawned them. Internal tools with modest traffic, documentation search, runbook servers, internal wikis serving dozens of people rather than thousands, add nothing by running on a cluster.
Code intelligence services deployed inside a company's own infrastructure sit here as well. A self-hosted code search and AI context platform can run without a cluster and keeps source code inside the network perimeter the entire time. That matters directly in regulated environments where source code isn't allowed to leave company infrastructure at all; Compose-on-prem satisfies that constraint by construction, not by policy exception.
The same logic extends to agentic AI infrastructure. An entire agent stack — LLMs, MCP servers, tool integrations — can be defined in a single compose.yaml and started with one docker compose up command. Compose handles the container lifecycle underneath, keeping iteration fast and teardown simple. No cluster required, fast to iterate on, easy to tear down and rebuild. Across every example here, the common thread holds: Kubernetes would add configuration surface without buying back any resilience or scale these services will ever use.
Services where Kubernetes complexity pays for itself
The mirror-image criterion applies on the other side: multi-tenant traffic, high concurrency, a real need for self-healing, rolling deploys that can't afford downtime, or the service is itself the platform that product teams deploy against. Here the instinct to reach for Kubernetes stops being reflex and starts being correct.
If an IDP provisions namespaces or clusters for product engineering teams, Kubernetes is the substrate the rest of that layer sits on, full stop. The self-service provisioning control plane, whether that's a Backstage backend or a platform like Port or Humanitec, takes traffic from every developer in the company at once, and a rolling update that doesn't interrupt that traffic is worth the Deployment and ReplicaSet machinery it costs to get one. Artifact registries and build caches at real scale hit the same wall Compose can't clear: high write concurrency, persistence across nodes, and multi-region replication all need a cluster underneath them. Preview environments spun up per pull request tell the same story; a namespace-per-PR pattern is a natural fit for Kubernetes' multi-tenant model in a way that a single-host orchestrator cannot replicate at scale. Any IDP service carrying an external SLA, one a product team's deployment pipeline actually depends on, needs the self-healing and PodDisruptionBudget guarantees Kubernetes provides, because downtime there doesn't stay contained to one team.
GitOps maturity is a useful signal for where a team already sits on this spectrum. CNCF's 2025 data shows teams it calls "cloud native innovators" run GitOps extensively at more than double the rate of the average "adopter," and teams at that layer of platform maturity already have the toolchain in place to run Kubernetes well. That matters because most Kubernetes incidents trace back to configuration changes, not infrastructure failure; at this tier, investing in policy engines, admission control, and disciplined change management is justified, because a misconfiguration has a large blast radius. There's a cost case here too: organizations running Kubernetes report meaningfully lower compute spend per transaction than equivalent workloads on standard infrastructure, according to industry FinOps data. That efficiency case holds at real scale, but it does not extend down to a documentation search tool serving forty people, and treating it like it does is how platform budgets get wasted.
The hybrid operating model: how mature IDP teams run both in production
The pattern across mature platform teams combines both tools by design: customer-facing and multi-tenant services on Kubernetes, with monitoring stacks, CI runners, and internal tools running on Compose alongside them. That's a documented production configuration, chosen deliberately rather than left over from an unfinished migration. Anyone treating this as an either-or decision has already misread the problem.
What keeps that split from turning into two disconnected systems is discipline about where the boundary lives. Compose files and Kubernetes manifests sit in the same GitOps repository, so the tier a service runs on is visible and version-controlled rather than tribal knowledge passed between engineers. Docker's Compose-for-Kubernetes bridge means a team isn't maintaining two incompatible syntaxes side by side; a Compose file can move to a cluster deployment without a rewrite when a service's requirements change. K3s or MicroK8s can sit in between, single-node but Kubernetes-API-compatible, useful for a service that needs the primitives before it needs the full multi-node cluster.
The split does real operational work, too. Since most Kubernetes issues trace back to configuration changes, keeping auxiliary services off the cluster makes them immune to cluster-level incidents entirely; a bad ConfigMap change can't take down the internal wiki if the internal wiki was never on the cluster to begin with. Backstage or a similar portal can sit on top of the whole arrangement and give developers one consistent self-service experience, and none of them need to know or care which tier their request lands on underneath. For organizations with real data sovereignty requirements, self-hosted services running on Compose inside the company's own network perimeter satisfy compliance obligations that a shared cluster on managed cloud infrastructure may not. Sourcebot, for instance, is an on-premises code search and AI context tool that runs exactly this way, keeping source code inside the network boundary without requiring a cluster. That's how regulated environments keep sensitive services on-premises while still using Kubernetes where it's genuinely needed.
A decision framework for placing each IDP service on the right tier
Five questions, asked per service, do most of the work, and the order matters: get the first one wrong and the rest don't need asking.
Does the service need to span more than one host? If not, Compose already covers it, no further discussion needed. Is downtime for this service a blocker for product engineering teams downstream? If so, Kubernetes' self-healing and PodDisruptionBudgets are worth the overhead they carry. Does the service handle sensitive data, source code, secrets, credentials, that can't leave a defined network boundary? If yes, a self-hosted Compose deployment inside that perimeter is simpler to audit than a node inside a shared cluster. Will concurrency or scale exceed what one well-resourced host can serve? Below roughly 20 containers, Kubernetes' complexity premium buys no throughput advantage; above that line, cluster economics deserve a real look. Does the team that will operate this service have the headcount and Kubernetes expertise to absorb the overhead? A six-figure annual engineering-time cost is a fair gut check here: if a service can't justify that spend, Compose keeps the same team running it without the tax.
Services aren't locked to their starting tier. A service can start on Compose and graduate to Kubernetes the moment one of those five answers flips, and the Compose-for-Kubernetes bridge makes that an incremental move rather than a rebuild from scratch. CNCF's 2025 survey names organizational alignment, not tool complexity, as the leading obstacle to cloud-native adoption, and that points to the real risk here: relitigating the same decision service by service with no documented standard to point back to, instead of picking the wrong tier once and moving on. Every service placed on Kubernetes without a real operational need is engineering time that isn't going toward product features. That's platform debt, plain and simple, and it compounds the same way any other kind does. A framework like this one earns its keep by making that trade-off visible and repeatable, rather than leaving it to whichever engineer happened to set up the service last.
