Designing Multi-Tenant Kubernetes
Clusters on EKS

Running multiple product teams or customers on a shared EKS cluster reduces infrastructure overhead and lowers per-tenant costs at scale. But shared clusters require intentional design. Without proper isolation, one tenant's workload can consume resources belonging to another, access data it should not see, or create blast radius across the entire cluster during incidents.

This article covers the four layers of multi-tenant isolation on EKS: namespace isolation, RBAC, resource controls, and network policies. It also discusses when a shared cluster makes sense versus when per-tenant clusters are the right call.

When a Shared Cluster Is Appropriate

Multi-tenancy works well when tenants are internal teams (different engineering teams within the same organization) or when you are running a SaaS product where customers share infrastructure but their data is logically separated at the application layer. It also works for environments (dev/staging/prod) that need different resource profiles but benefit from shared cluster management.

A shared cluster is a poor fit when:

• Tenants have strict regulatory requirements for physical isolation (e.g., some FedRAMP workloads require separate infrastructure per agency)
• Tenants have significantly different scaling patterns that would cause node contention
• You cannot enforce a consistent security posture across all tenants
• One tenant requires Kubernetes API access that would conflict with another tenant's security requirements

Layer 1: Namespace Isolation

The namespace is the primary isolation boundary in Kubernetes. Each tenant gets one or more dedicated namespaces. Naming conventions matter for management at scale. A consistent pattern like tenant-{id}-{env} (e.g., acme-prod, acme-staging) makes it easier to apply policies programmatically.

Namespace isolation controls:

• Scope of RBAC permissions (a RoleBinding in namespace A does not apply in namespace B)
• Scope of NetworkPolicies (policies apply at the namespace level)
• Scope of ResourceQuotas and LimitRanges
• Default service account tokens and secrets

Namespaces alone do not provide strong isolation. A pod in namespace A can still reach pods in namespace B over the cluster network unless NetworkPolicies are applied. RBAC and network policies are required to enforce meaningful boundaries.

Layer 2: RBAC

Role-Based Access Control determines what each identity can do within the cluster. For multi-tenant clusters, the key principle is: tenants get Role and RoleBinding (namespace-scoped), never ClusterRole with broad permissions.

A minimal tenant RBAC setup:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: tenant-developer
  namespace: acme-prod
rules:
- apiGroups: ["apps"]
  resources: ["deployments", "replicasets"]
  verbs: ["get", "list", "watch", "create", "update", "patch"]
- apiGroups: [""]
  resources: ["pods", "pods/log", "services", "configmaps"]
  verbs: ["get", "list", "watch"]
- apiGroups: [""]
  resources: ["secrets"]
  verbs: ["get", "list"]  # restrict to read; writes go through CI/CD
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: acme-developers
  namespace: acme-prod
subjects:
- kind: Group
  name: acme-dev-team   # maps to IAM role via aws-auth ConfigMap
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: tenant-developer
  apiGroup: rbac.authorization.k8s.io

AWS EKS uses the aws-auth ConfigMap (or the newer EKS Access Entries API on clusters running Kubernetes 1.29+) to map IAM roles and users to Kubernetes groups. Each tenant team gets a dedicated IAM role that maps to their Kubernetes group. Developers assume their team's IAM role to get cluster access scoped to their namespaces only.

Cluster-level resources (Nodes, PersistentVolumes, StorageClasses, ClusterRoles) should remain restricted to platform engineering. Tenants should not be able to list Nodes, which exposes other tenants' workload placement, or modify ClusterRoles, which could escalate their permissions.

Layer 3: Resource Controls

Without resource controls, a single tenant's workload can consume all available CPU and memory in the cluster, starving other tenants. Two Kubernetes objects address this: ResourceQuota and LimitRange.

ResourceQuota

ResourceQuota sets a hard ceiling on total resource consumption within a namespace:

apiVersion: v1
kind: ResourceQuota
metadata:
  name: tenant-quota
  namespace: acme-prod
spec:
  hard:
    requests.cpu: "8"
    requests.memory: 16Gi
    limits.cpu: "16"
    limits.memory: 32Gi
    pods: "50"
    services: "10"
    persistentvolumeclaims: "20"

LimitRange

LimitRange sets default and maximum resource requests/limits per container. Without LimitRange, a developer can deploy a pod with no resource requests, which makes it invisible to the scheduler for capacity planning and allows it to consume unlimited resources:

apiVersion: v1
kind: LimitRange
metadata:
  name: tenant-limits
  namespace: acme-prod
spec:
  limits:
  - type: Container
    default:
      cpu: "500m"
      memory: 256Mi
    defaultRequest:
      cpu: "100m"
      memory: 128Mi
    max:
      cpu: "4"
      memory: 8Gi

Apply both objects to every tenant namespace at creation time, ideally via a Helm chart or Terraform module that creates the full namespace stack (namespace, RBAC, quota, limit range) as a single unit.

Layer 4: Network Policies

By default, all pods in an EKS cluster can communicate with all other pods across all namespaces. NetworkPolicies restrict this. A tenant's pods should not be able to initiate connections to pods in other tenant namespaces.

A default-deny policy that blocks all ingress and egress within a namespace, then selectively allows required traffic:

# Default deny all ingress and egress
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny
  namespace: acme-prod
spec:
  podSelector: {}
  policyTypes:
  - Ingress
  - Egress
---
# Allow intra-namespace traffic
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-same-namespace
  namespace: acme-prod
spec:
  podSelector: {}
  policyTypes:
  - Ingress
  - Egress
  ingress:
  - from:
    - podSelector: {}
  egress:
  - to:
    - podSelector: {}
---
# Allow DNS resolution (kube-dns)
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-dns
  namespace: acme-prod
spec:
  podSelector: {}
  policyTypes:
  - Egress
  egress:
  - to:
    - namespaceSelector:
        matchLabels:
          kubernetes.io/metadata.name: kube-system
    ports:
    - protocol: UDP
      port: 53

NetworkPolicies require a CNI plugin that supports them. The default AWS VPC CNI supports NetworkPolicies starting in EKS 1.25 with the ENABLE_NETWORK_POLICY_CONTROLLER flag enabled. Calico and Cilium are also commonly used and offer more advanced policy capabilities, including cluster-wide policies and L7 rules.

Node Isolation

Namespace-level isolation protects against logical cross-tenant access but does not prevent pod scheduling on shared nodes. If tenants require node-level isolation (e.g., for compliance or performance reasons), use node groups with taints and tenant-specific tolerations, or consider Karpenter NodePools to provision dedicated node groups per tenant on demand.

For most internal multi-team use cases, shared nodes with proper namespace isolation are acceptable. For SaaS scenarios where tenants are external customers with data privacy requirements, dedicated node groups per tenant tier (or per high-value tenant) are worth the added cost.

Reference Architecture Summary

A production-ready multi-tenant EKS cluster typically includes:

• Cluster-level: EKS with private API endpoint, aws-auth or EKS Access Entries for IAM mapping, AWS Load Balancer Controller, Karpenter or Cluster Autoscaler
• Per-tenant namespace stack: Namespace with labels, Role + RoleBinding scoped to the namespace, ResourceQuota, LimitRange, default-deny NetworkPolicy + allowlist policies, dedicated service accounts with IRSA roles
• GitOps management: ArgoCD or Flux managing per-tenant namespace manifests from dedicated Git paths, with tenant teams having write access to their own path only
• Observability: Namespace-scoped Prometheus scrape configs and CloudWatch Container Insights with namespace dimensions for per-tenant cost and usage visibility