Retiring the Ingress Controller: Modernizing Kubernetes Traffic Routing with the Gateway API and Cilium

For years, the standard Kubernetes Ingress resource has been the faithful workhorse of cluster traffic management. It provided a way to expose HTTP and HTTPS routes from outside the cluster to services within the cluster. However, as Kubernetes architectures have matured and microservices have grown in complexity, the cracks in the legacy Ingress model have become impossible to ignore.

If you have ever found yourself drowning in a sea of vendor-specific annotations just to configure a simple redirect or struggling to implement traffic splitting for a canary deployment, you know the pain. The ecosystem needed a revolution, not just an evolution.

That revolution has arrived in the form of the Kubernetes Gateway API paired with the power of Cilium. This combination isn't just a replacement; it represents a fundamental shift in how we think about networking, decoupling the role of the platform engineer from the application developer, and leveraging eBPF for unprecedented performance. In this post, we will explore why it is time to retire the traditional Ingress Controller and how Nohatek is helping clients modernize their routing infrastructure.

The Problem with Legacy Ingress

To understand the future, we must first look at the limitations of the past. The original Ingress API (networking.k8s.io/v1) was designed with simplicity in mind. It defined a minimal set of fields to route HTTP traffic. While this was sufficient for basic monolithic applications, it failed to standardize the complex networking requirements of modern cloud-native environments.

The major issues with the legacy Ingress model include:

• Annotation Hell: Because the Ingress spec is the lowest common denominator, advanced features like rate limiting, header rewriting, or authentication are handled via annotations. These are unstructured strings specific to the controller (e.g., NGINX vs. HAProxy vs. AWS ALB), leading to vendor lock-in and fragile configurations.
• Lack of Role Separation: In the Ingress model, the infrastructure configuration (load balancer settings) and the application routing (paths and services) are often mashed into a single object. This forces developers to understand infrastructure details they shouldn't need to touch.
• Limited Expressiveness: Native support for traffic splitting (essential for A/B testing and canary releases) or protocol matching beyond basic HTTP is non-existent without complex custom resource definitions (CRDs).

The Ingress resource was a victim of its own simplicity. It forced complexity into the annotations, creating a 'Wild West' of configuration standards.

For enterprise-grade clusters, these limitations result in operational overhead and brittle deployment pipelines. This is where the Gateway API steps in.

Enter the Gateway API: A Role-Oriented Standard

The Gateway API is an open-source project managed by the SIG-NETWORK community. Unlike Ingress, it is designed from the ground up to be expressive, extensible, and role-oriented. It acknowledges that running a Kubernetes cluster involves different personas: the Infrastructure Provider (who manages the load balancers) and the Application Developer (who defines the routing logic).

The API breaks networking down into distinct resources:

• GatewayClass: Defines the type of controller available (similar to StorageClass). This is usually managed by the cluster operator.
• Gateway: Describes the instance of a load balancer or proxy. This defines the listeners (ports) and is managed by the platform team.
• HTTPRoute (and others): Defines the routing rules (matches, filters, backends) and attaches to a Gateway. This is managed by the developers.

This separation of concerns is transformative. A platform engineer at a company using Nohatek's managed services can configure a Gateway with strict TLS policies, while individual development teams can attach their own HTTPRoutes to it without risking the integrity of the underlying infrastructure.

Furthermore, the Gateway API standardizes advanced routing. Traffic splitting is now a first-class citizen:

kind: HTTPRoute
metadata:
  name: canary-route
spec:
  parentRefs:
  - name: my-gateway
  rules:
  - backendRefs:
    - name: v1-service
      weight: 90
    - name: v2-service
      weight: 10

This native support eliminates the need for service meshes just to achieve basic traffic weighting, simplifying the stack significantly.

Cilium: The Supercharged Implementation

The Gateway API is just a specification; it needs a controller to implement it. While you could use NGINX or Envoy, Cilium has emerged as the premier implementation for performance-conscious environments. Cilium utilizes eBPF (Extended Berkeley Packet Filter) to handle networking logic directly within the Linux kernel, bypassing much of the overhead associated with traditional TCP/IP stacks.

Why is Cilium the ideal partner for the Gateway API?

• Unified Networking Layer: Cilium is already handling your CNI (Container Network Interface) and network policies. By using Cilium for the Gateway API, you eliminate the need for a separate Ingress Controller installation (like `ingress-nginx`). This reduces the number of moving parts in your cluster.
• Performance and Latency: Traditional ingress controllers often require traffic to hop through a sidecar or a dedicated proxy pod, involving context switches between user space and kernel space. Cilium's eBPF datapath allows for highly efficient packet processing, resulting in lower latency and higher throughput.
• Seamless L2/L3 Integration: With Cilium's L2 Announcement or BGP features, you can expose your Gateway IP addresses to your physical network without needing expensive external load balancers for on-premise deployments.

Enabling Gateway API support in Cilium is surprisingly straightforward. Once the Gateway API CRDs are installed, it is often as simple as enabling a Helm value:

gatewayAPI:
  enabled: true

Once enabled, Cilium watches for Gateway and HTTPRoute resources and programs the eBPF datapath accordingly. For complex operational requirements, Cilium creates a lightweight Envoy proxy only when L7 processing (like HTTP header modification) is strictly necessary, keeping the happy path as efficient as possible.

Strategic Migration and Business Value

For CTOs and decision-makers, the shift to Gateway API and Cilium is not just a technical upgrade—it is a strategic alignment. It reduces Technical Debt by removing proprietary annotations that lock you into specific vendors. It improves Security Posture by allowing platform teams to enforce policies at the Gateway level that developers cannot override. And it optimizes Cost by potentially removing the need for heavy, resource-intensive ingress controller fleets.

However, modernization does not mean a "rip and replace" operation. One of the strengths of Kubernetes is that the Ingress API and Gateway API can coexist in the same cluster. At Nohatek, we recommend a phased migration strategy:

1. Install Gateway API CRDs: Ensure your cluster supports the new standard.
2. Greenfield Adoption: Mandate that all new microservices use HTTPRoute instead of Ingress.
3. Brownfield Migration: Identify complex Ingress resources that rely on heavy annotations and refactor them to Gateway API to gain immediate stability benefits.
4. Retirement: Once the legacy routes are migrated, remove the old Ingress Controller to reclaim cluster resources.

By adopting this architecture, organizations position themselves to leverage future Kubernetes networking enhancements, such as GAMMA (Gateway API for Mesh Management and Administration), which aims to unify north-south and east-west traffic management under a single API surface.

The Kubernetes ecosystem is maturing, moving away from the "move fast and break things" era of custom annotations toward stable, role-oriented APIs. The combination of the Gateway API's expressive design and Cilium's eBPF-powered performance provides a robust foundation for the next generation of cloud-native applications.

Retiring your Ingress Controller might seem daunting, but the benefits in manageability, performance, and standardization are undeniable. It is time to treat your cluster networking with the architectural rigor it deserves.

Ready to modernize your Kubernetes infrastructure? Nohatek specializes in cloud-native transitions, AI integration, and high-performance computing. Contact our team today to discuss how we can streamline your traffic routing and future-proof your tech stack.