Apigee ingress gateway pods show 1 of 2 containers running

Symptom

Your apigee-ingressgateway pods show only 1 of 2 containers running when you get the pod listing.

For example, when you run the following command:

 kubectl -n apigee get pods -l app=apigee-ingressgateway 

Output:

 NAME READY STATUS RESTARTS AGE apigee-ingressgateway-ext-ingress-myorg-hyb-8f2c412-shl9r 1/2 Running 0 6m48s apigee-ingressgateway-ext-ingress-myorg-hyb-8f2c412-vvzsf 1/2 Running 0 123m 

Additionally, performing a describe command on one of the above pods will show the readiness probe failing with a 503 status code.

For example, in the Events section of the pod description, you might see the following message:

 Events: Type Reason Age From Message ---- ------ ---- ---- ------- Warning Unhealthy 76s (x32903 over 18h) kubelet Readiness probe failed: HTTP probe failed with statuscode: 503

Possible cause

Cause: Apigee ingress is configured to listen on port 80

This issue is caused by the Apigee ingress gateway being configured to listen on port 80, which is no longer supported starting with Apigee Hybrid 1.9.

This can happen if you upgraded from an earlier version of Apigee Hybrid that allowed port 80, or if there's another misconfiguration that enabled it.

Diagnosis

1. Get a listing of your apigee-ingressgateway pods.

   Run the following command:

   kubectl -n apigee get pods -l app=apigee-ingressgateway

   Sample output

    NAME READY STATUS RESTARTS AGE apigee-ingressgateway-ext-ingress-myorg-hyb-8f2c412-shl9r 1/2 Running 0 6m48s apigee-ingressgateway-ext-ingress-myorg-hyb-8f2c412-vvzsf 1/2 Running 0 123m 

2. Describe one of the listed pods to check the events:

   kubectl -n apigee describe pod APIGEE_INGRESSGATEWAY_POD

   Where APIGEE_INGRESSGATEWAY_POD is an apigee-ingressgateway pod listed in the previous command output.

   Sample output:

    Events: Type Reason Age From Message ---- ------ ---- ---- ------- Warning Unhealthy 76s (x32903 over 18h) kubelet Readiness probe failed: HTTP probe failed with statuscode: 503 

3. Get the logs for the APIGEE_INGRESSGATEWAY_POD pod.

   kubectl -n apigee logs APIGEE_INGRESSGATEWAY_POD

   You may see a log entry showing that port 80 failed to bind due to a permission denied error followed by a message that envoy is not ready.

    2025-09-11T06:16:45.457621Z error envoy config external/envoy/source/common/listener_manager/listener_manager_impl.cc:1186 listener '0.0.0.0_80' failed to bind or apply socket options: cannot bind '0.0.0.0:80': Permission denied ... 2025-09-11T06:16:46.365818Z warn Envoy proxy is NOT ready: config received from XDS server, but was rejected: cds updates: 1 successful, 0 rejected; lds updates: 0 successful, 1 rejected 

4. Review the current apigeeroute configuration to find the one that has port 80 (HTTP) enabled.
   • Get a list of the defined apigee routes.

     kubectl -n apigee get apigeeroute

     Sample output:

      NAME STATE AGE myorg-hyb-dev-grp-000-33620d0 running 2d1h non-sni running 17s 

   • Check each apigeeroute for where port 80 is defined.

     Run the following command for each apigeeroute listed in the previous command output:

     kubectl -n apigee get apigeeroute APIGEE_ROUTE_NAME -o yaml

     Where APIGEE_ROUTE_NAME is the name of an individual apigeeroute.

     Sample Command:

      kubectl -n apigee get apigeeroute non-sni -o yaml 

     Sample output:

      apiVersion: apigee.cloud.google.com/v1alpha2 kind: ApigeeRoute metadata: name: non-sni namespace: apigee resourceVersion: "240441468" spec: enableNonSniClient: true hostnames: - '*' ports: - number: 443 protocol: HTTPS tls: credentialName: myorg-hyb-dev-grp minProtocolVersion: TLS_AUTO mode: SIMPLE - number: 80 protocol: HTTP selector: app: apigee-ingressgateway status: lastAppliedGeneration: 1 state: running 

     The non-sni apigeeroute shows that port 80 is enabled as part of this route.

Resolution

To resolve this issue, disable port 80 in the original apigeeroute yaml file by removing the following lines.

 - number: 80 protocol: HTTP 

If you do not have the original apigeeroute yaml file, you can follow the following steps:

1. Export the current configuration with the following command:

   kubectl -n apigee get apigeeroute APIGEE_ROUTE_NAME -o yaml > APIGEE_ROUTE_FILENAME.yaml

   Where APIGEE_ROUTE_NAME is the apigeeroute being updated to remove the port 80 configuration.

   Where APIGEE_ROUTE_FILENAME is the output file name containing the apigeeroute yaml content.

2. Edit the apigeeroute yaml file to remove the port 80 configuration.

   Open the APIGEE_ROUTE_FILENAME file in a text editor and remove the following lines:

    - number: 80 protocol: HTTP 

3. Replace the existing apigeeroute with the modified YAML file:

   kubectl -n apigee replace -f APIGEE_ROUTE_FILENAME.yaml 

   Where APIGEE_ROUTE_FILENAME is the output file containing the updated apigeeroute configuration.

4. The apigee-ingressgateway pods should now run with 2/2 containers. If they do not, delete the pods to allow new ones to be created automatically.

    NAME READY STATUS RESTARTS AGE apigee-ingressgateway-ext-ingress-myorg-hyb-8f2c412-shl9r 2/2 Running 0 25h apigee-ingressgateway-ext-ingress-myorg-hyb-8f2c412-vvzsf 2/2 Running 0 26h 

Must gather diagnostic information

• Overrides.yaml
• Output of the following commands:
  •  kubectl -n apigee get pods -l app=apigee-ingressgateway

  • kubectl -n apigee logs APIGEE_INGRESSGATEWAY_POD

  • kubectl -n apigee get apigeeroutes

  • For each of the stated routes, run:

    kubectl -n apigee get apigeeroute APIGEE_ROUTE_NAME -o yaml

• As an alternative, you may run and provide the Apigee hybrid must-gather information.