Too many redirects on ingress-controller-CodePudding

I am trying to setup an Ingress Controller based upon:
https://kubernetes.github.io/ingress-nginx/deploy/#aws
It works fine for ELB, but for some reason, if I set the following in NLB:

nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
nginx.ingress.kubernetes.io/ssl-redirect: "true"

then I am getting a Too many redirects error.
If I set the above to false then I can access both HTTP and HTTPS separately but there is no redirection.

In my Service annotations for NLB I have:

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-backend-protocol: http
    service.beta.kubernetes.io/aws-load-balancer-cross-zone-load-balancing-enabled: 'true'
    service.beta.kubernetes.io/aws-load-balancer-type: nlb
    service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "443"
    service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: '3600'
    service.beta.kubernetes.io/aws-load-balancer-proxy-protocol: "*"
    service.beta.kubernetes.io/aws-load-balancer-ssl-cert: arn:aws:acm:eu-west-1:12345:certificate/xyz
    service.beta.kubernetes.io/aws-load-balancer-ssl-negotiation-policy: ELBSecurityPolicy-FS-1-2-2019-08
...
spec:
  type: LoadBalancer
  externalTrafficPolicy: Local
  ports:
    - name: http
      port: 80
      protocol: TCP
      targetPort: http
      appProtocol: http
    - name: https
      port: 443
      protocol: TCP
      targetPort: http
      appProtocol: https

for ELB where it works ok I have:

    service.beta.kubernetes.io/aws-load-balancer-backend-protocol: http
    service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: '60'
    service.beta.kubernetes.io/aws-load-balancer-cross-zone-load-balancing-enabled: 'true'
    service.beta.kubernetes.io/aws-load-balancer-ssl-cert: arn:aws:acm:eu-west-1:12345:certificate/xyz
    service.beta.kubernetes.io/aws-load-balancer-ssl-ports: https
    service.beta.kubernetes.io/aws-load-balancer-type: elb
...
spec:
  type: LoadBalancer
  externalTrafficPolicy: Local
  ports:
    - name: http
      port: 80
      protocol: TCP
      targetPort: tohttps
      appProtocol: http
    - name: https
      port: 443
      protocol: TCP
      targetPort: http
      appProtocol: https

I've tried many combinations but I can't get NLB to act in the same way like with ELB.

CodePudding user response：

Try with removing the appProtocol: https and offload SSL at the LB level

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-backend-protocol: http
    service.beta.kubernetes.io/aws-load-balancer-cross-zone-load-balancing-enabled: 'true'
    service.beta.kubernetes.io/aws-load-balancer-type: nlb
    service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "443"
    service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: '3600'
    service.beta.kubernetes.io/aws-load-balancer-proxy-protocol: "*"
    service.beta.kubernetes.io/aws-load-balancer-ssl-cert: arn:aws:acm:eu-west-1:12345:certificate/xyz
    service.beta.kubernetes.io/aws-load-balancer-ssl-negotiation-policy: ELBSecurityPolicy-FS-1-2-2019-08
...
spec:
  type: LoadBalancer
  externalTrafficPolicy: Local
  ports:
    - name: http
      port: 80
      protocol: TCP
      targetPort: http
    - name: https
      port: 443
      protocol: TCP
      targetPort: HTTP

You can check config at : https://aws.amazon.com/blogs/opensource/network-load-balancer-nginx-ingress-controller-eks/

Also, check from AWS console LB having 80 and TLS 443 Listeners.

SSL offloading & terinating : https://aws.amazon.com/premiumsupport/knowledge-center/terminate-https-traffic-eks-acm/

CodePudding user response：

If backend protocol set to "ssl" everything works fine, except the fact that we're doing double TLS offloading for no reason (on NLB first, then on ingress). If backend protocol set to "tcp", we'll get "Plain HTTP request sent to TLS port" error. If we map https to http port to address the above then HTTP -> HTTPS redirects stop working.

So to make it working with NLB I needed set the backend protocol to ssl: service.beta.kubernetes.io/aws-load-balancer-backend-protocol: ssl and then:

spec:
  type: LoadBalancer
  externalTrafficPolicy: Local
  ports:
    - name: http
      port: 80
      protocol: TCP
      targetPort: http
    - name: https
      port: 443
      protocol: TCP
      targetPort: https

CodePudding user response：

For the In my Service annotations for NLB...block, The Too many redirects error. was a result that your NLB already performed the TLS termination and forward to your nginx-ingress as http - but your nginx-ingress keep doing the ssl-redirect again... and again. A few things you can do, since you only want https at the nginx-ingress, your NLB does not need to listen to 80 (http). Your NLB should let https pass-thru to the nginx-ingress which will do the ssl termination and route request to backend services.

This way there's no need to do ssl-redirect. In fact, even if you do expose both http and https at NLB and forward to nginx-ingress, the ssl-redirect at nginx-ingress will also work.