Google Cloud Kubernetes Load Balancer Service: ERR_CONNECTION

I have this problem that's driving me insane. I have two deployment and two service yaml files created by kompose convert from a docker-compose. The app that I'm trying to run in Google Cloud is a Spring Boot web app with a mariadb backend. After I apply the four yamls with kubectl, I expose the frontend deployment (on port 8081) by running

CodePudding user response：

TL;DR for anyone coming to this question via search:

OP's service was a ClusterIP and not LoadBalancer. Setting this as LoadBalancer still did not fix issue. Checking logs of pod determined code was unable to connect to DB, so never actually started up successfully.

Output from OP:

kubectl get svc -n default

NAME          TYPE          CLUSTER-IP   EXTERNAL-IP    PORT(S)       AGE
kubernetes    ClusterIP     10.72.0.1    <none>         443/TCP       147m
load-balancer LoadBalancer  10.72.15.246 34.69.204.138  80:30870/TCP  86s
mysqldb       ClusterIP     10.72.3.186  <none>         3308/TCP      3m20s
web-app       ClusterIP     10.72.13.41  <none>         8081/TCP      3m19s

Please share those in your question, making sure to prepend "```" before and after each yaml to preserve formatting

Now, a few points.

You specified you have service yamls. If you have a yaml that describes a service such as a LoadBalancer, you shouldn't need to kubectl expose afterwards, as your yaml should have done that for you.
Assuming the service named "load-balancer" is the one you have created via your yamls, the IP:port combination you should be using is 34.69.204.138:80. What IP have you been trying to access? Are you trying to access this IP and port? Or a different one?

UPDATE

Based on the pasted yamls, I see this:

In your docker-compose yaml:

  web-app:
    build: .
    image: mihaialexandruteodor/featherwriter
    ports:
      - "8081:8081"
    expose:
      - "8081"

This is exposing port 8081 and connecting it to the underlying container.

This is reflected in the service yaml:

apiVersion: v1
kind: Service
metadata:
  ...
  name: web-app
spec:
  ports:
    - name: "8081"
      port: 8081
      targetPort: 8081
  selector:
    io.kompose.service: web-app
status:
  loadBalancer: {}

However, I do not see a service called "web-app" in your listing. It's possible therefore, you may have deployed it into a different namespace.

Try kubectl get svc --all-namespaces and see where the service "web-app" is. Find the IP from that, the port should be 8081 and you can then do x.x.x.x:8081 to access the service.

UPDATE 2

The web-app service is of type ClusterIP (documentation) which cannot be accessed outside of the cluster, you need to change the service to be a LoadBalancer type, or use port-forwarding.

To make the service a LoadBalancer, change the service yaml as follows (documentation here):

apiVersion: v1
kind: Service
metadata:
  annotations:
    kompose.cmd: kompose convert
    kompose.version: 1.26.1 (HEAD)
  creationTimestamp: null
  labels:
    io.kompose.service: mysqldb
  name: mysqldb
spec:
  ports:
    - name: "3308"
      port: 3308
      targetPort: 3306
  selector:
    io.kompose.service: mysqldb
  type: LoadBalancer

This will provision a service that will have an external IP you can use.

Alternatively, use port forwarding to connect a local port with the port being listened on by the service:

kubectl port-forward -n {namespace} svc/web-app 8081:8081

Then you can use localhost:8081 to connect to your service. This option does not require an externally-accessible endpoint, but you will need to run the port forward command (and have it active) each time you want to access the service via the localhost endpoint.

If you want to be able to access the service from somewhere outside of your cluster, that is not your local machine, and is not within the same cluster, you will need to use a LoadBalancer service type.

UPDATE 3

Right, I can't build that Dockerfile as I do not have the src folder, but I can run the image from mihaialexandruteodor/featherwriter, and can see it is indeed listening on 8081

Tomcat initialized with port(s): 8081 (http)

so the next thing to see is see if there's any issues with the pod functionality itself. First check the pod status:

kubectl get pods -n {namespace}

The pod should be called web-app-xxxxx where xxxxx is a random sequence of letters and numbers.

Is the web-app pod running? Does it have a restart counter that is not zero like some of the pods in my prometheus namespace:

$ kubectl get pods -n prometheus
NAME                                                     READY   STATUS    RESTARTS   AGE
alertmanager-prometheus-kube-prometheus-alertmanager-0   2/2     Running   0          3h51m
prometheus-grafana-66cb8bcf4f-428d8                      3/3     Running   0          3h51m
prometheus-kube-prometheus-operator-749fc8899b-dnvft     1/1     Running   0          3h51m
prometheus-kube-state-metrics-77698656df-btq4k           1/1     Running   20         3h51m
prometheus-prometheus-kube-prometheus-prometheus-0       2/2     Running   0          3h51m
prometheus-prometheus-node-exporter-jj9z5                1/1     Running   30         3h51m
prometheus-prometheus-node-exporter-lbk6p                1/1     Running   0          3h51m
prometheus-prometheus-node-exporter-vqfhk                1/1     Running   20         3h51m

Next get the logs from the pods like so:

kubectl logs -n {namespace} web-app-xxxxx

See if you can find any errors.

My hunch, given that we've connected everything through on 8081 and Tomcat is indeed running on 8081, is that the spring app is crashing repeatedly and Kubernetes is restarting it, the app then fails again, and it tries again over and over, eventually failing into a CrashLoopBackOff state where Kubernetes will delay restarting by a longer period.