Home > Software design >  Spring Booot Application gets shutdown immediately after start in Kubernetes env
Spring Booot Application gets shutdown immediately after start in Kubernetes env

Time:12-03

I have one application which is running fine in local, dev env. In dev it runs as an image in kubernetes in AWS cloud. Upto this everything went fine. Now I am trying to deploy the app in PP env. However in PP, the applications is starting and then immediately shutdown.

Here the configurations :

pom.xml

<?xml version="1.0" encoding="UTF-8"?>

<project xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.7.3</version>

        <relativePath /> <!-- lookup parent from repository -->
    </parent>
    <groupId>com.xyz</groupId>
    <artifactId>my-app</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>my-app</name>
    <description>My App System</description>
    <properties>
        <java.version>11</java.version>
        <skipTests>true</skipTests>
    </properties>
    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-jpa</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>
        <dependency>
            <groupId>org.flywaydb</groupId>
            <artifactId>flyway-core</artifactId>
        </dependency>
        <dependency>
            <groupId>org.postgresql</groupId>
            <artifactId>postgresql</artifactId>
            <scope>runtime</scope>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>io.awspring.cloud</groupId>
            <artifactId>spring-cloud-starter-aws-parameter-store-config</artifactId>
            <version>2.4.1</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-bootstrap</artifactId>
            <version>3.1.3</version>
        </dependency>
        <dependency>
            <groupId>com.amazonaws</groupId>
            <artifactId>aws-java-sdk-sts</artifactId>
            <version>1.12.263</version>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
                <configuration>
                    <excludes>
                        <exclude>
                            <groupId>org.projectlombok</groupId>
                            <artifactId>lombok</artifactId>
                        </exclude>
                    </excludes>
                </configuration>
            </plugin>
        </plugins>
    </build>

</project>

Main Class:

@SpringBootApplication
public class MyApplication {

    public static void main(String[] args) {
        SpringApplication.run(MyApplication.class, args);
    }
    
    @Bean
    public RestTemplate getRestTemplate() {     
        return new RestTemplate();
    }

}

Start up log:

2022-11-30 13:10:53.112  INFO 1 --- [           main] j.LocalContainerEntityManagerFactoryBean : Initialized JPA EntityManagerFactory for persistence unit 'default'
2022-11-30 13:11:03.604  WARN 1 --- [           main] JpaBaseConfiguration$JpaWebConfiguration : spring.jpa.open-in-view is enabled by default. Therefore, database queries may be performed during view rendering. Explicitly configure spring.jpa.open-in-view to disable this warning
2022-11-30 13:11:11.305  INFO 1 --- [           main] o.s.b.a.e.web.EndpointLinksResolver      : Exposing 16 endpoint(s) beneath base path '/actuator'
2022-11-30 13:11:11.816  INFO 1 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port(s): 8080 (http) with context path ''
2022-11-30 13:11:12.110  INFO 1 --- [           main] c.a.f.MyApplication                      : Started MyApplication in 96.297 seconds (JVM running for 104.706)
2022-11-30 13:11:12.513  INFO 1 --- [ionShutdownHook] j.LocalContainerEntityManagerFactoryBean : Closing JPA EntityManagerFactory for persistence unit 'default'
2022-11-30 13:11:12.607  INFO 1 --- [ionShutdownHook] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Shutdown initiated...
2022-11-30 13:11:12.797  INFO 1 --- [ionShutdownHook] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Shutdown completed.

Please help me if you have any clue as where it may go wrong.

UPDATE:

DEBUG log :-

Application availability state ReadinessState changed to REFUSING_TRAFFIC


2022-11-30 14:45:14.420  INFO 1 --- [           main] c.a.f.MyApplication                      : Started FMyApplication in 98.611 seconds (JVM running for 107.522)
2022-11-30 14:45:14.512 DEBUG 1 --- [ionShutdownHook] o.s.b.a.ApplicationAvailabilityBean      : Application availability state ReadinessState changed to REFUSING_TRAFFIC
2022-11-30 14:45:14.514 DEBUG 1 --- [ionShutdownHook] ConfigServletWebServerApplicationContext : Closing org.springframework.boot.web.servlet.context.AnnotationConfigServletWebServerApplicationContext@5733f295, started on Wed Nov 30 14:44:06 GMT 2022, parent: org.springframework.context.annotation.AnnotationConfigApplicationContext@78ffe6dc
2022-11-30 14:45:14.516 DEBUG 1 --- [ionShutdownHook] o.s.c.support.DefaultLifecycleProcessor  : Stopping beans in phase 2147483647
2022-11-30 14:45:14.601 DEBUG 1 --- [ionShutdownHook] o.s.c.support.DefaultLifecycleProcessor  : Bean 'webServerGracefulShutdown' completed its stop procedure
2022-11-30 14:45:14.601 DEBUG 1 --- [ionShutdownHook] o.s.c.support.DefaultLifecycleProcessor  : Stopping beans in phase 2147483646
2022-11-30 14:45:14.616 DEBUG 1 --- [           main] o.s.b.a.ApplicationAvailabilityBean      : Application availability state LivenessState changed to CORRECT
2022-11-30 14:45:14.715 DEBUG 1 --- [           main] o.s.b.a.ApplicationAvailabilityBean      : Application availability state ReadinessState changed from REFUSING_TRAFFIC to ACCEPTING_TRAFFIC
2022-11-30 14:45:14.716 DEBUG 1 --- [ionShutdownHook] o.s.c.support.DefaultLifecycleProcessor  : Bean 'webServerStartStop' completed its stop procedure
2022-11-30 14:45:14.812 DEBUG 1 --- [ionShutdownHook] o.s.s.concurrent.ThreadPoolTaskExecutor  : Shutting down ExecutorService 'applicationTaskExecutor'
2022-11-30 14:45:14.904  INFO 1 --- [ionShutdownHook] j.LocalContainerEntityManagerFactoryBean : Closing JPA EntityManagerFactory for persistence unit 'default'
2022-11-30 14:45:14.904 DEBUG 1 --- [ionShutdownHook] o.hibernate.internal.SessionFactoryImpl  : HHH000031: Closing
2022-11-30 14:45:14.904 DEBUG 1 --- [ionShutdownHook] o.h.type.spi.TypeConfiguration$Scope     : Un-scoping TypeConfiguration [org.hibernate.type.spi.TypeConfiguration$Scope@6534185f] from SessionFactory [org.hibernate.internal.SessionFactoryImpl@d499c13]
2022-11-30 14:45:14.904 DEBUG 1 --- [ionShutdownHook] o.h.s.i.AbstractServiceRegistryImpl      : Implicitly destroying ServiceRegistry on de-registration of all child ServiceRegistries
2022-11-30 14:45:14.915 DEBUG 1 --- [ionShutdownHook] o.h.b.r.i.BootstrapServiceRegistryImpl   : Implicitly destroying Boot-strap registry on de-registration of all child ServiceRegistries
2022-11-30 14:45:14.917  INFO 1 --- [ionShutdownHook] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Shutdown initiated...
2022-11-30 14:45:14.917 DEBUG 1 --- [ionShutdownHook] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Before shutdown stats (total=10, active=0, idle=10, waiting=0)
2022-11-30 14:45:15.009 DEBUG 1 --- [nnection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.postgresql.jdbc.PgConnection@14a049f9: (connection evicted)
2022-11-30 14:45:15.011 DEBUG 1 --- [nnection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.postgresql.jdbc.PgConnection@82010d7: (connection evicted)
2022-11-30 14:45:15.013 DEBUG 1 --- [nnection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.postgresql.jdbc.PgConnection@47daf598: (connection evicted)
2022-11-30 14:45:15.018 DEBUG 1 --- [nnection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.postgresql.jdbc.PgConnection@4ff4f31c: (connection evicted)
2022-11-30 14:45:15.021 DEBUG 1 --- [nnection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.postgresql.jdbc.PgConnection@76d717a6: (connection evicted)
2022-11-30 14:45:15.104 DEBUG 1 --- [nnection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.postgresql.jdbc.PgConnection@37134f98: (connection evicted)
2022-11-30 14:45:15.106 DEBUG 1 --- [nnection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.postgresql.jdbc.PgConnection@27f50ad7: (connection evicted)
2022-11-30 14:45:15.109 DEBUG 1 --- [nnection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.postgresql.jdbc.PgConnection@4ff16126: (connection evicted)
2022-11-30 14:45:15.111 DEBUG 1 --- [nnection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.postgresql.jdbc.PgConnection@1d53559: (connection evicted)
2022-11-30 14:45:15.113 DEBUG 1 --- [nnection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.postgresql.jdbc.PgConnection@67077c93: (connection evicted)
2022-11-30 14:45:15.201 DEBUG 1 --- [ionShutdownHook] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - After shutdown stats (total=0, active=0, idle=0, waiting=0)
2022-11-30 14:45:15.201  INFO 1 --- [ionShutdownHook] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Shutdown completed.

CodePudding user response:

Here is the point :

sometimes our apps may need a little bit of time before being able to accept requests. The kubelet can make sure that the application receives requests only when it's ready. Also, if the main process of a pod crashes for any reason, the kubelet will restart the container.

Reference : Spring-Boot-Liveness-And-Readiness

In order to fulfill these responsibilities, Kubernetes has two probes: liveness probes and readiness probes

In my case as soon as the app was starting the ReadinessState changed to REFUSING_TRAFFIC. Hence added some delay seconds to the property in kubernetes service config yml to give some time to the app to make ReadinessState changed from REFUSING_TRAFFIC to ACCEPTING_TRAFFIC

In kustomization.yml file added the below 2 config properties which resolved the issue :

1. readinessProbe/initialDelaySeconds : 60
2. livenessProbe/initialDelaySeconds : 60

Also same can be added in application.yml file as below :

livenessProbe:
  httpGet:
    path: /actuator/health/liveness
    port: 8080
    initialDelaySeconds: 30
    periodSeconds: 30
  • Related