I have one application which is running fine in local, dev env. In dev it runs as an image in kubernetes in AWS cloud. Upto this everything went fine. Now I am trying to deploy the app in PP env. However in PP, the applications is starting and then immediately shutdown.
Here the configurations :
pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.7.3</version>
<relativePath /> <!-- lookup parent from repository -->
</parent>
<groupId>com.xyz</groupId>
<artifactId>my-app</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>my-app</name>
<description>My App System</description>
<properties>
<java.version>11</java.version>
<skipTests>true</skipTests>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>org.flywaydb</groupId>
<artifactId>flyway-core</artifactId>
</dependency>
<dependency>
<groupId>org.postgresql</groupId>
<artifactId>postgresql</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<optional>true</optional>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>io.awspring.cloud</groupId>
<artifactId>spring-cloud-starter-aws-parameter-store-config</artifactId>
<version>2.4.1</version>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-bootstrap</artifactId>
<version>3.1.3</version>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-sts</artifactId>
<version>1.12.263</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<excludes>
<exclude>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
</exclude>
</excludes>
</configuration>
</plugin>
</plugins>
</build>
</project>
Main Class:
@SpringBootApplication
public class MyApplication {
public static void main(String[] args) {
SpringApplication.run(MyApplication.class, args);
}
@Bean
public RestTemplate getRestTemplate() {
return new RestTemplate();
}
}
Start up log:
2022-11-30 13:10:53.112 INFO 1 --- [ main] j.LocalContainerEntityManagerFactoryBean : Initialized JPA EntityManagerFactory for persistence unit 'default'
2022-11-30 13:11:03.604 WARN 1 --- [ main] JpaBaseConfiguration$JpaWebConfiguration : spring.jpa.open-in-view is enabled by default. Therefore, database queries may be performed during view rendering. Explicitly configure spring.jpa.open-in-view to disable this warning
2022-11-30 13:11:11.305 INFO 1 --- [ main] o.s.b.a.e.web.EndpointLinksResolver : Exposing 16 endpoint(s) beneath base path '/actuator'
2022-11-30 13:11:11.816 INFO 1 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port(s): 8080 (http) with context path ''
2022-11-30 13:11:12.110 INFO 1 --- [ main] c.a.f.MyApplication : Started MyApplication in 96.297 seconds (JVM running for 104.706)
2022-11-30 13:11:12.513 INFO 1 --- [ionShutdownHook] j.LocalContainerEntityManagerFactoryBean : Closing JPA EntityManagerFactory for persistence unit 'default'
2022-11-30 13:11:12.607 INFO 1 --- [ionShutdownHook] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Shutdown initiated...
2022-11-30 13:11:12.797 INFO 1 --- [ionShutdownHook] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Shutdown completed.
Please help me if you have any clue as where it may go wrong.
UPDATE:
DEBUG log :-
Application availability state ReadinessState changed to REFUSING_TRAFFIC
2022-11-30 14:45:14.420 INFO 1 --- [ main] c.a.f.MyApplication : Started FMyApplication in 98.611 seconds (JVM running for 107.522)
2022-11-30 14:45:14.512 DEBUG 1 --- [ionShutdownHook] o.s.b.a.ApplicationAvailabilityBean : Application availability state ReadinessState changed to REFUSING_TRAFFIC
2022-11-30 14:45:14.514 DEBUG 1 --- [ionShutdownHook] ConfigServletWebServerApplicationContext : Closing org.springframework.boot.web.servlet.context.AnnotationConfigServletWebServerApplicationContext@5733f295, started on Wed Nov 30 14:44:06 GMT 2022, parent: org.springframework.context.annotation.AnnotationConfigApplicationContext@78ffe6dc
2022-11-30 14:45:14.516 DEBUG 1 --- [ionShutdownHook] o.s.c.support.DefaultLifecycleProcessor : Stopping beans in phase 2147483647
2022-11-30 14:45:14.601 DEBUG 1 --- [ionShutdownHook] o.s.c.support.DefaultLifecycleProcessor : Bean 'webServerGracefulShutdown' completed its stop procedure
2022-11-30 14:45:14.601 DEBUG 1 --- [ionShutdownHook] o.s.c.support.DefaultLifecycleProcessor : Stopping beans in phase 2147483646
2022-11-30 14:45:14.616 DEBUG 1 --- [ main] o.s.b.a.ApplicationAvailabilityBean : Application availability state LivenessState changed to CORRECT
2022-11-30 14:45:14.715 DEBUG 1 --- [ main] o.s.b.a.ApplicationAvailabilityBean : Application availability state ReadinessState changed from REFUSING_TRAFFIC to ACCEPTING_TRAFFIC
2022-11-30 14:45:14.716 DEBUG 1 --- [ionShutdownHook] o.s.c.support.DefaultLifecycleProcessor : Bean 'webServerStartStop' completed its stop procedure
2022-11-30 14:45:14.812 DEBUG 1 --- [ionShutdownHook] o.s.s.concurrent.ThreadPoolTaskExecutor : Shutting down ExecutorService 'applicationTaskExecutor'
2022-11-30 14:45:14.904 INFO 1 --- [ionShutdownHook] j.LocalContainerEntityManagerFactoryBean : Closing JPA EntityManagerFactory for persistence unit 'default'
2022-11-30 14:45:14.904 DEBUG 1 --- [ionShutdownHook] o.hibernate.internal.SessionFactoryImpl : HHH000031: Closing
2022-11-30 14:45:14.904 DEBUG 1 --- [ionShutdownHook] o.h.type.spi.TypeConfiguration$Scope : Un-scoping TypeConfiguration [org.hibernate.type.spi.TypeConfiguration$Scope@6534185f] from SessionFactory [org.hibernate.internal.SessionFactoryImpl@d499c13]
2022-11-30 14:45:14.904 DEBUG 1 --- [ionShutdownHook] o.h.s.i.AbstractServiceRegistryImpl : Implicitly destroying ServiceRegistry on de-registration of all child ServiceRegistries
2022-11-30 14:45:14.915 DEBUG 1 --- [ionShutdownHook] o.h.b.r.i.BootstrapServiceRegistryImpl : Implicitly destroying Boot-strap registry on de-registration of all child ServiceRegistries
2022-11-30 14:45:14.917 INFO 1 --- [ionShutdownHook] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Shutdown initiated...
2022-11-30 14:45:14.917 DEBUG 1 --- [ionShutdownHook] com.zaxxer.hikari.pool.HikariPool : HikariPool-1 - Before shutdown stats (total=10, active=0, idle=10, waiting=0)
2022-11-30 14:45:15.009 DEBUG 1 --- [nnection closer] com.zaxxer.hikari.pool.PoolBase : HikariPool-1 - Closing connection org.postgresql.jdbc.PgConnection@14a049f9: (connection evicted)
2022-11-30 14:45:15.011 DEBUG 1 --- [nnection closer] com.zaxxer.hikari.pool.PoolBase : HikariPool-1 - Closing connection org.postgresql.jdbc.PgConnection@82010d7: (connection evicted)
2022-11-30 14:45:15.013 DEBUG 1 --- [nnection closer] com.zaxxer.hikari.pool.PoolBase : HikariPool-1 - Closing connection org.postgresql.jdbc.PgConnection@47daf598: (connection evicted)
2022-11-30 14:45:15.018 DEBUG 1 --- [nnection closer] com.zaxxer.hikari.pool.PoolBase : HikariPool-1 - Closing connection org.postgresql.jdbc.PgConnection@4ff4f31c: (connection evicted)
2022-11-30 14:45:15.021 DEBUG 1 --- [nnection closer] com.zaxxer.hikari.pool.PoolBase : HikariPool-1 - Closing connection org.postgresql.jdbc.PgConnection@76d717a6: (connection evicted)
2022-11-30 14:45:15.104 DEBUG 1 --- [nnection closer] com.zaxxer.hikari.pool.PoolBase : HikariPool-1 - Closing connection org.postgresql.jdbc.PgConnection@37134f98: (connection evicted)
2022-11-30 14:45:15.106 DEBUG 1 --- [nnection closer] com.zaxxer.hikari.pool.PoolBase : HikariPool-1 - Closing connection org.postgresql.jdbc.PgConnection@27f50ad7: (connection evicted)
2022-11-30 14:45:15.109 DEBUG 1 --- [nnection closer] com.zaxxer.hikari.pool.PoolBase : HikariPool-1 - Closing connection org.postgresql.jdbc.PgConnection@4ff16126: (connection evicted)
2022-11-30 14:45:15.111 DEBUG 1 --- [nnection closer] com.zaxxer.hikari.pool.PoolBase : HikariPool-1 - Closing connection org.postgresql.jdbc.PgConnection@1d53559: (connection evicted)
2022-11-30 14:45:15.113 DEBUG 1 --- [nnection closer] com.zaxxer.hikari.pool.PoolBase : HikariPool-1 - Closing connection org.postgresql.jdbc.PgConnection@67077c93: (connection evicted)
2022-11-30 14:45:15.201 DEBUG 1 --- [ionShutdownHook] com.zaxxer.hikari.pool.HikariPool : HikariPool-1 - After shutdown stats (total=0, active=0, idle=0, waiting=0)
2022-11-30 14:45:15.201 INFO 1 --- [ionShutdownHook] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Shutdown completed.
CodePudding user response：
Here is the point :
sometimes our apps may need a little bit of time before being able to accept requests. The kubelet can make sure that the application receives requests only when it's ready. Also, if the main process of a pod crashes for any reason, the kubelet will restart the container.
Reference : Spring-Boot-Liveness-And-Readiness
In order to fulfill these responsibilities, Kubernetes has two probes: liveness probes and readiness probes
In my case as soon as the app was starting the ReadinessState changed to REFUSING_TRAFFIC. Hence added some delay seconds to the property in kubernetes service config yml to give some time to the app to make ReadinessState changed from REFUSING_TRAFFIC to ACCEPTING_TRAFFIC
In kustomization.yml file added the below 2 config properties which resolved the issue :
1. readinessProbe/initialDelaySeconds : 60
2. livenessProbe/initialDelaySeconds : 60
Also same can be added in application.yml file as below :
livenessProbe:
httpGet:
path: /actuator/health/liveness
port: 8080
initialDelaySeconds: 30
periodSeconds: 30