How do I diagnose failure to start jenkins after changing JVM options?-CodePudding

I'm trying to set some JVM for jenkins so that I can poke around with garbage collection logs, based on recommendations here

My jenkins server runs on amazon linux on EC2, so I edited the config file at /etc/sysconfig/jenkins ton include:

JENKINS_JAVA_OPTIONS="-Djava.awt.headless=true -XX: AlwaysPreTouch -XX: HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/var/log/jenkins -XX: UseG1GC -XX: UseStringDeduplication -XX: ParallelRefProcEnabled -XX: DisableExplicitGC -XX: UnlockDiagnosticVMOptions -XX: UnlockExperimentalVMOptions -verbose:gc -Xloggc:/var/log/jenkins/gc.log -XX:NumberOfGCLogFiles=2 -XX: UseGCLogFileRotation -XX:GCLogFileSize=100m -XX: PrintGC -XX: PrintGCDateStamps -XX: PrintGCDetails -XX: PrintHeapAtGC -XX: PrintGCCause -XX: PrintTenuringDistribution -XX: PrintReferenceGC -XX: PrintAdaptiveSizePolicy -XX:ErrorFile=/var/log/jenkins/hs_err_%p.log -XX: LogVMOutput -XX:LogFile=/var/log/jenkins/jvm.log"

But then when I restart the service (sudo systemctl restart jenkins.service), it looks like it's started up:

[ec2-user@ip-10-0-0-30 ~]$ sudo systemctl status jenkins.service
● jenkins.service - LSB: Jenkins Automation Server
   Loaded: loaded (/etc/rc.d/init.d/jenkins; bad; vendor preset: disabled)
   Active: active (exited) since Mon 2021-11-08 18:49:28 UTC; 7s ago
     Docs: man:systemd-sysv-generator(8)
  Process: 18318 ExecStop=/etc/rc.d/init.d/jenkins stop (code=exited, status=0/SUCCESS)
  Process: 20797 ExecStart=/etc/rc.d/init.d/jenkins start (code=exited, status=0/SUCCESS)

Nov 08 18:49:28 ip-10-0-0-30.ec2.internal systemd[1]: Starting LSB: Jenkins Automation Server...
Nov 08 18:49:28 ip-10-0-0-30.ec2.internal jenkins[20797]: Starting Jenkins [  OK  ]
Nov 08 18:49:28 ip-10-0-0-30.ec2.internal systemd[1]: Started LSB: Jenkins Automation Server.

I'd normally expect to see "Active: active (running)" and I can't access it (connections are refused on port 8080), and there are no new log messages in /var/log/jenkins/jenkins.log

None of the other log files are created either:

[ec2-user@ip-10-0-0-30 ~]$ sudo ls -la /var/log/jenkins
total 5856
drwxr-x--- 2 jenkins jenkins     242 Nov  4 11:17 .
drwxr-xr-x 9 root    root       4096 Nov  8 03:45 ..
-rw-r--r-- 1 jenkins jenkins 2007179 Nov  8 17:29 jenkins.log
-rw-r--r-- 1 jenkins jenkins  312046 Oct 15 03:40 jenkins.log-20211015.gz
-rw-r--r-- 1 jenkins jenkins  546457 Oct 23 03:36 jenkins.log-20211023.gz
-rw-r--r-- 1 jenkins jenkins  352006 Oct 28 03:09 jenkins.log-20211028.gz
...

Where else can I look to figure out why the service isn't starting?

Update: A more minimal set of options that does work:

JENKINS_JAVA_OPTIONS="-Djava.awt.headless=true -XX: UseG1GC -XX: AlwaysPreTouch -XX: UseStringDeduplication -XX: ParallelRefProcEnabled -XX: DisableExplicitGC"

CodePudding user response：

It turns out that many of the options recommended in the cloudbees post are specific to Java 8, and not compatible with Java 11. Based on this blog post, I got it working with the following JVM options:

JENKINS_JAVA_OPTIONS="-Djava.awt.headless=true -XX: AlwaysPreTouch -XX: HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/var/log/jenkins/ -XX: UseG1GC -XX: UseStringDeduplication -XX: ParallelRefProcEnabled -XX: DisableExplicitGC -XX: UnlockDiagnosticVMOptions -XX: UnlockExperimentalVMOptions -Xlog:gc*,gc heap=trace,gc age*=trace,gc ref*=trace,gc ergo*=trace:file=/var/log/jenkins/gc-%t.log:utctime:filecount=2,filesize=100m -XX:ErrorFile=/var/log/jenkins/hs_err_%p.log -XX: LogVMOutput -XX:LogFile=/var/log/jenkins/hs_%p.log"