How do I get Java/HtmlUnit to accept an old or bad website certificate?-CodePudding

If you go to https://eagletw.mohavecounty.us/treasurer/treasurerweb/search.jsp using Firefox on Linux, you will see that you can browse the website just fine.

But when I compile and run the following program:

import com.gargoylesoftware.htmlunit.html.*;
import com.gargoylesoftware.htmlunit.javascript.*;
import java.io.*;

public class BadWebsiteCertificate {
    public static void BadWebsiteCertificate () {
        try (final WebClient webClient = new WebClient()) {
            System.getProperties().put("org.apache.commons.logging.simplelog.defaultlog", "fatal");
            webClient.getOptions().setThrowExceptionOnScriptError(false);
            webClient.getOptions().setThrowExceptionOnFailingStatusCode(false);

            webClient.getOptions().setCssEnabled(false);
            webClient.setJavaScriptErrorListener(new SilentJavaScriptErrorListener());
            webClient.setCssErrorHandler(new SilentCssErrorHandler());
            webClient.setAjaxController(new NicelyResynchronizingAjaxController());
            HtmlPage page = webClient.getPage("https://eagletw.mohavecounty.us/treasurer/treasurerweb/search.jsp");
            webClient.waitForBackgroundJavaScriptStartingBefore(10000);
            page = (HtmlPage) page.getEnclosingWindow().getEnclosedPage();
            webClient.getOptions().setThrowExceptionOnScriptError(false);
            webClient.setJavaScriptErrorListener(new SilentJavaScriptErrorListener());
            HtmlTable grdTaxHistory = (HtmlTable) page.getElementById("grdTaxHistory");
            HtmlTableDataCell cpCell = (HtmlTableDataCell) grdTaxHistory.getCellAt(4,6);
            ((HtmlAnchor) cpCell.getFirstChild().getNextSibling()).click();

            webClient.waitForBackgroundJavaScriptStartingBefore(1_000);
            page = (HtmlPage) page.getEnclosingWindow().getEnclosedPage();
        }

        catch (Exception e) {
            System.out.println("Error: "  e);
        }

    }

    public static void main(String[] args) {
        File file = new File("validParcelIDs.txt");
        BadWebsiteCertificate();
    }

}

using the following commands:

javac -classpath ".:/opt/htmlunit_2.69.0/*" BadWebsiteCertificate.java
java -classpath ".:/opt/htmlunit_2.69.0/*" BadWebsiteCertificate

I get the following runtime error message:

Error: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

I tried the following solution proposed at Resolving javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed Error?

echo -n | openssl s_client -connect eagletw.mohavecounty.us:443 | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > ~/eagletw.mohavecounty.us.crt
sudo keytool -import -v -trustcacerts -alias eagletw.mohavecounty.us -file ~/eagletw.mohavecounty.us.crt -keystore /usr/lib/jvm/java-11-openjdk-amd64/lib/security/cacerts -keypass changeit -storepass changeit

But that didn't fix the problem. I am still getting the same runtime error message.

Any ideas of what else I can try?

CodePudding user response：

For some reason your Java runtime environment does not trust that certificate. There are two things you can try:

add that website's certificate to your truststore manually
upgrade to the latest JVM and see if the truststore in that version already solves your problem

I'd go first for upgrade, if that does not work check how to add certificates to Java's truststore. See also

CodePudding user response：

You can disable the SSL check by

webClient.getOptions().setUseInsecureSSL(true);

This might also help if you use a proxy like Charles Web Proxy to inspect the traffic.