I'm trying to create a self-signed server certificate for my client/server application (written in Java using the Netty library). From my understanding, I have to create a CA keystore and a server keystore, sign the server certificate using the CA keystore and place the CA certificate in the client's truststore. For this purpose, I have written the following windows script:
call keytool -genkey -alias ca -keyalg RSA -keystore ca.keystore -storetype JKS -storepass changeit -keypass changeit
:: create CA keystore
call keytool -exportcert -rfc -alias ca -file truststore.pem -keystore ca.keystore -storepass changeit
:: export CA certificate
call keytool -genkey -alias server -keyalg RSA -keystore server.keystore -storetype JKS -storepass changeit -keypass changeit
:: create server keystore
call keytool -certreq -alias server -keystore server.keystore -file server_signing_request.csr -storepass changeit
:: ask CA to sign server certificate
call keytool -gencert -infile server_signing_request.csr -outfile signed_server_cert.crt -keystore ca.keystore -alias ca -storepass changeit
:: sign server certificate
call keytool -importcert -alias ca -keystore server.keystore -file truststore.pem -storepass changeit
:: import root CA certificate into server keystore
call keytool -import -alias server -keystore server.keystore -trustcacerts -file signed_server_cert.crt -storepass changeit
:: import signed server certificate into server keystore
Server code to create the SSL engine:
final KeyStore keyStore;
try (FileInputStream input = new FileInputStream(new File("server.keystore"))) {
keyStore = KeyStore.getInstance("JKS");
keyStore.load(input, "changeit".toCharArray());
}
final KeyManagerFactory factory = KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());
factory.init(keyStore, "changeit".toCharArray());
final SSLContext context = SSLContext.getInstance("TLS");
context.init(factory.getKeyManagers(), null, null);
final SSLEngine sslEngine = context.createSSLEngine();
sslEngine.setUseClientMode(false);
sslEngine.setNeedClientAuth(false);
And to create the SslContext on client side:
final SslContext sslContext = SslContextBuilder.forClient().trustManager(new File("truststore.pem")).build();
However, connection attempts from client to server result in the following exception:
io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: PKIX path validation failed: sun.security.validator.ValidatorException: TrustAnchor with subject "CN=CA, OU=Unknown, O=Unknown, L=Unknown, ST=Unknown, C=Unknown" is not a CA certificate
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:477)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at de.klenze_kk.chess.client.netty.MyLoggingHandler.channelRead(MyLoggingHandler.java:28)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:719)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:655)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:581)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: javax.net.ssl.SSLHandshakeException: PKIX path validation failed: sun.security.validator.ValidatorException: TrustAnchor with subject "CN=CA, OU=Unknown, O=Unknown, L=Unknown, ST=Unknown, C=Unknown" is not a CA certificate
at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:131)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:326)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:269)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:264)
at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.checkServerCerts(CertificateMessage.java:1339)
at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.onConsumeCertificate(CertificateMessage.java:1214)
at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.consume(CertificateMessage.java:1157)
at java.base/sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:392)
at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:444)
at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:1061)
at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:1048)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask.run(SSLEngineImpl.java:995)
at io.netty.handler.ssl.SslHandler.runDelegatedTasks(SslHandler.java:1550)
at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1396)
at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1237)
at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1286)
at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:507)
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:446)
... 21 more
Caused by: sun.security.validator.ValidatorException: PKIX path validation failed: sun.security.validator.ValidatorException: TrustAnchor with subject "CN=CA, OU=Unknown, O=Unknown, L=Unknown, ST=Unknown, C=Unknown" is not a CA certificate
at java.base/sun.security.validator.PKIXValidator.doValidate(PKIXValidator.java:369)
at java.base/sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:263)
at java.base/sun.security.validator.Validator.validate(Validator.java:264)
at java.base/sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:313)
at java.base/sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:276)
at java.base/sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:141)
at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.checkServerCerts(CertificateMessage.java:1317)
... 35 more
Caused by: sun.security.validator.ValidatorException: TrustAnchor with subject "CN=CA, OU=Unknown, O=Unknown, L=Unknown, ST=Unknown, C=Unknown" is not a CA certificate
at java.base/sun.security.validator.PKIXValidator.verifyTrustAnchor(PKIXValidator.java:393)
at java.base/sun.security.validator.PKIXValidator.toArray(PKIXValidator.java:333)
at java.base/sun.security.validator.PKIXValidator.doValidate(PKIXValidator.java:366)
... 41 more
Can someone help, please? What am I doing wrong?
CodePudding user response：
It looks like you are in the right direction but missing couple of essential parts when creating the CA and the Server certificates.
I would recommend to use the option genkeypair instead of genkey and you also need some extension to give your certificates some additional capabilities.
For the CA I would recommend:
-ext KeyUsage=digitalSignature,keyCertSign -ext BasicConstraints=ca:true,PathLen:3
And for the server I would recommend:
-ext KeyUsage=digitalSignature,dataEncipherment,keyEncipherment,keyAgreement -ext ExtendedKeyUsage=serverAuth,clientAuth -ext SubjectAlternativeName:c=DNS:localhost,IP:127.0.0.1
Next to that you need to make sure to temporally import the CA certificate into the Server certificate before replacing the unsigned server certificate with the signed one. This is something which is weird in my opinion but oracle/sun just designed it this way. In your steps you have done this correctly, but you can remove the CA certificate when you have imported the signed server certificate.
You also don't need the -trustcacerts option when importing the signed server certificate. And I would recommend to use importcert instead of import command when importing the signed server certificate.
So to sum up I would recommend to execute the following steps:
1. Create CA keystore
keytool -genkeypair -alias ca -keyalg RSA -keystore ca.keystore -storetype JKS -storepass changeit -keypass changeit -ext KeyUsage=digitalSignature,keyCertSign -ext BasicConstraints=ca:true,PathLen:3
2. Create server keystore
keytool -genkeypair -alias server -keyalg RSA -keystore server.keystore -storetype JKS -storepass changeit -keypass changeit -ext KeyUsage=digitalSignature,dataEncipherment,keyEncipherment,keyAgreement -ext ExtendedKeyUsage=serverAuth,clientAuth -ext SubjectAlternativeName:c=DNS:localhost,IP:127.0.0.1
3. Create certificate signing request (CSR) for the server
keytool -certreq -alias server -keystore server.keystore -file server_signing_request.csr -storepass changeit -keyalg rsa
4. Sign the server CSR with the CA
keytool -gencert -infile server_signing_request.csr -outfile signed_server_cert.crt -keystore ca.keystore -alias ca -storepass changeit -ext KeyUsage=digitalSignature,dataEncipherment,keyEncipherment,keyAgreement -ext ExtendedKeyUsage=serverAuth,clientAuth -ext SubjectAlternativeName:c=DNS:localhost,IP:127.0.0.1 -rfc
5. Export the CA certificate
keytool -exportcert -rfc -alias ca -file truststore.pem -keystore ca.keystore -storepass changeit
6. Import the CA certificate into the server keystore
keytool -importcert -alias ca -keystore server.keystore -file truststore.pem -storepass changeit
7. Import the signed server certificate
keytool -importcert -alias server -keystore server.keystore -file signed_server_cert.crt -storepass changeit
8. Delete the imported CA certificate from the server keystore
keytool -delete -alias ca -keystore server.keystore -storepass changeit
Could you retry with these commands and share the results?
By the way I have written a tutorial here which might be helpful for you :) See: https://github.com/Hakky54/mutual-tls-ssl