Can't set up MQTT (Mosquito Docker) SSL MQTTNet (C#) Dapr.io binding-CodePudding

I am trying to set up a local MQTT broker using mosquito image and connect to it with MQTTNet as publisher and subscribe using Dapr MQTT binding. Everything works fine if I am using anonymous mode (both in Docker self-hosted and in K8S). But we need to use SSL... and here things stopped working. And I am literally out of ideas.

Anyone done this setup already?

I am following this guide for creating certificates: Mosquito SSL setup

This is my broker Dockerfile:

FROM eclipse-mosquitto:latest

COPY ./mqtt/mosquitto/mosquitto.conf /mosquitto/config/mosquitto.conf
COPY ./mqtt/mosquitto/certificates /mosquitto/certs
COPY ./mqtt/mosquitto/certificates /mosquitto/ca_certificates

and mosquitto.conf file:

listener 8883
protocol mqtt

listener 9001
protocol websockets

allow_anonymous false

cafile ./mosquitto/certs/ca.crt
keyfile ./mosquitto/certs/server.key
certfile ./mosquitto/certs/server.crt

require_certificate true
use_identity_as_username true

tls_version tlsv1.2

Broker is starting fine:

2023-01-02 23:44:16 1672699456: mosquitto version 2.0.15 starting
2023-01-02 23:44:16 1672699456: Config loaded from /mosquitto/config/mosquitto.conf.
2023-01-02 23:44:16 1672699456: Opening ipv4 listen socket on port 8883.
2023-01-02 23:44:16 1672699456: Opening ipv6 listen socket on port 8883.
2023-01-02 23:44:16 1672699456: Opening websockets listen socket on port 9001.
2023-01-02 23:44:16 1672699456: mosquitto version 2.0.15 running

Now, I am trying to use MQTTNet library and I wrote a simple c# console app:

using MQTTnet;
using MQTTnet.Client;
using System.Security.Cryptography.X509Certificates;
using System.Text;
using System.Text.Json;

var caCert = new X509Certificate2("ca.crt");
var clientCert = new X509Certificate2("client.pfx", "password");

var url = "localhost";
var factory = new MqttFactory();
var client = factory.CreateMqttClient();
var options = new MqttClientOptionsBuilder()
    .WithTcpServer(url, 8883)
    .WithClientId("Device1")
    .WithTls(new MqttClientOptionsBuilderTlsParameters
    {
        UseTls = true,
        SslProtocol = System.Security.Authentication.SslProtocols.Tls12,
        CertificateValidationHandler = (certContext) =>
        {
            X509Chain chain = new X509Chain();
            chain.ChainPolicy.RevocationMode = X509RevocationMode.NoCheck;
            chain.ChainPolicy.RevocationFlag = X509RevocationFlag.ExcludeRoot;
            chain.ChainPolicy.VerificationFlags = X509VerificationFlags.NoFlag;
            chain.ChainPolicy.VerificationTime = DateTime.Now;
            chain.ChainPolicy.UrlRetrievalTimeout = new TimeSpan(0, 0, 0);
            chain.ChainPolicy.CustomTrustStore.Add(caCert);
            chain.ChainPolicy.TrustMode = X509ChainTrustMode.CustomRootTrust;

            var x5092 = new X509Certificate2(certContext.Certificate);

            return chain.Build(x5092);
        },
        AllowUntrustedCertificates = true,
        Certificates = new List<X509Certificate2>
        {
            clientCert, caCert
        },
    })
    .Build();

await client.ConnectAsync(options);

var message = new MqttApplicationMessageBuilder()
    .WithTopic("sensor/reporting")
    .WithPayload(Encoding.UTF8.GetBytes("test"))
    .Build();
await client.PublishAsync(message);

And I am not able to connect with client error:

MQTTnet.Exceptions.MqttCommunicationException: ' Received an unexpected EOF or 0 bytes from the transport stream.'

And server error:

2023-01-02 23:47:00 1672699620: New connection from 172.19.0.1:37912 on port 8883.
2023-01-02 23:47:00 1672699620: Client disconnected due to protocol error.

Tried different options and never succeded.

On the subscriber side, I am following this guide: Dapr MQTT Binding with below config file:

apiVersion: dapr.io/v1alpha1
kind: Component
metadata:
  name: sensorreporting
spec:
  type: bindings.mqtt
  version: v1
  metadata:
  - name: url
    value: ssl://localhost:8883
  - name: topic
    value: sensor/reporting
  - name: consumerID
    value: "{uuid}"
  - name: caCert
    value: -----BEGIN CERTIFICATE-----\nMIIEGzCCAwOgAwIBAgIUYbZRJpxF3knVBs4A9FIc9KU6YskwDQYJKoZIhvcNAQEL\nBQAwgZwxCzAJBgNVBAYTAkNBMRAwDgYDVQQIDAdBbGJlcnRhMRAwDgYDVQQHDAdD\nYWxnYXJ5MRMwEQYDVQQKDAptQ2xvdWRDb3JwMRYwFAYDVQQLDA1DZXJ0QXV0aEFk\nbWluMQ0wCwYDVQQDDARhZGNhMS0wKwYJKoZIhvcNAQkBFh5taWNoYWwua29zb3dz\na2lAbWNsb3VkY29ycC5jb20wHhcNMjMwMTAyMjE0MzU5WhcNMjgwMTAyMjE0MzU5\nWjCBnDELMAkGA1UEBhMCQ0ExEDAOBgNVBAgMB0FsYmVydGExEDAOBgNVBAcMB0Nh\nbGdhcnkxEzARBgNVBAoMCm1DbG91ZENvcnAxFjAUBgNVBAsMDUNlcnRBdXRoQWRt\naW4xDTALBgNVBAMMBGFkY2ExLTArBgkqhkiG9w0BCQEWHm1pY2hhbC5rb3Nvd3Nr\naUBtY2xvdWRjb3JwLmNvbTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEB\nAMxZASmaTBj 8nVnQwngdjQ3632AVforWdZWSW7a/Z9mhlCis8Lmg/B7Hun9NUh3\nlwg3GxB9RN0K3A aeHObWfSUl R1NiTVAMWh CtUVibjWnDJGk TNQ9Drq5Lh/iX\nNPc2ztUm iaFyWdXcthWjeYqsnLD4NHbD0470F4mNidg93cPvB66c0Eam01pAAkR\nQ/jUU0W8gncN3SEHc/FAUahGp1xZzxWhawBAr/oa7xjDMZsz4cLBHjnUH/wNuTrZ\nxQ7g/ArO9DsDaITj7 tzKvOLkCza3LzjW7Ye19XL5l9DisX3xBNCeERIIdndwkOm\ndkEif6QIMgZjaYyyMH6JSsECAwEAAaNTMFEwHQYDVR0OBBYEFDDUsLcvER/gEJgl\nstJ3VutBQ/t MB8GA1UdIwQYMBaAFDDUsLcvER/gEJglstJ3VutBQ/t MA8GA1Ud\nEwEB/wQFMAMBAf8wDQYJKoZIhvcNAQELBQADggEBALzcZ32VWo37Da5wfFmONo1I\n bnnP4dc6i90/6MVuLoco6nt63XE DwL53Ib8tPYVMssKCLOBCI y tRtpIAIF y\ntg41SP5iLIU4eV4rjzdd3iirZOavvcx2cLVfMRHFVr1/8V5o4LaQOpiwVycndcHY\noRTXW6h49YpZt914oEwezSRBrN5h8Rc4cJB/alVpj/2FMq0 C6qQmgJ56xLT9Yu \nAFz2X4zoFX69WsGm h/cOfrMjqdR96UoS8cUgEjeNPZCxrZLWGEvphnjmjCom SQ\nJ qvQQBCGlNW6Hajd6yBybynB4ImwsqySlWYuX/JwfCDxq1rt  lWjha7hzGSdY=\n-----END CERTIFICATE-----
  - name: clientCert
    value: -----BEGIN CERTIFICATE-----\nMIIDwDCCAqgCFA7Zd6l6GgZYMtdtTwKHM7b3jn0HMA0GCSqGSIb3DQEBCwUAMIGc\nMQswCQYDVQQGEwJDQTEQMA4GA1UECAwHQWxiZXJ0YTEQMA4GA1UEBwwHQ2FsZ2Fy\neTETMBEGA1UECgwKbUNsb3VkQ29ycDEWMBQGA1UECwwNQ2VydEF1dGhBZG1pbjEN\nMAsGA1UEAwwEYWRjYTEtMCsGCSqGSIb3DQEJARYebWljaGFsLmtvc293c2tpQG1j\nbG91ZGNvcnAuY29tMB4XDTIzMDEwMjIxNTcwMVoXDTIzMTIyODIxNTcwMVowgZsx\nCzAJBgNVBAYTAkNBMRAwDgYDVQQIDAdBbGJlcnRhMRAwDgYDVQQHDAdDYWxnYXJ5\nMRMwEQYDVQQKDAptQ2xvdWRDb3JwMRIwEAYDVQQLDAlhZC1jbGllbnQxEDAOBgNV\nBAMMB2RldmljZTExLTArBgkqhkiG9w0BCQEWHm1pY2hhbC5rb3Nvd3NraUBtY2xv\ndWRjb3JwLmNvbTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAOZ2d3jm\n240jQ9JdTAoAEe/LaoFD 0q7TisXLeHBruCZWjMCbiqinT GtvyhOvNVVJk8/0jH\n1pTkAIads1hIqve6AUNinrZd9LRbW8CNeESz w29GcOcdZ9fMsQf42PqHd Y8Aes\nF/2TSU9Qu dVpplrdHOfz5WjmC88/AD9btMDrrJQOhi4MFD8Buy4S5Lrw4ZRQ4be\n72hHAuD3nVT /WS9EJtVSANHaIHsOPDmyJAPFVS6wWZRAHv5BEf5D/UtjmGg3VRn\nGN2krK1/MLMSv20kjePi7dErOtAE1Q2fsvEfs0zko3/qrGZlELVlxKqYgnAMfY2w\nsVf8sGvTz9sJIKkCAwEAATANBgkqhkiG9w0BAQsFAAOCAQEAc/ wE8Efgmi1GB5G\nSlPpFbDmM pAXm7cAbELTkQK4i70TZ4R7aRS9QulxSxL1MwpHkyQqvW9bN0s WtX\nUqh3ERyXCyeuOc6KpskROryyCD03rL2j83KIeTvbLSwsx3gwzE0nBy336Y2A1fYT\nJVkYUhi4Gh9LqSovw8yn O/DjGhzfBPo1MaPZihtVGmTlhvY9ypUsefggC FE8E8\n8UmvesY7H /h5TammDoHTFBupQG9JIx76DZJFQlyyGGanttfBooya4ZLzpZ zmeW\nFOvz RmekQzjt Jrtrzv1zkyrhFB129kr9/cNzZBtR2Hte2ezSgmPHFjIPZ0Z4ZC\nTW47cw==-----END CERTIFICATE-----
  - name: clientKey
    value: -----BEGIN RSA PRIVATE KEY-----\nMIIEpAIBAAKCAQEA5nZ3eObbjSND0l1MCgAR78tqgUP7SrtOKxct4cGu4JlaMwJu\nKqKdP4a2/KE681VUmTz/SMfWlOQAhp2zWEiq97oBQ2Ketl30tFtbwI14RLP7Db0Z\nw5x1n18yxB/jY od35jwB6wX/ZNJT1C751WmmWt0c5/PlaOYLzz8AP1u0wOuslA6\nGLgwUPwG7LhLkuvDhlFDht7vaEcC4PedVP79ZL0Qm1VIA0dogew48ObIkA8VVLrB\nZlEAe/kER/kP9S2OYaDdVGcY3aSsrX8wsxK/bSSN4 Lt0Ss60ATVDZ y8R zTOSj\nf qsZmUQtWXEqpiCcAx9jbCxV/ywa9PP2wkgqQIDAQABAoIBAQDP28dztxwMFuNj\nx5v ZQPIeHot7oemZnthJ/3M4Sh EDInUajVMWeMVU TWUPXFn4/26EQHpVuIppK\nz89i79 roQrkoP0u1F0RkliVucLgjEgBoy2qnOIFhRI/i9dGVXDuK31KHjBhF8eG\nDvX69uhU9XLJmjja6Psol NebTXYgIISUz0S39KGqH3BZ9IcRre/peNDh4LqUot9\n Wmrj/4kIgwC2FXLWw5Bz5wH4cvZbrSrf p4jf8MZehBBTlkJ2270aEwU2Di rDx\nS4o1rzDEB3FO6eoDJk vxseZhqUjAX7Vm2HDZhEM y1rXuOXY11ya2phIA7Y4YzX\nBuXvNQKBAoGBAP8PI4wk8XjVt5mD6KUbvPzcmEtTG43PrTKh9vXdsP9bkW8 RUEp\npwpguSxPYCXRGSl4A3Ava/8uT8grAx1978LY7l65uCK9EUTdYs5a3BrjhxBVE3Bj\np/e3VSmE8qD1ixlnfcRUf0Dv8CrivKmW4Lz226A3hS6AlWfzI9amEfG5AoGBAOdQ\nGcKSUijcNkASud4VZyOqNqjKtj5SKx1HblnOq7HERblPW53wD8Mg9jeQ8X7Flaox\n52m1W3EcO1VVLveQ6FYsBXVz 9FwHblztizVlty48XmzpQDbAL1xI G0WT7cpDt8\nDdnlOmZX4hxBsmrkOFz/gP4yn6sL0u0wbRTk995xAoGAcBe1Jaguv04djEgefEF \ngxpni MQXviAQ5pOsKsnWHJjriINyZ Gm9b6SnDv7m7AwirOSAVvTwyJX40Xp2Rr\ndJRl3hd8xzeUWVF6VPo1dVO9Tf41ttT1Qra9pKua/gYrhZSn/xqrelx4jkBrbZSz\nXBiZWQZNb6xu/OosO 9sP3ECgYEA3KoVaQCdUE2Dk2xJ1Asm8WcX7b40TQ mAbB7\nHUGDmXpb6YRDKlAxOTzgpfGn8AAd3tbspHNWYHlGQQmaXKHogwxLyNh0ikAimyA/\n0lWQLdBgzHEMH9zMv yHBnI ETH7mqOlsxByvd9o3PvvcPl2q4EzVUyF9W66MXvZ\nE 26aIECgYBtDK 8hffO3N1yXW tMFuCbbzMPrPJC5o4VII8pm oNcsvW1Ao9kpZ\nL7yVVJCozo0omvuY2ohVchiQ5jC1nxd17Xa0DuIrKgtcEmpoae16e2h7kcSjv63y\nbKEEUIKRyWfhx6iJb8o03lq1UUxRDhaB5leiUUddibJyhh7EDfCHfQ==\n-----END RSA PRIVATE KEY-----

and dapr sidecar is failing with following error:

mqtt-sample-mqtt-api-dapr-1 | time="2023-01-02T23:19:05.428410597Z" level=fatal msg="process component sensorreporting error: [INIT_COMPONENT_FAILURE]: initialization error occurred for sensorreporting (bindings.mqtt/v1): mqtt binding error: invalid ca certificate" app_id=mqtt-api instance=2c64804e20f1 scope=dapr.runtime type=log ver=1.9.5 mqtt-sample-mqtt-api-dapr-1 exited with code 1

I do not know where I am making mistake. I am going to test that on AKS now (maybe the problem is with running it locally? Anonymous mode is working with AKS).

I was able to set up the whole thing without SSL, but need this to go to prod.

CodePudding user response：

You have the listener on port 8883 configured to use websockets; you appear to be connecting using plain MQTT over TCP, not over websockets.

Your client needs to use the same protocol as the broker is configured for on the port the client connects to.

Try commenting out the websockets line in the mosquitto config file, restarting the broker and connecting again.

CodePudding user response：

The TLS certificate setup is per listener, so if you want both the listener on port 8883 and the Websocker listener on port 9001 to use TLS then you need to specify the certificates twice e.g.

listener 8883
protocol mqtt
cafile ./mosquitto/certs/ca.crt
keyfile ./mosquitto/certs/server.key
certfile ./mosquitto/certs/server.crt
require_certificate true
use_identity_as_username true
tls_version tlsv1.2

listener 9001
protocol websockets

allow_anonymous false
cafile ./mosquitto/certs/ca.crt
keyfile ./mosquitto/certs/server.key
certfile ./mosquitto/certs/server.crt


tls_version tlsv1.2

You also need to be very careful about requiring a client certificate for the websocker listener and using it as a the username, because if you are accessing from a Web Browser it is VERY tricky to set up client certificates. Hence I have removed that requirement from the WebSocket listener