stream truncated error on HTTPS TLS handshake via a HTTP proxy using boost::asio (synchronous calls)-CodePudding

I've working on some HTTP code to connect to REST endpoints. I'm working within a company network and the connection to the outside world must go through a HTTP proxy, whereas the endpoint is on HTTPS.

Now, I've been successful in making these calls using both Python requests and curl, however making the same calls in C has been a pain.

I'm attempting to perform a handshake through this proxy with the code below (connecting to an example Postmen endpoint), which has been adapted from the following examples:

#include <boost/asio.hpp>
#include <boost/asio/ssl.hpp>

#include <boost/beast.hpp>

#include <boost/beast/http.hpp>
#include <boost/beast/http/empty_body.hpp>
#include <boost/beast/http/fields.hpp>
#include <boost/beast/http/message.hpp>
#include <boost/beast/http/string_body.hpp>

#include<boost/certify/https_verification.hpp>

#include <iostream>

using tcp = boost::asio::ip::tcp;
namespace http = boost::beast::http;

// https://postman-echo.com/post see https://docs.postman-echo.com/?version=latest
static const std::string
        server_endpoint = "/post",
        hostname = "postman-echo.com",
        target = "https://postman-echo.com:443",
        port_no = "443",
        authorization_token =
        "Auth: "
        "c3RhdGljIGNvbnN0IHN0ZDo6c3RyaW5nIGF1dGhvcml6YXRpb"
        "25fdG9rZW4gPSAiQXV0aDogIj"
        "sK",
        client_name = "User-Agent: demo program 0.01",
        req_str = R"(name=blabla&password=bloblo)";

int main() {
    boost::asio::io_service io_service;
    boost::asio::ssl::context ctx(boost::asio::ssl::context::sslv23);

    // Establish TCP base socket to Proxy
    tcp::socket base_sock{io_service};
    tcp::resolver resolver(io_service);

    {
        // Connect socket via proxy
        const boost::asio::ip::basic_resolver_results<boost::asio::ip::tcp> proxy_hosts = resolver.resolve("PROXY_ADDR", "PROXY_PORT");
        if (proxy_hosts.empty()) {
            throw std::range_error("Proxy cannot be resolved.");
        }
        std::cout << "Proxy IP address: " << proxy_hosts->endpoint().address() << std::endl;

        //boost::asio::ip::tcp::socket::lowest_layer_type& socket = base_sock.lowest_layer();
        base_sock.connect(proxy_hosts->endpoint());
    }

    {
        // Converts request to transparent TCP/IP tunnel. To facilitate TLS communications
        // through HTTP proxy.
        http::request<http::string_body> req1{http::verb::connect, target, 11};
        req1.set(http::field::host, target);
        http::write(base_sock, req1);

        boost::beast::flat_buffer buffer;

        http::response<http::empty_body> res;
        http::parser</* isRequest */ false, http::empty_body> http_parser(res);

        /** Set the skip parse option. */
        http_parser.skip(true); // see https://stackoverflow.com/a/49837467/10904212

        http::read_header(base_sock, buffer, http_parser);

        // Write the message to standard out
        std::cout << "target_host response: " << res << std::endl;
    }

    ctx.set_verify_mode(boost::asio::ssl::verify_peer);
    //ctx.load_verify_file("/** ssl/cacert from python requests library */");
    //ctx.add_verify_path("/** CA certificates from Curl via strace https://serverfault.com/questions/485597/default-ca-cert-bundle-location */");
    //ctx.load_verify_file("/** CA certs from curl via calling endpoints with --verbose */");
    //ctx.set_default_verify_paths(); 
    boost::certify::enable_native_https_server_verification(ctx); // from lib https://github.com/djarek/certify
    ctx.set_options(boost::asio::ssl::context::default_workarounds |
                    boost::asio::ssl::context::no_sslv2 |
                    boost::asio::ssl::context::no_sslv3);

    boost::asio::ssl::stream<boost::asio::ip::tcp::socket&> ssl_socket(base_sock,
                                                                       ctx);

    {
        //ssl_socket.handshake(boost::asio::ssl::stream_base::client);
        ssl_socket.handshake(boost::asio::ssl::stream<tcp::socket&>::client);

        /** Make REST calls here */
    }
}

Running the above code fails on the handshake step, with a "stream truncated" error.

My attempts thus far have been:

Verifying certificates used were correct, of which I've fed in the certificates from Python, Curl and from certify. This hasn't resolved the problem.
Checking Python and Curl traces for their protocol steps, however I'm unable to validate this in C due to the limited error messages from boost::asio; whilst gdb traces has been difficult to follow.
Lots of stackoverflow digging, unfortunately I was unable to piece together a working prototype.

Given my limited experiences in writing network code, though, it's likely that the error is in my logic somewhere rather than an incorrect environment.

Wondering if anyone could give any pointers to where might the code be incorrect?

CodePudding user response：

After more experimenting, I noticed that the problem is in the target variable. Setting target = "postmen-echo.com:443"; - dropping the https prefix, and passing in the appropriate CA certs (e.g. from either Python or Curl) resolves this issue.