Unable to break down logstash message field-CodePudding

The input looks like

...,
"message": [
      ",{\"Timestamp\":\"2022-07-10T15:19:26.5172555Z\",\"Level\":\"Error\",\"MessageTemplate\":\"this is an error\",\"RenderedMessage\":\"this is an error\",\"Properties\":{\"RequestId\":\"0HMJ2FOA0IL5B:00000002\",\"RequestPath\":\"/\",\"ConnectionId\":\"0HMJ2FOA0IL5B\"}}\r"
],
...

I've tried the following:

filter {
    split {
        field => "message"
    }
}

and

filter {
    json {
        source => "message"
    }
}

I think there might be an issue with how it's being formatted, as there is a comma , at the start of the message string

below is an example of the minimal API I've made to test this

using Elastic.CommonSchema.Serilog;
using Serilog;

var builder = WebApplication.CreateBuilder(args);
builder.Host
    .UseSerilog((ctx, lc) => lc
        .WriteTo.Http(textFormatter: new EcsTextFormatter(), requestUri: "http://localhost:5000", queueLimitBytes: null)
        .Enrich.FromLogContext()
    );

var app = builder.Build();
app.MapGet("/", (Serilog.ILogger logger) =>
{
    logger.Error("this is an error");
    return "logged an error";
});
app.Run("http://localhost:3000");

CodePudding user response：

The issue I was having is due to an incorrect logstash input setup, previously I had:

input {
    beats {
        port => 5044
    }

    tcp {
        port => 5000
        tags => ["API"]
    }
}

And I was writing HTTP logs to port 5000 via Serilog.Sinks.Http

The correct logstash input is as below:

input {
    http {
        #default host 0.0.0.0:8080
        codec => json
    }
}

And to then write logs to 8080, very obvious in hindsight but it's something I was stuck with for a few days.

Doubt people make this mistake often (if ever) as I couldn't find anything for it online, but I'm writing this answer anyway!