I am using .NET Core 6 and I'm looking to convert the request and response to JSON for logging. When I use Newtonsoft.Json to convert the object I am getting an out of memory exception. What is the simplest and or best way to convert it without having to go through field by field?

Exception I got when trying to simply convert it.

Code of my logging middleware

public class RequestResponseLoggerMiddleware
{
    private readonly RequestDelegate _next;
    private readonly AppSettings _appSettings;
    private readonly ILogger<RequestResponseLoggerMiddleware> logger;

    public RequestResponseLoggerMiddleware(
        RequestDelegate next,
        IOptions<AppSettings> appSettings,
        ILogger<RequestResponseLoggerMiddleware> logger)
    {
        _next = next;
        _appSettings = appSettings.Value;
        this.logger = logger;
    }

    public async Task Invoke(HttpContext context)
    {
        logger.LogInformation($"{JsonConvert.SerializeObject(context.Request, Formatting.None, new JsonSerializerSettings(){ReferenceLoopHandling = ReferenceLoopHandling.Ignore})}");

        await _next(context);

        logger.LogInformation($"{JsonConvert.SerializeObject(context.Response, Formatting.None, new JsonSerializerSettings(){ReferenceLoopHandling = ReferenceLoopHandling.Ignore})}");
    }
}

Error:

Something went wrong: System.OutOfMemoryException: Insufficient memory to continue the execution of the program.

at System.Text.StringBuilder.ExpandByABlock(Int32 minBlockCharCount)
at System.Text.StringBuilder.Append(Char* value, Int32 valueCount)
at System.Text.StringBuilder.AppendHelper(String value)
at System.IO.StringWriter.Write(String value)
at Newtonsoft.Json.JsonWriter.AutoCompleteClose(JsonContainerType type)
at Newtonsoft.Json.JsonWriter.WriteEnd(JsonContainerType type)
at Newtonsoft.Json.JsonWriter.WriteEnd()
at Newtonsoft.Json.JsonWriter.AutoCompleteAll()
at Newtonsoft.Json.JsonWriter.Close()
at Newtonsoft.Json.JsonTextWriter.Close()
at Newtonsoft.Json.JsonWriter.Dispose(Boolean disposing)
at Newtonsoft.Json.JsonWriter.System.IDisposable.Dispose()
at Newtonsoft.Json.JsonConvert.SerializeObjectInternal(Object value, Type type, JsonSerializer jsonSerializer)
at Newtonsoft.Json.JsonConvert.SerializeObject(Object value, Type type, Formatting formatting, JsonSerializerSettings settings)
at Newtonsoft.Json.JsonConvert.SerializeObject(Object value, Formatting formatting, JsonSerializerSettings settings)
at API.Middleware.RequestResponseLoggerMiddleware.Invoke(HttpContext context)

What is the suggested / better way to do this?

CodePudding user response：

Another option:

According to Microsoft documentation in .Net 6 you can log request and response by adding app.UseHttpLogging(); middleware.

HTTP Logging is a middleware that logs information about HTTP requests and HTTP responses. HTTP logging provides logs of:

• HTTP request information
• Common properties
• Headers
• Body
• HTTP response information

HTTP Logging is valuable in several scenarios to:

• Record information about incoming requests and responses.
• Filter which parts of the request and response are logged.
• Filtering which headers to log.

CodePudding user response：

Looks like the information you're trying to convert is too huge. Newtonsoft.Json uses StringBuilder internally and StringBuilder generally has a limit of Int32.MaxValue. Again, this is implementation specific. Check the StringBuilder.MaxCapacity for your implementation.

There are two issues here:

1. The maximum number of characters that StringBuilder can hold.
2. The huge pressure on the Garbage Collector when StringBuilder dynamically keeps expanding (which I think is causing the OOM exception)

You have two options:

1. Find an alternative to NewtonSoft.Json that can handle large data elegantly. Check out System.Text.Json which Microsoft claims to be faster and efficient.
2. Reduce the size of your payload.