I am using .net core 6 and I'm looking to convert the request and response to json for logging. When I use Newtonsoft.Json to convert the object I am getting an out of memory exception. What is the simplest and or best way to convert it without having to go through field by field?

Exception I got when trying to simply convert it.

Code of my logging middleware

public class RequestResponseLoggerMiddleware
    {
        private readonly RequestDelegate _next;
        private readonly AppSettings _appSettings;
        private readonly ILogger<RequestResponseLoggerMiddleware> logger;

        public RequestResponseLoggerMiddleware(
            RequestDelegate next,
            IOptions<AppSettings> appSettings,
            ILogger<RequestResponseLoggerMiddleware> logger
        )
        {
            _next = next;
            _appSettings = appSettings.Value;
            this.logger = logger;
        }

        public async Task Invoke(HttpContext context)
        {
            logger.LogInformation($"{JsonConvert.SerializeObject(context.Request, Formatting.None, new JsonSerializerSettings(){ReferenceLoopHandling = ReferenceLoopHandling.Ignore})}");

            await _next(context);

            logger.LogInformation($"{JsonConvert.SerializeObject(context.Response, Formatting.None, new JsonSerializerSettings(){ReferenceLoopHandling = ReferenceLoopHandling.Ignore})}");
        }
    }

Error:

Something went wrong: System.OutOfMemoryException: Insufficient memory to continue the execution of the program.
         at System.Text.StringBuilder.ExpandByABlock(Int32 minBlockCharCount)
         at System.Text.StringBuilder.Append(Char* value, Int32 valueCount)
         at System.Text.StringBuilder.AppendHelper(String value)
         at System.IO.StringWriter.Write(String value)
         at Newtonsoft.Json.JsonWriter.AutoCompleteClose(JsonContainerType type)
         at Newtonsoft.Json.JsonWriter.WriteEnd(JsonContainerType type)
         at Newtonsoft.Json.JsonWriter.WriteEnd()
         at Newtonsoft.Json.JsonWriter.AutoCompleteAll()
         at Newtonsoft.Json.JsonWriter.Close()
         at Newtonsoft.Json.JsonTextWriter.Close()
         at Newtonsoft.Json.JsonWriter.Dispose(Boolean disposing)
         at Newtonsoft.Json.JsonWriter.System.IDisposable.Dispose()
         at Newtonsoft.Json.JsonConvert.SerializeObjectInternal(Object value, Type type, JsonSerializer jsonSerializer)
         at Newtonsoft.Json.JsonConvert.SerializeObject(Object value, Type type, Formatting formatting, JsonSerializerSettings settings)
         at Newtonsoft.Json.JsonConvert.SerializeObject(Object value, Formatting formatting, JsonSerializerSettings settings)
         at API.Middleware.RequestResponseLoggerMiddleware.Invoke(HttpContext context)

What is the suggested / better way to do this?

CodePudding user response：

Another option:

According to Microsoft documentation in .Net 6 you can log request and response by adding app.UseHttpLogging(); middleware.

HTTP Logging is a middleware that logs information about HTTP requests and HTTP responses. HTTP logging provides logs of:

• HTTP request information
• Common properties
• Headers
• Body
• HTTP response information

HTTP Logging is valuable in several scenarios to:

• Record information about incoming requests and responses.
• Filter which parts of the request and response are logged.
• Filtering which headers to log.

CodePudding user response：

Looks like the information you're trying to convert is too huge. Newtonsoft.Json uses StringBuilder internally and StringBuilder generally has a limit of Int32.MaxValue. Again, this is implementation specific. Check the StringBuilder.MaxCapacity for your implementation.

There are two issues here:

1. The maximum number of characters that StringBuilder can hold.
2. The huge pressure on the Garbage Collector when StringBuilder dynamically keeps expanding (which I think is causing the OOM exception)

You have two options:

1. Find an alternative to NewtonSoft.Json that can handle large data elegantly. Check out System.Text.Json which Microsoft claims to be faster and efficient.
2. Reduce the size of your payload.