I am required to save logs into a MySQL database of each request and response made to the backend. The issue is that we are migrating to microservices architecture. The backend was made with NodeJS and Express, and it has a middleware that does this task. Currently, it has this middleware attached to each microservice.

I would like to isolate this middleware as its own microservice. The issue is that I don't know how to redirect the traffic this way. This is how I would like to manage it:

I would like to do it this way, because we can make changes or add features to the middleware without having to implement it in each microservice. This is the middleware's code:

const connection = require("../database/db");

const viewLog = (req, res, next) => {
  const oldWrite = res.write,
    oldEnd = res.end,
    chunks = [],
    now = new Date();

  res.write = function (chunk) {
    chunks.push(chunk);
    oldWrite.apply(res, arguments);
  };

  res.end = function (chunk, error) {
    if (chunk) chunks.push(chunk);

    const bodyRes = Buffer.concat(chunks).toString("utf8");

    connection.query("CALL HospitalGatifu.insertLog(?,?,?,?,?)", [
      `[${req.method}] - ${req.url}`,
      `${JSON.stringify(req.body) || "{}"}`,
      bodyRes,
      res.statusCode === 400 ? 1 : 0,
      now,
    ]);

    oldEnd.apply(res, arguments);
  };

  next();
};

module.exports = viewLog;

I think there might be a way to manage this with Nginx which is the reverse proxy that we are using. I would like to get an approach of how to change the logs middleware.

CodePudding user response：

Let me give you not exactly what you requested but something to think about.

I can think on 3 solutions for the issue of logging in microservices, each 1 have its own advantages and disadvantages:

1. Create a shared library that handles the logs, I think its the best choice in must cases. An article I wrote about shared libraries
2. You can create API gateway, it is great solution for shared logic to all the requests. So it will probably be more work but then can be used for other shared logic. Further read (not written by me :) )
3. A third option (which I personally don't like) is create a log microservice that listens to LogEvent or something like that. Then from your MSs publish this event whenever needed.

CodePudding user response：

Perhaps you might want to take a look at the sidecar pattern which is used in microservice architectures for common tasks (like logging).

In short, a sidecar runs in a container besides your microservice container. One task of the sidecar could be intercepting network traffic and logging requests and responses (and a lot of other possible tasks). The major advantage of this pattern is that you don't need to change any code in your microservices and you don't have to manage traffic redirection yourself. The latter will be handled by the sidecar itself.

The disadvantage is that you are required to run your microservices containerized and use some kind of container orchestration solution. I assume this being the case since you are moving towards a microservices based application.

One question about the log service in between of the webapp and the NGNIX server. What if the logging services goes down for some reason, is it acceptable for the entire application to go down?