HTTP gzip compression does not save any time-CodePudding

I have a Spring Boot application that sometimes has to serve a very big JSON payload (several MB) over a REST API, which takes considerable time to download.

The data is read from a DB, serialized into JSON and sent back to the client. The DB read operation is fast, even for big datasets, usually below 1 second. So my conclusion was that the most time consuming part is the HTTP exchange.

I've enabled GZIP compression for the HTTP exchange so the payload should be compressed before being sent. It seems this works (the returned payload is indeed compressed), however, there is no noticeable performance gain.

A curl request to the application's endpoint without compression takes 49 sec and yields a ~10 MB JSON payload:

curl -H "Content-Type: application/json" -H "Accept: application/json" -H "Authorization: Basic <REDACTED>" --data-binary @priorities-request.json  'https://<REDACTED>/api/rest/priorities' > priorities-response.json                
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 10.0M    0  9.9M  100 85081   205k   1715  0:00:49  0:00:49 --:--:--  239k

With GZIP compression enabled, the same request takes 42 sec and yields a ~260 kb GZip compressed JSON payload:

curl -H "Content-Type: application/json" -H "Accept: application/json" -H "Accept-Encoding: gzip,deflate,br" -H "Authorization: Basic <REDACTED>" --data-binary @priorities-request.json  'https://<REDACTED>/api/rest/priorities' > priorities-response.json 
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  259k    0  176k  100 85081   4221   1991  0:00:42  0:00:42 --:--:-- 14408

My expectation would be that downloading a compressed 260K payload would take considerably less time than an uncompressed 10 MB download.

What's my mistake?

Edit: Because it's been asked in the comments how I set up the GZIP compression: I set compression="on" and compressableMimeType="application/json" in the server.xml of Tomcat. That's it. The rest is done by Tomcat's org.apache.coyote.http11.filters.GzipOutputFilter class.

Edit 2: To rule out that serializing the data into JSON is where the time is lost, I tested locally with Jackson2JsonMessageConverter, but it took only about 0.5 seconds to write even a huge data structure into a 10MB JSON string.

Edit 3: What I find most puzzling is that the client application that consumes the API, which is running on another Tomcat instance on the same physical machine, still experiences the same delay when retrieving the data.

CodePudding user response：

When you use gzip, there is a trade-off between CPU usage and bandwidth usage that needs to be considered. What comes as advantage for the end user (less transfer time i.e. faster download speed) will require additional CPU cycles at server side.
The performance gain would certainly be visible if you run a load test. Also if your API is used internally by other APIs in your application, it will reduce the network usage drastically. The CPU usage may not be that significant.

CodePudding user response：

If you plan to send over the network a compressed data than may be compress it not when you read it from DB but when you write it to DB. That way the compression time will not be at all part of the request and you save on your storage space. Than disable HTTP compressin and read your compressed binary data from DB and send it over