Please explain if this is a Zlib bug or I misunderstand the use of Zlib.

I am trying to do the following:

-I have two strings - data from which I need to compress: string_data_1 and string_data_2 and which I compress with Zlib as raw data.

-Next, I create a third string and copy the already compressed data into this single row.

-Now I'm decompressing this combined compressed data and there is a problem.

Zlib decompressed only the "first" part of the compressed data, did not decompress the second part. Is that how it should be?

For an example in the facebook/zstd:Zstandard library - exactly the same action - leads to unpacking - all compressed data and the first and second parts.

Here is a simple code:

#include <iostream>
#include <string>
#include <zlib.h>




int my_Zlib__compress__RAW(std::string& string_data_to_be_compressed, std::string& string_compressed_result, int level_compressed)
{

    //-------------------------------------------------------------------------
    uLong zlib_uLong = compressBound(string_data_to_be_compressed.size());         

    string_compressed_result.resize(zlib_uLong);
    //-------------------------------------------------------------------------


     //this is the standard Zlib compress2 function - with one exception: the deflateInit2 function is used instead of the deflateInit function and the windowBits parameter is set to "-15" so that Zlib compresses the data as raw data:
    int status = my_compress2((Bytef*)&string_compressed_result[0], &zlib_uLong, (const Bytef*)&string_data_to_be_compressed[0], string_data_to_be_compressed.size(), level_compressed);



    if (status == Z_OK)
    {
        string_compressed_result.resize(zlib_uLong);     

        return 0;
    }
        else
        { 
        return 1;
        }

}

int my_Zlib__uncompress__RAW(std::string& string_data_to_be_uncompressed, std::string& string_compressed_data, size_t size_uncompressed_data)
{

    //-------------------------------------------------------------------------
    string_data_to_be_uncompressed.resize(size_uncompressed_data);
    //-------------------------------------------------------------------------


    //this is the standard Zlib uncompress function - with one exception: the inflateInit2 function is used instead of the inflateInit function and the windowBits parameter is set to "-15" so that Zlib uncompresses the data as raw data:
    int status = my_uncompress((Bytef*)&string_data_to_be_uncompressed[0], (uLongf*)&size_uncompressed_data, (const Bytef*)&string_compressed_data[0], string_compressed_data.size());   

    if (status == Z_OK)
    {
        return 0;
    }
}




int main()
{

int level_compressed = 9;     


//------------------------------------------Compress_1-------------------------------------------
std::string string_data_1 = "Hello12_Hello12_Hello125";              //The data to be compressed.

std::string string_compressed_result_RAW_1;                             //Compressed data will be written here

int status = my_Zlib__compress__RAW(string_data_1 , string_compressed_result_RAW_1, level_compressed);
//----------------------------------------------------------------------------------------------


//--------------------------------------Compress_2----------------------------------------------
std::string string_data_2= "BUY22_BUY22_BUY223";              //The data to be compressed.

std::string string_compressed_result_RAW_2;                         //Compressed data will be written here

status = my_Zlib__compress__RAW(string_data_2 , string_compressed_result_RAW_2, level_compressed);
//----------------------------------------------------------------------------------------------


std::string Total_compressed_data = string_compressed_result_RAW_1   string_compressed_result_RAW_2; //Combine two compressed data into one string

//Now I want to uncompress the data in a string - "Total_compressed_data"


//--------------------------------------Uncompress--------------------------------
std::string string_uncompressed_result_RAW;                         //Uncompressed data will be written here

int size_that_should_be_when_unpacking = string_data_1.size()   string_data_2.size();

status = my_Zlib__uncompress__RAW(string_uncompressed_result_RAW, Total_compressed_data, size_that_should_be_when_unpacking , level_compressed);
//--------------------------------------------------------------------------------

std::cout<<string_uncompressed_result_RAW<<std::endl;  //Hello12_Hello12_Hello125

}

Zlib decompressed only the "first" part of the compressed data, did not decompress the "second" part.

Is that how it should be?

CodePudding user response：

As noted in the comments, a concatenation of zlib streams is not a zlib stream. You need to uncompress again for the second zlib stream. Or compress the whole thing as one zlib stream in the first place.

You would need to use a variant of uncompress2(), not uncompress(), since the former will return the size of the first decompressed zlib stream in the last parameter, so that you know where to start decompressing the second one.

Better yet, you should use the inflate() functions instead for your application. The retention of the uncompressed size for use in decompression means that you'd need that on the other end. How do you get that? Are you transmitting it separately? You do not need that. You should use inflate() to decompress a chunk at a time, and then you don't need to know the uncompressed size ahead of time.

You should also use the deflate() functions for compression. Then you can keep the stream open, and keep compressing until you're done. Then you will have a single zlib stream.