I want to create a static array of instances of a struct, which has a member -- that points to a static stack-based variable-sized byte array. (can be represented as an array)

Due to the large size of the array and the fact that it holds static data -- I want to avoid heap allocations and keep the data stack-based.

A concrete example:

struct DataBlock
{
    size_t id;
    const unsigned char* data;
};

void test()
{
    // creating an array of DataBlock, with static data of different size
    DataBlock dataArray [] = {
        {
            .id = 1;
            .data = {0xFF, 0xCC, 0XDD};
        },
        {
            .id = 1;
            .data = {0xFF};
        }
    };
}

If the data was of the same size, I could've just defined data member like that –

const unsigned char data[STATIC_SIZE];

But due to the fact that the length can be different for each member, this approach isn't relevant.

An obvious solution would be to replace data with an std::vector -- but again, I want to avoid heap allocations, due to the fact that the data is 100% static and known in compile-time.

I've explored some template-based solutions -- which eventually produce a separate function for every array size, which is also not a desired results.

I'm sure C has a solution for that sort of scenario.

CodePudding user response：

The impulse to try and avoid superfluous allocations in dynamic memory ("heap") is good, but in this case it will lead to problems. Firstly, C does not have variable sized arrays. Secondly, if you want to use it outside the function you create it in, it will no longer exist. But most importantly, third, if your data is indeed very large, putting it in automatic storage will fill up the tiny stack-space that typical processes run with.

So, if I understand you correctly you have a large array of DataBlock objects, each of which may hold a small number of data items. The compromise then is to

• not have an std::vector in each individual struct, because that will create a lot of small dynamic memory allocations. Instead allocate one large std::vector and have your data items carry iterator-pairs pointing into that vector.
• Furthermore don't have your primary array in automatic memory either. For the same reasons.

Maybe you would go with something like this:

using DataBlock = std::pair<std::vector<unsigned char>::const_iterator>;
template <std::size_t N>
struct DataBlocks {
  std::vector<unsigned char> data;
  DataBlock array[N];
};
auto makeDataBlocks() {
  // return std::unique_ptr<DataBlocks<N>> for some N
} 

CodePudding user response：

You could store all the data together in a single array whose size needs to be at least the total size of all the data. For that we have a helper class:

struct Data {
  static const int MAX_SIZE = 100;
  uint8_t data[MAX_SIZE];
  int index = 0;

  const uint8_t* initBlock(const std::vector<uint8_t>& src) {
    int at = index;
    for (auto x: src) {
      data[index  ] = x;
    }
    return data   at;
  }
};

Then just call the initBlock method to put some data in that array and obtain a pointer that points at the data:

void test()
{
  Data data;
  
  // creating an array of DataBlock, with static data of different size
  DataBlock dataArray [] = {
    {
      .id = 1,
      .data = data.initBlock({0xFF, 0xCC, 0XDD})
    },
    {
      .id = 1,
      .data = data.initBlock({0xFF})
    }
  };
}