I want to use protocol buffer to send and receive the following type using gRPC

std:array<std::complex<int>, 2> bar_array;

• Sources used to get idea: 1, 2

What I have done so far

My approach (Intentionally I am omitting here the unnecessary code)

• proto file

syntax = "proto3";

package expcmake;

message child_data {
    repeated int32 child_data_list = 1;
}

message Address {
    child_data child_data_var = 8;
    repeated child_data parent_data_list = 9;
}

• server.cpp

  • Here, at first I have made a dummy std:array<std::complex<int>, 2> data. Then, I have filled the child_data_list with the std::complex<int> data. After each filling of real and imaginary part I have pushed them in the parent_data_list. Also, at this moment I have cleared the child_data_list.
  • Client message name is NameQuerry, while Server message name is Address
  • In Server side both message are passed as pointer

class AddressBookService final : public expcmake::AddressBook::Service {
    public:
        virtual ::grpc::Status GetAddress(::grpc::ServerContext* context, const ::expcmake::NameQuerry* request, ::expcmake::Address* response)
{

// omitting unnecessary lines

// populate bar_array with std::complex<int> data
std::complex<int> z4 = 1.   2i, z5 = 1. - 2i; // conjugates
bar_array = {z4, z5};

std::cout << "bar array printing whose size: " << bar_array.size() << std::endl;
for(int i = 0; i < bar_array.size(); i  ) {
    std::cout << bar_array[i] << " ";
}
std::cout << std::endl;

// Use parent_data_list protocol buffer message type to fill with the content of bar_array
for (int i = 0; i < bar_array.size(); i  ){
    
    // Use child_data protocol buffer message type to fill with the content of complex int data
    response->mutable_child_data_var()->add_child_data_list(real(bar_array[i]));
    response->mutable_child_data_var()->add_child_data_list(imag(bar_array[i]));
    
    // Use parent_data_list protocol buffer message type to fill with the content of child_data -> child_data_list data
    response->add_parent_data_list() -> child_data_list();
    
    // clear the child_data message. Reason to keep child_data_list new in every iteration otherwise add_child_data_list will append new data (eg: 1,2,1,-2) which is wrong. Expected is (1,2) then (1,-2)
    response->mutable_child_data_var()->clear_child_data_list();
}

// This is zero which I have got. Without clearing it is 4 which is also correct I believe as per the concept of protocol buffer message type
std::cout << "response->mutable_child_data_var()->child_data_list_size(): " << response->mutable_child_data_var()->child_data_list_size() << std::endl; 

// This is 2 which meets my requirement
std::cout << "response->parent_data_list_size(): " << response->parent_data_list_size() << std::endl;

// omitting unnecessary lines

}
};

• client.cpp

int main(int argc, char* argv[])
{
    // Setup request
    expcmake::NameQuerry query;
    expcmake::Address result;
    // printing the content of child_data -> child_data_list data array/container. There I have seen 1,2,1,-2 if I don't do the clear operation on child_data_list in server side. So, I guess it is correctly got the data 
          for(int i = 0; i < result.mutable_child_data_var()->child_data_list_size(); i  )
            std::cout << "Child Data at index [" << i << "]: " << result.mutable_child_data_var()->child_data_list(i) << std::endl;

    // This one making problem
    // printing the content of parent_data_list type/container
    // for(int i = 0; i < result.parent_data_list_size(); i  ){
    //   std::cout << "Parent Data at index [" << i << "]: " << result.mutable_parent_data_list(i) << std::endl; // This give me the memory address

    // Tried others to fetch the data but failed. Eg: result.parent_data_list(i) // failed
    // 
    // }
}

• Snippet from the generated pb file

  // repeated int32 child_data_list = 1;
  int child_data_list_size() const;
  private:
  int _internal_child_data_list_size() const;
  public:
  void clear_child_data_list();
  private:
  ::PROTOBUF_NAMESPACE_ID::int32 _internal_child_data_list(int index) const;
  const ::PROTOBUF_NAMESPACE_ID::RepeatedField< ::PROTOBUF_NAMESPACE_ID::int32 >&
      _internal_child_data_list() const;
  void _internal_add_child_data_list(::PROTOBUF_NAMESPACE_ID::int32 value);
  ::PROTOBUF_NAMESPACE_ID::RepeatedField< ::PROTOBUF_NAMESPACE_ID::int32 >*
      _internal_mutable_child_data_list();
  public:
  ::PROTOBUF_NAMESPACE_ID::int32 child_data_list(int index) const;
  void set_child_data_list(int index, ::PROTOBUF_NAMESPACE_ID::int32 value);
  void add_child_data_list(::PROTOBUF_NAMESPACE_ID::int32 value);
  const ::PROTOBUF_NAMESPACE_ID::RepeatedField< ::PROTOBUF_NAMESPACE_ID::int32 >&
      child_data_list() const;
  ::PROTOBUF_NAMESPACE_ID::RepeatedField< ::PROTOBUF_NAMESPACE_ID::int32 >*
      mutable_child_data_list();


  // .expcmake.child_data child_data_var = 8;
  bool has_child_data_var() const;
  private:
  bool _internal_has_child_data_var() const;
  public:
  void clear_child_data_var();
  const ::expcmake::child_data& child_data_var() const;
  PROTOBUF_MUST_USE_RESULT ::expcmake::child_data* release_child_data_var();
  ::expcmake::child_data* mutable_child_data_var();
  void set_allocated_child_data_var(::expcmake::child_data* child_data_var);
  private:
  const ::expcmake::child_data& _internal_child_data_var() const;
  ::expcmake::child_data* _internal_mutable_child_data_var();
  public:
  void unsafe_arena_set_allocated_child_data_var(
      ::expcmake::child_data* child_data_var);
  ::expcmake::child_data* unsafe_arena_release_child_data_var();


  // repeated .expcmake.child_data parent_data_list = 9;
  int parent_data_list_size() const;
  private:
  int _internal_parent_data_list_size() const;
  public:
  void clear_parent_data_list();
  ::expcmake::child_data* mutable_parent_data_list(int index);
  ::PROTOBUF_NAMESPACE_ID::RepeatedPtrField< ::expcmake::child_data >*
      mutable_parent_data_list();
  private:
  const ::expcmake::child_data& _internal_parent_data_list(int index) const;
  ::expcmake::child_data* _internal_add_parent_data_list();
  public:
  const ::expcmake::child_data& parent_data_list(int index) const;
  ::expcmake::child_data* add_parent_data_list();
  const ::PROTOBUF_NAMESPACE_ID::RepeatedPtrField< ::expcmake::child_data >&
      parent_data_list() const;

I guess

• Is the filling of the message field is incorrectly done !! Though the size is not saying that
• I am not catching the protobuf syntax(which is generated in the pb file) in right way to fetch the data

Need suggestions(helpful if can provide the syntax too).

CodePudding user response：

The mutable_* prefixed APIs are for mutating (modifying/adding) elements and they return a pointer. The mutable_* and set_* APIs should only be used while filling the data. Once data is filled, you should not be using mutable_* APIs to check sizes. After receiving the response from the server, the client will most probably be consuming it, not mutating. You need to update the client-side accordingly.

Also, you can use a range-based for loop to iterate over child_data_list of child_data_var like this:

const auto child_data_list = result.child_data_var().child_data_list();
for (const auto element : child_data_list)
{
    // process element
}

---

Apart from that, you're using std::array (a fixed-size array) i.e. std:array<std::complex<int>, 2>, alternatively here's another way to model this without repeated which does not represent a fixed-size array.

complex.proto

syntax = "proto3";

package pb;

message Complex {
    int32 real = 1;
    int32 imag = 2;
}

message Compound {
    Complex c1 = 1;
    Complex c2 = 2;
}

The message Complex emulates an std::complex type and Compound an std::array with only 2 elements c1 and c2.

With this, you can easily convert the protobuf message to std::complex and vice versa. Once converted to std::complex, you can perform its supported operations as needed.

• Compile complex.proto:

protoc --cpp_out=. ./complex.proto

For the C API, look for the accessors in the generated complex.pb.h file under public. The official doc Protocol Buffer Basics: C also lists the C API examples under The Protocol Buffer API section.

Here's a complete C example with the above complex.proto:

main.cpp

#include <iostream>
#include <complex>
#include <array>
#include <cstdlib>

#include "complex.pb.h"

namespace my {
    using complex = std::complex<int>;
    using compound = std::array<std::complex<int>, 2>;
}

int main()
{
    const auto my_c1 = my::complex{ 1, 2 };
    const auto my_c2 = my::complex{ 2, 4 };
    const auto my_compound_1 = my::compound{ my_c1, my_c2 };

    std::cout << "my_compound_1 [size: " << my_compound_1.size() << "]\n";
    std::cout << "my_c1: " << my_c1 << '\n';
    std::cout << "my_c2: " << my_c2 << '\n';

    pb::Compound pb_compound;
    pb_compound.mutable_c1()->set_real(my_compound_1[0].real());
    pb_compound.mutable_c1()->set_imag(my_compound_1[0].imag());
    pb_compound.mutable_c2()->set_real(my_compound_1[1].real());
    pb_compound.mutable_c2()->set_imag(my_compound_1[1].imag());

    std::cout << "\npb_compound:\n";
    pb_compound.PrintDebugString();

    const auto serialized_compound = pb_compound.SerializeAsString();

    // send

    // receive

    pb::Compound deserialized_compound;
    if (!deserialized_compound.ParseFromString(serialized_compound))
    {
        std::cerr << "[ERROR] Parsing failed!\n";
        return EXIT_FAILURE;
    }
    
    std::cout << "\n\npb_compound (deserialized):\n";
    deserialized_compound.PrintDebugString();

    const auto pb_c1 = deserialized_compound.c1();
    const auto pb_c2 = deserialized_compound.c2();

    const auto my_c3 = my::complex{ pb_c1.real(), pb_c1.imag() };
    const auto my_c4 = my::complex{ pb_c2.real(), pb_c2.imag() };
    const auto my_compound_2 = my::compound{ my_c3, my_c4 };

    std::cout << "my_compound_2 [size: " << my_compound_2.size() << "]\n";
    std::cout << "my_c3: " << my_c3 << '\n';
    std::cout << "my_c4: " << my_c4 << '\n';

    const auto sum = my_c3   my_c4;
    std::cout << "sum: " << sum << '\n';

    return EXIT_SUCCESS;
}

• Compile with g :

g   main.cpp complex.pb.cc -o pb_complex `pkg-config --cflags --libs protobuf`

• Run:

$ ./pb_complex
my_compound_1 [size: 2]
my_c1: (1,2)
my_c2: (2,4)

pb_compound:
c1 {
  real: 1
  imag: 2
}
c2 {
  real: 2
  imag: 4
}


pb_compound (deserialized):
c1 {
  real: 1
  imag: 2
}
c2 {
  real: 2
  imag: 4
}
my_compound_2 [size: 2]
my_c3: (1,2)
my_c4: (2,4)
sum: (3,6)

CodePudding user response：

The way I was accessing the filed parent_data_list was wrong. I am posting the approach which has solved the issue.

• A little change in proto file

syntax = "proto3";

package expcmake;

message child_data {
    repeated int32 child_data_list = 1 [packed=true];
}

message Address {
    repeated child_data parent_data_list = 1;
}

Reason to use packed see this

• server.cpp

class AddressBookService final : public expcmake::AddressBook::Service {
    public:
        virtual ::grpc::Status GetAddress(::grpc::ServerContext* context, const ::expcmake::NameQuerry* request, ::expcmake::Address* response){

// omitting unnecessary lines

// populate bar_array with std::complex<int> data
std::complex<int> z4 = 1.   2i, z5 = 1. - 2i; // conjugates
bar_array = {z4, z5};

// Now goal is to pass this bar_array as a protobuf message

// Use parent_data_list protocol buffer message type to fill with the content of bar_array
for (int i = 0; i < bar_array.size(); i  ){

    // make a 2D array. Eg: std::array<std::complex<int>>. In each iteration, new sub_content will be added (here sub_content means child_data_list)
    response->add_parent_data_list() -> child_data_list();

    // Followings are filling the child_data_list with the required data(Real and Imag part of the complex data which is already generated)
    response->mutable_parent_data_list(i)->add_child_data_list(real(bar_array[i]));
    response->mutable_parent_data_list(i)->add_child_data_list(imag(bar_array[i]));
        }
    }
    return grpc::Status::OK;
};

After the completion of the iteration, A 2D type of message will be generated which is parent_data_list = [[child_data_list], [child_data_list]]. Don't consider it as a real scenario. Just for presentation I am using it.

• client.cpp

std::cout <<"Using Mutable" << std::endl;
for(int i = 0; i < result.parent_data_list_size(); i  ){
    for(int j = 0 ; j<result.mutable_parent_data_list(i)->child_data_list_size(); j  ){
        std::cout << "Value [" << i << "][" << j << "]: " << result.mutable_parent_data_list(i)->child_data_list(j) << std::endl;
    }
}

// Following is same as before. Added as I was in turmoil to understand the properties generated by protobuf. Hope could be helpful for beginners like me 
std::cout <<"Without Mutable property" << std::endl;
for(int i = 0; i < result.parent_data_list_size(); i  ){
    for(int j = 0 ; j < result.parent_data_list(i).child_data_list_size(); j  ){
        std::cout << "Value [" << i << "][" << j << "]: " << result.parent_data_list(i).child_data_list(j) << std::endl;
    }
}

In client side I am accessing the child_data_list with index which is not properly mimic the C style. Would be great if I can in one line print the value of parent_data_list[n] data. As far I have studied the generated pb.h file my understanding is that it is not possible as no return function I have not found without index for the repeated field. Or may be I am missing something important again.