I am using nanopb to do logging on an embedded system: my logging messages will be .proto messages.
Speed of encoding is the most important factor; I have plenty of RAM and flash.

QUESTION
Specific to nanopb, how do I minimize encoding time?

I know of all the C optimizations I could make: inline functions, set pb_encode functions in RAM instead of flash, etc

Would it make a difference if I use all fixed-size types in my .proto file?

CodePudding user response：

The easy methods I'm aware of are:

• Encode to memory buffer and enable PB_BUFFER_ONLY at compile time.
• If platform is little-endian and size of byte is 8 bits, you can define PB_LITTLE_ENDIAN_8BIT. It should be detected automatically on most compilers, though.
• In your message definition, minimize the amount of submessages. They require separate size calculations which take time.

These could result in speed improvements up to 2x.

It is further possible to speed up encoding by programming direct calls to the encoding functions, instead of going through the message structure and descriptor loops. I expect this to increase encoding speed up to 5x. Some knowledge of protobuf encoding spec will be needed.

For example, for this message:

message LogMessage {
    uint64 timestamp = 1;
    float batterylevel = 2;
    string logmessage = 3;
}

you could do:

void writelog(const char *logmsg)
{
    uint8_t buffer[256];
    pb_ostream_t stream = pb_ostream_from_buffer(buffer, sizeof(buffer));

    // Get system state
    uint64_t timestamp = get_system_timestamp();
    float batterylevel = get_batterylevel();

    // Encode timestamp
    pb_encode_tag(&stream, PB_WT_VARINT, LogMessage_timestamp_tag);
    pb_encode_varint(&stream, timestamp);

    // Encode battery level
    pb_encode_tag(&stream, PB_WT_32BIT, LogMessage_batterylevel_tag);
    pb_encode_fixed32(&stream, &batterylevel);

    // If we have explicit message, encode it also.
    if (logmsg)
    {
       pb_encode_tag(&stream, PB_WT_STRING, LogMessage_logmessage_tag);
       pb_encode_string(&stream, logmsg, strlen(logmsg));
    }

    // Save the encoded message data to storage
    save_log(buffer, stream.bytes_written);
}

While this results in hard-coding part of the message definition, the backward- and forward compatibility of protobuf messages works as usual. The tag numbers can be accessed by the Message_field_tag macros that are generated by nanopb generator. The field types are defined in the C code, but they are also embedded in the encoded messages so any mismatches will be detected on the decoding side.