I have data from the DB that is in json.RawMessage format. The specific column is jsonb.

I can't really find a way to unmarshal the data to a property that on proto is defined as protobuf.Any like so.

repeated google.protobuf.Any list = 1;

When I try to unmarshal the data from the db using json.Unmarshal() then list is empty. The documentation mention something like this:

foo := &pb.Foo{...}
 any, err := anypb.New(foo)
 if err != nil {
   ...
 }
 ...
 foo := &pb.Foo{}
 if err := any.UnmarshalTo(foo); err != nil {
   ...
 }

But in this example foo is of type proto.Message which I can't convert since I have json.RawMessage.

Is there any way I can do this?

CodePudding user response：

First of all, you should understand what is stored in the DB column. json.RawMessage is simply defined as type RawMessage []byte (see the doc). And it does not carry enough information to answer your question.

I will provide a demo to show how google.protobuf.Any works, which should help you understand your question better.

Notes:

1. Any is for embedding other types in a message. So I define two other messages (Foo and Bar) in the demo.

   The Any message type lets you use messages as embedded types without having their .proto definition. An Any contains an arbitrary serialized message as bytes, along with a URL that acts as a globally unique identifier for and resolves to that message's type.

2. Actually, your question depends on what is stored in the DB. See the comments in main.go.

Folder structure of the demo:

├── go.mod
├── main.go
└── pb
    ├── demo.pb.go
    └── demo.proto

go.mod:

module github.com/ZekeLu/demo

go 1.19

require (
    github.com/golang/protobuf v1.5.2
    google.golang.org/protobuf v1.28.1
)

pb/demo.proto:

syntax = "proto3";
package pb;

import "google/protobuf/any.proto";

option go_package = "github.com/ZekeLu/demo/pb";

message MyMessage {
  repeated google.protobuf.Any list = 1;
}

message Foo {
  int32 v = 1;
}

message Bar {
  string v = 1;
}

main.go:

package main

import (
    "encoding/json"
    "fmt"

    "google.golang.org/protobuf/types/known/anypb"

    "github.com/ZekeLu/demo/pb"
)

func main() {
    // If the db stores an instance of pb.Foo, then unmarshal it first.
    buf := json.RawMessage([]byte(`{"v":10}`))
    var foo pb.Foo
    err := json.Unmarshal(buf, &foo)
    if err != nil {
        panic(err)
    }

    // And then marshal it into a new Any instance, which can be used to
    // create a slice that can be assigned to pb.MyMessage.List.
    a1, err := anypb.New(&foo)
    if err != nil {
        panic(err)
    }

    bar := &pb.Bar{V: "10"}
    a2, err := anypb.New(bar)
    if err != nil {
        panic(err)
    }

    // Initialize the List field.
    m := pb.MyMessage{List: []*anypb.Any{a1, a2}}

    buf, err = json.Marshal(&m)
    if err != nil {
        panic(err)
    }
    fmt.Printf("%s\n", buf)
    // Output: {"list":[{"type_url":"type.googleapis.com/pb.Foo","value":"CAo="},{"type_url":"type.googleapis.com/pb.Bar","value":"CgIxMA=="}]}

    // If the db stores the output above, it can be unmarshal directly
    var m2 pb.MyMessage
    err = json.Unmarshal(buf, &m2)
    if err != nil {
        panic(err)
    }

    fmt.Printf("%v\n", m2.List)
    // Output: [[type.googleapis.com/pb.Foo]:{v:10} [type.googleapis.com/pb.Bar]:{v:"10"}]
}

Steps to run the demo:

$ protoc --proto_path=pb --go_out=pb --go_opt=paths=source_relative demo.proto
$ go mod tidy
$ go run main.go