Home > Back-end >  Not the same output when using Messagepack encoding on Golang and Rust
Not the same output when using Messagepack encoding on Golang and Rust

Time:02-27

I am trying to make communicate a Go server with a Rust client (and vice-versa), and to do so, I want to serialize (or Marshal as you would say in Go) a struct in order to send it. Here are my codes :

package main

import (
    "fmt"
    "github.com/vmihailenco/msgpack/v5"
)

func ExampleMarshal() {
    type Human struct {
        Age byte
    }
    var x = Human{Age: 42}
    b, err := msgpack.Marshal(x)
    if err != nil {
        panic(err)
    }
    fmt.Println("b:", b)
    
}

func main () {
    ExampleMarshal()
} // took from https://github.com/vmihailenco/msgpack
extern crate serde;
#[macro_use]
extern crate rmp_serde as rmps;

use std::collections::HashMap;
use serde::{Deserialize, Serialize};
use rmps::{Deserializer, Serializer};

#[derive(Debug, PartialEq, Deserialize, Serialize)]
struct Human {
    age: u8,
}

fn main() {
    let mut buf = Vec::new();
    let val = Human {
        age: 42,
    };

    val.serialize(&mut Serializer::new(&mut buf)).unwrap();
    println!("{:?}", buf);
} // took from https://docs.rs/rmp-serde/latest/rmp_serde/

The problem is that with the exact same values, I don't get the same serialized value

  • Go -> b: [129 163 65 103 101 204 42]
  • Rust -> [145, 42]

Can someone explain me why I don't get the exact same values ? My goal is to have the Go Output the same as the Rust one

CodePudding user response:

Full answer for my own question.

The rust part was good, it was only the Go one that had a "problem", if we can call this a problem.

After digging the internet, I have found that I should use "Compact Encoding" in order to have the desired result.

The solution was to import the v4.0.4 version of msgpack in golang, and set to true, the UseCompactEncoding flag.

To conclude this thread, here are the codes that returns the exact same output using msgpack :

use serde::{Deserialize, Serialize};
use rmp_serde::{Deserializer, Serializer};

#[derive(Debug, PartialEq, Deserialize, Serialize)]
struct Human {
    name: String,
    age: u8,
}

fn main() {
    let x = Human { name: "nice".to_string(), age: 69 };
    let buf = rmp_serde::to_vec(&x).unwrap();

    println!("{:?}", buf);
}
package main

import (
    "bytes"
    "fmt"
    "github.com/vmihailenco/msgpack"
)

func main() {
    type Human struct {
        Name string
        Age  uint64
    }
    
    var buf bytes.Buffer
    enc := msgpack.NewEncoder(&buf).StructAsArray(true).UseCompactEncoding(true)
    err := enc.Encode(&Human{Name: "nice", Age: 69})
    if err != nil {
        panic(err)
    }
    fmt.Println(buf.Bytes())
}

Golang output : [146 69 164 110 105 99 101]

Rust output: [146, 69, 164, 110, 105, 99, 101]

  • Related