I was trying to test a bit with Redis for the first time and I bumped into some confusion with HGET/HSET/HGETALL. My main problem was that I needed to store time, and I wanted to use a hash as I'll continuously update the time.
At first I read about how a MarshalBinary function such as this would save me:
func (f Foo) MarshalBinary() ([]byte, error) {
return json.Marshal(f)
}
What that did was that it saved the struct as a json string, but only as a string and not as an actual Redis hash. What I ended up doing in the end was a fairly large boilerplate code that makes my struct I want to save into a map, and that one is properly stored as a hash in Redis.
type Foo struct {
Number int `json:"number"`
ATime time.Time `json:"atime"`
String string `json:"astring"`
}
func (f Foo) toRedis() map[string]interface{} {
res := make(map[string]interface{})
rt := reflect.TypeOf(f)
rv := reflect.ValueOf(f)
if rt.Kind() == reflect.Ptr {
rt = rt.Elem()
rv = rv.Elem()
}
for i := 0; i < rt.NumField(); i {
f := rt.Field(i)
v := rv.Field(i)
switch t := v.Interface().(type) {
case time.Time:
res[f.Tag.Get("json")] = t.Format(time.RFC3339)
default:
res[f.Tag.Get("json")] = t
}
}
return res
}
Then to parse back into my Foo struct when calling HGetAll(..).Result(), I'm getting the result as a map[string]string and create a new Foo with these functions:
func setRequestParam(arg *Foo, i int, value interface{}) {
v := reflect.ValueOf(arg).Elem()
f := v.Field(i)
if f.IsValid() {
if f.CanSet() {
if f.Kind() == reflect.String {
f.SetString(value.(string))
return
} else if f.Kind() == reflect.Int {
f.Set(reflect.ValueOf(value))
return
} else if f.Kind() == reflect.Struct {
f.Set(reflect.ValueOf(value))
}
}
}
}
func fromRedis(data map[string]string) (f Foo) {
rt := reflect.TypeOf(f)
rv := reflect.ValueOf(f)
for i := 0; i < rt.NumField(); i {
field := rt.Field(i)
v := rv.Field(i)
switch v.Interface().(type) {
case time.Time:
if val, ok := data[field.Tag.Get("json")]; ok {
if ti, err := time.Parse(time.RFC3339, val); err == nil {
setRequestParam(&f, i, ti)
}
}
case int:
if val, ok := data[field.Tag.Get("json")]; ok {
in, _ := strconv.ParseInt(val, 10, 32)
setRequestParam(&f, i, int(in))
}
default:
if val, ok := data[field.Tag.Get("json")]; ok {
setRequestParam(&f, i, val)
}
}
}
return
}
The whole code in its ungloryness is here
I'm thinking that there must be a saner way to solve this problem? Or am I forced to do something like this? The struct I need to store only contains ints, strings and time.Times.
*edit The comment field is a bit short so doing an edit instead:
I did originally solve it like 'The Fool' suggested in comments and as an answer. The reason I changed to the above part, while more complex a solution, I think it's more robust for changes. If I go with a hard coded map solution, I'd "have to" have:
- Constants with hash keys for the fields, since they'll be used at least in two places (from and to Redis), it'll be a place for silly mistakes not picked up by the compiler. Can of course skip that but knowing my own spelling it's likely to happen
- If someone just wants to add a new field and doesn't know the code well, it will compile just fine but the new field won't be added in Redis. An easy mistake to do, especially for junior developers being a bit naive, or seniors with too much confidence.
- I can put these helper functions in a library, and things will just magically work for all our code when a time or complex type is needed.
My intended question/hope though was: Do I really have to jump through hoops like this to store time in Redis hashes with go? Fair, time.Time isn't a primitive and Redis isn't a (no)sql database, but I would consider timestamps in cache a very common use case (in my case a heartbeat to keep track of timed out sessions together with metadata enough to permanently store it, thus the need to update them). But maybe I'm misusing Redis, and I should rather have two entries, one for the data and one for the timestamp, which would then leave me with two simple get/set functions taking in time.Time and returning time.Time.
CodePudding user response：
I suggest keeping it simple. You know your data, so there should be no need to do reflection.
For example, for storing data, this would be fine.
type Foo struct {
Number int `json:"number"`
ATime time.Time `json:"atime"`
String string `json:"astring"`
}
func (f *Foo) Slice() []any {
return []any{
"number", f.Number,
"atime", f.ATime,
"string", f.String,
}
}
rdb.HSet(ctx, "foo", f.Slice())
You could also drop this and write some helper function in the following fashion. Perhaps you have a controller or similar.
func (f *Foo) HSet(ctx context.Context, rdb *redis.Client, key string) *redis.IntCmd {
return rdb.HSet(ctx, key, "number", f.Number, "atime", f.ATime, "string", f.String)
}
To retrieve the data, you could use the map that is returned by StringStringMapCmd.Result()
func fooFromStringStringMap(m map[string]string) (f Foo, err error) {
f.Number, err = strconv.Atoi(m["number"])
if err != nil {
return f, err
}
f.ATime, err = time.Parse(time.RFC3339, m["atime"])
if err != nil {
return f, err
}
f.String = m["string"]
return f, nil
}
m, _ := rdb.HGetAll(ctx, "foo").Result()
f2, _ := fooFromStringStringMap(m)
I think this is alright, performance wise. If you use StringStringMapCmd.Scan, it does some reflection under the hood, because it actually also needs to convert from a string map.
Alternatively you could use a helper struct whichs time field is string, you so scan into that as an intermediate step and then assign the values to the main struct converting the time.
type scanFoo struct {
Number int `redis:"number"`
ATime string `redis:"atime"`
String string `redis:"string"`
}
func fooFromStringStringMapCmd(cmd *redis.StringStringMapCmd) (f Foo, err error) {
if err = cmd.Err(); err != nil {
return
}
sf := scanFoo{}
if err := cmd.Scan(&sf); err != nil {
return f, err
}
f.Number = sf.Number
f.ATime, err = time.Parse(time.RFC3339, sf.ATime)
if err != nil {
return f, err
}
f.String = sf.String
return f, nil
}
You could also store the timestamp as Unix epoch. That way, you only need to store an integer. It should be easily to work with epoch instead of time.Time or convert in the places you really need it.
CodePudding user response：
You can use redigo/redis#Args.AddFlat to convert struct to redis hash we can map the value using redis tag.
package main
type Foo struct {
Number int64 `json:"number" redis:"number"`
Atime time.Time `json:"atime" redis:"atime"`
Astring string `json:"astring" redis:"astring"`
}
func main() {
c, err := redis.Dial("tcp", ":6379")
if err != nil {
fmt.Println(err)
return
}
defer c.Close()
t1 := time.Now().UTC()
var foo Foo
foo.Number = 10000000000
foo.Atime = t1
foo.Astring = "Hello"
tmp := redis.Args{}.Add("id1").AddFlat(&foo)
if _, err := c.Do("HMSET", tmp...); err != nil {
fmt.Println(err)
return
}
v, err := redis.StringMap(c.Do("HGETALL", "id1"))
if err != nil {
fmt.Println(err)
return
}
fmt.Printf("%#v\n", v)
}
Then to update Atime you can use redis HSET
if _, err := c.Do("HMSET", "id1", "atime", t1.Add(-time.Hour * (60 * 60 * 24))); err != nil {
fmt.Println(err)
return
}
And to retrieve it back to struct we have to do some reflect magic
func mapToStruct(src map[string]string, dst interface{}) error {
re := reflect.ValueOf(dst).Elem()
for k, v := range src {
key := []byte(k)
key[0] -= 32
rf := re.FieldByName(string(key))
switch rf.Interface().(type) {
case time.Time:
format := "2006-01-02 15:04:05 -0700 MST"
ti, err := time.Parse(format, v)
if err != nil {
return err
}
rf.Set(reflect.ValueOf(ti))
case int, int64:
x, err := strconv.ParseInt(v, 10, rf.Type().Bits())
if err != nil {
return err
}
rf.SetInt(x)
default:
rf.SetString(v)
}
}
return nil
}
Final Code
package main
import (
"time"
"github.com/gomodule/redigo/redis"
"reflect"
"strconv"
"fmt"
)
type Foo struct {
Number int64 `json:"number" redis:"number"`
Atime time.Time `json:"atime" redis:"atime"`
Astring string `json:"astring" redis:"astring"`
}
func main() {
c, err := redis.Dial("tcp", ":6379")
if err != nil {
fmt.Println(err)
return
}
defer c.Close()
t1 := time.Now().UTC()
var foo Foo
foo.Number = 10000000000
foo.Atime = t1
foo.Astring = "Hello"
tmp := redis.Args{}.Add("id1").AddFlat(&foo)
if _, err := c.Do("HMSET", tmp...); err != nil {
fmt.Println(err)
return
}
v, err := redis.StringMap(c.Do("HGETALL", "id1"))
if err != nil {
fmt.Println(err)
return
}
fmt.Printf("%#v\n", v)
if _, err := c.Do("HMSET", "id1", "atime", t1.Add(-time.Hour * (60 * 60 * 24))); err != nil {
fmt.Println(err)
return
}
var foo2 Foo
mapToStruct(v, &foo2)
fmt.Printf("%#v\n", foo2)
}
func mapToStruct(src map[string]string, dst interface{}) error {
re := reflect.ValueOf(dst).Elem()
for k, v := range src {
key := []byte(k)
key[0] -= 32
rf := re.FieldByName(string(key))
switch rf.Interface().(type) {
case time.Time:
format := "2006-01-02 15:04:05 -0700 MST"
ti, err := time.Parse(format, v)
if err != nil {
return err
}
rf.Set(reflect.ValueOf(ti))
case int, int64:
x, err := strconv.ParseInt(v, 10, rf.Type().Bits())
if err != nil {
return err
}
rf.SetInt(x)
default:
rf.SetString(v)
}
}
return nil
}
Note: For mapToStruct function we have to make sure the key are title case for struct.
for eg: redis:"atime" map to Atime