Should all data from the database be mapped to the model?-CodePudding

I am doing homework on restAPI using Go and MongoDB. But I'm still wondering: As for whether I should create a dictionary to store data at the model level, it will help me to retrieve data much faster without accessing MongoDB. But the big problem here is to synchronize the data under MongoDB and in the dictionary that I created.

In file models/account.go I have a struct Account and in MongoDB I also have a collection Account to save all account information of the website. Should I create Accountlist to store all the data in the database to increase performance?.

Source as below:

var AccountList map[int]*Account

type Account struct {
    ID       int
    UserName string
    Password string
    Email    string
    Role     string
}

CodePudding user response：

As with many things in software, "It Depends".

There's not enough information about the systems involved, how often the data is being queried, mutated, and so on to give a concrete answer. But because this is for homework, we can give scenarios.

The root of your question is this: should you cache results from the database?

Is it really needed?

Academically, it's OK to over-optimize. You get to play with technologies and understand how they work. In the real world, we should understand where the need for something is before implementing it. The more complex a solution is, the more important making a correct trade-off becomes.

Caching is best when you're going to use the results more often than they're going to change, and fetching from storage is expensive.

"Expensive" can vary. One operation measured in seconds can be expensive. But so can tens, hundreds, or thousands of operations close together measured in 100ms.

How should you do it?

You called out a couple drawbacks. Most importantly:

But the big problem here is to synchronize the data under MongoDB and in the dictionary that I created.

Synchronization is the most important thing for any distributed system.

It doesn't matter how you cache values if you have one server instance. But once you start adding instances, things get complex.

A common pattern for caching is to use a distributed key-value store. They allow you to store results which can be shared across applications — and invalidate them.

Application checks to see if the key exists in the store.
If so, use it.
If not, fetch from origin and update the cache for next time.
Separately, invalidate the key any time data needs updating.

There are a bunch of products to use. Redis is popular, memcached works. But since you're using Go, checkout groupcache: https://github.com/mailgun/groupcache. It was written by Google to simplify dl.google.com, and extended by Mailgun to support TTLs.