I wonder how can I write consecutively to the same file in Go? Do I have to use os.WriteAt()?
The JSON is basically just an array filled with structs:
[
{
"Id": "2817293",
"Data": "XXXXXXX"
},
{
"Id": "2817438",
"Data": "XXXXXXX"
}
...
]
I want right data to it consecutively i.e. append to that JSON array more than once before closing the file. The data I want to write to the file is a slice of said structs:
dataToWrite := []struct{
Id string
Data string
}{}
What is the proper way to write consecutively to a JSON array in Go?
My current approach creates multiple slices in the JSON file and thus is not what I want. The write process (lying in a for loop) looks like this:
...
// Read current state of file
data := []byte{}
f.Read(data)
// Write current state to slice
curr := []Result{}
json.Unmarshal(data, &curr)
// Append data to the created slice
curr = append(curr, *initArr...)
JSON, _ := JSONMarshal(curr)
// Empty data container
initArr = &[]Result{}
// Write
_, err := f.Write(JSON)
if err != nil {
log.Fatal(err)
}
...
CodePudding user response：
If you don't care about the existing file you can just use Encoder.Encode on the whole slice as @redblue mentioned.
If you have an existing file you want to append to, the simplest way is to do what you've shown in your edit: Unmarshal or Decoder.Decoder the whole file into a slice of structs, append the new struct to the slice, and re-decode the whole lot using Marshal or Encoder.Encode.
If you have a large amount of data, you may want to consider using JSON Lines to avoid the trailing , and ] issue, and write one JSON object per line. Or you could use regular JSON, seek back from the end of the file so you're writing over the final ], then write a ,, the new JSON-encoded struct, and finally a ] to make the file a valid JSON array again.
So it depends on a bit on your use case and the data size which approach you take.
CodePudding user response：
Write the opening [ to the file. Create an encoder on the file. Loop over slices and the elements of each slice. Write a comma if it's not the first slice element. Encode each slice element with the encoder. Write the closing ].
_, err := f.WriteString("[")
if err != nil {
log.Fatal(err)
}
e := json.NewEncoder(f)
first := true
for i := 0; i < 10; i {
// Create dummy slice data for this iteration.
dataToWrite := []struct {
Id string
Data string
}{
{fmt.Sprintf("id%d.1", i), fmt.Sprintf("data%d.1", i)},
{fmt.Sprintf("id%d.2", i), fmt.Sprintf("data%d.2", i)},
}
// Encode each slice element to the file
for _, v := range dataToWrite {
// Write comma separator if not the first.
if !first {
_, err := f.WriteString(",\n")
if err != nil {
log.Fatal(err)
}
}
first = false
err := e.Encode(v)
if err != nil {
log.Fatal(err)
}
}
}
_, err = f.WriteString("]")
if err != nil {
log.Fatal(err)
}