Need to obtain data through interfaces data volume is a little bit more in this way when parsing error

How to solve is better is to let each other less points or update the data interface batch update

CodePudding user response:

Reduce data, adding paging or compressed zip downloads interaction data,

CodePudding user response:

reference 1/f, fat uncle write code reply:

reduce data, adding paging or zip downloads interaction data,

the received data as json format string can not directly by the method using or partial parsing add paging or zip download what way

CodePudding user response:

Partial preach not, have to be so big, gzip, partial parsing is not appropriate, don't know where to begin points

CodePudding user response:

reference maailili reply: 3/f

batch no, have to be so big, gzip, partial parsing is not appropriate, don't know where to begin points

To receive a total of more than one hundred ten thousand data this time is about 6 w received for interface is adjust once a day so there's no good method, may be a little hurry,,,

CodePudding user response:

Amount of data is so big, that should be a json array, if an object has a so big
Then the array type should be a json object

So you can write a simple split of the array is to get rid of the fore and aft []
Then use {} a simple processing method to pick up the first object (attention should be paid to deal with the {} in the string!!!!! ), and deserialization
Then after one treatment object

CodePudding user response:

Source data format, a line of a json data, text, gzip, come here, in the form of file server and line-by-line parsing

CodePudding user response:

Partial transfer, too much

CodePudding user response:

Batch batch values, a bowl of rice can not eat

CodePudding user response:

You can know about the protobuf json or relatively slow!

CodePudding user response:

Saw, JavaScriptserialize did not block the function of reading, it seems only to write their own algorithm
Estimation is array
Can use a stack, which in turn read the json string, will [all pushed against the first], began to stack,...

CodePudding user response:

Range of int said, the number of assembly, partial

CodePudding user response:

Paging transmission,

CodePudding user response:

Multiple files from the source data points, and then compressed package, you receive the file, unzip the use,