I want to write the data from the redis to mysql, from the redis tens of thousands of the value of a list, one-time into mysql timeout problems may arise, although you can turn the timeout time is big, but if the amount of data is more and more big unrealistic
Want to send this list section, use the cycle don't feel very good judgment

CodePudding user response:

Solved using list fragmentation and while loop

 
Start=0 
End=0 
While True: 
End +=500 
Values=redis_util. Get_values_batch_keys (keys [start: end]) 
Insert_list=[] 
For the row in values: 
Params=row 
The row=json. Loads (row, encoding="utf-8") 
A_url=row [' action '] 
Insert_list. Append ((2, a_url, params)) 
If len (insert_list) & gt; 0: 
Db_util. Executemany_no_commit (insert_sql insert_list) 
Redis_util. Delete_batch (keys [start: end]) 
MIT () 
 db_util.comStart=end 
If the end & gt; Len (keys) : 
Break