JAVA implementation timing task interface, timing to synchronization time increment to the third party system, third-party interface data volume limitation (1000), the third-party interface in a call, please get to the 1000 records, are stored in the proprietary database, such as article 200 record abnormal storage failure, how do you deal with at this moment? Please some directions!

CodePudding user response:

Bulk storage, add transaction, an error rolled back, try again

CodePudding user response:

reference 1/f, qinghai siu-tong response:

bulk storage, plus transaction, error rolled back, try again

Retry, fail? Can not always try again,

CodePudding user response:

That you define a mechanism bai error after a few times to abnormalities in the table

CodePudding user response:

If the 3000 data, batch 1000, three group, the first no problem the rest of the error, there are three common mistakes:
1, the primary key conflict data repeat, solution: the data is divided into two wave, wave insert, wave the update
2, the data type mismatch, solution: completes the data type conversion, string/date/time/integer, floating point four types is enough
3, field length is insufficient, the solution: according to find corresponding anomaly information fields, execute the alter table XXX change... Statement
Our independent research and development across the three situations of ETL is solved by program automatically, without human intervention, and have a web interface
If there is a fourth kind of circumstance, can send email alert

CodePudding user response:

Usually read is batch, then detailed processing,
If a record when dealing with exception, typically the error information is preserved (log database or writing hard disk),
Error information to complete, that is to say, the context of the key data, the best preserved at the same time, of course, also including the exception information,
To help troubleshoot problems,
, write a for error message handler, all kinds of anomalies for error message processing, repair and improve the program gradually,
Retry mechanism, in general, do not have all the fault tolerance of abnormal situation, and to set a maximum retries,