Let's say there is a microservice, an API over http with several endpoints like:
- get the balance of an account
- deposit money into an account
- withdraw money from an account
Now let's say there are client applications, subsystems, etc that make requests to this API. For example: in an ATM a customer can withdraw money, bank web application (frontend app) that make requests to the API, external app like third party paymment apps, etc.
This is the scenario I'm thinking:
- two apps/systems needs to withdraw some amount of money from the same customer account
- app1 is an ATM, customer wants to withdraw $200
- app2 is a payment system, it needs to withdraw $100 for some bill
- customer balance is $250
timeline: 1- app1 request balance, api returns $250 2- app2 request balance, api returns $250 3- app1 makes a request for withdraw $200 4- app2 makes a request for withdraw $100
At this point the balance could be less than 0, if (3) and (4) were successful. Let's say before trying to make the withdraw the API check the balance. Then (3) will be allowed but (4) will return an error.
1- app1 request balance, api returns $250 2- app2 request balance, api returns $250 3- app1 makes a request for withdraw $200 3.1- api get account balance (=$250) 3.2- $250 > $200 => withdraw allowed 4- app2 makes a request for withdraw $100 4.1- api get account balance (=$50) 4.2- $50 < $100 => withdraw NOT allowed
Ok. But, what if another app/system changed the balance in between (3.1) and (3.2) from $250 to $150. If the withdraw is allowed that obviously be bad.
Is this a possible scenario? How can be prevented or avoided?
CodePudding user response：
It's better to make all methods responsible for account balance change synchronized so one system thread would wait for the other to finish its request before doing its own stuff.
You should have one microservice responsible for such types of transactions. Since all balance validations happen on the backend you should be good. If app1, app2 & some 3rd party withdraw money at the same time - they will wait for each other since they are performing operations on the same account method that checks the balance withdraws the money is synchronized.
I believe not all of the PLs have such a thing as process synchronization so I would also recommend another option where you basically create some object holding info about the API request and for every request you put such object into the FIFO queue. Then you configure a thread that will constantly poll the queue for new records and if there are any -> pass the API request object into the method that checks the account balance and performs the balance operation.
In any of these options, you should abort the operation and return an error to the user if the account balance is less than what was requested to withdraw.
CodePudding user response：
This is not clearly a microservice issue. This may also occur in Monolith application as well with same endpoint.
Scenario 1:
This is more a issue of concurrency and transaction.
Concurrency issue can be solved by locking either pessimistic or optimistic.
Mainly in Web Application scenario or current level of concurrent system pessimist locking is not good. In this kind of locking It will lock entire row till operation finish by one client for that account.
Another is optimistic locking.
For your scenario let's assume that you have one extra field associated with Account row that hold balance for account. This field called RowVersion ( It may maintain by DB automatically)
Now let's say if you have System A or System B ( any system)
When system request record for balance, It will also has this rowversion.
Now when any system request to withdrawl, In that request it has this RowVersion.
When any request try to update account balance by withdrawl amount it will also check RowVersion.
update account set balance = balance - requestwithdrawlamount where accountId = requestaccountId and rowversion = requestrowversion
Now if rowversion is changed by other system then this operation will not get success.
If rowversion not change then update success and rowversion automatically update to new rowversion ( This value mostly control by DB and atomic fashion)
Along with lock there is also good things to consider is transaction isolation level. It should be serializable to give better consistency for such type of operation.
Scenario 2:
In this scenario, you can use some messaging system. Need to make sure that for particular accountId , all message goes to same partition. By this way it will maintain order. This is async operation and give more throughput. In this scenario two you have to consider locking and transaction. Here too if there are two consumer that process same message on same row then issue occur. ( Mostly avoided using partition but still good to consider for financial system)