I am fairly new to Azure cloud development.

I have a function app coded in C# that:

1. Gets a record from a storage table
2. Deletes that record
3. Updates fields on that record (including the partition key)
4. inserts the new record into the storage table

I am experiencing data loss, when an exception is thrown, on the insert portion.

I am wondering how, if step 4 throws an exception, I can then rollback step 2. If that is not possible how would I prevent the data loss, as I'm unable to use the built in Table Operations that would replace the entity because I am changing the partition key?

I understand that the hard part in all of this to be the partition key update, as I know the system was designed so that each transaction or operation is operating on records with the same partition key.

I have looked through the Table Service REST API and looked at all the Table Operations I thought could be helpful: Insert Entity Update Entity Merge Entity Insert or Update Entity Insert or Replace Entity

CodePudding user response：

You can't do transactions due to partition key. So you'll have to look at a solution outside of the table storage.

What you could do is create the record before deleting it. That way you're assured that you won't lose any data (as long as you make sure the request to create a record succeeded).

You could take it one step further by making it an async process. Having a storage queue or service bus queue up your message containing the information of the request and having a function app (or anything else) handle the requests. That way you can assure the request remains retryable in case any transient errors occur over a larger timespan.

CodePudding user response：

As per question, we are able to reproduce the data loss issue. In table, have below highlighted record.

Once exception occurred on Insert data got loss as mentioned in question

To update value of PartitionKey CosmosDb doesn't allow direct update to value of partitionkey. First we need to delete the record and then create new record with new partitionkey value.

To prevent data loss using built in TableOperations you can perform/call Execute() once prior steps got completed successfully.

TableOperation delOperation = TableOperation.Delete(getBatchCustomer);

You can clone or create a deep copy of first object using copy constructor.

public Customer(Customer customer)
{
    PartitionKey = customer.PartitionKey;
    RowKey = customer.RowKey;
    customerName = customer.customerName;
}

Creating copy of the object

Customer c = new(getCustomer)
{
    PartitionKey = "India"
};

In step 4 mentioned in question by you is completed successfully then we can commit delete operation.

Got exception on insert step

but when looked data table no data has been lost.

Below is code snippet to prevent data loss.

TableOperation _insOperation = TableOperation.Insert(c);
var insResult = _table.Execute(_insOperation);
if (insResult.HttpStatusCode == 204)
{
    var delResult = _table.Execute(delOperation);
}