I have about 5,000 read-only records.
Typically I store records in an Azure blob, and a function will retrieve the data, search through them, and then return matching records to the end-user.
Is it efficient and safe to, instead, retrieve the data from the Azure blob once, store it in a static variable, and then search through that?
I've looked and looked, but I can't find a comparison or discussion between (a) repeatedly retrieving data from an Azure blob vs. (b) just maintaining it in a static variable, and why that might be undesirable.
CodePudding user response:
Issues with repeatedly retrieving data from an Azure blob:
- Every retrieve (read) operation is a chargeable transaction so your Azure bill will go up.
- Every retrieve (read) operation is a network call which might fail so you have to take that into consideration. Furthermore, it will add time to your overall request process.
Issues with maintaining it in a static variable:
- Static variables will consume server memory.
Not sure how big these files are but if they are not big, then I would recommend storing them in static variables if your blobs are truly read-only (i.e. will not change).
CodePudding user response:
This sounds like a Cache-Aside pattern use-case. Load data on demand into a cache from a data store. This can improve performance and also helps to maintain consistency between data held in the cache and data in the underlying data store.
https://docs.microsoft.com/en-us/azure/architecture/patterns/cache-aside
CodePudding user response:
I would go with what Gaurav suggested. I have used similar pattern with my azure functions. I generally load the variables in static variables and keep using it. you should keep in mind that your azure function would run from a container and the azure function runtime can load multiple containers on demand. So please remember to do a null check on your static variable. considering that your 5000 records are not very big (few Kb each), you will not incur a lot of bill. Also the performance would be much better.