I am looking for an efficient solution to iterate through large files located on blob storage in C#. I want to use the blob storage as a local storage. For now, I find only solutions where the file has to be downloaded or streamed, maybe there are some external tools that could be integrated. Is it possible to directly iterate through a file on blob storage without downloading it locally or to a stream?

CodePudding user response：

Is it possible to directly iterate through a file on blob storage without downloading it locally or to a stream?

Simple answer: No. You can list the blobs, i.e. their properties. But if you want to actually work with the content, you need to download it from the storage account. The storage is a pure object store.

CodePudding user response：

As already mentioned, this is not possible. Maybe you could split up your files into smaller chunks and up-/downoad these for manipulations. If for example you have a 1MB file, split it up into 10 KB chunks and add an integer suffix to your original name to distinct the individual chunks. If you need a specific part of the file, calculate the needed chunk ids and download these files only.

If this approach works for you, highly depends on your data. Can it be split by size or some other criteria (e.g. number of lines, json elements), does changing the data within one chunk have impact on other chunks (e.g. split by rows and you insert a row into a chunk). So be careful with this approach and choose wisely.