Home > Software engineering >  Saving CSV output from Azure function to Azure container (Python 3X)
Saving CSV output from Azure function to Azure container (Python 3X)

Time:09-30

I want to run an Azure function and save a CSV output to a Azure container. I currently have two blocks of code that

  1. Generates a CSV file.
  2. Loads a CSV file into my container.

Each blocks works on my local PC in a Jupyter Notebook.

But I am struggling to combine them to work together in an Azure function. So I am looking for help.

Block 1 (Generate the CSV)

import yfinance as yf
import pandas as pd
from datetime import date
import csv

#stock names
NZX =[["Ascension Capital Limited", "ACE"],["AFC Group Holdings Limited", "AFC"],["Z Energy Limited", "ZEL"]]

today = str(date.today().isoformat())
directory = "C:\\Users\\Etc...\\SharePrices\\CSVs\\"  

df_list = list()
for i in NZX:
    code =i[1]
    name =i[0]
    cmpy =  f"{code}.NZ"
    tickerStrings = [cmpy]
    for ticker in tickerStrings:
        data = yf.download(ticker, group_by="Ticker", period='1d')
        data['ticker'] = ticker  
        df_list.append(data)       
df = pd.concat(df_list)
df.to_csv(f"{directory}_{today}.csv")

Block 2

from azure.storage.blob import BlobClient

blob = BlobClient.from_connection_string(conn_str="Myconnectionstring", container_name="container1", blob_name="StevesBlob3.csv")
with open("./output.csv", "rb") as data:
    blob.upload_blob(data)

Can anyone point me in the right direction? Current issues I am struggling with

  1. Do I need to save the file in a temp folder in the Azure function before trying to move it, or can I push it directly to the container
  2. How do I reference the destination folder/container when I save the CSV?

Any guidance would be much appreciated. New to Azure functions.

CodePudding user response:

Example with a generated CSV file

#creates random csv in blob storage
import numpy as np
import pandas as pd
from datetime import datetime

from azure.storage.blob import ContainerClient

#Create dynamic filename
dateTimeObj = datetime.now()
timestampStr = dateTimeObj.strftime("%d%b%Y%H%M%S")
filename =f"{timestampStr}.csv"

df = pd.DataFrame(np.random.randn(5, 3), columns=['Column1','Column2','Colum3'])
df.to_csv(filename, index=False)  

blob = BlobClient.from_connection_string(
    conn_str="DefaultEndpointsProtocol=https;AccountName=storageaccountXXXXXXX;AccountKey=XXXXXXXXXXXXXXXX;EndpointSuffix=core.windows.net",
    container_name="container2",
    blob_name=filename)

with open(filename, "rb") as data:
    blob.upload_blob(data)
  • Related