Home > Blockchain >  Generate Multiple files Table Structure and create table
Generate Multiple files Table Structure and create table

Time:12-12

I'm trying to Generate Multiple files Table Structure to Snowflake via Python

I have list of files in Directory, I want to read data from files create the tables dynamically in snowflake using file names.

below is what I tried so far

# Generate Multiple files Table Structure to Snowflake via Python
import os
from os import path
import pandas as pd
import snowflake.connector
from snowflake.sqlalchemy import URL
from sqlalchemy import create_engine
from snowflake.connector.pandas_tools import write_pandas, pd_writer

dir = r"D:\Datasets\users_dataset"

engine = create_engine(URL(
    account='zr44507.ap-south-1',
    user='python',
    password='P@ssword_123',
    role='ACCOUNTADMIN',
    warehouse='COMPUTE_WH',
    database='DEM0_DB',
    schema='PUBLIC'
))

connection = engine.connect()
connection.execute("USE DATABASE DEMO_DB")
connection.execute("USE SCHEMA PUBLIC")
results = connection.execute('USE DATABASE DEMO_DB').fetchone()
print(results)
# read the files from the directory and split the filename and extension

for file in os.listdir(dir):
    name, extr = path.splitext(file)
    print(name)
    file_path = os.path.join(dir, file)
    print(file_path)
    df = pd.read_csv(file_path, delimiter=',')
    df.to_sql(name, con=engine, index=False)

I'm getting below error

sqlalchemy.exc.ProgrammingError: (snowflake.connector.errors.ProgrammingError) 090105 (22000): Cannot perform CREATE TABLE. This session does not have a current database. Call 'USE DATABASE', or use a qualified name.
[SQL: 
CREATE TABLE desktop (
    "[.ShellClassInfo]" FLOAT
)

]
(Background on this error at: https://sqlalche.me/e/14/f405)

I checked for permission issues on snowflake, and I haven't found any issues.

Can someone please help with this error

CodePudding user response:

At the create_engine the database is called DEM0_DB instead of DEMO_DB:

engine = create_engine(URL(
    ...
    #database='DEM0_DB',
    database='DEMO_DB',
    schema='PUBLIC'
))
  • Related