Wrong variable is loaded with os.getenv(), not the variable in .env-CodePudding

.env

STAGING_BUCKET=our-staging-bucket

common.py (in same directory as .env)

import os
STAGING_BUCKET = os.getenv('STAGING_BUCKET')
print(STAGING_BUCKET)

When we run our airflow project locally, we receive a different value diff-staging-bucket for our STAGING_BUCKET variable. We recently changed this variable in our .env from diff-staging-bucket to our-staging-bucket. It seems like os.getenv('STAGING_BUCKET') is grabbing a globally saved STAGING_BUCKET variable, rather than the one in the .env file, as we can run this code locally in python from our home directory and the old diff-staging-bucket is still returned.

How can we straighten this out so that we receive the correct .env variable here

EDIT I ran env from the command line and yes there is a global STAGING_BUCKET variable that is diff-staging-bucket

CodePudding user response：

os.getenv gets environment variables. Environment variables are usually set, in unix, by shell command such as export or setenv (depending on the shell). Those commands may be in shell init files, such as .bashrc or .cshrc

.env is not one of them. What you put in that file has no specific meaning. It is just a file. It has nothing to do with os.getenv.

I guess that confusion comes from the fact that in docker (but you never mentioned docker, neither in your question nor in any tags. I just surmise that previous experience with docker is the reason why you, wrongly, believe that .env contains environment variable), .env files are used to set the environment variable that are inserted in the generated docker images; so that inside those images, there are such bashrc files that contain those variables. But if you are not in an OS generated by a tool like docker-compose, from config files like .env, there is no reason to expect that os.getenv should know what is in a .env file.

CodePudding user response：

os.getenv or os.environ.get both get environment variables which are usually set, in your kernel.

But if you're interested in reading environment variables from .env file you can use python-dotenv package; It reads What you put in that file as your config:

First, install it via pip:

pip install python-dotenv

and for example, you have foo.py you want to read .env variables in it and its directory structure is like below:

.
├── .env
└── foo.py

So your foo.py module should be like this:

from dotenv import load_dotenv

load_dotenv()  # take environment variables from .env.

# Code of your application, which uses environment variables (e.g. from `os.environ` or
# `os.getenv`) as if they came from the actual environment.

Reference:

The above example was taken from python-dotenv-docs; you can find out more about other use cases of this package in its documentation.