I'm working on creating a Python/PySpark library using VS Code. When I debug, I face the ModuleNotFoundError. Despite of that, I'm able to generate a .whl file, load it in Databricks and import it using from dqengine import *

Why VS Code throws this error?

CodePudding user response：

Because your module dqengine is not in the top level folder, it is probably not in your PYTHONPATH variable, which VSCode has probably added the path to DATA QUALITY ENGINE

Either:

• Move it to the top level folder (Data quality engine)
• add the path to check_engine_lib to PYTHONPATH.
• Or as @franjefriten says, add an __init__ to check-engine-lib and do

from check-engine-lib.dqengine.validate_df import *

CodePudding user response：

From what I see, you are working in ./DATA-QUALITY-ENGINE/check-engine-lib/dqengine/validate_df. Basically you have to import it the following way.

from check-engine-lib.dqengine.validate_df import *

That should work
Also you need to create a __init__.py file to import other files as modules