How can I run `catalog.load` in a non-IPython context?-CodePudding

In IPython I can run data = catalog.load('my_dataset') in order to load a dataset specified as 'my_dataset' in the catalog.yml file. What's the equivalent command in a pthon script? What do I need to import?

CodePudding user response：

When using kedro you don't need to load the data explicitly, just make sure the name of the input used in your pipeline definition matches the one you are using in catalog.yml

node(
    func=create_model_input_table,
    inputs=["my_dataset"],
    outputs="model_input_table",
    name="create_model_input_table_node",
),

You can follow tutorial in official documentation -> create pipelines

CodePudding user response：

So you can see how to do it here: https://kedro.readthedocs.io/en/stable/get_started/hello_kedro.html

It's also important to only use this if you're not going to violate some of Kedro's core assumptions (i.e. that nodes are functionally pure and have no idea of IO). 99% of the time, the right way to extend the life cycle of a run is via hooks.