In Databricks repos, say I want to programmatically run a notebook titled 'ExampleNotebook_45', but I want only use the part of the title 'ExampleNotebook', how do I write this in the command:
dbutils.notebook.run()
Similar language that comes to mind is the LIKE operator in sql where I can write along the lines of:
select * from table where name LIKE 'ar%'
CodePudding user response:
After doing some research, this is the answer that solved the problem, getting the list of files in the folder via the workspace API
import requests
import json
ctx = json.loads(dbutils.entry_point.getDbutils().notebook().getContext.toJson())
my_pat = ctx['extraContext']['api_token']
workspace_url = 'https://'
def list_files(base_path: str):
lst = requests.request(method='get',
url=f"{workspace_url}/api/2.0/workspace/list",
headers={"Authentication": f"Bearer {my_pat}"},
json={"path": base_path}).json()["objects"]
results = []
for i in lst:
if i["object_type"] == "DIRECTORY" or i["object_type"] == "REPO":
results.extend(list_files(i["path"]))
else:
results.append(i["path"])
return results
all_files = list_files("base_path")
for i in all_files:
if 'ExampleNotebook' in i:
file_name = i
else:
continue
This will give me the file name (and path), that is similar to the LIKE sql operator.