Home > Back-end >  Programmatically running notebook with LIKE operator
Programmatically running notebook with LIKE operator

Time:04-09

In Databricks repos, say I want to programmatically run a notebook titled 'ExampleNotebook_45', but I want only use the part of the title 'ExampleNotebook', how do I write this in the command:

dbutils.notebook.run()

Similar language that comes to mind is the LIKE operator in sql where I can write along the lines of:

select * from table where name LIKE 'ar%'

CodePudding user response:

After doing some research, this is the answer that solved the problem, getting the list of files in the folder via the workspace API

import requests
import json

ctx = json.loads(dbutils.entry_point.getDbutils().notebook().getContext.toJson()) 
my_pat = ctx['extraContext']['api_token'] 
workspace_url = 'https://'

def list_files(base_path: str):
  lst = requests.request(method='get',
    url=f"{workspace_url}/api/2.0/workspace/list",
    headers={"Authentication": f"Bearer {my_pat}"}, 
    json={"path": base_path}).json()["objects"]
  results = []
  for i in lst:
    if i["object_type"] == "DIRECTORY" or i["object_type"] == "REPO":
      results.extend(list_files(i["path"]))
    else:
      results.append(i["path"])
  
  return results
  
all_files = list_files("base_path")

for i in all_files:
  if 'ExampleNotebook' in i:
    file_name = i
  else:
    continue

This will give me the file name (and path), that is similar to the LIKE sql operator.

  • Related