Python how to open multiple json files in a directory-CodePudding

I am trying to open a directory in which there are multiple json files, to then make a data frame of the data in each of them, I try this:

    for file in os.listdir('Datasets/'):
        json_read = pd.read_json(file)

However, it gives me an error: ValueError: Expected object or value

As I inspect the type of the files, it says they are class str. When opening a single file in the directory with read_json it does work correctly as the file is recognized as json. I am not quite sure why the files are turned into strings nor how to solve it. Do you have any tips?

Thanks in advance!

CodePudding user response：

import os
import pandas as pd
base_dir = '/path/to/dir'

#Get all files in the directory

data_list = []
for file in os.listdir(base_dir):

    #If file is a json, construct it's full path and open it, append all json data to list
    if file.endswith('json'):
        json_path = os.path.join(base_dir, file)
        json_data = pd.read_json(json_path, lines=True)
        data_list.append(json_data)

print(data_list)

CodePudding user response：

You probably need to build a list of DataFrames. You may not be able to process every file in the given directory so try this:

import pandas as pd
from glob import glob
from os.path import join
BASEDIR = 'Datasets'
dataframes = []
for file in glob(join(BASEDIR, '*.json')):
    try:
        dataframes.append(pd.read_json(file))
    except ValueError:
        print(f'Unable to process {file}')
print(f'Successfully constructed {len(dataframes)} dataframes')

CodePudding user response：

import os

import json
#heres some information about get list of file in a folder: <https://www.geeksforgeeks.org/python-list-files-in-a-directory/>
#heres some information about how to open a json file: <https://www.geeksforgeeks.org/read-json-file-using-python/>
path = "./data"
file_list = os.listdir(path) #opens the directory form path and gets name of file

for i in range(len(file_list)): #loop the list with index number
  
  current = open(path "/" file_list[i]) #opens the file in order by current index number
  data = json.load(current) #loads the data form file

  for k in data['01']: # 
      print(k)

output

main.json :
{'name': 'Nava', 'Surname': 'No need for to that'}
data.json :
{'name': 'Nava', 'watchs': 'Anime'}

heres a link to run code online https://replit.com/@Nava10Y/opening-open-multiple-json-files-in-a-directory