I have a python script that takes data from vector positions and directions from csv files and then generate plots using the quiver method from Matplotlib. The columns from the csv files coordinates and directions are extracted using a Pandas dataframe and then plotted using the quiver method from Matplotlib. It works quite nice for small number of files but gets "Killed" when I try to process more than 1000 files. Any ideas of how I could solve this issue will be highly appreciated.

# Import libraries
import os
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd

directory = './'     

for filename in os.listdir(directory):

    if filename.endswith(".csv"):

        print(os.path.join('./', filename))

        df = pd.read_csv(os.path.join(directory, filename))

        # import column of CSV file with x_pos info
        x_pos = df['x'].tolist()
        y_pos = df['y'].tolist()
        x_direct = df['x_dir'].tolist()
        y_direct = df['y_dir'].tolist()
        scale = df['scalar'].tolist()

        # Creating plot
        fig, ax = plt.subplots(figsize = (10, 10))
        ax.quiver(x_pos, y_pos, x_direct, y_direct, angles='xy', scale_units='xy', scale=1)

        plt.savefig(os.path.splitext(os.path.join(directory, filename))[0]   '.png')

        continue
    else:
        continue

CodePudding user response：

It's important to close your figure after saving it:

for filename in os.listdir(directory):
    if filename.endswith(".csv"):
        ...

        fig, ax = plt.subplots(...)
        ax.quiver(...)

        plt.savefig(...)

        plt.close(fig)  # HERE: close figure

Tip to parse directory:

import pathlib

directory = '.'

for filename in pathlib.Path(directory).glob('*.csv'):
    print(filename)
    df = pd.read_csv(filename)
    ...
    plt.savefig(filename.parent / f"{filename.stem}.png")