I have a python script which needs to run every 5 minutes. I am calling this script via a bash file named activate.sh:

#!/bin/bash
../../env/bin/python3 ./run.py

Then to run this task every 5 minutes, I have a crontab job configured as such:

*/5 * * * * cd /var/www/usa/api/BackgroundTasks && /bin/bash ./activate.sh

I have literally the same setup running on 3 different servers 2 of which work just fine and activate the task only once. The one on the USA server however spawns multiple instances of run.py. This went so far that it nearly filled up the entire memory causing performance issues.

Here is the app.py code but I don't think the issue is there, I think the issue is with cron itself:

import time, sys, os, datetime
try:
    sys.path.append("../")

    from Flabstraction.Flabstraction import Pysqlalchemy, MailingService
    from Flabstraction.constants import constants_dict

    from AutomatedReports import schedules_builder_2
    from DeleteOldJobs import delete_old_jobs
    from StallTimers import stop_timers

    constants_selector = "LOCAL"
    path_to_self = os.getcwd()
    selectors = ['/au/','/eu/','/nz/','/usa/','/demo/']
    for sel in selectors:
        if sel in path_to_self:
            constants_selector = sel.upper().replace('/', '')
            break
    constants = constants_dict[constants_selector]

    mail = MailingService()
    sql = Pysqlalchemy('mysql pymysql', constants["mysql_user"], constants["mysql_pw"], constants["server_ip"], constants["mysql_db"])

    file = open('./background.log', 'w')
    file.write(str(datetime.datetime.now())   " : running started. \n")

    if constants_selector in ["LOCAL", "US"]:
        schedules_builder_2(sql, constants["timezone"], constants_selector, mail, constants)    
    if constants_selector in ["AU", "NZ", "EU"]:
        delete_old_jobs(sql, constants["timezone"])

    stop_timers(sql, constants["timezone"], constants["_instance_id"])

    file.write(str(datetime.datetime.now())   " : running stapped. \n")
    file.close()

    except Exception as e:
        file = open("./background.log", "w")
        file.write("Error occured: ", e)
        file.close()
        raise e

Any idea how to resolve this and why it would be happening on only 1 of 3 servers?

CodePudding user response：

Okay so I went through my code in detail and found that one of my constructors was giving a warning message which essentially froze the script while waiting for an input by the user.

The way I resolved this was by using the getcwd() function imported from os in order to check which folder is calling the python script where this happened.

Having:

├── BackgroundTasks
│   ├── AutomatedReports.py
│   ├── DeleteOldJobs.py
│   ├── StallProjects.py
│   ├── StallTimers.py
│   ├── activate.sh
│   ├── background.log
│   └── run.py
├── Config
│   ├── Config.py
│   ├── common.py
│   ├── connect.py
│   ├── constants.py
├── app.py

I was executing run.py from my activate.sh file. When I run the application I call the app.py.

Using the getcwd() function, I was able to check if I call the script from BackgroundTasks and therefore add a case in my connect.py script to ignore the user input when invoked from that folder.