Situation:

I have a function which runs at the start of my code load_content:

async function load_content() {
    console.log("I GOT HERE 1");
    await load_js_files("./cmds/","commands")
    console.log("I GOT HERE 2");
    await load_js_files("./events/","events");
}

This function calls load_js_files twice, load_js_files is a recursive function which calls itself for each directory in the specified directory, 'requiring' each file found and doing different things if type = commands or type = events.

The function load_js_files looks like:

function load_js_files(dir,type){
    fs.readdir(dir, (e, files) => {
        if(e) console.error(e);

        let jsfiles = files.filter(f => f.split(".").pop() === "js");
        if(jsfiles.length <= 0){
            console.log(`No commands to load from ${dir}!`);
            return;
        }
        for(const file of files){
            if(fs.lstatSync(dir file).isDirectory()){
                load_js_files(dir file "/",type)
            }
        }
        if(type === "commands"){
            console.log("\x1b[4m%s\x1b[0m",`Loading ${jsfiles.length} commands from ${dir} ...`);
            jsfiles.forEach((f,i) => {
                let command = require(`${dir}${f}`);
                console.log(`${i   1}: ${f} loaded!`);
                bot.commands.set(command.info.name, command);
            });
        } else if (type === "events"){
            console.log("\x1b[4m%s\x1b[0m",`Loading ${jsfiles.length} events from ${dir} ...`);
            jsfiles.forEach((f,i) => {
                let event = require(`${dir}${f}`);
                console.log(`${i   1}: ${f} loaded!`);
                let commands = [];
                for(const cmd of bot.commands){
                    if(cmd[1].data) commands.push(cmd[1].data.toJSON());
                }
                if(event.once){
                    bot.once(event.name, (...args) => event.execute(...args, commands));
                } else {
                    bot.on(event.name, (...args) => event.execute(...args, commands));
                }
            });
        } else {
            console.log(log_Red,"FUNCTION 'load_js_files' CALLED WITH INCORRECT 'type'.")
        }
    });
    return new Promise((resolve,reject) => resolve("DONE"));
}

I would expect in load_content that the events occur in this order:

1. console logs I GOT HERE 1

2. load_js_files occurs with commands parameter (granted I haven't solved recursion promises yet it should run once at least)

3. console logs I GOT HERE 2

4. load_js_files occurs again but with events parameter.

Issue:

Upon running load_js_files requiring type = event the variable (bot.commands) is undefined. bot.commands is assigned values based in step 2 above, during the load_js_files call.

From what I can debug to, the initial function load_content does not respect (my understanding) of async/await, so I assume I am doing something incorrectly with promises.

In my console however the two console.log statements execute immidiatley & before the function is finished:

I GOT HERE 1
I GOT HERE 2
Loading 3 commands from ./cmds/ ...
1: createtables.js loaded!
2: ping.js loaded!
3: sqltest.js loaded!
Loading 1 events from ./events/ ...
1: ready.js loaded!
Loading 1 commands from ./cmds/settings/ ...
1: set.js loaded!

What I've tried:

I've tried the code noted above, additionally I have tried wrapping the second run of load_js_files in a .then(), I've tried a callback function & I've also tried nesting Promises but run into issues as load_js_files is calling itself recursively.

I'm having a hard time understanding if these Promises are going to work with this type of recursion (all recursions of load_js_files must finish before the second load_js_files is called within load_content).

Bonus points:

Bonus points if you can help me understand promises within a recursive function. I've read

• https://blog.scottlogic.com/2017/09/14/asynchronous-recursion.html
• https://www.bennadel.com/blog/3201-exploring-recursive-promises-in-javascript.htm and
• https://medium.com/@wrj111/recursive-promises-in-nodejs-769d0e4c0cf9

But it's not quite getting through.

Attempt at implementing David's answer:

This results in error, I believe related to fs.readdir(dir) requiring a callback.

Error: TypeError [ERR_INVALID_CALLBACK]: Callback must be a function. Received undefined

async function load_js_files_async_test(dir,type){
    const files = fs.readdir(dir);
    for (const file of files) {
        const file_info = await lstat(dir   file);
        if(file_info.isDirectory()){
            await load_jsfiles_async_test(dir   file   "/", type);
        } else {
            console.log("Thanks David!");
        }
    }
}

CodePudding user response：

does not respect/wait for promise

Sure it does. This is the Promise it's awaiting:

new Promise((resolve,reject) => resolve("DONE"))

That Promise of course finishes very quickly (and synchronously) and then the code moves on to the next task. But what isn't being awaited anywhere in the code is this asynchronous operation:

fs.readdir(dir, (e, files) => {
  //...
});

This call to readdir is an asynchronous operation, and as such that callback function won't be invoked until after the current thread finishes everything it's doing. Which includes the "promises" being awaited (which don't do anything asynchronous) as well as the console.log statements and the next invocation of load_js_files.

Fortunately, Node provides Promise-based versions of these operations as well.

Simplifying the original code a bit, imagine instead this structure:

async function load_js_files(dir, type) {
  const files = await readdir(dir);
  for (const file of files) {
    const fileInfo = await lstat(dir   file);
    if(fileInfo.isDirectory()) {
      await load_js_files(dir   file   "/", type)
    }
  }
  // etc.
}

As you can see, this "reads" a lot more like synchronous operations. The idea here is to remove the usage of callback functions, which are essentially causing you confusion and making "awaiting" much more difficult. Now all of the logic in the load_js_files function is directly in the load_js_files function, not in other anonymous callback functions. And that logic proceeds, step by step, awaiting each asynchronous operation.

Then you can await the calls to load_js_files as expected.

CodePudding user response：

The function

fs.readdir

Is a non blocking function which means that the code that you inserted there is not being 'awaited' to be done, you could try with fs.readdirSync and remove your return new Promise().