How to deal with rate limited API services using React/Node.js? (ReferenceError)-CodePudding

Hi I'm new to React and NodeJS and currently I am trying to work my way around pulling from a rate limited API service. My idea is to have NodeJS pull data from the API service every 30 minutes or so and send that to my express "/youtube" endpoint for my frontend to be able to infinitely pull from. However I don't know if this is the best practice and I am also running into an ReferenceError with my code. Any help would be greatly appreciated.

I'm able to console.log(data) so I know the variable is defined but I am unsure on how to pass it to app.get() properly.

ERROR

ReferenceError: data is not defined

CODE

const express = require('express')
const axios = require('axios')
const app = express()


const url = "**Youtube API URL**"

axios.get(url)
        .then((response) => {
            let data = response.data.items
            console.log("Posted data")
            console.log(data)
        })
        .catch(
            err => console.log(err)
        )

app.get("/youtube", (req,res) => {
    res.send(data)
})

app.listen(5000, () => {console.log("server started on port 5000")})

CodePudding user response：

Data is undefined in your route because it is defined within the callback of your axios get callback. Even if it were to be defined prior to the axios call, it would still be undefined because the api call is an async operation...

A simple solution would be to make the axios call within your route callback, and send the response within the axios callback. Although there are tons of coding techniques/patterns to solve this issue, this helps illustrate your problem:

app.get("/youtube", (req,res) => {

    axios.get(url)
    .then((response) => {
        let data = response.data.items
        console.log("Posted data")
        console.log(data)
        res.send(data)
    })
    .catch(
        err => console.log(err)
    )
    
})

Of course, you could create an async function and call it from within the route:

async function makeCall() {
    const data = await axios.get(url)
    .then((response) => {
        let resdata = response.data.items
        return resdata
    })
    .catch(
        err => console.log(err)
    )
    return data;
}

Then in your route:

 app.get("/youtube", async (req,res) => {
     const data = await makeCall();
     res.send(data);
 }

Caching the data from the response to be made readily available is another issue altogether...

CodePudding user response：

Thank you everyone for your insight. I've ended up taking the Caching approach to be able to deal with API rate limit of the service that I'm pulling from. I make a call from the API in the beginning of my server.js file and added a setInterval function to pull from the API every 30 minutes to update the data and then store it in a cache. I then pull that data from the cache and send it to my "/youtube" express endpoint.

Here's my working code in hope that it helps anyone else that runs across the same issues as I have.

server.js

const axios = require('axios')
const NodeCache = require('node-cache')
const express = require('express')
const cors = require('cors')
const app = express()
app.use(cors())

const port = process.env.PORT || 4000
const url = " **Youtube API url** "
const myCache = new NodeCache();

function initialCall(){
    axios.get(url)
        .then((response) => {
            const result = response.data.items
            myCache.set("key", result)
            console.log("Retrived from API")
        })
        .catch(
            err => console.log(err)
        )
}

initialCall()
setInterval(initialCall, 1800000)

app.get("/", (req,res) => {
    const welcome = "Welcome"
    res.send(welcome)
})

app.get("/youtube", (req,res) => {
    if(myCache.get("key")){
        res.send(myCache.get("key"))
        console.log("Retrieved from cache")
    }
})

app.listen(port, () => {console.log(`server started on port: ${port} `)})