Socket.io application costing me too much bandwith what could be the problem?-CodePudding

I have a socket.io application running on my aws ec2 instance and today when i opened my billing report I saw:

Bandwidth$44.58 $0.000 per GB - data transfer in per month 6.956 GB $0.00 $0.000 per GB - data transfer out under the monthly global free tier 100.000 GB $0.00 $0.000 per GB - regional data transfer under the monthly global free tier 0.009 GB $0.00 $0.090 per GB - first 10 TB/month data transfer out beyond the global free tier 495.385 GB $44.58

Its saying that i have transfered 495.385 GB of data, which is too much for my for me to run my small experimental website. Now that i think of it could it be because my code is not efficient?

I have 20 .html files that display different exchange rates charts which all of them request data from the socket using this code

const sock = io.connect();
sock.on('USDPrice', price => {
    //console.log(price)
    document.querySelector('#livePrice').innerHTML = `${(price[price.length - 1].value).toFixed(4)} Br`;
    document.querySelector('#date').innerHTML = `${d.getDate()}/${d.getMonth()   1}/${d.getFullYear()} - ${d.getHours()}:${d.getMinutes()}`;

    if(price != null)
        lineSeries.setData(price.sort(function(a, b) {
    return parseFloat(a.time) - parseFloat(b.time);
}));
});
let i = [] 
sock.on('allLastprice', allLastprice => {
    
    document.querySelector('#EUR').innerHTML = `${allLastprice.EUR.toFixed(2)} Br`;
    document.querySelector('#GBP').innerHTML = `${allLastprice.GBP.toFixed(2)} Br`;
    document.querySelector('#CNY').innerHTML = `${allLastprice.CNY.toFixed(2)} Br`;
    document.querySelector('#CAD').innerHTML = `${allLastprice.CAD.toFixed(2)} Br`;
    document.querySelector('#CHF').innerHTML = `${allLastprice.CHF.toFixed(2)} Br`;
    document.querySelector('#SEK').innerHTML = `${allLastprice.SEK.toFixed(2)} Br`;
    document.querySelector('#AED').innerHTML = `${allLastprice.AED.toFixed(2)} Br`;
    document.querySelector('#KWD').innerHTML = `${allLastprice.KWD.toFixed(2)} Br`;
    document.querySelector('#SAR').innerHTML = `${allLastprice.SAR.toFixed(2)} Br`;
    document.querySelector('#TRY').innerHTML = `${allLastprice.TRY.toFixed(2)} Br`;
    document.querySelector('#OMR').innerHTML = `${allLastprice.OMR.toFixed(2)} Br`;
    document.querySelector('#ILS').innerHTML = `${allLastprice.ILS.toFixed(2)} Br`;
    
    document.querySelector('#QAR').innerHTML = `${allLastprice.QAR.toFixed(2)} Br`;
    document.querySelector('#JOD').innerHTML = `${allLastprice.JOD.toFixed(2)} Br`;
    document.querySelector('#BHD').innerHTML = `${allLastprice.BHD.toFixed(2)} Br`;
    document.querySelector('#EGP').innerHTML = `${allLastprice.EGP.toFixed(2)} Br`;
    document.querySelector('#YER').innerHTML = `${allLastprice.YER.toFixed(2)} Br`;
    document.querySelector('#AUD').innerHTML = `${allLastprice.AUD.toFixed(2)} Br`;
    document.querySelector('#INR').innerHTML = `${allLastprice.INR.toFixed(2)} Br`;
    document.querySelector('#PKR').innerHTML = `${allLastprice.PKR.toFixed(2)} Br`;
});

and from the server side i have different socket.emit functions for each currency

setInterval(function () {
  (async function () {
    const d = new Date();
    io.on("connection", (sock) => {});
    let allCurrencyInBlack = await getCurrencyInBlack();
    let usdCurrentBlackPrice = allCurrencyInBlack.USD;
    let usdPrices = prices
      .map((Object) => {
        if (Object.value != undefined || Object.value != null)
          return {
            time: Object.time,
            value: Object.value.USD,
          };
      })
      .filter((notUndefined) => notUndefined !== undefined);

    if (allCurrencyInBlack != null) {
      prices.push({
        time: Math.round(d.getTime() / 1000),
        value: allCurrencyInBlack,
      });

      io.sockets.emit("allLastprice", allCurrencyInBlack);
      io.sockets.emit(
        "USDPrice",
        prices
          .map((Object) => {
            if (Object.value != undefined || Object.value != null)
              return {
                time: Object.time,
                value: Object.value.USD,
              };
          })
          .filter((notUndefined) => notUndefined !== undefined)
          .slice(-1000)
      );
      io.sockets.emit(
        "EURPrice",
        prices
          .map((Object) => {
            if (Object.value != undefined || Object.value != null)
              return {
                time: Object.time,
                value: Object.value.EUR,
              };
          })
          .filter((notUndefined) => notUndefined !== undefined)
          .slice(-1000)
      );
      io.sockets.emit(
        "CHFPrice",
        prices
          .map((Object) => {
            if (Object.value != undefined || Object.value != null)
              return {
                time: Object.time,
                value: Object.value.CHF,
              };
          })
          .filter((notUndefined) => notUndefined !== undefined)
          .slice(-1000)
      );
      io.sockets.emit(
        "CADPrice",
        prices
          .map((Object) => {
            if (Object.value != undefined || Object.value != null)
              return {
                time: Object.time,
                value: Object.value.CAD,
              };
          })
          .filter((notUndefined) => notUndefined !== undefined)
          .slice(-1000)
      );
      io.sockets.emit(
        "AUDPrice",
        prices
          .map((Object) => {
            if (Object.value != undefined || Object.value != null)
              return {
                time: Object.time,
                value: Object.value.AUD,
              };
          })
          .filter((notUndefined) => notUndefined !== undefined)
          .slice(-1000)
      );
      io.sockets.emit(
        "CNYPrice",
        prices
          .map((Object) => {
            if (Object.value != undefined || Object.value != null)
              return {
                time: Object.time,
                value: Object.value.CNY,
              };
          })
          .filter((notUndefined) => notUndefined !== undefined)
          .slice(-1000)
      );
      io.sockets.emit(
        "GBPPrice",
        prices
          .map((Object) => {
            if (Object.value != undefined || Object.value != null)
              return {
                time: Object.time,
                value: Object.value.GBP,
              };
          })
          .filter((notUndefined) => notUndefined !== undefined)
          .slice(-1000)
      );
      io.sockets.emit(
        "SEKPrice",
        prices
          .map((Object) => {
            if (Object.value != undefined || Object.value != null)
              return {
                time: Object.time,
                value: Object.value.SEK,
              };
          })
          .filter((notUndefined) => notUndefined !== undefined)
          .slice(-1000)
      );
      io.sockets.emit(
        "KWDPrice",
        prices
          .map((Object) => {
            if (Object.value != undefined || Object.value != null)
              return {
                time: Object.time,
                value: Object.value.KWD,
              };
          })
          .filter((notUndefined) => notUndefined !== undefined)
          .slice(-1000)
      );
      io.sockets.emit(
        "AEDPrice",
        prices
          .map((Object) => {
            if (Object.value != undefined || Object.value != null)
              return {
                time: Object.time,
                value: Object.value.AED,
              };
          })
          .filter((notUndefined) => notUndefined !== undefined)
          .slice(-1000)
      );
      io.sockets.emit(
        "SARPrice",
        prices
          .map((Object) => {
            if (Object.value != undefined || Object.value != null)
              return {
                time: Object.time,
                value: Object.value.SAR,
              };
          })
          .filter((notUndefined) => notUndefined !== undefined)
          .slice(-1000)
      );
      io.sockets.emit(
        "QARPrice",
        prices
          .map((Object) => {
            if (Object.value != undefined || Object.value != null)
              return {
                time: Object.time,
                value: Object.value.QAR,
              };
          })
          .filter((notUndefined) => notUndefined !== undefined)
          .slice(-1000)
      );
      io.sockets.emit(
        "OMRPrice",
        prices
          .map((Object) => {
            if (Object.value != undefined || Object.value != null)
              return {
                time: Object.time,
                value: Object.value.OMR,
              };
          })
          .filter((notUndefined) => notUndefined !== undefined)
          .slice(-1000)
      );
      io.sockets.emit(
        "JODPrice",
        prices
          .map((Object) => {
            if (Object.value != undefined || Object.value != null)
              return {
                time: Object.time,
                value: Object.value.JOD,
              };
          })
          .filter((notUndefined) => notUndefined !== undefined)
          .slice(-1000)
      );
      io.sockets.emit(
        "BHDPrice",
        prices
          .map((Object) => {
            if (Object.value != undefined || Object.value != null)
              return {
                time: Object.time,
                value: Object.value.BHD,
              };
          })
          .filter((notUndefined) => notUndefined !== undefined)
          .slice(-1000)
      );
      io.sockets.emit(
        "TRYPrice",
        prices
          .map((Object) => {
            if (Object.value != undefined || Object.value != null)
              return {
                time: Object.time,
                value: Object.value.TRY,
              };
          })
          .filter((notUndefined) => notUndefined !== undefined)
          .slice(-1000)
      );
      io.sockets.emit(
        "EGPPrice",
        prices
          .map((Object) => {
            if (Object.value != undefined || Object.value != null)
              return {
                time: Object.time,
                value: Object.value.EGP,
              };
          })
          .filter((notUndefined) => notUndefined !== undefined)
          .slice(-1000)
      );
      io.sockets.emit(
        "YERPrice",
        prices
          .map((Object) => {
            if (Object.value != undefined || Object.value != null)
              return {
                time: Object.time,
                value: Object.value.YER,
              };
          })
          .filter((notUndefined) => notUndefined !== undefined)
          .slice(-1000)
      );
      io.sockets.emit(
        "ILSPrice",
        prices
          .map((Object) => {
            if (Object.value != undefined || Object.value != null)
              return {
                time: Object.time,
                value: Object.value.ILS,
              };
          })
          .filter((notUndefined) => notUndefined !== undefined)
          .slice(-1000)
      );
      io.sockets.emit(
        "INRPrice",
        prices
          .map((Object) => {
            if (Object.value != undefined || Object.value != null)
              return {
                time: Object.time,
                value: Object.value.INR,
              };
          })
          .filter((notUndefined) => notUndefined !== undefined)
          .slice(-1000)
      );
      io.sockets.emit(
        "PKRPrice",
        prices
          .map((Object) => {
            if (Object.value != undefined || Object.value != null)
              return {
                time: Object.time,
                value: Object.value.PKR,
              };
          })
          .filter((notUndefined) => notUndefined !== undefined)
          .slice(-1000)
      );
    }

    if (d.getHours() == 21 && d.getMinutes() == 00) {
      openingPrice = usdCurrentBlackPrice;
      openingTime = Math.round(d.getTime() / 1000);
      //console.log("opening time "   openingTime);
    }

    if (d.getHours() == 20 && d.getMinutes() == 59) {
      closingPrice = usdCurrentBlackPrice;
    }

    let dailyPercentage =
      ((usdCurrentBlackPrice - openingPrice) / openingPrice) * 100;

    if (openingTime != null) {
      //console.log(prices);
      todaysPrices = usdPrices
        .map((Object) => {
          if (Object.time > openingTime) {
            if (Object.value != undefined || Object.value != null)
              return Object.value;
          }
        })
        .filter((notUndefined) => notUndefined !== undefined);
    }

    if (usdCurrentBlackPrice != null) {
      maxPrice = Math.max(...todaysPrices).toFixed(2);
      minPrice = Math.min(...todaysPrices).toFixed(2);
      let range = null;
      if (maxPrice != Infinity || maxPrice != -Infinity) {
        range = { max: maxPrice, min: minPrice };
      }

      io.sockets.emit("dailyPercentage", dailyPercentage);
      io.sockets.emit("openingPrice", openingPrice);
      io.sockets.emit("closingPrice", closingPrice);
      io.sockets.emit("range", range);

      //io.sockets.emit("price", prices.slice(-1000));

      // priceDb.insert({
      //   time: Math.round(d.getTime() / 1000),
      //   value: allCurrencyInBlack,
      // });
    }

    console.log(usdPrices);
  })();
}, 10000);

Could the hight bandwith transferes be caused because i have emitting multiple sockets? if yes is there a way i could make this code more efficient?

im expecting my code to use less data

CodePudding user response：

Could the hight bandwith transferes be caused because i have emitting multiple sockets?

Your server is doing a whole bunch of io.sockets.emit() (at least 22 of them) every 10 seconds for every client that is connected. With a browser just open to your website, that will eat up bandwidth 24 hours a day (actually 190,080 emits per day). And, that's just for one browser window open to your site.

Does it only emit to connected clients? so a good thing to do would be minimize the socket.emmits and the interval?

if yes is there a way i could make this code more efficient?

Yes, it emits to connected clients. I would first consolidate the emits so instead of 22 in a row, you're just doing one combined emit. That should save some overhead.

Then, I'd examine the payloads and see if you can make them smaller. JSON, on it's own (which is what is used for the socket.io data format) can be a little inefficient with data size. Even just shortening object property names makes a difference in JSON.

Then, you'll have to decide if you can lengthen the interval when you send the data. One trick that some people use is to detect when a given client is "active" and when not by either watching for mouse movements, keyboard usage or pages and when the page has gone inactive, you either stop or slow the data interval. That way, you can keep a quick interval when people are actively using the page, but not waste bandwidth when nobody is there. I believe modern browsers will also tell you when a given page is viewable and when not (like when it's not the active, visible tab).

To implement this variable interval, you could put all new socket.io connections into the "active" socket.io room. Then, the client can detect when it's not being used or not visible and send the server a "inactive" message. On the server, you can then remove that client from the "active" room and put them in the "inactive" room.

Meanwhile, your regular interval could emit every time to those in the "active" room with something like io.to("active").emit(...) and then once every 50 intervals (you pick that number how you like), you emit to the "inactive" connections io.to("inactive").emit(...).

When a client detects it has now become active, it sends the "active" message and your server immediately sends them the last data it generated and moves them to the "active" room.

If need be, you can indicate in your UI that the page is "inactive" (and thus doesn't have the absolute latest datea) and offer a "refresh" button to make it become active again, but you might also detect mouse movement on your page and make it active automatically.