JS sync function to handle async activities-CodePudding

I have a function write2Bytes which appends to a buffer. If its full, it has to write to wire. If I make write2Bytes as async function, caller can do await and it works. But since write2Bytes is mostly sync and it just writes to memory, buf but not on to socket. so making it async just causes promises to be created/deleted which are not needed . With below sync implementation, the second call to write2Bytes is not ensured that the first one has finished without using async and caller doing await.

Please suggest if there is a better way to design this API write2Bytes.. Thanks...

following is the sample


var buf = Buffer.alloc(1024);
var offset = 0;

function write2Bytes(val) {
    if (buf.length - offset   1 > 2) { // buf is full 
        // write buf data to socket
        socket.write(buf, (err) => {
            if (err) {
               console.log("failure");
            }   
            console.log("success");
        }); 
    }
    buf[offset  ] = val 
    buf[offset  ] = val   1;
}

write2Bytes(4);
write2Bytes(4);

above is the code I was writing and not able to make it sync and still retain performance.

CodePudding user response：

You can use async await for making the socket.write function work like synchronous operation.

async write2Bytes(val) {

if (buf.length - offset   1 > 2) { // buf is full 
   // write buf data to socket
        try{
           await socket.write(buf)
        }catch(e){
           consol.log(e);
        }
}

 buf[offset  ] = val 
 buf[offset  ] = val   1;

}

For more information check this link

CodePudding user response：

I don't know what exactly your socket does, but since I'd assume that it consumes an entire buffer, I'd go with 2 buffer in tandem. That way you can "overflow" the first buffer without losing data.

This also means that, since you can write past the buffer size, you don't need to check before writing if there's enough space (to a reasonable degree) and instead just write some data and only occasionally check if the buffer is full and needs to be flushed into the socket.

pro - this keeps the consuming API simple, because it doesn't have to plan ahead (too much); like checking if there's enough space (if not, await the socket, ...) before every single writing step.

contra - the higher memory consumption, or smaller chunk-size that you send in one go through the socket.

// keep the buffer size at a power of 2, 
// then the math stays trivial and fast.
const BUFFER_SIZE = 1024, 
  SHIFT = Math.log2(BUFFER_SIZE), 
  MASK = BUFFER_SIZE - 1;
console.assert(BUFFER_SIZE & MASK === 0, "BUFFER_SIZE needs to be a power of 2");

const buffers = [ Buffer.alloc(BUFFER_SIZE), Buffer.alloc(BUFFER_SIZE) ];
let offset = 0;

const empty = () => !offset;
const isFull = () => offset > MASK;

async function flushBuffer() {
  const buf = buffers.shift();

  // fill the buffer with zeroes untill the end ...
  // if (offset < BUFFER_SIZE) {
  //   buf.fill(0, offset);
  // }

  offset = Math.max(0, offset - BUFFER_SIZE);

  // that's up to you to make this async
  await socket.write(buf);

  // ... or empty the buffer before reuse
  // buffer.fill(0);

  // reuse the buffer
  buffers.push(buf);
}

function writeByte(val) {
  // automagically writes into the second buffer when the offset >= BUFFER_SIZE
  buffers[offset >> SHIFT][MASK & offset  ] = val;
}

function write2Bytes(val) {
  // just guessing
  writeByte(val & 0xFF);
  writeByte(val >> 8 & 0xFF);
}

then you can do something like

write2Bytes(1);
write2Bytes(2);
// write some more ...

// every now and then
if (isFull()) {
  await flushBuffer();
}
// and write some more ...



// and at the end
if (!empty()) {
  // flush the rest of the data
  await flushBuffer();
}

or like in a loop

while (youHaveData) {
  while (!isFull()) {
    write2Bytes(1);
    write2Bytes(2);
    // write some more ...
  }

  await flushBuffer();
}

if (!empty()) {
  await flushBuffer();
}

Do you understand where I'm going with this?