Read N last lines of a large shared file with NodeJs-CodePudding

I have two codes one for reading shared file using smb2:

// load the library
var SMB2 = require('smb2');
 
// create an SMB2 instance
var smb2Client = new SMB2({
  share:'\\\\192.168.0.111\\folder'
, domain:'WORKGROUP'
, username:'username'
, password:'password'
});

// read the file
smb2Client.readFile('path\\to\\the\\file.txt', "utf-8", function(error, data){
    if(error) throw error;    
    console.log(data);
});

And the other one for reading the last 20 lines of a local file using read-last-line:

// load the library
const readLastLine = require('read-last-line');

// read the file
readLastLine.read('path\\to\\the\\file.txt', 20).then(function (lines) {
    console.log(lines)
}).catch(function (err) {
    console.log(err.message);
});

I didn't know how to combine the two of them. Do you have any suggestions. Thanks.

CodePudding user response：

If smb2 and read-last-line both supported streams, it would be easy. According to their documentation at least, neither does, but pv-node-smb2 has a method createReadStream.

Here is a stream transformer that processes its input line by line, but keeps only the most recent n lines and outputs them at the end. This avoids keeping huge files entirely in memory:

class Tail extends stream.Transform {
  constructor(n) {
    super();
    this.input = new stream.PassThrough();
    this.tail = [];
    readline.createInterface({input: this.input, crlfDelay: Infinity})
    .on("line", function(line) {
      if (this.tail.length === n) this.tail.shift();
      this.tail.push(line);
    }.bind(this));
  }
  _transform(chunk, encoding, callback) {
    this.input.write(chunk);
    callback();
  }
  _flush(callback) {
    callback(null, this.tail.join("\n"));
  }
}

(Likely, there is already an NPM package for that.)

Now you can pipe the SMB2 read stream through that transformer:

new require("pv-node-smb2").PvNodeSmb2(...).createReadStream(...)
.pipe(new Tail(20))
.pipe(process.stdout);

CodePudding user response：

I couldn't install pv-node-smb2 package, it gave me too much errors so I tried a Powershell command that works like the tail command in Linux

Get-Content \\192.168.0.111\path\to\the\file.txt -Tail 20

Which returns the last 20 lines of the file and then run that script in NodeJs

let last20lines = "";
let spawn = require("child_process").spawn,child;
child = spawn("powershell.exe",["c:\\wamp64\\www\\powershell_script.ps1"]);
child.stdout.on("data",function(data){
   last20lines  = data
});
child.stdin.end();