How to pipe from standard input to an http request in NodeJS?-CodePudding

I have an http server listening on port 9090 - piping the request to stdout like so:

let server  = http.createServer((req, res) => {req.pipe(process.stdout)})
server.listen(9090)

When I send it something with curl like so:

curl -XGET -T - 'http://localhost:9090' < /tmp/text-input

it works, and I see the output on the server's terminal

but when I try the following in node:

const http = require('http')
const nurl = new URL("http://localhost:9090")
let request = http.request(nurl)

request.on('response', (res) => {
  process.stdin.pipe(request)
})

request.end() // If I emit this, nothing happens. If I keep this, I get the below error

and try to run it like so: node request.js < /tmp/text-input, I'm getting the following error:

node:events:368
      throw er; // Unhandled 'error' event
      ^

Error [ERR_STREAM_WRITE_AFTER_END]: write after end
    at new NodeError (node:internal/errors:371:5)
    at write_ (node:_http_outgoing:748:11)
    at ClientRequest.write (node:_http_outgoing:707:15)
    at ClientRequest.<anonymous> (/home/tomk/workspace/js-playground/http.js:17:7)
    at ClientRequest.emit (node:events:390:28)
    at HTTPParser.parserOnIncomingClient (node:_http_client:623:27)
    at HTTPParser.parserOnHeadersComplete (node:_http_common:128:17)
    at Socket.socketOnData (node:_http_client:487:22)
    at Socket.emit (node:events:390:28)
    at addChunk (node:internal/streams/readable:324:12)
Emitted 'error' event on ClientRequest instance at:
    at emitErrorNt (node:_http_outgoing:726:9)
    at processTicksAndRejections (node:internal/process/task_queues:84:21) {
  code: 'ERR_STREAM_WRITE_AFTER_END'
}

I want to pipe my stdin to an http server the same way I can with curl -T -. What is wrong with my request code?

CodePudding user response：

Short answer

To send chunked encoding messages in node, use the POST method:

let request = http.request(nurl, { method: 'POST' })
process.stdin.pipe(request)

Slightly longer (yet partial) answer

I opened a listening netcat (listen on plain tcp) like so nc -l 9090 to view how the request from curl differs from my code and found a few key differences in the headers.

In curl, the header Transfer-Encoding: chunked appeared, but was missing from the request my code sent out. Also, my code had a header Connection: closed

I logged the request object and found that useChunkedEncodingByDefault is set to false, which was confusing given the quote from the nodejs http docs:

Sending a 'Content-Length' header will disable the default chunked encoding.

Implying that it should be the default.

But then I found this in the source of node

  if (method === 'GET' ||
      method === 'HEAD' ||
      method === 'DELETE' ||
      method === 'OPTIONS' ||
      method === 'TRACE' ||
      method === 'CONNECT') {
    this.useChunkedEncodingByDefault = false;
  } else {
    this.useChunkedEncodingByDefault = true;
  }

So, in conclusion, node doesn't allow sending GET requests with chunked encoding, but curl does. Odd, and unfortunately not documented (as far as I could find), but the important thing I got it working