I am able to successfully log into a website using Firefox but having trouble replicating that log in using node-fetch. There are three stages to the log-in process:

1. go to /login and website responds with sessionToken (this part is working OK)
2. enter email and password alongside sessionToken and website responds with sessionToken and authToken (this is the part I am having trouble with)
3. request with sessionToken and authToken to /portal and website responds with the HTML I am trying to access

in Firefox Dev Tools, I can see both the working headers and working request body. When I click on "Request" in Dev Tools, I see a 'Form data' heading with the proper values of 'email' and 'password'. When I click on "Headers" in Dev Tools, here is the successful Firefox request:

POST /login/action HTTP/2
Host: www.website.com
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:93.0) Gecko/20100101 Firefox/93.0
Accept: text/html,application/xhtml xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
Content-Type: application/x-www-form-urlencoded
Content-Length: 46
Origin: https://www.website.com
Connection: keep-alive
Referer: https://www.website.com/login
Cookie: _sessiontoken=sessionTokenHere
Upgrade-Insecure-Requests: 1
Sec-Fetch-Dest: document
Sec-Fetch-Mode: navigate
Sec-Fetch-Site: same-origin
Sec-Fetch-User: ?1
TE: trailers

On Firefox, that returns a 302 (redirect) response with these headers:

HTTP/2 302 Found
date: Sun, 31 Oct 2021 17:25:03 GMT
content-type: text/html; charset=utf-8
cache-control: no-cache
location: https://www.website.com/portal
x-runtime: 9
set-cookie: auth_token=authtokenHere; path=/
set-cookie: 
set-cookie: _sessiontoken=sessionTokenHere; path=/; HttpOnly
cf-cache-status: DYNAMIC
[omitted for brevity: expect-ct, report-to, nel, server, cf-ray, alt-svc, X-Firefox-Spdy]

Now, here are the POST options I am trying to use with node-fetch in node.js:

{
method: 'POST', headers: {
Host: 'www.website.com'
,'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:93.0) Gecko/20100101 Firefox/93.0'
,Accept: 'text/html,application/xhtml xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8',
'Accept-Language': 'en-US,en;q=0.5',
'Accept-Encoding': 'gzip, deflate, br',
'Content-Length': 46,
Origin: 'https://www.website.com',
Connection: 'keep-alive',
Referer: 'https://www.website.com/login',
Cookie: '_sessiontoken=sessionTokenHere'
'Upgrade-Insecure-Requests': '1',
'Sec-Fetch-Dest': 'document',
'Sec-Fetch-Mode': 'navigate',
'Sec-Fetch-Site': 'same-origin',
'Sec-Fetch-User': '?1',
TE: 'trailers',
body: URLSearchParams { 'email' => 'myEmail','password' => 'myPassword'}

However, when using node-fetch with the above options it returns a 200, not a 302. Here are the headers of that response (printed via console.log() within the fetch() function):

 [Object: null prototype] {
date: [ 'Sun, 31 Oct 2021 18:12:01 GMT' ],
'content-type': [ 'text/html; charset=utf-8' ],
'transfer-encoding': [ 'chunked' ],
connection: [ 'keep-alive' ],
vary: [ 'Accept-Encoding' ],
'x-runtime': [ '5' ],
'cache-control': [ 'private, max-age=0, must-revalidate' ],
 'set-cookie': [
'_sessionToken=sessionTokenHere; path=/; HttpOnly'
],
'cf-cache-status': [ 'DYNAMIC' ],
'content-encoding': [ 'br' ],
[omitted for brevity: expect-ct, report-to, nel, server, cf-ray, alt-svc]
}

I am puzzled why it works in Firefox but not with node-fetch. A few notes:

• I suspect I am somehow including the email and password incorrectly, but I can't figure out how
• The working Firefox request includes "X-Firefox-Spdy" in the response header; the nonworking node-fetch request does not.
• I am not including "Content-Type" in my request per suggestion from this post
• The Firefox request seems to be using HTTP/2. Could this be part of the problem?

CodePudding user response：

node-fetch automatically follows redirects. It will automatically go to the next redirected url (Location http header value).

If you want to catch the 301 or 302 call, you can set redirect property to manual in the options like redirect: "manual", this way you manage the redirection yourself

For example the following code will catch the 301 call (it'll do the same for your 302 in theory, implemented here):

const fetch = require("node-fetch");

(async () => {
  const response = await fetch("https://stackoverflow.com/users/2614364", {
    redirect: "manual",
  });
  console.log(response.status);
  console.log(response.headers.get("set-cookie"));
})();

I suppose you want to get the cookies using response.headers.get("set-cookie")