Home > Back-end >  Scraping the data of the advent of code in js
Scraping the data of the advent of code in js

Time:12-11

i m trying to scrap the input of the advent of code. For that i use this code:

import got from 'got';


(async () =>{
    const url = 'https://adventofcode.com/2022/day/1/input'
    try {
        const response = await got.get(url);
        console.log(response.body)
        console.log(typeof(response.body))
    } catch (error) {
        console.log(error)
    }
})();

But i got the error: HTTPError: Response code 400 (Bad Request)

It work with other url and i see the data in my Web browser. So i don't understand what is the problem...

CodePudding user response:

This site requires a cookie in the headers to send you the data

Adding a cookie solves the issue

import got from 'got';


(async () =>{
    const url = 'https://adventofcode.com/2022/day/1/input'
    try {
        const response = await got.get(url, {
          headers: {
            "cookie": /* your cookie here */
          }
        });
        console.log(response.body)
        console.log(typeof(response.body))
    } catch (error) {
        console.log(error)
    }
})();
  • Related