Laravel GET Requests returns Resource not found for nse india website-CodePudding

I use this program to get the json data from https://www.nseindia.com/api/quote-equity?symbol=SBIN My code:

foreach (StockName::all() as $stockName) {
    $data = [];
    $headers = [
        "Host"=> "www.nseindia.com",
        "Referer"=> "https://www.nseindia.com/get-quotes/equity?symbol=SBIN",
        "X-Requested-With"=> "XMLHttpRequest",
        "pragma"=> "no-cache",
        "sec-fetch-dest"=> "empty",
        "sec-fetch-mode"=> "cors",
        "sec-fetch-site"=> "same-origin",
        "User-Agent"=> "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36",
        "Accept"=> "*/*",
        "Accept-Encoding"=> "gzip, deflate, br",
        "Accept-Language"=> "en-GB,en-US;q=0.9,en;q=0.8",
        "Cache-Control"=> "no-cache",
        "Connection"=> "keep-alive",
    ];
    $lastRecord = $stockName->latestMetadata();
    $eachSymbolResponse = Http::withHeaders($headers)->get("https://www.nseindia.com/api/quote-equity?symbol=".$stockName->symbol);

    $symbolData = json_decode($eachSymbolResponse->body());
    dd($eachSymbolResponse->body());

Here eachSymbolResponse->body() returns Resource not found

CodePudding user response：

It seems like you'll need to visit their frontend to obtain some of the cookies. Apparently, their cookies act as some kind of authorization token that the API uses to very weirdly prevent external access.

I cannot pinpoint which cookie exactly they need but it appears to be one of these: ak_bmsc, bm_sv, bm_mi. bm_sv appears to only be set after a second request to either their frontend or APIs, but still cannot be used as the sole authentication cookie. So the most important cookies are ak_bmsc and bm_mi.

So, before you attempt to access their API you'll have to obtain the cookies from their main site and pass them to any subsequent following API calls. This is the only way I managed to get the API requests working consistently.

Following is the code I used to verify my theory. It still needs a ton of work to be useful in any production environment. You might also get away with caching the entire CookieJar returned by getAuthorisationCookies for around two hours (according to the expiration date of the aforementioned cookies). The headers declared in prepareRequestWithHeaders seem to be the bare minimum to get a working request in the first place.

use GuzzleHttp\Cookie\CookieJar;
use Illuminate\Http\Client\PendingRequest;
use Illuminate\Http\Client\Response;

PendingRequest::macro('withCookieJar', function (CookieJar $cookieJar) {
    $this->options['cookies'] = $cookieJar;

    return $this;
});

function prepareRequestWithHeaders(): PendingRequest
{
    return Http::withHeaders([
        'Accept' => '*/*',
        'Accept-Encoding' => 'gzip, deflate, br',
        'Connection' => 'keep-alive',
        'User-Agent' => 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36',
    ]);
}

function getAuthorisationCookies(): CookieJar
{
    $response = prepareRequestWithHeaders()
        ->get('https://www.nseindia.com');

    return $response->cookies();
}

function getQuoteEquityResponse(string $symbol): Response
{
    $authorisationCookies = getAuthorisationCookies();

    return prepareRequestWithHeaders()
        ->withCookieJar($authorisationCookies)
        ->get("https://www.nseindia.com/api/quote-equity?symbol={$symbol}");
}

getQuoteEquityResponse('SBIN')->json();