curl not returning JSON data as expected-CodePudding

I have the following code, where i am basically just doing a curl request to get the JSON data back.. the URL to the request should be:

 https://www.instagram.com/reel/CiAbVWmpiWR/?__a=1&__d=dis

so even if i just copy and paste the link above into the browser i can see the JSON data, however when putting it into code and requesting via CURL.. it just returns Oops an error occured.. what am i doing wrong ?

$mediaUrl = 'https://www.instagram.com/reel/CiAbVWmpiWR';
$pos = strpos($mediaUrl, '?');
if ($pos) {
    $mediaUrl = substr($mediaUrl, 0, $pos);
}
$url = rtrim($mediaUrl, '/') . '/?__a=1&__d=dis';

//var_dump($url);

$proxy = '139.99.54.49:10163';
$proxyauth = 'aditya:xcQWzyfX7ybNM8d';

$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_PROXY, $proxy);     // PROXY details with port
curl_setopt($ch, CURLOPT_PROXYUSERPWD, $proxyauth);   // Use if proxy have username and password
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Accept: application/json') );
$data = curl_exec($ch);
if (curl_errno($ch)) {
    $error_msg = curl_error($ch);
    var_dump($error_msg);
}


$json_data = json_decode($data, true);
var_dump($data);

so var_dump data returns

string(25) "Oops, an error occurred.

curl_errno($ch) is always NULL

CodePudding user response：

I think the API returned was in GraphQl Format. Can you please consider the following code and refer More on GrapQL API here

$mediaUrl = 'https://www.instagram.com/reel/CiAbVWmpiWR';
        $pos = strpos($mediaUrl, '?');
        if ($pos) {
            $mediaUrl = substr($mediaUrl, 0, $pos);
        }
        $url = rtrim($mediaUrl, '/') . '/?__a=1&__d=dis';
        $endpoint = $url;
        $query = "query {
  
            }";
        $data = array('query' => $query);
        $data = http_build_query($data);
        $options = array(
            'http' => array(
                'header' => "Content-Type: application/json\r\n".
                    "Content-Length: ".strlen($query)."\r\n".
                    "User-Agent:MyAgent/1.0\r\n",
                'method' => 'GET',
                'content' => $data
            )
        );
        $context = stream_context_create($options);
        $result = file_get_contents($endpoint, false, $context);

        if ($result === FALSE) { /* Handle error */
        }
        print_r($result);

CodePudding user response：

First off you need to urlencode the request query string.

$url = rtrim($mediaUrl, '/') . urlencode('/?__a=1&__d=dis');

When you did the request with your Browser, did it use a request header 'Content-Type: application/json'? Do the Browser request again and look at the request header the Browser used. Make your curl request header identical to the Browser request.

Do you understand that curl has it's own UA?
So all instagram has to do is reject your request because it has the curl UA.

I copied the url into my Browser and this is the request header:

GET /reel/CiAbVWmpiWR/?__a=1&__d=dis HTTP/3
Host: www.instagram.com
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:105.0) Gecko/20100101 Firefox/105.0
Accept: text/html,application/xhtml xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
DNT: 1
Alt-Used: www.instagram.com
Connection: keep-alive
Cookie: csrftoken=e45LEqbwa8sn6oKsnHbrNgj709DZR0kC; mid=YxliAQALAAGsaab0CnbZVWNmGTrU; ig_did=A531D2DF-33D7-47A1-BAAB-55802C1F9CA0; ig_nrcb=1
Upgrade-Insecure-Requests: 1
Sec-Fetch-Dest: document
Sec-Fetch-Mode: navigate
Sec-Fetch-Site: same-origin
TE: trailers

So I would create the request headers like this:

$request = array();

$request[] = 'Host: www.instagram.com';
User-Agent: 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:105.0) Gecko/20100101 Firefox/105.0';
$request[] =' Accept: text/html,application/xhtml xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8';
$request[] = 'Accept-Language: en-US,en;q=0.5';
$request[] = 'Accept-Encoding: gzip, deflate, br';
$request[] = 'DNT: 1';
$request[] = 'Alt-Used: www.instagram.com';
$request[] = 'Connection: keep-alive';
$request[] = Cookie: csrftoken=e45LEqbwa8sn6oKsnHbrNgj709DZR0kC; mid=YxliAQALAAGsaab0CnbZVWNmGTrU; ig_did=A531D2DF-33D7-47A1-BAAB-55802C1F9CA0; ig_nrcb=1';
$request[] = 'Upgrade-Insecure-Requests: 1';
$request[] = 'Sec-Fetch-Dest: document';
$request[] = 'Sec-Fetch-Mode: navigate';
$request[] = 'Sec-Fetch-Site: same-origin';
$request[] = 'TE: trailers';
curl_setopt($ch, CURLOPT_HTTPHEADER, $request);