Elastic Search bulk request does not import all data, but shows no error-CodePudding

I use GuzzleHttp to send data via "_bulk" to an Elastic Search index. It is only a small dataset of 850 records. When I transfer the data record by record, I get an error message for 17 records. That's fine for me, so I can fix the errors.

But when I use _bulk, I do not get any error message at all. The 17 incorrect records are just ignored and are missing inside the index. How can I get an error message here? Are there some kind of options that I can use? Any ideas?

The endpoint is:

Here are my main code parts:

$jsonData = "xxxxx"; // the payload for the request
$elasticUrl = "https://xxxx.xx/xxxxx/_doc/_bulk";

$client = new Client([
        "verify" => false, // disable ssl certificate verification
        "timeout" => 600, // maximum timeout for requests
        "http_errors" => false // disable exceptions
]);

$header = ["Content-Type" => "application/json"];

$result = $client->post($elasticUrl,
          [
            "headers" => $header,
            "body" => $jsonData
          ]
);
        
if ($result->getStatusCode() != 200) {
    $ret = "Error ".$result->getStatusCode()." with message: ".$result->getReasonPhrase();
}

CodePudding user response：

A bulk request will always succeed with HTTP 200.

However, in the bulk response, you should see an indication whether each item succeeded or not. If you see errors: true in the response, then you know some of the items could not get indexed and looking into the items array, you'll find the error for the corresponding items.

CodePudding user response：

As @Val pointed out the use of $response->getBody() gives the needed information:

$body      = (string) $result->getBody();
$bodyArray = json_decode($body, true);

if ($bodyArray["errors"]) {
    $retArray = [];
    foreach ($bodyArray["items"] as $key => $item) {
        if (isset($item["create"]["error"])) {
            $retArray[] = $item["create"]["error"]["reason"].": ".json_encode($data[$key]);
        }
    }
    $ret = implode(", ", $retArray);
}

As side note: in $data I keep the data as php array before sending it to Elastic Search.