I am trying to run a curl command using the run command in the subprocess module in python. But it is failing. However, the curl command for my elasticsearch search query API is working fine-
curl -k XGET -H "Content-Type: application/json" -H "Authorization: ApiKey Q2txZ2FvSUJ6Nlcwa3pjbnh0NUM6enBidjdNLTdRQVNhc0tuTEk5UEZmZw==" 'https://internal-a02c1b85dade940ef871b0b6d1e65191-1270244841.us-west-2.elb.amazonaws.com:9200/_search?index=.fleet-enrollment-api-keys-7&size=20&pretty' -d '{"query": {"match": {"_id": "c8a537a9-7f98-489b-8f8b-de2450c67ce6"}},"fields" : ["_id", "_index"]}'
This is the code that I am trying to run-
import json
from builtins import int
from pydantic import BaseModel, Field
from typing import List, Dict
from subprocess import PIPE, run
def elasticsearch_search_query(host: str,
port: int,
api_key: str,
path: str,
query: dict,
index: str,
size: int,
fields: List):
es_path = host ":" str(port) path "?index=" index "&" str(size) "&pretty"
es_header = "Authorization: ApiKey" " " api_key
es_dict = {"query": query, "fields": fields}
es_json = json.dumps(es_dict)
es_replaced = " " "'" es_json.replace("'", "\"") "'"
result = run(["curl", "-k", "-XGET", "-H", "Content-Type: application/json", "-H",
es_header,
es_path,
"-d",
es_replaced]
# ,stdout=PIPE, stderr=PIPE, universal_newlines=True, shell=False
)
print(es_path, es_header, es_replaced)
print(result.stdout)
elasticsearch_search_query(host="https://internal-a02c1b85dade940ef871b0b6d1e65191-1270244841.us-west-2.elb.amazonaws.com:9200/_search", api_key="Q2txZ2FvSUJ6Nlcwa3pjbnh0NUM6enBidjdNLTdRQVNhc0tuTEk5UEZmZw==",
port=9200, size=5, query={"match": {"_id": "c8a537a9-7f98-489b-8f8b-de2450c67ce6"}}, fields=["_id", "_index"], path="/_search",index=".fleet-enrollment-api-keys-7")
Here is the error-
{
"error" : {
"root_cause" : [
{
"type" : "illegal_argument_exception",
"reason" : "request [/_search:9200/_search] contains unrecognized parameter: [5]"
}
],
"type" : "illegal_argument_exception",
"reason" : "request [/_search:9200/_search] contains unrecognized parameter: [5]"
},
"status" : 400
}
https://internal-a02c1b85dade940ef871b0b6d1e65191-1270244841.us-west-2.elb.amazonaws.com:9200/_search:9200/_search?index=.fleet-enrollment-api-keys-7&5&pretty Authorization: ApiKey Q2txZ2FvSUJ6Nlcwa3pjbnh0NUM6enBidjdNLTdRQVNhc0tuTEk5UEZmZw== '{"query": {"match": {"_id": "c8a537a9-7f98-489b-8f8b-de2450c67ce6"}}, "fields": ["_id", "_index"]}'
None
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 350 100 350 0 0 10294 0 --:--:-- --:--:-- --:--:-- 10294
curl: (3) nested brace in URL position 12:
'{"query": {"match": {"_id": "c8a537a9-7f98-489b-8f8b-de2450c67ce6"}}, "fields": ["_id", "_index"]}'
UPDATE
Output of repr(result):
CompletedProcess(args=['curl', '-k', '-XGET', '-H', 'Content-Type: application/json', '-H', 'Authorization: ApiKey Q2txZ2FvSUJ6Nlcwa3pjbnh0NUM6enBidjdNLTdRQVNhc0tuTEk5UEZmZw==', 'https://internal-a02c1b85dade940ef871b0b6d1e65191-1270244841.us-west-2.elb.amazonaws.com:9200:9200/_search?index=.fleet-enrollment-api-keys-7&size=5&pretty', '-d', '{"query": {"match": {"_id": "c8a537a9-7f98-489b-8f8b-de2450c67ce6"}}, "fields": ["_id", "_index"]}'], returncode=3)
CodePudding user response:
Since the error was
contains unrecognized parameter: [5]
The problem is not in the query but in the way you construct es_path
. Instead of
&" str(size)
it should be
&size=" str(size)
UPDATE:
Also from what I see your URL now looks like this
https://internal-a02c1b85dade940ef871b0b6d1e65191-1270244841.us-west-2.elb.amazonaws.com:9200:9200/_search?index=.fleet-enrollment-api-keys-7&size=5&pretty
and it's wrong for the following reasons:
- the port is specified twice
- the index is not at the right location
First, host
should just be this (i.e. without port and path)
host="https://internal-a02c1b85dade940ef871b0b6d1e65191-1270244841.us-west-2.elb.amazonaws.com"
Then you can build the URL like this instead:
es_path = host ":" str(port) "/" index path "?size=" str(size) "&pretty"
i.e. it should look like this instead:
https://internal-a02c1b85dade940ef871b0b6d1e65191-1270244841.us-west-2.elb.amazonaws.com:9200/.fleet-enrollment-api-keys-7/_search?size=5&pretty