I'm tasked to identify running queries in our ADX cluster and according to MS docs we should use the x-ms-app header to identify it.
I basically need to fill this info when checking the queries in our cluster
CodePudding user response:
At this point there is no support for populating the Application
field in .show queries
(might change in the future).
An alternative approach would be to send client request properties.
Here is a demo (minus the credentials):
Spark
Install com.microsoft.azure.kusto:kusto-spark_3.0_2.12:3.0.0
from Maven
kustoCluster = "https://***.westeurope.kusto.windows.net/"
kustoDatabase = "mydb"
kustoQuery = "AnotherStormEvents | take 10"
kustoAadAppId = "***"
kustoAadAppSecret = "***"
kustoAadAuthorityID = "***"
crp = sc._jvm.com.microsoft.azure.kusto.data.ClientRequestProperties()
crp.setOption("query_take_max_records", 3) # this actually impacts the returned result set
crp.setOption("MyApplicationName" ,"ThisIsMyApp")
crp.setOption("MyProperty" ,"SomeValue")
crp.setOption("MyOtherProperty" ,123.45)
df = (spark.read
.format("com.microsoft.kusto.spark.datasource")
.option("kustoCluster", kustoCluster)
.option("kustoDatabase", kustoDatabase)
.option("kustoQuery", kustoQuery)
.option("kustoAadAppId", kustoAadAppId)
.option("kustoAadAppSecret", kustoAadAppSecret)
.option("kustoAadAuthorityID", kustoAadAuthorityID)
.option("clientRequestPropertiesJson", crp.toString())
.load()
)
df.display()
ADX
.show queries
| top 1 by StartedOn
| project kv = ClientRequestProperties.Options
| mv-expand kind=array kv
| project k = kv[0], v = kv[1]
k | v |
---|---|
api_version | v2 |
request_readonly | true |
servertimeout | 2400000000 |
servertimeoutorigin | Gateway |
query_datascope | 1 |
query_fanout_nodes_percent | 100 |
query_fanout_threads_percent | 100 |
maxmemoryconsumptionperiterator | 5368709120 |
max_memory_consumption_per_query_per_node | 8589699072 |
truncationmaxsize | 67108864 |
truncationmaxrecords | 500000 |
MyProperty | SomeValue |
query_take_max_records | 3 |
MyOtherProperty | 123.45 |
MyApplicationName | ThisIsMyApp |