I am trying to find a way to create a custom application id for a pyspark app submitted via YARN on EMR.
Usually YARN application ids are in this format application_1632776696096_0006
Is there a way to modify them somehow to add a prefix or to make them completely custom?
Appreciate your response! Thanks in advance
CodePudding user response:
ApplicationId is a unique identifier generated internally by yarn,there is no way to customize it except modify the source code. You can achieve the purpose of distinguishing apps by specifying appname. E.g:
spark-submit --name