Home > Back-end >  How to set yarn application id in a PySpark app
How to set yarn application id in a PySpark app

Time:09-29

I am trying to find a way to create a custom application id for a pyspark app submitted via YARN on EMR. Usually YARN application ids are in this format application_1632776696096_0006

Is there a way to modify them somehow to add a prefix or to make them completely custom?

Appreciate your response! Thanks in advance

CodePudding user response:

ApplicationId is a unique identifier generated internally by yarn,there is no way to customize it except modify the source code. You can achieve the purpose of distinguishing apps by specifying appname. E.g:

spark-submit --name
  • Related