I wrote a pysaprk program, if it's not remote debugging, program on a remote machine can perform spark2 - submit way,
Now I use VS Code plugin PTVSD for remote debugging, the error is as follows,
My environment is CDH 5.14.4, spark2.4.0
Please, comment,
The Exception has occurred: IndexError
Pop the from an empty deque
The File "/opt/cloudera/parcels/SPARK2-2.4.0. Cloudera2-1. Cdh5.13.3. P0.1041012/lib/SPARK2/python/lib/py4j - 0.10.7 - SRC. Zip/py4j java_gateway. Py", line 929, in _get_connection
The connection=self. Deque. Pop ()
The File "/opt/cloudera/parcels/SPARK2-2.4.0. Cloudera2-1. Cdh5.13.3. P0.1041012/lib/SPARK2/python/lib/py4j - 0.10.7 - SRC. Zip/py4j java_gateway. Py", line 983, in send_command
The connection=self. _get_connection ()
The File "/opt/cloudera/parcels/SPARK2-2.4.0. Cloudera2-1. Cdh5.13.3. P0.1041012/lib/SPARK2/python/lib/py4j - 0.10.7 - SRC. Zip/py4j java_gateway. Py", line 179, in java_import
Answer=gateway_client. Send_command (command)
The File "/opt/cloudera/parcels/SPARK2-2.4.0. Cloudera2-1. Cdh5.13.3. P0.1041012/lib/SPARK2/python/lib/pyspark. Zip/pyspark/java_gateway py", line 124, in launch_gateway
The File "/opt/cloudera/parcels/SPARK2-2.4.0. Cloudera2-1. Cdh5.13.3. P0.1041012/lib/SPARK2/python/lib/pyspark. Zip/pyspark/context. Py", line 307, in _ensure_initialized
SparkContext. _gateway=gateway or launch_gateway (conf)
The File "/opt/cloudera/parcels/SPARK2-2.4.0. Cloudera2-1. Cdh5.13.3. P0.1041012/lib/SPARK2/python/lib/pyspark. Zip/pyspark/context. Py", line 124, in __init__
SparkContext. _ensure_initialized (self, gateway=gateway, the conf=conf)
The File "/home/devXu/project/HydrologicalYear/joinData py", line 124, in stationAndLand
The spark=SparkContext (appName="computeHydrolocialYear")
The File "/home/devXu/project/HydrologicalYear/joinData py", line 349, in & lt; module>
StationAndLand ()