I am getting this error while trying to write txt file to local path in windows.
Error: Caused by: java.lang.UnsatisfiedLinkError: org.apache.hadoop.io.nativeio.NativeIO$Windows.access0(Ljava/lang/String;I)Z
- Spark , hadoop versions : spark-3.0.3-bin-hadoop2.7.
- winutils is placed in C:\winutils\bin
- hadoop.dll is placed in C:\winutils\bin and c:\System32
- Environment Variables set HADOOP_HOME C:\winutils Path %HADOOP_HOME%\bin
- Tried restarting
CodePudding user response:
I found below cause and solution:
Root Cause: Gradle dependency was with higher version of spark. I have spark 3.0.3 installed but here it was 3.2.0 implementation 'org.apache.spark:spark-core_2.13:3.2.0'
Fix: Replaced with implementation 'org.apache.spark:spark-core_2.12:3.0.3'