Hadoop lib has been added under the corresponding version of the winutils
CMD input D: \ peizhi \ hadoop \ bin \ winutils exe chmod 777/TMP/hive
Can't successfully started the spark - shell




C: \ Windows \ system32 & gt; The spark - shell
Log4j: WARN No appenders could be found for logger (. Org. Apache hadoop. Metrics2. Li
B.M utableMetricsFactory).
Log4j: WARN both Please initialize the log4j system properly.
Log4j: WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more in
Xsl-fo.
Using the Spark 's repl log4j profile: org/apache/Spark/log4j - defaults - repl. Propertie
S
To adjust logging level use sc. SetLogLevel (" INFO ")
Welcome to
____ __
/__/__ _____ _____//__
/_ \ \ _ \ _ `/__/__/
'/___/__/\ _, _/_/_ \ _ \ version 1.6.2
/_

Using the Scala version 2.10.5 (Java HotSpot (TM), 64 - Bit VM Server, Java 1.8.0 comes with _111)
Type in expressions and them evaluated.
Type: help for more information.
The Spark context available as sc.
17/01/16 23:55:31 WARN General: the Plugin (Bundle) "org. Datanucleus. API. Jdo is alr
Eady registered. Ensure you dont have multiple JAR versions of the same plugin I
N the classpath. The URL "file:/D:/peizhi/spark - 1.6.2 - bin - hadoop2.6/bin/../lib/d
Atanucleus - API - jdo - 3.2.6. Jar "is already registered, and you are trying to Regis
Ter the an identical plugin located at the URL "file:/D:/peizhi/spark - 1.6.2 - bin - hadoop2.
6/lib/datanucleus - API - jdo - 3.2.6. Jar. "
17/01/16 23:55:31 WARN General: the Plugin (Bundle) "org. Datanucleus. Store. An RDBMS is
Already registered. Ensure you dont have multiple JAR versions of the same plug-in
In the in the classpath. The URL "file:/D:/peizhi/spark - 1.6.2 - bin - hadoop2.6/bin/../l
The ib/datanucleus - an RDBMS - 3.2.9. Jar "is already registered, and you are trying to reg
Ister an identical plugin located at the URL "file:/D:/peizhi/spark - 1.6.2 - bin - hadoop
2.6/lib/datanucleus - an RDBMS - 3.2.9. Jar. "
17/01/16 23:55:31 WARN General: the Plugin (Bundle) "org. Datanucleus" is already reg
Istered. Ensure you dont have multiple JAR versions of the same plugin in the cl
Asspath. The URL "file:/D:/peizhi/spark - 1.6.2 - bin - hadoop2.6/bin/../lib/datanucle
Us - core - 3.2.10. Jar "is already registered, and you are trying to register an ide
Ntical plugin located at the URL "file:/D:/peizhi/spark - 1.6.2 - bin - hadoop2.6/lib/data
Nucleus - core - 3.2.10. Jar. "
17/01/16 23:55:31 WARN Connection: BoneCP specified but not present in the CLASSPATH
(or one of dependencies)
Mon, Jan 16 23:55:32 CST 2017 Thread/main, 5, the main Java IO. FileNotFoundException:
The Derby. The log (access denied,)
17/01/16 23:55:32 WARN Connection: BoneCP specified but not present in the CLASSPATH
(or one of dependencies)
-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
Mon, Jan 16 23:55:32 CST 2017:
Booting The Derby version of The Apache Software Foundation - Apache Derby - 10.10.1.1
- (1458268) : the instance - a7fd a816c00e - 0159-9 c748a8 a8e - 000031
On the database directory C: \ Users \ yyy \ AppData \ Local \ Temp \ b97 d10c - d60a spark - 7698-4 - b
834 - f25607bb3b62 \ metastore with class loader sun. Misc. The Launcher $AppClassLoader @ 55
F96302
The Loaded from a file:/D:/peizhi/spark - 1.6.2 - bin - hadoop2.6/lib/spark - assembly - 1.6.2 -h
Adoop2.6.0. Jar
Java. Vendor=Oracle Corporation
Java. Runtime. Version=1.8.0 comes with _111 - b14
User. Dir=C: \ Windows \ system32
OS. Name=Windows 8.1
OS. The arch=amd64
OS. Version=6.3
The Derby. System. Home=null
The Database Class Loader started - the Derby. Database. The classpath='
17/01/16 23:56:00 WARN the ObjectStore: Version information not found in metastore.
Hive. Metastore. Schema. Verification is not enabled so recording the schema versio
N 1.2.0
17/01/16 23:56:00 WARN the ObjectStore: Failed to get the default, returning No
SuchObjectException
Java. Lang. RuntimeException: Java. Lang. NullPointerException
At org, apache hadoop. Hive. Ql. Session. The SessionState. Start (SessionState. Jar
A: 522)
The at org. Apache. Spark. SQL. Hive. Client. ClientWrapper. & lt; init> (ClientWrapper. S
Cala: 204)
The at org. Apache. Spark. SQL. Hive. Client. IsolatedClientLoader. CreateClient (Is
OlatedClientLoader. Scala: 238)
The at org. Apache. Spark. SQL. Hive. HiveContext. ExecutionHive $lzycompute (HiveCo
Ntext. Scala: 218)
The at org. Apache. Spark. SQL. Hive. HiveContext. ExecutionHive (HiveContext. Scala
: 208)
The at org. Apache. Spark. SQL. Hive. HiveContext. FunctionRegistry $lzycompute (Hiv
EContext. Scala: 462)
The at org. Apache. Spark. SQL. Hive. HiveContext. FunctionRegistry (HiveContext. Sc
Ala: 461)
The at org. Apache. Spark. SQL. UDFRegistration. & lt; init> (UDFRegistration. Scala: 40)

The at org. Apache. Spark. SQL. SQLContext. & lt; init> (SQLContext. Scala: 330)
The at org. Apache. Spark. SQL. Hive. HiveContext. & lt; init> (HiveContext. Scala: 90)
The at org. Apache. Spark. SQL. Hive. HiveContext. & lt; init> (HiveContext. Scala: 101)
At sun. Reflect. NativeConstructorAccessorImpl. NewInstance0 (Native Method)

nullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnull