How to write a Java code in scala-CodePudding

In scala, self-study recently found a connection in the project of spark code, want to try the hand out to implement in scala, but found a little won't work, the method of Java and scala does not seem to be the same, my own scala is also a rawness, methods for bosses that last JavaRDD U=gaaatem2. ToJavaRDD (). How to use scala map () should be implemented in the next grateful

 public class Gaaa_Wide_Yarn {

Public static void main (String [] args) {
SparkConf sc=new SparkConf (.) setAppName (" Gaaa_Wide_Yarn "); 
SparkContext JSC=new SparkContext (sc); 
HiveContext hc=new HiveContext (JSC); 
String day=args [0]; 
System. The out. Println (" Gaaa_Wide_Yarn day: "+ day); 
String gaaaSql="select the username, groupid, starttime, stoptime, timelen, octets, sourceno, nasip," 
+ "nasport frameip, downreason" 
+ "from shjs_wlpt. Interface_aaag_wide_day" 
='" + + "where day day +"' "; 
String gaaaSql_1="select m.c ityname as area_name, m.c itycrmid as area_code, username, groupid, starttime, stoptime, timelen, octets, sourceno, nasip," 
+ "nasport, frameip downreason, 'as reserve1,' as reserve2, 'as reserve3,' 'as reserve4" 
+ "from gaaaSql t, usi_odso itv_check_city m" 
+ "where t.s ourceno=m.c ityid"; 
System. The out. Println (" gaaaSql_1: "+ gaaaSql_1); 

DataFrame gaaatem1=hc. SQL (gaaaSql); 
GaaaSql gaaatem1. RegisterTempTable (" "); 
DataFrame gaaatem2=hc. SQL (gaaaSql_1); 
GaaaSql_1 gaaatem2. RegisterTempTable (" "); 
SimpleDateFormat df3=new SimpleDateFormat (" yyyy - MM - dd HH: MM: ss "); 
Final String today=df3. The format (new Date ()); 
@ SuppressWarnings (" serial ") 
JavaRDD U=gaaatem2. ToJavaRDD (). The map (
New Function () {
Public String call (Row Row) {
String STR=""; 
STR +=today + "" 001;//updated date first, convenient for subsequent add field 
for (int i=0; i STR +=row. Get string (I) + "" 001; 
} 
return str; 
} 
}); 
U.r epartition (20). SaveAsTextFile (args [1]). 
} 

}

CodePudding user response:

Spark is written Scala, you would use the Scala API, why want to use Java?

CodePudding user response:

reference 1st floor link0007 response:

Spark itself is written Scala, you would use the Scala API, why want to use Java?

This is a program written in Java code, I want to use scala implementation

CodePudding user response:

refer to the second floor Hexxxc response:

Quote: refer to 1st floor link0007 response:

Spark is written Scala, you would use the Scala API, why want to use Java?

This is a program written in Java code, I want to use scala implementation

Really crying,,,

 
Gaaatem2. RDD (). The map ()

Here the RDD is scala source raw RDD,,,

CodePudding user response:

reference link0007 reply: 3/f

Quote: refer to the second floor Hexxxc response:

Quote: refer to 1st floor link0007 response:

Spark is written Scala, you would use the Scala API, why want to use Java?

This is a program written in Java code, I want to use scala implementation

Really crying,,,

Gaaatem2. RDD (). The map ()

Here the RDD is scala source raw RDD ah,,,

Mean. As long as the direct gaaatem2 RDD line, which the method of string manipulation should write

CodePudding user response:

Hello, I'm shoving the your Java code into the scala version, see whether accord with you want

 

The object Gaaa_Wide_Yarn {

Def main (args: Array [String]) : Unit={
Val sc=new SparkConf (). SetAppName (" Gaaa_Wide_Yarn ") 
Val JSC=new SparkContext (sc) 
Val hc=new HiveContext (JSC) 
Val day=args (0) 
Println (" Gaaa_Wide_Yarn day: "+ day 

Val gaaaSql="select the username, groupid, starttime, stoptime, timelen, octets, sourceno, nasip," 
+ "nasport frameip, downreason" 
+ "from shjs_wlpt. Interface_aaag_wide_day" 
='" + + "where day day +"' "; 

Val gaaaSql_1="select m.c ityname as area_name, m.c itycrmid as area_code, username, groupid, starttime, stoptime, timelen, octets, sourceno, nasip," 
+ "nasport, frameip downreason, 'as reserve1,' as reserve2, 'as reserve3,' 'as reserve4" 
+ "from gaaaSql t, usi_odso itv_check_city m" 
+ "where t.s ourceno=m.c ityid"; 

Println (" gaaaSql_1: "+ gaaaSql_1) 

Val gaaatem1=hc. SQL (gaaaSql) 
Gaaatem1. RegisterTempTable (" gaaaSql ") 
Val gaaatem2=hc. SQL (gaaaSql_1); 
GaaaSql_1 gaaatem2. RegisterTempTable (" "); 

Val dfs3=new SimpleDateFormat (" MM - dd yyyy - HH: MM: ss ") 
Val today=dfs3. The format (new Date ()) 
Val u=gaaatem2. The map (
New Function/Row, the String () {
Def call (row, row) : String={
Val STR="" 
STR +=today + "\ 001 
"For (I & lt; - row. Length) {
STR +=row. Get string (I) + "\ 001 
"} 
STR 
} 
} 
) 
U.r epartition (20). SaveAsTextFile (args (1)) 
} 
}