Now is to use the logic of the paging performance is too bad,, and maximum limit article 5 w data
CodePudding user response:
Your top oneCodePudding user response:
CodePudding user response:
This is not a two words can say clear, give the building Lord two addresses a look will understand1, http://www.lupaworld.com/article-227170-2.html is
2, about HIVE principle
http://www.open-open.com/doc/view/1d90cabb4b5544a5b40a5d69009a5ab1My smart ideas, please support it [/b] [b] , thank you!
CodePudding user response:
Oh, good,CodePudding user response:
Int pageNo=1;Int pageSize=10;
long startTime=System.currentTimeMillis();
DbConnectionManager DBM=DbConnectionManager
GetInstance (JarConf. DBTYPE);
The Connection conn=DBM. GetConnection (JarConf. DBTYPE);
DBM. FreeConnection (JarConf DBTYPE, conn);
JdbcTemplate=new jdbcTemplate (conn);
Long pageStart=jdbcTemplate
QueryForLong (" select min (id) from 1 edata. Ammeter ");
Int size=(pageNo - 1) * pageSize;
PageStart +=size;
Long potalRecords=jdbcTemplate
QueryForLong (" select count (1) the from 1 edata. Ammeter ");
List the List=jdbcTemplate
QueryForList (" select * from 1 edata. Ammeter where id & gt;"
+ pageStart + "asc limit order by id" + pageSize);
long endTime=System.currentTimeMillis();
System. The out. Println (" time: "+ (endTime - startTime));
System. The out. Println (" potalRecords: "+ potalRecords);
System. The out. Println (" pageStart: "+ pageStart);
System. The out. Println (" list. Size: "+ list. The size ());
For (Object o: list) {
The Map m=(Map) o;
System. Out.println (" id: "+ m.g et (" id") + ", frist_id:
"Frist_id + m.g et (" ") + ", update_date:
"+ m.g et (" update_date ") + ", price: "+ m.g et (" price")
+ ", second_id: "+ m.g et (" second_id"));
}
CodePudding user response:
Sqoop import, add a rowid, and then use the above code, 110 million data, impala look up to 6-7 seconds,CodePudding user response:
And again three connection to youThought:
http://iyanlei.com/hive-page.html
Establish a rownum function
http://datasciencedavid.blog.163.com/blog/static/235853095201462575318/
http://www.cnblogs.com/ggjucheng/archive/2013/01/30/2868993.html
Ha ha, give points.