I want to read database data, there are about 10 million lines, only a string field, almost 10 characters, each data read out on the list after found huge memory, nearly more than 600 M, is there a way to less memory footprint? After reading out the main operation is whether a string in the list,

CodePudding user response:

Using SQL query isn't it better, why take into memory to find ah

CodePudding user response:

10 million rows read,,, was a little strange,

CodePudding user response:

Why read put within judgment, direct database where conditions are not sweet, please

CodePudding user response:

Method there is, of course, any qualified programmer way, oh, write "light code" programmer,

CodePudding user response:

Because my program to process the data, can't query the database every time, as more take up system resources

CodePudding user response:

I feel the people in the BBS some got now, if you approach, is any clue, I'm very grateful, don't want to help calculate, why waste your precious time, as if their very nb to coax,

CodePudding user response:

Should be to let the database to help you whether the query string in the table field!
This is not a download file, it is a general query

CodePudding user response:

You can read the database every time

CodePudding user response:

Or you on the news column

CodePudding user response:

Me detailed said my needs, I'm in the crawler, each climb a page than if the page has climbed, the url of the page through some processing after put into the database, is likely to climb around 5 page 1 SEC, I don't understand why everyone advice directly to check the database? Isn't a second database query must level 5 times than check memory efficiency?

CodePudding user response:

And this field is also not indexed, check certainly takes more than one seconds at a time,

CodePudding user response:

Initialize a List to one large capacity value try as far as possible,
For example:
List A=new List (10000000);

List is internal array, if the capacity is insufficient, the insert a new one each new array, the old array, GC will not immediately recycling, can cause waste of memory,