I met a mongo's problem in the project, a lot of days still could not be solved, hope to get Daniel's advice,
Specific question is: under the condition of long time not to access the database, query the database for the first time the time is very long, but after the query will soon,
Details:
(1) the entire database size probably around 1.9 TB;
(2) I query the collection of data for roughly 7 million;
(3) query once I get the data of 230000 or so;
(4) server memory for 120 gb;
(5) has been established in accordance with the query condition index, index data size to 600 MB or so;
6 querying time about 20 s for the first time, after the query within 1 s,
Now consider reason:
Since mongo will not be responsible for the management of the memory, so, when a long time not to access the database, the data in the memory is the cold data, the operating system's memory management program data released this part will be cold, cause the next query, you need to load data into memory, so the more time-consuming, at present, will not be able to determine the load index is more time-consuming, or loading the data more time-consuming, although mongo touch command (the command to specify a particular collection of index data or user data is loaded into memory), but I use WiredTiger storage engines, this command does not support the storage engine,
Need help:
(1) above reasons lead to this problem?
(2) if this is the reason, how to determine the load index is time consuming or load data take time?
(3) what is the better solution?

Note: as the most general assembly has reached around 25 gb, the collection and the many other database collection, so the collection of all the data stored in the memory is not desirable, if can confirm the load index is time-consuming, can consider to regular index will be loaded into memory, but to WiredTiger storage engines did not support this function method, it is a problem,

CodePudding user response:

If for such a big data have to be so strong real-time can consider to replace wiredtiger with the memory engine

CodePudding user response:

The landlord to solve, also encountered similar problems, long time no visit for the first time access speed is slow, and then access speed is normal

CodePudding user response:

Mark it I also found this problem because I am a web access data so I always thought that is the problem that the browser appears to be the mongo

CodePudding user response:

System start start automatically on a random a data query?

CodePudding user response:

Every day to preheat the query,

CodePudding user response:

I also encountered this problem at present, the use of first query need to explain the query found saveState and restoreState large amounts of data, for the second time this data will be much smaller, the building Lord, please solved this problem?

CodePudding user response:

Could you tell me the first query mongo slow problem solved?

CodePudding user response:

First visit must be to go disk, if slow disk I/o, this process can be slow, the cache contains two layers, a layer of cache is the operating system itself, the other one is WiredTiger internal cache, in this layer, the data will be unzipped (the default Settings, the data are stored in compressed)
So disk read/decompression, these will result in slow query for the first time
Index of problem should not, it is the amount of data is not large, the consumption of disk IO is not too big, the index is the default prefix compression, the internal cache in WiredTiger also do not need to unzip the
Confirmation method: you can build a same collection, the collection contains only index data, contrast the set with the original set of query performance for the first time