Data is too much, causing stuck, how to use the multithreading implementation query a large number of data, bosses, if can be attached to the code, thank you very much

CodePudding user response:

With the method of asynchronous, query and paging

CodePudding user response:

Can with code to explain, bosses

CodePudding user response:

SQL statements you should write, optimizing the SQL query, indexing, the query using paging can improve the query efficiency

CodePudding user response:

Hundreds of thousands of database, database, or easily

CodePudding user response:

If you still can't write multithreaded, it should find some posts into the door first, and this post a lot of, don't waste your rewrite again,
Waiting for you to know, have more specific questions, suitable for questions here,

CodePudding user response:

Multithreaded queries can not solve the problem of stuck, what is the cause of the main analysis should be jammed, which links are time consuming is higher, the bottleneck of the database in the disk, the disk is not suitable for multi-threaded concurrent read and write (solid state disk exception), general multi-thread query is: will inquire into multiple segments, according to the conditions (for example, according to the time of the query, the period will be subdivided into multiple), and then open multiple threads respectively query, finally the results summary, if the data distribution in different database servers, multithreading can improve efficiency, otherwise no function,

CodePudding user response:

With the similar data, what a multi-threaded, with paging, index and good, multi-threaded read tried before, because may be a need to read a larger quantity) (business is special, so read block, the results did not think of so good, like 6 #, database bottleneck in IO speed, using multiple threads to read and write must limit the maximum number of threads in a certain number, otherwise wait IO. A timeout error, multithreading to write, too,

CodePudding user response:

I'll ask
Article 1: why do you want to find hundreds of thousands of?
2 what is action? Display? Transmission?

CodePudding user response:

Multi-threaded read the problem bigger, easy to cause the lock table (read records much more so), also use this form of business is slow, you should avoid doing this in our business,

CodePudding user response:

Design a high concurrent high performance system, it is necessary to the overall thinking and sufficient practical experience, when you only in my brain "relational database to add and delete", put forward the problem is also only this level, it will not be able to solve the problem, because can't really ask questions,

CodePudding user response:

Relational data to solve the problem is inherently "oriented to the underlying storage, facing structural index, to lock protection affairs strictly", it is a very bottom, if only these in my brain, so there will be no tencent, there will be no, alas, ali, there won't be baidu, etc. Any company, any practical system as long as from the above beginner "to add and delete" way of thinking and design for the memory not be born of high-performance big concurrent solution of data structure.

CodePudding user response:

Use a thread pool.
The point is not more not multi-threading, you want to be well designed architecture, such as distributed storage, etc

CodePudding user response:

Hundreds of thousands of data, processing is not casually, why is your thing, I want to know

CodePudding user response:

This has nothing to do with multithreading, also has nothing to do with the code, hundreds of thousands of data is small data quantity, there are three key: 1 is your SQL query, may be written by unreasonable cause all data filtering, 2 is not the appropriate index, 3 is paging query, you can see if you try to query fields can add index

CodePudding user response:

For hundreds of thousands of data (rather than the tens of millions, billions of data) in the so-called "data is too much, causing stuck" large amounts of data for using multi-threading implementation "this kind of problem, we are aiming at high concurrency, big data operation and the" pain points ", just didn't know if you don't have experienced pain points, almost any commercial functions, for example, if you want to publish to the server, I am willing to it from the start is based on distributed parallel part that goes from the automatic service,

CodePudding user response:

Actually similar Orlans such things as a person beginners vb to write a form user controls as simple, write well so long as can combine into a different form for application of ever-changing, the same meaning, write good data service and then part that goes from the automatic can on the cluster distribution management (usually require telecommunication level of high performance, high stability), usually learn this stuff as long as 1, 2 days, Microsoft has traditionally is particularly simple particularly thoughtful!

The difference is that the need to begin to understand the concept, know for cluster rather than stand-alone oriented, always stand in the trend of road, this may be the hardest of all to begin to understand,

CodePudding user response:

As a DBA, want to tell you: really want to too much, don't take everything on that program,
If only hundreds of thousands of data table, then you have to do is to increase the paging, optimizing the query, add index, avoid blocking, avoid unreasonable business requirements,

This amount of data, is really isn't worth mentioning, multi-threaded of little help, probably still messed up,

CodePudding user response:

Or to get into the SQL above laid hands on him

CodePudding user response:

I take back my point of view,
Really only hundreds of thousands of data card, also the problem either in the database, either in your code to access data in a database, either in the database design problem, to solve the problem,
And with multithreading, just stick a snake oil, can not solve the problem, and if you want to access data, all in one table, that you couldn't multi-threading,

CodePudding user response:

Optimization of two aspects: 1 index, query the database tables have do 2 interface display is to cycle loading in binding way can't oneself read slowly, monitor https://item.taobao.com/item.htm? SPM=a1z10. 1 - c - s. 4004-22879184 324.20.2 d2d58fb7Ew9Cy & amp; Id=620683911377

CodePudding user response:

Basic to be sure, multithreading here doesn't make any sense. Should do: optimization of SQL statements, optimize the index, the primary key...

CodePudding user response:

Hundreds of thousands of data, the more... Article is the relationship between the DB, also, tens of millions of began demanding.
: ok, if you want to hundreds of thousands of data were found out, the vast majority of cases it is not necessary, is how much we need to check, usually a screen you can see the scope of the article is 20.