I have a WebUserControl that get Records from SQL and store inside a ViewState, this has worked fine so far. but now i have a DataSet with 100k records and i noticed that at each PostBack the XHR Request download all the data and is very slow, this not happen if i store data in Session, so i will ask is good to store this amount of data in a session variable?
My Test:
- Viewstate: ViewState Network Response
- The first request read the data and store inside the ViewState (Time: 11s, Size: 119MB)
- The second request is a generic postback that do nothing with the viewstate, despite this, it takes a long time to receive a reply and downloads a lot of data (Time: 15s, Size: 119MB)
- Total: 26s, 238MB
- Session: Session Network Response
- The first request read the data and store inside the Session (Time: 7s, Size: 123kB)
- The second request is a generic postback that do nothing with the Session and is fast (Time: 60ms, Size: 122kB)
- Total: 7s, 244kB
After this test, i mean, it would seem is a lot better to store data in Session instead of ViewState, is more fast to retrive data and download less data for each request, but i'm not totally sure is a good pratice, maybe there is e better method for do this that i ignore. i hope this is clear, English is not my first language sorry for that.
CodePudding user response：
You are quite correct. In fact, I have even found that the whole page size can exceed the limits for post backs set in web config.
However, two significant issues:
Loading up session() with that much data is as pointed out in comments ALSO a very bad idea.
Next up:
Do you REALLY need to store that much data in the session()? While you pointed out the huge performance increase - you still putting a VERY large load on the web server. that is going to eat up session() memory, and worse if using sql server based sessions, then you going to put quite a hard load on sql server (that blob of data is serialized by .net, and THEN saved to ONE row in sql server session state - and that again is going to be very hard load on sql server - even time to "serialize" into session going to take some time).
And how many rows of data can you really display on a page? 30 tops?
Now, for a few 1000 rows, I even then would not DARE use ViewState. It simply loads up and bloats up the web page far too much.
And with only say 1000 rows, and data paging of the grid view (or list view), then this can work.
But, beyond 1000 rows?
Not even close and on the same planet are you to load that 100k of rows. It is simply out of the question.
Think of Google, or even any other software - web or desktop.
we don't download the WHOLE internet, and then say have you use ctrl-f to search the huge monster page of data.
Same goes for this.
So, what does this REALLY mean?
You have to dump the built in data pager, and write custom code.
You can with most recent versions of sql server (2012 onwards) can use what is called sql server data paging. So, this means we want to let SQL server do the data paging, and NOT LOAD the whole big elephant of data, and THEN attempt to page that data. It just not practical to try and deal with that amount of data in one shot - at least from a UI point of view.
(how can a user deal with 100,000 rows at one time anyway - it not really possible.
This means, that if you page has 30 rows? Well, then you not even need ViewState or session. You pull ONLY 30 rows out of the 100,000 rows, and your performance will be instant. Your pages will load (and data page) WELL UNDER one second of time!!!
And you cut down huge memory use on the server, and you also as noted, cut down the size on view state. (you only ever have say 30 rows - and with that, you even find to let the gridview/list view continue to use view state (the automatic view state that they have).
So, check out the concept of sql server side paging. You can still cobble together what looks like a data pager, and you can get (count) the total number of rows in that table, but you only ever say pull 30 rows at a time.
How this works is outlined here:
What is the best way to paginate results in SQL Server
And the above link and post outlines some alternative approaches if you don't have sql server 2012 or later. I am of the view and position that using the newer "paging" features of 2012 is the simple approach, despite that some solutions posted in above are noted to be even faster.
So, you have to build a pager here. You can roll your own, but at the end of the day, the built in paging system is REALLY only for better UI performance, and really only works for about 1000 rows of data. after that, you need to adopt a paging system, and your software and users will love you for this - since everything will respond and run quite much as fast as you can click the mouse. You don't even need to adopt ajax calls for this, but you do have to cut down the rows pulled from SQL server - down to 30 or 20 or however many rows your grid displays at one time.