What's the best/most accurate way to measure a pages load time that would include the time to load all of the pages resources? (Basically trying to get a load time that a real end user might have).
Is it better to use Wget or cURL for this type of task? (The operating system in use will be Windows due to other dependencies)
CodePudding user response：
You can download all the resources requested by a page with wget, using the -p option:
wget -p https://www.example.com/
curl doesn't parse HTML so it can't be used for this. It will just print the initial page's HTML but if the HTML requests images or CSS or JS files, it won't know about that (because it doesn't parse HTML) so it won't download any of them.
wget won't be a very accurate measure of the page's load time if the page requests resources through JavaScript, because wget doesn't parse or execute JavaScript. A more accurate way to get user perceived load times is to open the page in Chrome and then look at how long it took. You can see an accurate breakdown by looking at the Network tab in the Dev Tools. If you're trying to automate it, you can use Chrome Headless through Puppeteer, something like this:
const puppeteer = require('puppeteer');
(async () => {
const browser = await puppeteer.launch();
// Get the first tab
const page = (await browser.pages())[0];
await page.goto('https://example.com/');
const loadTime = page.evaluate(() => window.performance.timing.loadEventEnd - window.performance.timing.navigationStart);
console.log(loadTime);
await browser.close();
})();
This on its own will also not be accurate due to caching of DNS results, the TLS handshake or resources by the browser.
CodePudding user response：
You can use curl to get the response time for the HTML. It will not give you much more.
curl will give you:
CURLINFO_SIZE_UPLOAD - Total number of bytes uploaded
CURLINFO_SIZE_DOWNLOAD - Total number of bytes downloaded
CURLINFO_SPEED_DOWNLOAD - Average download speed
CURLINFO_SPEED_UPLOAD - Average upload speed
The best way I know of to get EVERY detail of an HTTP request is to use 
Sample of CSV info
"type","id","request_id","ip_addr","full_url","is_secure","method","host","url","raw_id","frame_id","documentURL","responseCode","request_type","load_ms","ttfb_ms","load_start","load_start_float","bytesIn","objectSize","objectSizeUncompressed","expires","cacheControl","contentType","contentEncoding","socket","protocol","dns_start","dns_end","connect_start","connect_end","ssl_start","ssl_end","initiator","initiator_line","initiator_column","initiator_type","priority","initial_priority","server_rtt","bytesOut","score_cache","score_cdn","score_gzip","score_cookies","score_keep-alive","score_minify","score_combine","score_compress","score_etags","dns_ms","connect_ms","ssl_ms","gzip_total","gzip_save","minify_total","minify_save","image_total","image_save","cache_time","cdn_provider","server_count","created","http2_stream_id","http2_stream_dependency","http2_stream_weight","http2_stream_exclusive","tls_version","tls_resumed","tls_next_proto","tls_cipher_suite","netlog_id","server_port","final_base_page","is_base_page","load_end","ttfb_start","ttfb_end","download_start","download_end","download_ms","all_start","all_end","all_ms","index","number","cpu.EvaluateScript","cpu.v8.compile","cpu.FunctionCall","cpuTime","run","cached","renderBlocking","initiator_function",
"3","221020_AiDc05_4BQ","BE8FE33DABBBA0CF2B3AC5D78083C66E","151.101.193.69","https://stackoverflow.com/questions/74122109/curl-or-wget-for-measuring-page-load-time","1","GET","stackoverflow.com","/questions/74122109/curl-or-wget-for-measuring-page-load-time","BE8FE33DABBBA0CF2B3AC5D78083C66E","4476133D5C5FF9920B0E2BB49084B133","https://stackoverflow.com/questions/74122109/curl-or-wget-for-measuring-page-load-time","200","Document","346","198","533","533.000041","40562","40562","145896","","private","text/html","gzip","50","HTTP/2","0","172","173","344","344","533","","","","script","Highest","Highest","","2317","-1","100","100","-1","100","-1","-1","-1","-1","-1","171","189","41099","0","","","","","","Fastly","","4","1","0","256","1","TLS 1.2","False","h2","49199","41","443","1","1","879","533","731","731","879","148","173","879","706","0","1","135","26","2","164","1","0","","",
"3","221020_AiDc05_4BQ","6487.6","151.101.193.69","https://cdn.sstatic.net/Js/stub.en.js?v=0e3ada576039","1","GET","cdn.sstatic.net","/Js/stub.en.js?v=0e3ada576039","6487.6","4476133D5C5FF9920B0E2BB49084B133","https://stackoverflow.com/questions/74122109/curl-or-wget-for-measuring-page-load-time","200","Script","585","568","967","967.000067","18152","18152","53336","","max-age=604800","application/javascript","gzip","50","HTTP/2","787","967","-1","-1","-1","-1","https://stackoverflow.com/questions/74122109/curl-or-wget-for-measuring-page-load-time","25","","parser","High","High","","1831","50","100","100","-1","100","-1","-1","-1","-1","180","-1","-1","18152","0","","","","","34541","Fastly","","788","9","","","","","","","","67","","","","1552","967","1535","1535","1552","17","787","1552","765","1","2","34","8","443","485","1","0","blocking","",
"3","221020_AiDc05_4BQ","6487.5","151.101.193.69","https://cdn.sstatic.net/Js/third-party/npm/@stackoverflow/stacks/dist/js/stacks.min.js?v=facbc6b2f3b6","1","GET","cdn.sstatic.net","/Js/third-party/npm/@stackoverflow/stacks/dist/js/stacks.min.js?v=facbc6b2f3b6","6487.5","4476133D5C5FF9920B0E2BB49084B133","https://stackoverflow.com/questions/74122109/curl-or-wget-for-measuring-page-load-time","200","Script","599","568","968","968.000061","23746","23746","100193","","max-age=604800","application/javascript","gzip","50","HTTP/2","-1","-1","-1","-1","-1","-1","https://stackoverflow.com/questions/74122109/curl-or-wget-for-measuring-page-load-time","24","","parser","Low","Low","","1978","50","100","100","-1","100","-1","-1","-1","-1","-1","-1","-1","23746","0","","","","","567498","Fastly","","786","11","","","","","","","","61","","","","1567","968","1536","1536","1567","31","968","1567","599","2","3","81","2","38","121","1","0","potentially_blocking","",
"3","221020_AiDc05_4BQ","6487.7","151.101.193.69","https://cdn.sstatic.net/Shared/stacks.css?v=5ad0f45f4799","1","GET","cdn.sstatic.net","/Shared/stacks.css?v=5ad0f45f4799","6487.7","4476133D5C5FF9920B0E2BB49084B133","https://stackoverflow.com/questions/74122109/curl-or-wget-for-measuring-page-load-time","200","Stylesheet","363","188","968","968.000072","68692","68692","654577","","max-age=604800","text/css","gzip","50","HTTP/2","-1","-1","-1","-1","-1","-1","https://stackoverflow.com/questions/74122109/curl-or-wget-for-measuring-page-load-time","28","","parser","Highest","Highest","","1885","50","100","100","-1","100","-1","-1","-1","-1","-1","-1","-1","68692","0","","","","","542107","Fastly","","790","3","","","","","","","","72","","","","1331","968","1156","1156","1331","175","968","1331","363","3","4","","","","","1","0","blocking","",
"3","221020_AiDc05_4BQ","6487.8","151.101.193.69","https://cdn.sstatic.net/Sites/stackoverflow/primary.css?v=0cc30ee01b86","1","GET","cdn.sstatic.net","/Sites/stackoverflow/primary.css?v=0cc30ee01b86","6487.8","4476133D5C5FF9920B0E2BB49084B133","https://stackoverflow.com/questions/74122109/curl-or-wget-for-measuring-page-load-time","200","Stylesheet","550","374","968","968.000077","60467","60467","339815","","max-age=604800","text/css","gzip","50","HTTP/2","-1","-1","-1","-1","-1","-1","https://stackoverflow.com/questions/74122109/curl-or-wget-for-measuring-page-load-time","29","","parser","Highest","Highest","","1927","50","100","100","-1","100","-1","-1","-1","-1","-1","-1","-1","60467","0","","","","","134559","Fastly","","793","5","","","","","","","","77","","","","1518","968","1342","1342","1518","176","968","1518","550","4","5","","","","","1","0","blocking","",
"3","221020_AiDc05_4BQ","6487.9","151.101.193.69","https://cdn.sstatic.net/Shared/Channels/channels.css?v=d098999fc478","1","GET","cdn.sstatic.net","/Shared/Channels/channels.css?v=d098999fc478","6487.9","4476133D5C5FF9920B0E2BB49084B133","https://stackoverflow.com/questions/74122109/curl-or-wget-for-measuring-page-load-time","200","Stylesheet","551","393","968","968.000082","4168","4168","18913","","max-age=604800","text/css","gzip","50","HTTP/2","-1","-1","-1","-1","-1","-1","https://stackoverflow.com/questions/74122109/curl-or-wget-for-measuring-page-load-time","70","","parser","Highest","Highest","","1918","50","100","100","-1","100","-1","-1","-1","-1","-1","-1","-1","4168","0","","","","","477490","Fastly","","797","7","","","","","","","","82","","","","1519","968","1361","1361","1519","158","968","1519","551","5","6","","","","","1","0","blocking","",
"3","221020_AiDc05_4BQ","6487.4","142.251.163.95","https://ajax.googleapis.com/ajax/libs/jquery/1.12.4/jquery.min.js","1","GET","ajax.googleapis.com","/ajax/libs/jquery/1.12.4/jquery.min.js","6487.4","4476133D5C5FF9920B0E2BB49084B133","https://stackoverflow.com/questions/74122109/curl-or-wget-for-measuring-page-load-time","200","Script","370","175","1309","1309.000055","33951","33951","97163","Thu, 19 Oct 2023 17:18:45 GMT","public, max-age=31536000, stale-while-revalidate=2592000","text/javascript","gzip","87","HTTP/2","784","955","955","1128","1128","1309","https://stackoverflow.com/questions/74122109/curl-or-wget-for-measuring-page-load-time","23","","parser","High","High","","1870","100","100","100","-1","100","-1","-1","-1","-1","171","173","181","33951","0","","","","","31493759","Google","","783","1","0","220","1","TLS 1.3","False","h2","4865","55","443","","","1679","1309","1484","1484","1679","195","784","1679","895","6","7","64","12","31","107","1","0","blocking","",
"3","221020_AiDc05_4BQ","6487.10","151.101.193.69","https://cdn.sstatic.net/Img/teams/teams-illo-free-sidebar-promo.svg?v=47faa659a05e","1","GET","cdn.sstatic.net","/Img/teams/teams-illo-free-sidebar-promo.svg?v=47faa659a05e","6487.10","4476133D5C5FF9920B0E2BB49084B133","https://stackoverflow.com/questions/74122109/curl-or-wget-for-measuring-page-load-time","200","Image","176","175","1580","1580.000099","2368","2368","5950","","max-age=604800","image/svg xml","gzip","50","HTTP/2","-1","-1","-1","-1","-1","-1","https://stackoverflow.com/questions/74122109/curl-or-wget-for-measuring-page-load-time","485","","parser","Low","Low","","2101","50","100","100","-1","100","-1","-1","-1","-1","-1","-1","-1","2368","0","","","","","93767","Fastly","","1579","13","0","147","1","","","","","99","443","","","1756","1580","1755","1755","1756","1","1580","1756","176","7","8","","","","","1","0","","",
"3","221020_AiDc05_4BQ","6487.20","151.101.193.69","https://cdn.sstatic.net/Img/unified/sprites.svg?v=fcc0ea44ba27","1","GET","cdn.sstatic.net","/Img/unified/sprites.svg?v=fcc0ea44ba27","6487.20","4476133D5C5FF9920B0E2BB49084B133","https://stackoverflow.com/questions/74122109/curl-or-wget-for-measuring-page-load-time","200","Image","176","174","2163","2163.000126","2852","2852","7542","","max-age=604800","image/svg xml","gzip","50","HTTP/2","-1","-1","-1","-1","-1","-1","https://cdn.sstatic.net/Sites/stackoverflow/primary.css?v=0cc30ee01b86","","","parser","High","Low","","2173","50","100","100","-1","100","-1","-1","-1","-1","-1","-1","-1","2852","0","","","","","383944","Fastly","","2162","15","0","147","1","","","","","126","443","","","2339","2163","2337","2337","2339","2","2163","2339","176","8","9","","","","","1","0","","",
The CSV contains the following details on every request made to render the page.
type
id
request_id
ip_addr
full_url
is_secure
method
host
url
raw_id
frame_id
documentURL
responseCode
request_type
load_ms
ttfb_ms
load_start
load_start_float
bytesIn
objectSize
objectSizeUncompressed
expires
cacheControl
contentType
contentEncoding
socket
protocol
dns_start
dns_end
connect_start
connect_end
ssl_start
ssl_end
initiator
initiator_line
initiator_column
initiator_type
priority
initial_priority
server_rtt
bytesOut
score_cache
score_cdn
score_gzip
score_cookies
score_keep-alive
score_minify
score_combine
score_compress
score_etags
dns_ms
connect_ms
ssl_ms
gzip_total
gzip_save
minify_total
minify_save
image_total
image_save
cache_time
cdn_provider
server_count
created
http2_stream_id
http2_stream_dependency
http2_stream_weight
http2_stream_exclusive
tls_version
tls_resumed
tls_next_proto
tls_cipher_suite
netlog_id
server_port
final_base_page
is_base_page
load_end
ttfb_start
ttfb_end
download_start
download_end
download_ms
all_start
all_end
all_ms
index
number
cpu.EvaluateScript
cpu.v8.compile
cpu.FunctionCall
cpuTime
run
cached
renderBlocking
initiator_function