Home > other >  Data about the crawl snowball
Data about the crawl snowball

Time:09-28

In accordance with the method of online crawl data, but in the end it is wrong to return json error messages, but directly on the browser can return the json data, is it an error?
Crawl json returns url is
https://xueqiu.com/service/v5/stock/hq/longhu? The date=1581955200000 & amp; _=1582028332240
Post should return data

The following is a code, please see error

CodePudding user response:

The parameters of the get method has been in the url
Why pass params

CodePudding user response:

references on 1st floor day I reply:
the get method of the parameters has been in the url
Why pass params
in order to test, even if the same results without parameters

CodePudding user response:


The attached code, more violent
The import requests
The import json
Url="https://xueqiu.com/service/v5/stock/hq/longhu? The date=1581955200000 & amp; _=1582028332240 '
Headers={
"Accept", "text/HTML, application/XHTML + XML, application/XML. Q=0.9, image/webp image/apng, */*; Q=0.8, application/signed - exchange; V=b3,
""Accept - the Encoding", "gzip, deflate, br",
"Accept - Language" : "useful - CN, useful;
q=0.9,"Cache-control" : "Max - age=0,"
"Connection" : "keep - the alive,"
"Cookie", "s=bs1b2cmktp; Acw_tc=2760821 e15796836553553193e24c4d1bc0c7832a035e5aff0f32745c45ca4; Hm_lvt_1db88642e346389874251b5a1eded6e3=1579683673; __utma=1.648799459.1579683673.1579683673.1579683673.1; __utmz=1.1579683673.1.1. Utmcsr=(direct) | utmccn=(direct) | utmcmd=(none); U=671579683672710; Device_id=fe2392eb8f4d168dbd8187739d0a1b48; _ga=GA1.2.1034260887.1579683681; Aliyungf_tc=AQAAAL7PM1APDwUAVZZ5d + XAZMW45kiT; Xq_a_token=b2f87b997a1558e1023f18af36cab23af8d202ea; Xq_r_token=823123 c3118be244b35589176a5974c844687d5e; xq_id_token=eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9.eyJ1aWQiOi0xLCJpc3MiOiJ1YyIsImV4cCI6MTU4MzE0MzIwMCwiY3RtIjoxNTgyMDk0MDc4NjAxLCJjaWQiOiJkOWQwbjRBWnVwIn0.BV1SpKZIlr-6cTOE7rSqQpA3XJZRHhdCpGHWciM7-zcYbrVEdCQdZPjS-O40ZRIWH7IFFVCEx_ub_mDJdgElghdXtQcwyciN-Oj30GIBdc9rj9KLt2OHY7npH-MQAquk7L5HoOqdDbv7xdIf9oQPpdZVZV3qKlxMr6oDYurodOL3SPCGODVcySNXYmBSKJsneGmIOp7Nqe2OXSoz9BnSiuqRYdq1gir3BpRjcRVFpegqeqv37SGYCsxDpVjw57GNE3H7mv1Xp6E0vVM8qb6uiS5RBW7oZ9Gx5PD6LP0PNTD-5A0xvtvYWDdD-27DJygcrRmDamY3EylQtDaBcrdvZw",
"The Host" : "xueqiu.com",
"The Sec - Fetch - Mode" : "navigate",
"The Sec - Fetch - Site" : "none",
"The Sec - Fetch - User" : "? 1, "
"The Upgrade - the Insecure - Requests" : "1",
"The user-agent: Mozilla/5.0 (Windows NT 6.1; Win64. X64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.70 Safari/537.36 ",
}

The response=requests. Get (url=url, headers=headers)
Datas=json. Loads (response. The decode ())
Datas_values=datas (" data ") (" items ")
For the index, the data in enumerate (datas_values) :
The symbol=data [" symbol "]
Name=data/" name ", "
The close=data (" close ")
Percent=data [" percent "]
Volume=data (" volume ")
Amount=data (" amount ")
The type_name=data [" the type_name "] [0]
SQL="{} first lines of each number can be assembled into SQL statements here," \
"Respectively: {}, {}, {}, {}, {}, {}, {}".. \
Format (index + 1, symbol, name, close, percent, volume, amount, the type_name)
Print (SQL)

CodePudding user response:

Check the code line by line to see if there is a abnormal return data

CodePudding user response:

Thank you, has been solved

CodePudding user response:

reference 5 floor zhouguoguang reply:
thank you and have resolved the

What kind of method is used to solve, and sharing, knowledge has no limit
  • Related