Ever bosses climb a dove "search https://www.jiumodiary.com/)

Home page is the


Then I want to climb the search results page, using selenium

A results page


I want to use page_source get the source code of the page but the page's source code, source code is the home page is not in the code review elements that see so page_source is the home page of the source code





Have bosses know how to solve it or what advice without selenium to climb

CodePudding user response:

The data you want to in this url:

This is a post request

CodePudding user response:

 
The import requests 
The import time 
The import json 

The session=requests. The session () 
Headers={

} 
Keyword='django' 
Url_id='https://www.jiumodiary.com/init_hubs.php' 
Data={
 https://bbs.csdn.net/topics/'q: keyword, 
'remote_ip' : ' ', 
'time_int: int (time. Time ()) 
} 
Response=session. Post (url_id, data=https://bbs.csdn.net/topics/data). The decode () 
Id=json. Loads (response) [' id '] 

Url_detail='https://www.jiumodiary.com/ajax_fetch_hubs.php' 
Data={
 https://bbs.csdn.net/topics/'id: id, 
'set' : 0 
} 
The response=session. Post (url_detail, data=https://bbs.csdn.net/topics/data). The decode () 
Results=json. Loads (response) 
Sources=results [' sources'] 
CNT=1 
For I in sources: 
View_type=[' view_type] I 
If view_type=='view_normal: 
For j in the I [' details'] [' data '] : 
Title=j [' title '] 
The link=j/' link '
Print (CNT, the title and link) 
CNT +=1 


"' 
1 Learning Website Development with Django: A beginner 's tutorial to building web applications, quickly and cleanly, have the Django application framework 
http://b-ok.org/book/682360/9c4416 
2 the Django Design Patterns and Best Practices: Easily build maintainable websites with powerful and relevant Django Design Patterns 
http://b-ok.org/book/2734391/8211a0 
3 the Django By Example: Create your own line of successful web applications with Django 
http://b-ok.org/book/2734419/a7330d 
Song of 4 Django: simple blog build tutorial (Jane books novels). Azw3 
https://u15169360.ctfile.com/fs/15169360-371375575 
5 the Django Web development guide. PDF 
https://pan.baidu.com/share/link? Shareid=436951 & amp; UK=3190231936 
PDF 
 6 Django documentation in Chinese.https://pan.baidu.com/s/17I14 
7 the Django Django - Django Django 
https://pan.baidu.com/s/1hr1mVQ0 
8 the Django Django - Marble Skies (FLAC) (2018). Zip 
https://pan.baidu.com/s/1bqMiRIB 
9 Python Django lecture video - Taiwan fu jen university (Emerging Web Tech FJU CSIE Python Django) AutoCAD. Rar 
https://pan.baidu.com/s/1pJnvWHt 
10 the Django Django - Default - Single.zip 
https://pan.baidu.com/share/link? Shareid=176087 & amp; UK=925773429 & amp; Fid=339289867 
11 Django Django - Default - EP. Zip 
https://pan.baidu.com/share/link? Shareid=176085 & amp; UK=925773429 & amp; Fid=1825281524 
Song of 12 Django: simple blog build tutorial. Epub 
http://vdisk.weibo.com/s/aADaW4YRFvb2u 
13 Udemy - Python and Django Full Stack Web Developer Bootcamp 
http://kankan.yyupload.com/file/583850-207902895 
14 Hosting Django Amazon Web Services (AWS) Fundamentals 
http://kankan.yyupload.com/file/583850-199807956 
Upload 15 Django instance - dream my blog - if I was dreaming 
https://dreame.wodemo.net/entry/360377 
16 and proved video, original software 
https://xfqq.wodemo.net/
17 django advanced topic 
https://www.gitbook.com/book/tangxuguo/django 
18 a django application practice 
https://www.gitbook.com/book/wanghe4096/django-practice 
19 Python django project practical shopping system Python teaching of Web development... Doc 
https://wenku.baidu.com/view/1bc4a676ae1ffc4ffe4733687e21af45b207fe4b.html 
"' 

CodePudding user response:

the first step to get id
A post request to https://www2.jiumodiary.com/init_hubs.php

Request parameters:
{
"Q" : "search content
"}

From the return value id

the second step is to get a json content
A post request to https://www5.jiumodiary.com/ajax_fetch_hubs.php

Request parameters:
{
"Id" : input step on to get id,
"Set" : 0
}

The returned json data is the search results