Python request, succeed login, and then logout in different url-CodePudding

here my code :

session = requests.Session()

headers =  {'User-Agent': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:86.0) Gecko/20100101 Firefox/86.0', 
        'Accept': 'text/html,application/xhtml xml,application/xml;q=0.9,image/webp,*/*;q=0.8',
        'Accept-Language': 'en-US,en;q=0.5',
        'Connection': 'keep-alive',
        'Upgrade-Insecure-Requests': '1',
        'Cache-Control': 'max-age=0'} 

def generating_data():

    main_url='https://opencorporates.com/users/sign_in'    
    r1 = session.get(main_url, headers=headers)

    soup = BeautifulSoup(r1.text, 'html.parser')
    
    tokens = soup.find('meta', attrs={'name':'csrf-token'})
    token = tokens.get('content')
    print(f'token is : {token}')

    print('Login!')
    datas = {
        'utf8': '✓',
        'authenticity_token': token,
        'user[email]':'user',
        'user[password]':'pass',
        'submit':''
    }

    r2 = session.post('https://opencorporates.com/users/sign_in',headers=headers, data=datas, cookies=r1.cookies)
    r3 = session.get('https://opencorporates.com/companies?utf8=✓&q=above and beyond&commit=Go&jurisdiction_code=&utf8=✓&commit=Go&nbsp=&controller=searches&action=search_companies&inactive=false&mode=best_fields&search_fields[]=name&branch=false&nonprofit=&order=score', headers=headers, cookies=r1.cookies)

    f = open('./res.html', 'w ')
    f.write(r3.text)
    f.close

generating_data()

i already get the result of login if print the r2 line, but when change to next line r3, it show the page like we are not login yet, anyone can help ? thanks

CodePudding user response：

You need to remove the portion cookies=r1.cookies since you are already using a session. What this does is it overwrites the cookies collected from response of r2 that would have been sent along with the request, and which might been important for logging in. Same goes for the r2. In general, you do not need to deal with cookies yourself when you are using a session with requests. Your code for generating_data() then becomes:

def generating_data():

    main_url='https://opencorporates.com/users/sign_in'    
    r1 = session.get(main_url, headers=headers)

    soup = BeautifulSoup(r1.text, 'html.parser')
    
    tokens = soup.find('meta', attrs={'name':'csrf-token'})
    token = tokens.get('content')
    print(f'token is : {token}')

    print('Login!')
    datas = {
        'utf8': '✓',
        'authenticity_token': token,
        'user[email]':'user',
        'user[password]':'pass',
        'submit':''
    }

    r2 = session.post('https://opencorporates.com/users/sign_in',headers=headers, data=datas)
    r3 = session.get('https://opencorporates.com/companies?utf8=✓&q=above and beyond&commit=Go&jurisdiction_code=&utf8=✓&commit=Go&nbsp=&controller=searches&action=search_companies&inactive=false&mode=best_fields&search_fields[]=name&branch=false&nonprofit=&order=score', headers=headers)

f = open('./res.html', 'w ')
f.write(r3.text)
f.close