I use Selenium below method.

1. open chrome by using chromedriver selenium

2. manually login

3. get information of webpage

However, after doing this, Selenium seems to get the html code when not logged in.

Is there a solution?

CodePudding user response：

Try this code:

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.chrome.options import Options

options = Options()

# path of the chrome's profile parent directory - change this path as per your system
options.add_argument(r"user-data-dir=C:\\Users\\User\\AppData\\Local\\Google\\Chrome\\User Data")
# name of the directory - change this directory name as per your system
options.add_argument("--profile-directory=Default")
driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()), options=options)

You can get the chrome profile directory by passing this command - 'chrome://version/' in chrome browser.

Add the code for login after the above code block, then if you execute the code for the second time onwards you can see the account is already logged in.

Before running the code close all the chrome browser windows and execute.

CodePudding user response：

Instead of storing and maintaining the login session another easy approach would be to use pickle library to store the cookies post login.

As an example to store the cookies from Instagram after logging in and then to reuse them you can use the following solution:

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
import pickle

# first login
driver.get('http://www.instagram.org')
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input[name='username']"))).send_keys("_SelmanFarukYılmaz_")
driver.find_element(By.CSS_SELECTOR, "input[name='password']").send_keys("Selman_Faruk_Yılmaz")
driver.find_element(By.CSS_SELECTOR, "button[type='submit'] div").click()
pickle.dump( driver.get_cookies() , open("cookies.pkl","wb"))
driver.quit()

# future logins
driver = webdriver.Chrome(service=s, options=options)
driver.get('http://www.instagram.org')
cookies = pickle.load(open("cookies.pkl", "rb"))
for cookie in cookies:
    driver.add_cookie(cookie)
driver.get('http://www.instagram.org')