How to get the output that is generated after sending a form using Python?-CodePudding

This is the website that I am want to scrape from: https://tweeterid.com/

This is what I have done currently in a form of function

from bs4 import BeautifulSoup as bs
from selenium import webdriver

def getTwitterID(name):
  options = webdriver.ChromeOptions()
  options.add_argument('--no-sandbox')
  options.add_argument('start-maximized')
  options.add_argument('enable-automation')
  options.add_argument('--disable-infobars')
  options.add_argument('--disable-dev-shm-usage')
  options.add_argument('--disable-browser-side-navigation')
  options.add_argument("--remote-debugging-port=9222")
  # options.add_argument("--headless")
  options.add_argument('--disable-gpu')
  options.add_argument("--log-level=3")
  driver = webdriver.Chrome('chromedriver', options=options)
  driver.maximize_window()
  driver.get("https://tweeterid.com/")
  driver.refresh()
  driver.find_element("id","twitter").send_keys(name)
  driver.find_element("id","twitterButton").click()
  WebDriverWait(driver, 10)
  html = driver.page_source
  soup = bs(html)
  id = soup.select_one('div[class*="output"]')
  return id.text

name in the parameter contains the handle name of the Twitter. What I want is the output result that is displayed on the larger box on the right side however, when I run my code... all I got is "Enter a Twitter ID or @handle on the left above and it will be converted here" which is the default text when you haven't sent the form.

How should I fix this?

CodePudding user response：

To get an id, it is enough to write a simple request. In the body of which to pass the username

import requests


def get_converted_tweeter_id(handle):
    url = "https://tweeterid.com/ajax.php"
    payload = f"input={handle}"
    headers = {
        'content-type': 'application/x-www-form-urlencoded; charset=UTF-8'
    }
    response = requests.request("POST", url, headers=headers, data=payload)
    return response.text

print('@elonmusk->', get_converted_tweeter_id('@elonmusk'))
print('elonmusk->', get_converted_tweeter_id('elonmusk'))
print('eminem->', get_converted_tweeter_id('eminem'))

OUTPUT:

@elonmusk-> 44196397
elonmusk-> 44196397
eminem-> 22940219