Beautiful Soup 4 Python Webscraping price from website (not the same)-CodePudding

Someone automatically closed my post. This is not the same question as someone else. Do not close it please.

I am trying to get the price from the website. Can anyone please find the error in my code?

import requests
from bs4 import BeautifulSoup

def priceTracker():
    url = ' https://www.britishairways.com/travel/book/public/en_gb/flightList?onds=LGW-AMS_2022-3-11&ad=1&yad=0&ch=0&inf=0&cabin=M&flex=LOWEST&usedChangeSearch=true'
    response = requests.get(url)
    soup = BeautifulSoup(response.text,'lxml')
    #print(soup.prettify)
    price = soup.find(class_=" heading-sm ng-tns-c70-4 ng-star-inserted").next_sibling
    #print(soup) test if soup working properly
    print(price)
    
while True:
  priceTracker()

I have attached the DOM screen.

I have attached the DOM screen of the price. I have updated the url (in case the url does not work, to get the url you go to the main website and press the search button)

CodePudding user response：

Try this :

import requests
from bs4 import BeautifulSoup

def priceTracker():
    url = ' https://www.britishairways.com/travel/book/public/en_gb/flightList?onds=LGW-AMS_2022-3-11&ad=1&yad=0&ch=0&inf=0&cabin=M&flex=LOWEST&usedChangeSearch=true'
    response = requests.get(url)
    soup = BeautifulSoup(response.text,'lxml')
    #print(soup.prettify)

    price = soup.find(class_=" heading-sm ng-tns-c70-4 ng-star-inserted").text # changed
    #print(soup) test if soup working properly
    print(price)
    
while True:
  priceTracker()

CodePudding user response：

As mentioned in a comment, instead of .next_sibling you should apply .text method there.
This line

soup.find(class_=" heading-sm ng-tns-c70-4 ng-star-inserted")

is giving you a price web element.
To extract a text from it you can apply .text method.
So your code will be

price = soup.find(class_=" heading-sm ng-tns-c70-4 ng-star-inserted").text