I am trying to use values of column in web scraping to get results in a loop-CodePudding

I am practicing web scraping I am new to this and am trying to scrape amazon using their product ids. so, suppose I have a list of product ids

asin = ['B09DKZYBR5','B098QGLXXY','B09BQV32W2']

now I want to use each list value at a time to create a list of product name so I tried

for i in asin:
page = requests.get("https://www.amazon.com.au/dp/"   {i})
soup = page.content
doc = BeautifulSoup(soup, "html.parser")
data = []

data.append({
name = doc.find_all(class_="a-size-large product-title-word-break")[0].parent.find('span').string
})

then when I do this

pd.DataFrame(data)

I want something like this

    name
0   Auriko Cat Teaser Toy with Wooden Hand..
1   Cactus Cat Scratching Posts Pole Tree...
2   LNLtoy 8 Cat Toys Kitten Toys Assortme..

i am getting this error

File "C:\Users\Parth\AppData\Local\Temp/ipykernel_21420/2238400382.py", line 2
    page = requests.get("https://www.amazon.com.au/dp/"   {i})
    ^
IndentationError: expected an indented block

CodePudding user response：

I think you have to do this in append {name:doc.find_all(class_="a-size-large....}

CodePudding user response：

The main cause of your error is because the identation in your for loop is not properly done To get the solution you want, this should work:

asin = ['B09DKZYBR5','B098QGLXXY','B09BQV32W2']
data = []
for i in asin:
    page = requests.get("https://www.amazon.com.au/dp/"   str(i))
    soup = page.content
    doc = BeautifulSoup(soup, "html.parser")
    name = doc.find_all(class_="a-size-large product-title-word-break")[0].parent.find('span').string
    data.append(name)
df = pd.DataFrame({"name":data})