they show me these errors ValueError: All arrays must be of the same length how can I solve these error kindly anyone who give me solution of these problem I am trying many approaches but I can not solve these error so how can I handle these error my array is not same

import enum
import requests
from bs4 import BeautifulSoup
import json
import pandas as pd 

url="https://www.fleetpride.com/parts/otr-coiled-air-hose-otr6818"
headers = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.131 Safari/537.3"
}
r = requests.get(url)
soup = BeautifulSoup(r.content, "html5lib")
raw_json = ""
for table_index,table in enumerate( soup.find_all("script")):
    if('CCRZ.detailData.jsonProductData = {"' in str(table)):
        x=str(table).split('CCRZ.detailData.jsonProductData = {"')
        raw_json = "{\"" str(x[-1]).split('};')[0] "}"
        break
           
req_json = json.loads(raw_json)
# with open("text_json.json","w")as file:
#     x=json.dump(req_json,file,indent=4)

temp = req_json

name=[]
specs=[]


title=temp['product']['prodBean']['name']
name.append(title)


item=temp['specifications']['MARKETING']
for i in item:
    try:
        get=i['value']
    except:
        pass

    specs.append(get)


temp={'title':name,'Specification':specs}
df=pd.DataFrame(temp)
print(df)
    
  


    

CodePudding user response：

While error is quite clear, question and expected result is not.

The way you try to create the DataFrame has to deal with missing rows, thats why the error comes up. To fix this you could create DataFrame from dict:

pd.DataFrame.from_dict(temp, orient='index')

But that looks pretty ugly and could not processed well later on, so an alternative would be:

data = [{
    'title':temp['product']['prodBean']['name'],
    'specs':','.join([s.get('value') for s in temp['specifications']['MARKETING']])
}]
pd.DataFrame(data)

or following if you like to have each spec in a new row:

data = {
    'title':temp['product']['prodBean']['name'],
    'specs':[s.get('value') for s in temp['specifications']['MARKETING']]
}

pd.DataFrame.from_dict(data)

Example

import enum
import requests
from bs4 import BeautifulSoup
import json
import pandas as pd 

url="https://www.fleetpride.com/parts/otr-coiled-air-hose-otr6818"
headers = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.131 Safari/537.3"
}
r = requests.get(url)
soup = BeautifulSoup(r.content, "html5lib")
raw_json = ""
for table_index,table in enumerate( soup.find_all("script")):
    if('CCRZ.detailData.jsonProductData = {"' in str(table)):
        x=str(table).split('CCRZ.detailData.jsonProductData = {"')
        raw_json = "{\"" str(x[-1]).split('};')[0] "}"
        break

temp = json.loads(raw_json)

data = [{
    'title':temp['product']['prodBean']['name'],
    'specs':','.join([s.get('value') for s in temp['specifications']['MARKETING']])
}]

pd.DataFrame(data)

Output