NEW TO PYTHON*** Below is my code I am using to pull a zip file from a website but I am getting the error, "list index out of range". I was given this code by someone else who wrote it but I had to change the URL and now I am getting the error. When I print(list_of_documents) it is blank.

Can someone help me with this? The url requires access so you won't be able to try to input this code directly. I am trying to understand how to use beautiful soup in this and how I can get the list to populate correctly.

import datetime
import requests
import csv
from zipfile import ZipFile as zf
import os
import pandas as pd
import time
from bs4 import BeautifulSoup
import pyodbc
import re

#set download location

downloads_folder = r"C:\Scripts\"


##### Creating outage dataframe

#Get list of download links

res = requests.get('https://www.ercot.com/mp/data-products/data-product-details?id=NP3-233-CD')

ercot_soup = BeautifulSoup(res.text, "lxml")

list_of_documents = ercot_soup.findAll('td', attrs={'class': 'labelOptional_ind'})
list_of_links = ercot_soup.select('a')'

##create the url for the download 

loc = str(list_of_links[0])[9:len(str(list_of_links[0]))-9]
link = 'http://www.ercot.com'   loc
link = link.replace('amp;','')

# Define file name and set download path

file_name = str(list_of_documents[0])[30:len(str(list_of_documents[0]))-5]
file_path = downloads_folder   '/'   file_name

CodePudding user response：

You can't expect code tailored to scrape one website to work for a different link! You should always