Home > Blockchain >  List index out of range error when using Pandas and Yahoo_fin
List index out of range error when using Pandas and Yahoo_fin

Time:04-20

This is a modified version of a program from a tutorial that extracts data from all of the stocks in the S&P 500 and picks stocks that match the criteria you specify.

The issue is that when I run the program List index out of range [stock symbol] pops up and those stocks are skipped and aren't added to the final CSV file.

Example:

list index out of range for ABMD
list index out of range for ABT
list index out of range for ADBE
list index out of range for ADI

I'm not really sure what the issue is, I would greatly appreciate it if someone would explain it to me! Also, I am not applying any of the specifying criteria yet and am just trying to get all of the stock data into the CSV file. Make sure to create a database named stock_data if you try the program. Thanks!

My code:

import pandas_datareader as web
import pandas as pd
from yahoo_fin import stock_info as si
import datetime as dt


dow_list = si.tickers_dow()
sp_list = si.tickers_sp500()
tickers = sp_list
'''tickers = list(set(tickers))
tickers.sort()'''

start = dt.datetime.now() - dt.timedelta(days=365)
end = dt.datetime.now()

sp500_df = web.DataReader('^GSPC', 'yahoo', start, end)
sp500_df['Pct Change'] = sp500_df['Adj Close'].pct_change()
sp500_return = (sp500_df['Pct Change']   1).cumprod()[-1]

return_list = []
final_df = pd.DataFrame(columns=['Ticker', 'Latest_Price', 'Score', 'PE_Ratio', 'PEG_Ratio', 'SMA_150', 'SMA_200', '52_Week_Low', '52_Week_High'])

counter = 0


for ticker in tickers:
    df = web.DataReader(ticker, 'yahoo', start, end)
    df.to_csv(f'stock_data/{ticker}.csv')

    df['Pct Change'] = df['Adj Close'].pct_change()
    stock_return = (df['Pct Change']   1).cumprod()[-1]

    returns_compared = round((stock_return / sp500_return), 2)
    return_list.append(returns_compared)

    counter  = 1
    if counter == 100:
        break

best_performers = pd.DataFrame(list(zip(tickers, return_list)), columns=['Ticker', 'Returns Compared'])
best_performers['Score'] = best_performers['Returns Compared'].rank(pct=True) * 100
best_performers = best_performers[best_performers['Score'] >= best_performers['Score'].quantile(0)]  #picks stocks in top 25 percentile


for ticker in best_performers['Ticker']:
    try:
        df = pd.read_csv(f'stock_data/{ticker}.csv', index_col=0)
        moving_averages = [150, 200]
        for ma in moving_averages:
            df['SMA_'   str(ma)] = round(df['Adj Close'].rolling(window=ma).mean(), 2)

        latest_price = df['Adj Close'][-1]
        pe_ratio = float(si.get_quote_table(ticker)['PE Ratio (TTM)'])
        peg_ratio = float(si.get_stats_valuation(ticker)[1][4])
        moving_average_150 = df['SMA_150'][-1]
        moving_average_200 = df['SMA_200'][-1]
        low_52week = round(min(df['Low'][-(52*5):]), 2)
        high_52week = round(min(df['High'][-(52 * 5):]), 2)
        score = round(best_performers[best_performers['Ticker'] == ticker]['Score'].tolist()[0])

        condition_1 = latest_price > moving_average_150 > moving_average_200
        condition_2 = latest_price >= (1.3 * low_52week)
        condition_3 = latest_price >= (0.75 * high_52week)
        condition_4 = pe_ratio < 25
        condition_5 = peg_ratio < 2

        final_df = final_df.append({'Ticker': ticker,
                                    'Latest_Price': latest_price,
                                    'Score': score,
                                    'PE_Ratio': pe_ratio,
                                    'PEG_Ratio': peg_ratio,
                                    'SMA_150': moving_average_150,
                                    'SMA_200': moving_average_200,
                                    '52_Week_Low': low_52week,
                                    '52_Week_High': high_52week}, ignore_index=True)
    except Exception as e:
        print(f"{e} for {ticker}")


final_df.sort_values(by='Score', ascending=False)
pd.set_option('display.max_columns', 10)
print(final_df)
final_df.to_csv('final.csv')

CodePudding user response:

I have done the error shooting on your behalf. As a conclusion, I see that you have not checked the contents of the acquisition of the individual indicator data. They are being added to the dictionary format and empty data frames as they are in index and named series. I believe that is the root cause of the error.

  1. Specifying the last data and retrieving the values
  2. iloc is not used.
  3. 52*5 lookbacks for 253 data

In addition, when additional indicators are acquired for the acquired issue data, there are cases where they can be acquired for the same issue, and cases where they cannot. (The cause is unknown.) Therefore, it may be necessary to change the method of processing pe_ratio and peg_ratio after obtaining them in advance.

for ticker in best_performers['Ticker']:
    #print(ticker)
    try:
        df = pd.read_csv(f'stock_data/{ticker}.csv')#, index_col=0

        moving_averages = [150, 200]
        for ma in moving_averages:
            df['SMA_'   str(ma)] = round(df['Adj Close'].rolling(window=ma).mean(), 2)

        latest_price = df['Adj Close'][-1:].values[0]
        pe_ratio = float(si.get_quote_table(ticker)['PE Ratio (TTM)'])
            
        moving_average_150 = df['SMA_150'][-1:].values[0]
        moving_average_200 = df['SMA_200'][-1:].values[0]

        low_52week = round(min(df['Low'][-(52*1):]), 2)
        high_52week = round(min(df['High'][-(52*1):]), 2)
        #print(low_52week, high_52week)
        score = round(best_performers[best_performers['Ticker'] == ticker]['Score'].tolist()[0])
        #print(score)
        #print(ticker, latest_price,score,pe_ratio,moving_average_200,low_52week,high_52week)
        final_df = final_df.append({'Ticker': ticker,
                                    'Latest_Price': latest_price,
                                    'Score': score,
                                    'PE_Ratio': pe_ratio,
                                    'SMA_150': moving_average_150,
                                    'SMA_200': moving_average_200,
                                    '52_Week_Low': low_52week,
                                    '52_Week_High': high_52week}, ignore_index=True)
        #print(final_df)
    except Exception as e:
        print(f"{e} for {ticker}")

final_df
    Ticker  Latest_Price    Score   PE_Ratio        SMA_150     SMA_200     52_Week_Low     52_Week_High
0   A   123.839996  40  31.42       147.26  150.31  123.06  126.75
1   AAP     218.250000  70  22.23       220.66  216.64  190.79  202.04
2   AAPL    165.070007  80  29.42       161.85  158.24  150.10  154.12
3   ABC     161.899994  90  21.91       132.94  129.33  132.00  137.79
4   ADBE    425.470001  10  42.46       552.19  571.99  407.94  422.38

Note Some stocks are missing because additional indicators could not be obtained. (tickers = sp_list[:10] tested on the first 10)

  • Related