I'm trying to import all the tickers(security symbols) in a single array from the following site: https://indexes.nasdaqomx.com/Index/Weighting/NQUSS

I have found 2 main problems:

1. I had to switch page every time so that I cannot import all the values in a single array

2. I'm not sure if elements are written correctly (elements should be the single array where all the 2000 security symbols are stored

this is the code I have written

from builtins import print, input, int
from selenium import webdriver as wd

wd = wd.Chrome()
wd.implicitly_wait(10)
wd.get('https://indexes.nasdaqomx.com/Index/Weighting/NQUSS')

#this for allow the code to skip to the next page

for n in range(0,22):

#ticker is the xpath referring to the column of the security symbol

    ticker = wd.find_elements_by_xpath("/html/body/div[1]/div/div/section[2]/div/div/div[1]/table/tbody/tr/td[3]")
    try:
        #this for should take all value in the current page and store them in the list elements

        for i in range(0,100):
            elements = [elem.text for elem in ticker]
    finally:
    #this line is used to go the next page
        wd.find_element_by_xpath('/html/body/div[1]/div/div/section[2]/div/div/div[1]/div[1]/a[2]/img').click()

#at the end it should print all the values contained in the single array called elements

for i in range(0,3000):
    print(elements[i].text)

The end result should be a single array with all the security symbols stored as strings so that I can interact with them in my code, the array should be the list of Elements. Thanks in advance

CodePudding user response：

In order to put all the texts in the common array you need to define that array out of the loop.
Also, you can easily improve locators you are using here.
Also, there is no need to apply .text on elements[i] when you finally printing the collected data since you are already collected texts, not web elements.
This should work better:

from builtins import print, input, int
from selenium import webdriver as wd

wd = wd.Chrome()
wd.implicitly_wait(10)
wd.get('https://indexes.nasdaqomx.com/Index/Weighting/NQUSS')

#this for allow the code to skip to the next page
elements = []
for n in range(0,22):

#ticker is the xpath referring to the column of the security symbol

    tickers = wd.find_elements_by_xpath("//table[@id='weightingsTable']//td[3]")
    try:
        #this for should take all value in the current page and store them in the list elements

        for el in tickers:
            elements.append(el.text)
    finally:
    #this line is used to go the next page
        wd.find_element_by_xpath("//a[@class='paginate_button next']").click()

#at the end it should print all the values contained in the single array called elements

for i in range(0,3000):
    print(elements[i])

I would also advice here to use Expected Conditions explicit waits instead of implicit waits and not to use predefined hardcoded for ranges like for i in range(0,3000)
This will be better

for idx, val in enumerate(elements):
    print(elements[idx])