I am trying to scrape data from seetickets.us. I am clicking on each org and then all events by that org. The scraper correctly scrape data from each event but the problem is that when I come back to all events page web driver cannot find the css selector.

Here is the site structure:

https://ibb.co/WBjMDJf

clicking on World Cafe Live get me here:

https://ibb.co/cLbMP19

clicking on any event will move me toward further info about event.

Now when the driver is coming back from extracting each event , It is not able to go into other event. I have also tried explicit wait anf time.sleep()

Here is my code:

   #this is the func click on each event and extract data then come back to all event page
   def get_all_events_in_each_event(self):
        inner_events  = self.get_all_inner_events()
        print(len(inner_events))
        for event in inner_events:

            self.click_inner_event(event)
            self.get_event_loc()
            self.get_talent()
            self.get_facebook()
            self.get_date()
            self.get_showtime_city()
            self.get_ticket_doors()
            self.back()
            try:
                WebDriverWait(self, 10).until(
                    EC.element_to_be_clickable((By.CLASS_NAME, "event-images-box")))
            except Exception as e:
                print("Wait Timed out")
                print(e) 
   
#this is the func to click on each event in all event pages
def click_inner_event(self , inner_event):
        
        link = inner_event.find_element_by_css_selector('div[]')
        link.click()

Here is HTML of all events page: https://ibb.co/wcKWc68

Kindly help me with finding what's wrong here. Thanks

CodePudding user response：

As @Arundeep Chohan , correctly pointed that web driver loses reference when moving back and forth so I had to re grab all the elements. Correct code is:

def get_all_events_in_each_event(self):
        inner_events  = self.get_all_inner_events()
        
        for i in range(len(inner_events)):
            
            self.click_inner_event(inner_events[i])
            self.get_event_loc()
            self.get_talent()
            self.get_facebook()
            self.get_date()
            self.get_showtime_city()
            self.get_ticket_doors()
            self.back()
            inner_events = self.get_all_inner_events() #regrabbing the elements

Thanks arundeep for the answer.