Home > Enterprise >  how do i get all the href links in this div
how do i get all the href links in this div

Time：11-01

i am trying to get the links for each restaurant on the search results page for any city on food.grab.com using selenium. This is the div i need to target to get the links but when i run this

rl_elements = driver.find_elements_by_css_selector("ant-layout")

I get an error about the CSS element not being found. I was thinking I could grab that div then probably loop through or use the get("href") tag. This is the div section with the links of the restaurants

<div class="ant-layout">
   <div>
      <div class="ant-row-flex RestaurantListRow___1SbZY">
         <div class="ant-col-24 RestaurantListCol___1FZ8V  ant-col-md-12 ant-col-lg-6">
            <a href="/ph/en/restaurant/mcdonald-s-sta-cruz-church-delivery/AWiD5JwQfYWaYaQC4nY4" style="color: inherit; text-decoration: none;">
               <div class="ant-row-flex ant-row-flex-start ant-row-flex-top asList___1ZNTr">
                  <div class="ant-col-24 colPhoto___3vb-o ant-col-md-24 ant-col-lg-24">
                     <div class="promoTag___IYhfm">
                        <div class="placeholder___1xbBh restoPhoto___3nncy"><img alt="Order McDonald's - Sta Cruz Church" class="realImage___2TyNE show___3oA6B" src="https://d1sag4ddilekf6.azureedge.net/compressed/merchants/AWiD5JwQfYWaYaQC4nY4/hero/20a3b455671746a1b02a2cac627b5a67_1600322490671317355.jpg"></div>
                        <div class="promoTagHead___1bjRG">Promo</div>
                        <div class="promoTagTail___2Jy3D"></div>
                     </div>
                  </div>
                  <div class="ant-col-24 colInfo___3iLqj ant-col-md-24 ant-col-lg-24">
                     <h6 class="name___2epcT">McDonald's - Sta Cruz Church</h6>
                     <div >
                        <div >Fast Food, Burgers, American, #Combodeals, #ComboDealsBurgers</div>
                        <div >
                           <div >
                              <div  role="button" tabindex="0" style="display: inline-block; background-position: center center; background-repeat: no-repeat; background-size: contain; background-image: url(&quot;/static/images/icons/icon-star.svg&quot;);"></div>
                              4.4
                           </div>
                           <div >
                              <div  role="button" tabindex="0" style="display: inline-block; background-position: center center; background-repeat: no-repeat; background-size: contain; background-image: url(&quot;/static/images/icons/icon-clock.svg&quot;);"></div>
                              29 mins&nbsp;&nbsp;•&nbsp;&nbsp;0.8 km
                           </div>
                        </div>
                     </div>
                     <div >
                        <div  role="button" tabindex="0" style="display: inline-block; background-position: center center; background-repeat: no-repeat; background-size: contain; background-image: url(&quot;/static/images/icons/icon-promo-tag.svg&quot;);"></div>
                        <span >P150 OFF: Use code PADALOVE, min 600</span>
                     </div>
                  </div>
               </div>
            </a>
         </div>
         <div >
            <a href="/ph/en/restaurant/jollibee-raon-delivery/2-CZC1EZEYACBTCX" style="color: inherit; text-decoration: none;">
               <div >
                  <div >
                     <div >
                        <div ><img alt="Order Jollibee - Raon"  src="https://d1sag4ddilekf6.azureedge.net/compressed/merchants/2-CZC1EZEYACBTCX/hero/3189897a1a264165af3c554991970723_1634664559764732889.jpg"></div>
                     </div>
                  </div>
                  <div >
                     <h6 >Jollibee - Raon</h6>
                     <div >
                        <div >Filipino, Chicken, Fast Food</div>
                        <div >
                           <div >
                              <div  role="button" tabindex="0" style="display: inline-block; background-position: center center; background-repeat: no-repeat; background-size: contain; background-image: url(&quot;/static/images/icons/icon-star.svg&quot;);"></div>
                              4.3
                           </div>
                           <div >
                              <div  role="button" tabindex="0" style="display: inline-block; background-position: center center; background-repeat: no-repeat; background-size: contain; background-image: url(&quot;/static/images/icons/icon-clock.svg&quot;);"></div>
                              33 mins&nbsp;&nbsp;•&nbsp;&nbsp;0.1 km
                           </div>
                        </div>
                     </div>
                  </div>
               </div>
            </a>
         </div>
         <div >
            <a href="/ph/en/restaurant/mang-inasal-tayuman-delivery/2-CZCVUA42FBUTTX" style="color: inherit; text-decoration: none;">
               <div >
                  <div >
                     <div >
                        <div ><img alt="Order Mang Inasal - Tayuman"  src="https://d1sag4ddilekf6.azureedge.net/compressed/merchants/2-CZCVUA42FBUTTX/hero/40d7b143ab2846a490c52bec6b823cfe_1634227146999336525.png"></div>
                        <div >Promo</div>
                        <div ></div>
                     </div>
                  </div>
                  <div >
                     <h6 >Mang Inasal - Tayuman</h6>
                     <div >
                        <div >Filipino, Chicken, Fast Food</div>
                        <div >
                           <div >
                              <div  role="button" tabindex="0" style="display: inline-block; background-position: center center; background-repeat: no-repeat; background-size: contain; background-image: url(&quot;/static/images/icons/icon-star.svg&quot;);"></div>
                              4.5
                           </div>
                           <div >
                              <div  role="button" tabindex="0" style="display: inline-block; background-position: center center; background-repeat: no-repeat; background-size: contain; background-image: url(&quot;/static/images/icons/icon-clock.svg&quot;);"></div>
                              31 mins&nbsp;&nbsp;•&nbsp;&nbsp;2.3 km
                           </div>
                        </div>
                     </div>
                     <div >
                        <div  role="button" tabindex="0" style="display: inline-block; background-position: center center; background-repeat: no-repeat; background-size: contain; background-image: url(&quot;/static/images/icons/icon-promo-tag.svg&quot;);"></div>
                        <span >Promo Discount - Paa Large Buddy Size   Pork BBQ Buddy Size</span>
                     </div>
                  </div>
               </div>
            </a>
         </div>
         <div >
            <a href="/ph/en/restaurant/angel-s-pizza-legarda-available-for-long-distance-delivery-delivery/PHGFSTI000000zw" style="color: inherit; text-decoration: none;">
               <div >
                  <div >
                     <div >
                        <div ><img alt="Order Angel's Pizza - Legarda [Available for LONG-DISTANCE DELIVERY]" hljs-string">B" src="https://d1sag4ddilekf6.azureedge.net/compressed/merchants/PHGFSTI000000zw/hero/2166e88cc82b4a8fb3193f1b555002fe_1624797732725799594.jpg"></div>
                     </div>
                  </div>
                  <div hljs-number">24 colInfo___3iLqj ant-col-md-24 ant-col-lg-24">
                     <h6 hljs-string">">Angel's Pizza - Legarda [Available for LONG-DISTANCE DELIVERY]</h6>
                     <div hljs-string">">
                        <div hljs-string">">Quick Bites, Pizza, Italian, #ComboDealsPizza</div>
                        <div hljs-string">">
                           <div hljs-string">">
                              <div hljs-string">" role="button" tabindex="0" style="display: inline-block; background-position: center center; background-repeat: no-repeat; background-size: contain; background-image: url(&quot;/static/images/icons/icon-star.svg&quot;);"></div>
                              4.6
                           </div>
                           <div hljs-string">">
                              <div hljs-string">" role="button" tabindex="0" style="display: inline-block; background-position: center center; background-repeat: no-repeat; background-size: contain; background-image: url(&quot;/static/images/icons/icon-clock.svg&quot;);"></div>
                              43 mins&nbsp;&nbsp;•&nbsp;&nbsp;1.4 km
                           </div>
                        </div>
                     </div>
                  </div>
               </div>
            </a>
         </div>
         <div hljs-number">24 RestaurantListCol___1FZ8V  ant-col-md-12 ant-col-lg-6">
            <a href="/ph/en/restaurant/wendy-s-dapitan-available-for-long-distance-delivery-delivery/PHGFSTI0000017w" style="color: inherit; text-decoration: none;">
               <div hljs-string">r">
                  <div hljs-number">24 colPhoto___3vb-o ant-col-md-24 ant-col-lg-24">
                     <div hljs-string">">
                        <div hljs-string">"><img alt="Order Wendy's - Dapitan [Available for LONG-DISTANCE DELIVERY]"  src="https://d1sag4ddilekf6.azureedge.net/compressed/merchants/PHGFSTI0000017w/hero/9cf2d2e2170d4c288706e4ec08740199_1604489462614383246.jpeg"></div>
                        <div >Promo</div>
                        <div ></div>
                     </div>
                  </div>
                  <div >
                     <h6 >Wendy's - Dapitan [Available for LONG-DISTANCE DELIVERY]</h6>
                     <div class="basicInfoContainer___1DcNs">
                        <div class="basicInfoRow___UZM8d cuisine___T2tCh">Fast Food, Burgers, American, #ComboDealsBurgers</div>
                        <div class="basicInfoRow___UZM8d numbers___2xZGn">
                           <div class="numbersChild___2qKMV">
                              <div class="medium___3F_Er ratingStar infoItemIcon___23Zvv" role="button" tabindex="0" style="display: inline-block; background-position: center center; background-repeat: no-repeat; background-size: contain; background-image: url(&quot;/static/images/icons/icon-star.svg&quot;);"></div>
                              4.4
                           </div>
                           <div class="numbersChild___2qKMV">
                              <div class="medium___3F_Er deliveryClock infoItemIcon___23Zvv" role="button" tabindex="0" style="display: inline-block; background-position: center center; background-repeat: no-repeat; background-size: contain; background-image: url(&quot;/static/images/icons/icon-clock.svg&quot;);"></div>
                              30 mins&nbsp;&nbsp;•&nbsp;&nbsp;1.5 km
                           </div>
                        </div>
                     </div>
                     <div class="basicInfoRow___UZM8d discount___3h-0m">
                        <div class="medium___3F_Er discountIcon infoItemIcon___23Zvv" role="button" tabindex="0" style="display: inline-block; background-position: center center; background-repeat: no-repeat; background-size: contain; background-image: url(&quot;/static/images/icons/icon-promo-tag.svg&quot;);"></div>
                        <span class="discountText___GQCkj">P150 OFF: Use code PADALOVE, min 600</span>
                     </div>
                  </div>
               </div>
            </a>
         </div>
         <div class="ant-col-24 RestaurantListCol___1FZ8V  ant-col-md-12 ant-col-lg-6">
            <a href="/ph/en/restaurant/kfc-sta-cruz-delivery/2-CYUZC8BTGJ51GJ" style="color: inherit; text-decoration: none;">
               <div class="ant-row-flex ant-row-flex-start ant-row-flex-top asList___1ZNTr">
                  <div class="ant-col-24 colPhoto___3vb-o ant-col-md-24 ant-col-lg-24">
                     <div class="promoTag___IYhfm">
                        <div class="placeholder___1xbBh restoPhoto___3nncy"><img alt="Order KFC - Sta Cruz" class="realImage___2TyNE show___3oA6B" src="https://d1sag4ddilekf6.azureedge.net/compressed/merchants/2-CYUZC8BTGJ51GJ/hero/680a96e5303d4434af6c9ea5751b211e_1593015057999865942.jpeg"></div>
                        <div class="promoTagHead___1bjRG">Promo</div>
                        <div class="promoTagTail___2Jy3D"></div>
                     </div>
                  </div>
                  <div class="ant-col-24 colInfo___3iLqj ant-col-md-24 ant-col-lg-24">
                     <h6 class="name___2epcT">KFC - Sta Cruz</h6>
                     <div class="basicInfoContainer___1DcNs">
                        <div class="basicInfoRow___UZM8d cuisine___T2tCh">American, Chicken, Fast Food</div>
                        <div class="basicInfoRow___UZM8d numbers___2xZGn">
                           <div class="numbersChild___2qKMV">
                              <div class="medium___3F_Er ratingStar infoItemIcon___23Zvv" role="button" tabindex="0" style="display: inline-block; background-position: center center; background-repeat: no-repeat; background-size: contain; background-image: url(&quot;/static/images/icons/icon-star.svg&quot;);"></div>
                              4.1
                           </div>
                           <div class="numbersChild___2qKMV">
                              <div class="medium___3F_Er deliveryClock infoItemIcon___23Zvv" role="button" tabindex="0" style="display: inline-block; background-position: center center; background-repeat: no-repeat; background-size: contain; background-image: url(&quot;/static/images/icons/icon-clock.svg&quot;);"></div>
                              27 mins&nbsp;&nbsp;•&nbsp;&nbsp;0.8 km
                           </div>
                        </div>
                     </div>
                     <div class="basicInfoRow___UZM8d discount___3h-0m">
                        <div class="medium___3F_Er discountIcon infoItemIcon___23Zvv" role="button" tabindex="0" style="display: inline-block; background-position: center center; background-repeat: no-repeat; background-size: contain; background-image: url(&quot;/static/images/icons/icon-promo-tag.svg&quot;);"></div>
                        <span class="discountText___GQCkj">P150 OFF: Use code PADALOVE, min 600</span>
                     </div>
                  </div>
               </div>
            </a>
         </div>
         <div class="ant-col-24 RestaurantListCol___1FZ8V  ant-col-md-12 ant-col-lg-6">
            <a href="/ph/en/restaurant/chowking-sta-cruz-manila-delivery/2-CYMHN7DWAGL2WE" style="color: inherit; text-decoration: none;">
               <div class="ant-row-flex ant-row-flex-start ant-row-flex-top asList___1ZNTr">
                  <div class="ant-col-24 colPhoto___3vb-o ant-col-md-24 ant-col-lg-24">
                     <div class="">
                        <div class="placeholder___1xbBh restoPhoto___3nncy"><img alt="Order Chowking - Sta Cruz Manila" class="realImage___2TyNE show___3oA6B" src="https://d1sag4ddilekf6.azureedge.net/compressed/merchants/2-CYMHN7DWAGL2WE/hero/370cba7acb8645d99e045d93121555cd_1632276886062046084.jpg"></div>
                     </div>
                  </div>
                  <div class="ant-col-24 colInfo___3iLqj ant-col-md-24 ant-col-lg-24">
                     <h6 class="name___2epcT">Chowking - Sta Cruz Manila</h6>
                     <div class="basicInfoContainer___1DcNs">
                        <div class="basicInfoRow___UZM8d cuisine___T2tCh">Chinese, Chicken, Fast Food</div>
                        <div class="basicInfoRow___UZM8d numbers___2xZGn">
                           <div class="numbersChild___2qKMV">
                              <div class="medium___3F_Er ratingStar infoItemIcon___23Zvv" role="button" tabindex="0" style="display: inline-block; background-position: center center; background-repeat: no-repeat; background-size: contain; background-image: url(&quot;/static/images/icons/icon-star.svg&quot;);"></div>
                              4.5
                           </div>
                           <div class="numbersChild___2qKMV">
                              <div class="medium___3F_Er deliveryClock infoItemIcon___23Zvv" role="button" tabindex="0" style="display: inline-block; background-position: center center; background-repeat: no-repeat; background-size: contain; background-image: url(&quot;/static/images/icons/icon-clock.svg&quot;);"></div>
                              31 mins&nbsp;&nbsp;•&nbsp;&nbsp;0.6 km
                           </div>
                        </div>
                     </div>
                  </div>
               </div>
            </a>
         </div>
         <div class="ant-col-24 RestaurantListCol___1FZ8V  ant-col-md-12 ant-col-lg-6">
            <a href="/ph/en/restaurant/greenwich-binondo-delivery/2-CYKTRUEHNAUGEX" style="color: inherit; text-decoration: none;">
               <div class="ant-row-flex ant-row-flex-start ant-row-flex-top asList___1ZNTr">
                  <div class="ant-col-24 colPhoto___3vb-o ant-col-md-24 ant-col-lg-24">
                     <div class="promoTag___IYhfm">
                        <div class="placeholder___1xbBh restoPhoto___3nncy"><img alt="Order Greenwich - Binondo" class="realImage___2TyNE show___3oA6B" src="https://d1sag4ddilekf6.azureedge.net/compressed/merchants/2-CYKTRUEHNAUGEX/hero/upload-photo-Hero_Photo_66bf96e0fcff47da8b6aacb925f913a3_1554863645049377072.jpeg"></div>
                        <div class="promoTagHead___1bjRG">Promo</div>
                        <div class="promoTagTail___2Jy3D"></div>
                     </div>
                  </div>
                  <div class="ant-col-24 colInfo___3iLqj ant-col-md-24 ant-col-lg-24">
                     <h6 class="name___2epcT">Greenwich - Binondo</h6>
                     <div class="basicInfoContainer___1DcNs">
                        <div class="basicInfoRow___UZM8d cuisine___T2tCh">American, Pizza, Fast Food</div>
                        <div class="basicInfoRow___UZM8d numbers___2xZGn">
                           <div class="numbersChild___2qKMV">
                              <div class="medium___3F_Er ratingStar infoItemIcon___23Zvv" role="button" tabindex="0" style="display: inline-block; background-position: center center; background-repeat: no-repeat; background-size: contain; background-image: url(&quot;/static/images/icons/icon-star.svg&quot;);"></div>
                              4.5
                           </div>
                           <div class="numbersChild___2qKMV">
                              <div class="medium___3F_Er deliveryClock infoItemIcon___23Zvv" role="button" tabindex="0" style="display: inline-block; background-position: center center; background-repeat: no-repeat; background-size: contain; background-image: url(&quot;/static/images/icons/icon-clock.svg&quot;);"></div>
                              34 mins&nbsp;&nbsp;•&nbsp;&nbsp;1.6 km
                           </div>
                        </div>
                     </div>
                     <div class="basicInfoRow___UZM8d discount___3h-0m">
                        <div class="medium___3F_Er discountIcon infoItemIcon___23Zvv" role="button" tabindex="0" style="display: inline-block; background-position: center center; background-repeat: no-repeat; background-size: contain; background-image: url(&quot;/static/images/icons/icon-promo-tag.svg&quot;);"></div>
                        <span class="discountText___GQCkj">Lasagna Supreme Flash Sale: two (2) Greenwich Lasagna Supreme Small Pans for only P399!</span>
                     </div>
                  </div>
               </div>
            </a>
         </div>
      </div>
      <button type="button" class="ant-btn ant-btn-block"><span>Load More</span></button>
   </div>
</div>

I am currently using this, but it gets all the links on the search result page, i want the ones in the div above.

url_elements = driver.find_elements_by_xpath("//a[contains(@href, '/ph/en/restaurant')]") 
url_elements_list = []
for url_element in url_elements:
    url_elements_list.append(url_element.get_attribute("href"))
sliced_list=url_elements_list[:2]
print(sliced_list)

CodePudding user response：

Use those line of code Edited:

url_elements_list = []
for url_element in driver.find_elements_by_class_name('ant-layout')::
    url_elements_list.append(url_element.get_attribute("href"))
    print(url_elements_list)

CodePudding user response：

To get the restaurants elements inside the specific div element you should do as following:

  1. Apply find_elements_by_xpath method on rl_elements web element instead of applying it on web driver instance driver.
  2. The XPath locator should start with a dot . meaning "search from this node, not form the beginning of the whole page.
rl_elements = driver.find_elements_by_css_selector("ant-layout")
url_elements = rl_elements.find_elements_by_xpath(".//a[contains(@href, '/ph/en/restaurant')]") 
url_elements_list = []
for url_element in url_elements:
    url_elements_list.append(url_element.get_attribute("href"))
sliced_list=url_elements_list[:2]
print(sliced_list)

Page link：https//www.codepudding.com/Enterprise/174386.html

Prev:Is there a way to get line-by-line console output and print it in a separate script?
Next:Why does axios post method stop working after 6 consecutive post calls?
  • Related

About Us:  Contact Us      Terms of Service       Privacy Policy

Copyright © 2010-2023，Powered By CodePudding