The problem here is i have a "show more" with infinite scroll, i need to detect when the button opacity becomes 0, or in other words when the button becomes not visible so i can use pagination until the button becomes invisible, which means there's no more info to scrape, i'm talking about over 200 pages to scrape, and no other way to know how many pages are there and use range to iterate the pages, so i need to get the trigger of the opacity attribute. Since the button is always there i can't just use is not None to find the last page.

The button is like this when there are more pages:

<a class="btn btn-primary btn-lg MaisArtigos">SHOW MORE</a>

it turns to this on the last page

<a class="btn btn-primary btn-lg MaisArtigos" style="opacity: 0;">SHOW MORE</a>

I need to trigger when the style attribute is present

I'm using BeautifulSoup and requests library

CodePudding user response：

you can use the Element.getAttribute()

    const div1 = document.getElementById('div1');
    const exampleAttr= div1.getAttribute('style');
    console.log("the attribute is: "   exampleAttr)

    <div id="div1" style="opacity:0;">Hi Champ!</div>

<iframe name="sif1" sandbox="allow-forms allow-modals allow-scripts" frameborder="0"></iframe> well you may want to add an event listener with "scroll" to trigger the get attribute, also if the element has no "style" attribute it will return a NULL, exampleAttr = NULL;

CodePudding user response：

Getting the attribute you can use the css selector:

soup.select_one('a.MaisArtigos[style="opacity: 0;"]')

Checking if the style is set you can use bool() that will give you True if exists and False while not existing:

bool(soup.select_one('a.MaisArtigos[style="opacity: 0;"]'))

Example

To demonstrate, this will only select the styled <a>.

from bs4 import BeautifulSoup

html='''
<a  style="opacity: 0;">SHOW MORE</a>
<a  style="opacity: 100;">SHOW MORE</a>
<a  >SHOW MORE</a>
'''

soup=BeautifulSoup(html,'lxml')

soup.select('a.MaisArtigos[style="opacity: 0;"]')