Home > Back-end >  Regex issue in Python printing entire URL
Regex issue in Python printing entire URL

Time:05-20

I am trying to pull all the urls that contain "https://play.google.com/store/" and print the entire string. When I run my current code, it only prints "https://play.google.com/store/" but I am looking for the entire URL. Can someone point me in the right direction? Here is my code:

import pandas as pd
import os
import requests
from bs4 import BeautifulSoup
import re



URL = "https://www.pocketgamer.com/android/best-tycoon-games-android/?page=3"
page = requests.get(URL)
soup = BeautifulSoup(page.text, "html.parser")

links = []
for link in soup.findAll("a", target="_blank"):
    links.append(link.get('href'))

x = re.findall("https://play.google.com/store/", str(links))
print(x)

CodePudding user response:

re.findall just returns the part of the text that matches the regex, so all you are getting is the https://play.google.com/store/ that is in the regex. You could modify the regex, but given what you are searching is a list of links, it's easier to just check if they start with https://play.google.com/store/. For example:

x = [link for link in links if link.startswith('https://play.google.com/store/')]

Output (for your query):

[
 'https://play.google.com/store/apps/details?id=com.auxbrain.egginc',
 'https://play.google.com/store/apps/details?id=net.kairosoft.android.gamedev3en',
 'https://play.google.com/store/apps/details?id=com.pixodust.games.idle.museum.tycoon.empire.art.history',
 'https://play.google.com/store/apps/details?id=com.AdrianZarzycki.idle.incremental.car.industry.tycoon',
 'https://play.google.com/store/apps/details?id=com.veloxia.spacecolonyidle',
 'https://play.google.com/store/apps/details?id=com.uplayonline.esportslifetycoon',
 'https://play.google.com/store/apps/details?id=com.codigames.hotel.empire.tycoon.idle.game',
 'https://play.google.com/store/apps/details?id=com.mafgames.idle.cat.neko.manager.tycoon',
 'https://play.google.com/store/apps/details?id=com.atari.mobile.rctempire',
 'https://play.google.com/store/apps/details?id=com.pixodust.games.rocket.star.inc.idle.space.factory.tycoon',
 'https://play.google.com/store/apps/details?id=com.idlezoo.game',
 'https://play.google.com/store/apps/details?id=com.fluffyfairygames.idleminertycoon',
 'https://play.google.com/store/apps/details?id=com.boomdrag.devtycoon2',
 'https://play.google.com/store/apps/details?id=com.TomJarStudio.GamingShop2D',
 'https://play.google.com/store/apps/details?id=com.roasterygames.smartphonetycoon2'
]
  • Related