I am trying to pull all the urls that contain "https://play.google.com/store/" and print the entire string. When I run my current code, it only prints "https://play.google.com/store/" but I am looking for the entire URL. Can someone point me in the right direction? Here is my code:
import pandas as pd
import os
import requests
from bs4 import BeautifulSoup
import re
URL = "https://www.pocketgamer.com/android/best-tycoon-games-android/?page=3"
page = requests.get(URL)
soup = BeautifulSoup(page.text, "html.parser")
links = []
for link in soup.findAll("a", target="_blank"):
links.append(link.get('href'))
x = re.findall("https://play.google.com/store/", str(links))
print(x)
CodePudding user response:
re.findall
just returns the part of the text that matches the regex, so all you are getting is the https://play.google.com/store/
that is in the regex. You could modify the regex, but given what you are searching is a list of links, it's easier to just check if they start with https://play.google.com/store/
. For example:
x = [link for link in links if link.startswith('https://play.google.com/store/')]
Output (for your query):
[
'https://play.google.com/store/apps/details?id=com.auxbrain.egginc',
'https://play.google.com/store/apps/details?id=net.kairosoft.android.gamedev3en',
'https://play.google.com/store/apps/details?id=com.pixodust.games.idle.museum.tycoon.empire.art.history',
'https://play.google.com/store/apps/details?id=com.AdrianZarzycki.idle.incremental.car.industry.tycoon',
'https://play.google.com/store/apps/details?id=com.veloxia.spacecolonyidle',
'https://play.google.com/store/apps/details?id=com.uplayonline.esportslifetycoon',
'https://play.google.com/store/apps/details?id=com.codigames.hotel.empire.tycoon.idle.game',
'https://play.google.com/store/apps/details?id=com.mafgames.idle.cat.neko.manager.tycoon',
'https://play.google.com/store/apps/details?id=com.atari.mobile.rctempire',
'https://play.google.com/store/apps/details?id=com.pixodust.games.rocket.star.inc.idle.space.factory.tycoon',
'https://play.google.com/store/apps/details?id=com.idlezoo.game',
'https://play.google.com/store/apps/details?id=com.fluffyfairygames.idleminertycoon',
'https://play.google.com/store/apps/details?id=com.boomdrag.devtycoon2',
'https://play.google.com/store/apps/details?id=com.TomJarStudio.GamingShop2D',
'https://play.google.com/store/apps/details?id=com.roasterygames.smartphonetycoon2'
]