How can i specify what output i want from my webscraper and not return every single thing-CodePudding

So i have tried alot of codes to try and get this to work. If it's not the mod sequrity blocking my request it's something else. But now finally i found a solution but the "problem" is it's returning every single request now. But the only thing that i want to return is all the names from the output.

*code

import requests


url = ('https://api-mainnet.magiceden.io/all_organizations?edge_cache=true')

headers ={
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:84.0) Gecko/20100101 Firefox/84.0',
    'Accept-Language': 'en-GB,en;q=0.5',
    'Referer': 'https://google.com',
    'DNT': '1'
    
}

r = requests.get(url, headers=headers)

print(r.text)

CodePudding user response：

You can fetch the required element by using x-path or css selectors. Beautiful soup library will be helpful on this.

Try Parsehub it will be very efficient with no code

https://www.google.com/amp/s/www.parsehub.com/blog/web-scraper-tutorial/amp/

CodePudding user response：

Since the response is a block of JSON, try this:

for item in r.json():
    print(item['name'])