How can I get more than 20 youtube search results using urllib?-CodePudding

I am trying to make a python script that fetches youtube channel URLs with a keyword input. I'm using urlib to request the html of the search result page and then filtering out the channel IDs using RE. I can't seem to find a way to get the script to fetch more than 20 results. Can anyone help me out here? Here's the code so far

import urllib.request
import re

search_keyword = input("Search Keyword \n")
html = urllib.request.urlopen("https://www.youtube.com/results?search_query="   search_keyword   "&sp=EgIQAg%3D%3D")
regex = r"\"channelId\": (\S{24})"
#print(html.read().decode())
ids = re.findall(r"\"channelId\"\:(\S{25})", html.read().decode())
idsLen = len(ids)
for i in range(idsLen):
    ids[i] = ids[i][1:]
    ids[i] = "https://www.youtube.com/channel/"   ids[i]
    print(ids[i])

I have looked up the urllib APIs to find something to get the job done, but I can't find anything related to it. I'm expecting someone can tell me how this can be achieved, with or without urllib.

CodePudding user response：

For more results you need to implement in some way pagination, as you are working with the YouTube UI, as I did in my open-source API.

Note that you are anyway limited to 500 results, if you use YouTube Data API v3 Search: list endpoint or the YouTube UI.

CodePudding user response：

Can you try fetching two times? Maybe there's a limit.