Get JSON response from GraphQL queries-CodePudding

I need to get some data from this subgraph that uses GraphQL: https://v4.subgraph.polygon.oceanprotocol.com/subgraphs/name/oceanprotocol/ocean-subgraph/graphql

In order to achieve this, I run a script from selenium that execute the following query: https://v4.subgraph.polygon.oceanprotocol.com/subgraphs/name/oceanprotocol/ocean-subgraph/graphql?query={ poolSnapshots(orderBy: baseTokenLiquidity, orderDirection: desc, block: {number: 29357051}) { id baseTokenLiquidity } }

And then convert the output shown in the div placed on the right from text to JSON. Do you know a lighter/faster method than selenium that allow me to get the same output? I would use BS but be aware that sometimes in order to get the entire response I need to scroll the output-div.

This is the code I actually use:

def find_datasets():
    datasets = []

    s=Service(ChromeDriverManager().install())
    options = Options()
    options.headless = True
    options.add_argument("--start-maximized")
    options.add_argument('window-size=2560,1440')
    driver = webdriver.Chrome(service=s, options=options)
    urls = ["https://v4.subgraph.polygon.oceanprotocol.com/subgraphs/name/oceanprotocol/ocean-subgraph/graphql?query={
  pools(orderBy: createdTimestamp, orderDirection: desc) {
    id
    datatoken {
      address
    }
    publishMarketSwapFee
    liquidityProviderSwapFee
  }
}
", "https://v4.subgraph.mainnet.oceanprotocol.com/subgraphs/name/oceanprotocol/ocean-subgraph/graphql?query={
  pools(orderBy: createdTimestamp, orderDirection: desc) {
    id
    datatoken {
      address
    }
    publishMarketSwapFee
    liquidityProviderSwapFee
  }
}
"]

    
    for url in urls:
        driver.get(url)
        sleep(20)
        
        driver.find_element(by=By.XPATH, value="//button[contains(@class, 'execute-button')]").click()
        sleep(20)
        
        polygon = json.loads(driver.find_element(by=By.XPATH, value="//div[contains(@class, 'result-window')]").text.replace('\n,', '\n'))
        print(polygon)
        for pool in polygon['data']['pools']:
            if url == urls[0]:
                datasets.extend([[pool['id'],  pool['datatoken']['address'], 'polygon', pool["liquidityProviderSwapFee"], pool["publishMarketSwapFee"]]])
            if url == urls[1]:
                datasets.extend([[pool['id'], pool['datatoken']['address'], 'ethereum', pool["liquidityProviderSwapFee"], pool["publishMarketSwapFee"]]])

    return datasets

CodePudding user response：

You don't need selenium. You can call GraphQL directly.

import requests
data = '{"query":"{poolSnapshots(orderBy: baseTokenLiquidity, orderDirection: desc, block: {number: 29357051}) {id baseTokenLiquidity}}","variables":null,"operationName":null}'
response = requests.post('https://v4.subgraph.polygon.oceanprotocol.com/subgraphs/name/oceanprotocol/ocean-subgraph', data=data)

for pool in response.json()['data']['poolSnapshots']:
    print("ID: "   pool['id']   " Token: "   pool['baseTokenLiquidity'])

OUTPUT:

ID: 0x95f57249e6dd394318025068a8bfc841ac6ec0dd-1654732800 Token: 10554.045809993899394931
ID: 0xa6fe1df0e3ae0aa0d84f4beb0c626add65d971b1-1654732800 Token: 1961.544133255355556327
ID: 0xa6fe1df0e3ae0aa0d84f4beb0c626add65d971b1-1654646400 Token: 1024