Python API calls return same value despite page changes, Postman does not-CodePudding

I've been beating my head over this one for a few days now.

I'm making API calls to a webstore that has multiple pages of item listings.

If I iterate through API calls per page in Python I get duplicates of the response. I don't get that in Postman.

In other words... Lets say the webstore has 5 pages of listings

If I iterate through in Python I get this:

Response for Page 1:
Item1: 'Book: Python For Idiots'
Item2: 'Book: Why don't my API calls work? An exercise in frustration'
Item3: 'Book: Talking you off the ledge, a guide to debugging'

Response for Page 2:
Item1: 'Book: Python For Idiots'
Item2: 'Book: Why don't my API calls work? An exercise in frustration'
Item3: 'Book: Talking you off the ledge, a guide to debugging'

and so on, Unless I put in a large delay... Like 20 seconds.

In Postman, I get....

Response for Page 1:
Item1: 'Book: Python For Idiots'
Item2: 'Book: Why don't my API calls work? An exercise in frustration'
Item3: 'Book: Talking you off the ledge, a guide to debugging'

Response for Page 2:
Item1: 'Book: Zen and the Art of Working Code'
Item2: 'Book: I should have been a business major, A programmers memoir'
Item3: 'Book: Curse,Cry,Code'

Despite sending a request roughly every 2 seconds

Below is my code. It's ugly for right now cause I've been debugging, trying different things, deleting and adding etc. etc.

def ListingAPIQuery(JsonData):
    headers = CaseInsensitiveDict()
    headers["Content-Type"] = "application/json"
    url = "https://fakewebstore.com/api/Search/ItemLIsting"
    resp = requests.post(url, headers=headers, data=JsonData)
    return resp.json()

def main():
    filePath = 'ListingQueryTemplate.json'
    with open(filePath, 'r') as file:
        QT = json.load(file)

    NumberOfListings = ListingAPIQuery(QT)['searchResults']['itemCount']

    NumberOfPages = NumberOfListings / 40

    QTs = []

    for x in range(1, int(NumberOfPages   1)):
        QT['page'] = x
        QTs.append(QT.copy())

    ListingList = []

    for x in range(1, int(NumberOfPages   1)):
        ListingList.append(ListingAPIQuery(QTs[x])['searchResults']['items'])

I should add the "Lets say it has 5 pages" was an example. It really has like 5000, so adding a 20 second timeout is not really a good way to go about it.

I'm fairly newish to Python so I wouldn't be surprised if it's a quirk in the language that I'm messing up on.

Any help or a point in the right direction would be a Godsend.

CodePudding user response：

This is a guess that you need to keep some sort of cookie information for the api.

session = None
def ListingAPIQuery(JsonData):
    headers = CaseInsensitiveDict()
    headers["Content-Type"] = "application/json"
    url = "https://fakewebstore.com/api/Search/ItemLIsting"
    resp = session.post(url, headers=headers, data=JsonData)
    return resp.json()

def main():
    global session
    session = requests.Session()
    filePath = 'ListingQueryTemplate.json'
    with open(filePath, 'r') as file:
        QT = json.load(file)

    NumberOfListings = ListingAPIQuery(QT)['searchResults']['itemCount']

    NumberOfPages = NumberOfListings / 40

    QTs = []

    for x in range(1, int(NumberOfPages   1)):
        QT['page'] = x
        QTs.append(QT.copy())

    ListingList = []

    for x in range(1, int(NumberOfPages   1)):
        ListingList.append(ListingAPIQuery(QTs[x])['searchResults']['items'])

CodePudding user response：

I figured it out. So I figured I'd share in case anyone stumbles on this.

Looking deeper into the Python Requests library I found this....

Any dictionaries that you pass to a request method will be merged with the session-level values that are set. The method-level parameters override session parameters.

Requests Advanced Usage

It's under the sessions objects portion.

The way I read that was... If I pass the method a dictionary it just keeps the values I already set the first time. So it stays on page 1.

So instead of using this as my call

ListingAPIQuery(QTs[x])

I changed it to

ListingAPIQuery(json.dumps(QTs[x]))

to pass the JSON string instead.

Moral of the Story.... Dig into the documentation.