The command below loops through a file which contains activity ID numbers, and sends a request that checks what members attended this activity. It's a PowerShell command from within a batch file.

Powershell "Get-Content .\idsonly.txt | ForEach-Object {Invoke-WebRequest -Method Get -Headers @{ \"api-version\" = \"2\" ; \"Authorization\" = \"Bearer %token%\"} -Uri %uri%/$_/Members?pagesize=100`&fields=memberid`,hasattended | Select-Object -Expand Content}" ">> memberssact.txt"

The problem is that some activities have more than 100 members attending, and this will only give met the first 100 members for each activity. How do I make it ask for additional pages, if there is one? The maximum number of results that may be returned is 100 and this is a server-side limitation.

Possibilities (but I don't know how to implement them):

1. I have a list of the activity IDs next to the number of attendees, from which I can work out how many pages the list of attendees will be.

2. There is a "next page url" in the response header if there are more pages (but how do I tell PowerShell to GET it, if it's there?)

3. Least favourite, but could just brute-force increasing the number of the page requested and tell it to stop when no response received, but not sure how.

I have provided a sample response header below:

{
  "access-control-expose-headers": "Request-Context",
  "cache-control": "no-cache",
  "content-length": "2",
  "content-security-policy": "default-src 'self' ;base-uri 'self';  object-src 'none'; connect-src 'self'; worker-src 'none'; upgrade-insecure-requests; frame-ancestors 'self'; child-src 'self'; frame-src 'self'; manifest-src 'self'; prefetch-src 'self'; ; font-src 'self' fonts.gstatic.com data:; img-src 'self' data: *.ac.uk *.swagger.io; media-src 'self' ; script-src 'self' 'unsafe-inline' 'report-sample'; style-src 'self' fonts.googleapis.com 'unsafe-inline' 'report-sample' https://fonts.googleapis.com; report-uri /log",
  "content-type": "application/json; charset=utf-8",
  "date": "Mon, 23 May 2022 15:37:06 GMT",
  "expires": "0",
  "last-modified": "Mon, 23 May 2022 15:37:07 GMT",
  "pragma": "no-cache",
  "request-context": "appId=cid-v1:3abf1c0e-2963-49e7-bc48-18ed8b04daa3",
  "strict-transport-security": "max-age=31536000",
  "x-content-type-options": "nosniff",
  "x-frame-options": "DENY",
  "x-pagination": "{\"currentPage\":3,\"pageSize\":100,\"totalCount\":178,\"totalPages\":2,\"previousPageLink\":\"https://url.uk/activities/597850/students?page=2&pageSize=100&sort=hasattended&fields=studentID,hasattended\",\"nextPageLink\":\"\"}",
  "x-xss-protection": "1; mode=block"
}

CodePudding user response：

Since you've provided a sample response header, I've updated the code sample below as well as provided an explanation for how it works:

Note: This example is not invoking PowerShell as a one-liner. You should move this piece to a script with a parameter for each %VARIABLE% you use in your original command. $Token and $Uri are the variables I've changed to a PowerShell format. I have also removed otherwise-unnecessary string escapes. This could be modified to be a one-liner of course, but for any real scripting logic it should be part of its own script, function, etc.

Param(
  [string]$Token,
  [string]$Uri,
  [string]$IdsFile
)

$ids = Get-Content $idsFile

foreach( $id in $ids ) {

  $nextUri = "$Uri/$id/Members?pagesize=100&fields=memberid,hasattended"

  while( $nextUri ) {
    $response = Invoke-WebRequest -Method Get -Headers @{
      "api-version" = "2"
      Authorization = "Bearer $Token"
    } -Uri $nextUri

    $response.Content >> membersacct.txt
  
    $paginationHeader = $response.Headers.'x-pagination' | ConvertFrom-Json
    $nextUri = $paginationHeader.nextPageLink
  }
}

Essentially, what this does is:

• Added input parameters to facilitate the CMD variables you were using as well as allowing you to provide the file path to the idsonly.txt file. This can be used with a PowerShell script or within a PowerShell function definition.
• Set $nextUri to the initial URL you want to get results from
• Perform the web request, assign to the $response variable
• Writen (append) $response.Content to file
• Convert the x-pagination header value to a PowerShell object using ConvertFrom-JSON.
  • This results in a larger object than you may need but it handles the parsing of the JSON so you don't have to.
• Set $nextUri to the next page, even if there is no next page
• If there is no $nextPage, the while loop will exit
  • This relies on the truthiness of $nextUri; in other words, an empty string will evaluate to $False while any other string value will evaluate to $True.