Parsing JSON with multiple pages in Ruby-CodePudding

I understand how to parse JSON, but I don’t understand how to parse it if it contains links to other pages.

I would be grateful for your help!

api.example.com/v0/accounts

On the first request for a JSON file, we get:

{
  "response": "OK",
  "nfts": [
    {
      "token_id": "35507806371588763669298464310896317145981867843055556101069010709538683224114"
    }
  ],
  "total": null,
  "continuation": "1634866413000"
}

There is a line: continuation, which is a link to the next request, and so it repeats many more times.

On next request, the link changes to api.example.com/v0/accounts&continuation=1634866413000

My code now looks like this:

class Source
  include Mongoid::Document
  include Mongoid::Timestamps
  require 'json'

  after_save :add_items

  def add_items
    json= HTTParty.get("https://api.example.com/v0/accounts")
    json.dig('nfts')
    load_items_ethereum.each do |item|
      Item.create!(
        :token_id => item['token_id'],
      )
    end
  end
end

CodePudding user response：

Low-level HTTP clients like HTTParty typically don't handle iteration. You'll need to do it yourself, using a loop until there's no continuation field, e.g.:

begin
  continuation_param = "?continuation=#{continuation_id}" if continuation_id
  json = HTTParty.get("https://api.example.com/v0/accounts#{continuation_param}")
  continuation_id = json.dig('continuation');
  # process latest payload, append it to a running list, etc.
end while continuation_id

(And for production, best practice would be to keep a counter so you can bail after N iterations, to avoid an infinite loop.)