I am attempting to scrape the JSON from this endpoint https://prices.runescape.wiki/api/v1/osrs/latest .

#[derive(Serialize, Deserialize, Debug)]
struct Latest {
    high: u32,
    highTime: String,
    low: String,
    lowTime: String,
}

#[derive(Serialize, Deserialize, Debug)]
struct Data {
    #[serde(with = "serde_with::json::nested")]
    data: Vec<Latest>,
}

#[derive(Serialize, Deserialize, Debug)]
struct Datav2 {
    #[serde(with = "serde_with::json::nested")]
    data: HashMap<u32, Vec<Latest>>,
}
#[cfg(not(target_arch = "wasm32"))]
#[tokio::main]
async fn main() -> Result<(), reqwest::Error> {

     let res = reqwest::get(url).await?;
    let response = &res.json::<Datav2>().await?;

}

I've tried two versions of the datastructure. Data is using a vector of latest, but i noticed each object has a unique ID, so in DataV2 i tried using a hashmap but i get the same error. I've also tried unnested versions without using Serde_with.

I get the error Error: reqwest::Error { kind: Decode, source: Error("invalid type: map, expected valid json object", line: 1, column: 8)

It seems my datastructure is messed up, but have been trying for hours to figure out the correct data structure to use.

CodePudding user response：

There are multiple issues with your current code.

1. Datav2 is closer, but still not correct. It is not a HashMap<u32, Vec<Latest>>but a HashMap<u32, Latest>. There is no need to have another Vec, since each number is assigned a value in the JSON.
2. highTime, low, lowTime are not of type String (since they have no quotes in the JSON), but are unsigned numbers (to be on the safe side I just assumed them to be u64).
3. Apparently the fields of Latest can be null, so they need to be Options.
4. I would still use snake_case for the field names in the structs and then rename them to camelCase using the serde macro

I modified your code like I would do this, in order to give you an example of how it could be done:

use std::collections::HashMap;

use serde::{Serialize, Deserialize};

#[derive(Serialize, Deserialize, Debug)]
#[serde(rename_all = "camelCase")]
struct Latest {
    high: Option<u64>,
    high_time: Option<u64>,
    low: Option<u64>,
    low_time: Option<u64>,
}

#[derive(Serialize, Deserialize, Debug)]
struct Data {
    data: HashMap<u64, Latest>,
}

#[cfg(not(target_arch = "wasm32"))]
#[tokio::main]
async fn main() -> Result<(), reqwest::Error> {
    let url = "https://prices.runescape.wiki/api/v1/osrs/latest";
    let res = reqwest::get(url).await?;
    let response = &res.json::<Data>().await?;

    println!("{:#?}", response);

    Ok(())
}