Aeson does not find a key that I believe is present-CodePudding

I'm trying to parse a JSON blob that looks like this:

"{\"order_book\":{\"asks\":[[\"0.06777\",\"0.00006744\"],[\"0.06778\",\"0.01475361\"], ... ]],\"bids\":[[\"0.06744491\",\"1.35\"],[\"0.06726258\",\"0.148585363\"], ...]],\"market_id\":\"ETH-BTC\"}}"

Those lists of pairs of numbers are actually much longer; I've replaced their tails with ellipses.

Here's my code:

{-# LANGUAGE OverloadedStrings #-}

module Demo where

import Data.Aeson
import Data.ByteString.Lazy hiding (putStrLn)
import Data.Either (fromLeft)
import Network.HTTP.Request

data OrderBook = OrderBook
  { orderBook_asks     :: [[(Float,Float)]]
  , orderBook_bids     :: [[(Float,Float)]]
  , orderBook_marketId :: String
  }

instance FromJSON OrderBook where
  parseJSON = withObject "order_book" $ \v -> OrderBook
    <$> v .: "asks"
    <*> v .: "bids"
    <*> v .: "market_id"

demo :: IO ()
demo = do
  r <- get "https://www.buda.com/api/v2/markets/eth-btc/order_book"
  let d = eitherDecode $ fromStrict $ responseBody r :: Either String OrderBook
  putStrLn $ "Here's the parse error:"
  putStrLn $ fromLeft undefined d
  putStrLn $ "\n\nAnd here's the data:"
  putStrLn $ show $ responseBody r

Here's what running demo gets me:

Here's the parse error:
Error in $: key "asks" not found

And here's the data:
"{\"order_book\":{\"asks\":[[\"0.06777\",\"0.00006744\"],[\"0.06778\",\"0.01475361\"], ... ]],\"bids\":[[\"0.06744491\",\"1.35\"],[\"0.06726258\",\"0.148585363\"], ...]],\"market_id\":\"ETH-BTC\"}}"

The "asks" key looks clearly present to me -- it's the first one nested under the "order_book" key.

CodePudding user response：

withObject "order_book" does not look into the value at key "order_book". In fact, the "order_book" argument is ignored apart from appearing in the error message; actually you should have withObject "OrderBook" there.

All withObject does is confirm that what you have is an object. Then it proceeds using that object to look for the keys "asks", "bids" and "market_id" – but the only key that's there at this level is order_book.

The solution is to only use this parser with the {"asks":[["0.06777"...]...]...} object. The "order_book" key tells no information anyway, unless there are other keys present there as well. You can represent that outer object with another Haskell type and its own FromJSON instance.

CodePudding user response：

The key is present, but it's wrapped inside another nested object, so you have to unwrap the outer object before you can parse the keys.

The smallest-diff way to do this is probably just inline:

instance FromJSON OrderBook where
  parseJSON = withObject "order_book" $ \outer -> do
    v <- outer .: "order_book"
    OrderBook
      <$> v .: "asks"
      <*> v .: "bids"
      <*> v .: "market_id"

Though you might want to consider introducing another wrapping type instead. This would really depend on the semantics of the data format you have.

I guess you were probably assuming that this is what withObject "order_book" would do, but that's not what it does. The first parameter of withObject is just a human-readable name of the object being parsed, used to create error messages. Customarily that parameter should name the type that is being parsed - i.e. withObject "OrderBook". See the docs.

Separately, I think your asks and bids fields are mistyped.

First, your JSON input looks like they are supposed to be arrays of tuples, but your Haskell type says doubly nested arrays of tuples. So this will fail to parse.

Second, your JSON input has strings as elements of those tuples, but your Haskell type says Float. This will also fail to parse.

The correct type, according to your JSON input, should be:

  { orderBook_asks     :: [(String,String)]
  , orderBook_bids     :: [(String,String)]

Alternatively, if you really want the floats, you'll have to parse them from strings:

instance FromJSON OrderBook where
  parseJSON = withObject "order_book" $ \outer -> do
    v <- outer .: "order_book"
    OrderBook
      <$> (map parseTuple <$> v .: "asks")
      <*> (map parseTuple <$> v .: "bids")
      <*> v .: "market_id"
    where
      parseTuple (a, b) = (read a, read b)

(note that this ☝️ code is not to be copy&pasted: I'm using read for parsing strings into floats, which will crash at runtime if the strings are malformatted; in a real program you should use a better way of parsing)