I hope someone can help me with this -

I make a number of API calls to various services, all of which return a structured JSON response. Normally I just convert to a table, and extract the one or two values i need, but I’d really like to be able to do a full comparison check to see everything that’s changed since I retrieved the last one.

I’ve already gone down the path of writing a block of code to specifically compare each entry within the associated tables generated, (I’m on Lua version 5.1) but I wanted to ask is there a more dynamic way of doing this, one which is not so rigid that I could even apply against any other json calls (2 x Lua tables) i want to compare?

(I’ve looked at various SO posts about comparing Lua tables but most just look to report if there’s a difference overall (true/false), none of them seem to itemise the differences, or can adapt to different json/table structures - which is important considering the variety of json structures API services return)

Below is an example api json/response, and a sample of the code I’ve written to compare an entry in both tables and report the differences…

Is there a dynamic version that can do the same thing, avoiding the need to script in detail, every comparison step ?

local json = require("dkjson")
local snapshot1 = [[
{
   "ActTime" : 1501360852,
   "ServerTime" : "2017-07-29 22:40:52",
   "Sunrise" : "05:50",
   "Sunset" : "21:28",
   "result" : [
      {
         "AddjMulti" : 1.0,
         "AddjValue" : 0.0,
         "BatteryLevel" : 255,
         "Data" : "73 Lux",
         "Description" : "",
         "HardwareID" : 4,
         "HardwareName" : "Dummies",
         "HardwareType" : "Dummy",
         "ID" : "82089",
         "LastUpdate" : "2017-07-29 21:16:22",
         "Name" : "ESP8266C_Licht1",
         "Notifications" : "false",
         "ShowNotifications" : true,
         "SignalLevel" : "334",
         "Timers" : "false",
         "Type" : "Lux",
         "YOffset" : "0",
         "idx" : "89"
      }
   ],
   "status" : "OK",
   "title" : "Devices"
}
]]

local snapshot2 = [[
{
   "ActTime" : 1501360852,
   "ServerTime" : "2017-07-31 11:40:52",
   "Sunrise" : "05:52",
   "Sunset" : "21:29",
   "result" : [
      {
         "AddjMulti" : 1.0,
         "AddjValue" : 0.0,
         "BatteryLevel" : 255,
         "Data" : "73 Lux",
         "Description" : "",
         "HardwareID" : 4,
         "HardwareName" : "Dummies",
         "HardwareType" : "Dummy",
         "ID" : "82089",
         "LastUpdate" : "2017-07-30 21:16:22",
         "Name" : "ESP8266C_Licht1",
         "Notifications" : "false",
         "ShowNotifications" : true,
         "SignalLevel" : "545",
         "Timers" : "true",
         "Type" : "Lux",
         "YOffset" : "0",
         "idx" : "89"
      }
   ],
   "status" : "OK",
   "title" : "Devices"
}
]]

local tableA = json.decode(snapshot1)
local tableB = json.decode(snapshot2)
if tableA.ActTime ~= tableB.ActTime then
    print("ActTime has changed from [ "..tableA.result[1].ActTime.." ] to [ "..tableB.result[1].ActTime.." ]") end
if tableA.ServerTime ~= tableB.ServerTime then
    print("ActTime has changed from [ "..tableA.ServerTime.." ] to [ "..tableB.ServerTime.." ]") end
if tableA.result[1].LastUpdate ~= tableB.result[1].LastUpdate then
    print("result:LastUpdate has changed from [ "..tableA.result[1].LastUpdate.." ] to [ "..tableB.result[1].LastUpdate.." ]") end

As always, many thanks to this community for their help..

CodePudding user response：

Since these are JSON tables, we can make the following assumptions greatly simplifying the problem of comparing two tables:

1. There are no cycles, backreferences, duplicate references etc.; the object is a proper tree;
2. Tables are either dictionaries/objects (string keys) or arrays/lists (integer keys from 1 to n, possibly holes depending on how your JSON library handles nulls)
3. Values are either booleans, numbers, strings, or one of the aforementioned two table types

we can now write a recursive function that returns the difference in table form:

local function table_changes(original, modified)
    if original == modified then return nil end -- no changes
    if not (type(original) == "table" and type(modified) == "table") then return {from = original, to = modified} end -- primitive types
    local result = {}
    for k, v in pairs(original) do
        result[k] = table_changes(v, modified[k])
    end
    for k, v in pairs(modified) do
        if original[k] == nil then
            result[k] = table_changes(nil, v)
        end
    end
    return next(result) and result -- return nil for an empty table (no changes)
end

this will return a table like the following:

{
    ServerTime = {
        from = "2017-07-29 22:40:52",
        to = "2017-07-31 11:40:52"
    },
    Sunrise = {
        from = "05:50",
        to = "05:52"
    },
    Sunset = {
        from = "21:28",
        to = "21:29"
    },
    result = {
        {
            LastUpdate = {
                from = "2017-07-29 21:16:22",
                to = "2017-07-30 21:16:22"
            }
        }
    }
}

CodePudding user response：

The easiest way is to deserialize the JSON string and compare the resulting table element by element.

Expanding mentioned solutions that just deliver true/false so something that lists the differences is trivial.

table fields can be added, removed or changed. This is just a matter of comparing two tables recursively.

Comparing the json strings would be much more complex.