Getting individual key values from Perl JSON one at a time-CodePudding

I have JSON code that I'm pulling with key names that are the same and I'm trying to pull the values from the keys one at a time and pass them to variables (in a loop) in a perl script but it pulls all of the values at one time instead of iterating through them. I'd like to pull a value from a key and pass it to a variable then iterate through the loop again for the next value. The amount of data changes in JSON so the amount of identical keys will grow.

Perl Script Snippet

#!/usr/bin/perl

use warnings;
use strict;
use JSON::XS;

my $res = "test.json";
my $txt = do {                             
    local $/;                              
    open my $fh, "<", $res or die $!;
    <$fh>;                                 
};

my $json = decode_json($txt);

for my $mdata (@{ $json->{results} }) {
    my $sitedomain = "$mdata->{custom_fields}->{Domain}";
    my $routerip   = "$mdata->{custom_fields}->{RouterIP}";

    #vars
    my $domain  = $sitedomain;
    my $host    = $routerip;

   print $domain;
   print $host;
}

Print $host variable

print $host;

192.168.201.1192.168.202.1192.168.203.1

Print $domain variable

print $domain;

site1.global.localsite2.global.localsite3.global.local

JSON (test.json)

{
"results": [
    {
            "id": 37,
            "url": "http://global.local/api/dcim/sites/37/",
            "display": "Site 1",
            "name": "Site 1",
            "slug": "site1",
            "custom_fields": {
                "Domain": "site1.global.local",
                "RouterIP": "192.168.201.1"
            }
     },
     {
            "id": 38,
            "url": "http://global.local/api/dcim/sites/38/",
            "display": "Site 2",
            "name": "Site 2",
            "slug": "site2",
            "custom_fields": {
                "Domain": "site2.global.local",
                "RouterIP": "192.168.202.1"
            }
      },
      {
            "id": 39,
            "url": "http://global.local/api/dcim/sites/39/",
            "display": "Site 3",
            "name": "Site 3",
            "slug": "site3",
            "custom_fields": {
                "Domain": "site3.global.local",
                "RouterIP": "192.168.203.1"
            }
      }
   ]
}

CodePudding user response：

Your code produces expected result if you add \n to print statement. You can utilize say instead of print if there is no format required.

use warnings;
use strict;
use feature 'say';

use JSON::XS;

my $res = "test.json";
my $txt = do {                             
    local $/;                              
    open my $fh, "<", $res or die $!;
    <$fh>;                                 
};

my $json = decode_json($txt);

for my $mdata (@{ $json->{results} }) {
    my $sitedomain = "$mdata->{custom_fields}->{Domain}";
    my $routerip   = "$mdata->{custom_fields}->{RouterIP}";

    #vars
    my $domain  = $sitedomain;
    my $host    = $routerip;

    say "$domain $host";
}

The code can be re-written in shorter form as following

use strict;
use warnings;
use feature 'say';

use JSON;

my $fname = 'router_test.json';
my $txt   = do {                             
    local $/;                              
    open my $fh, "<", $fname or die $!;
    <$fh>;                                 
};

my $json = from_json($txt);

say "$_->{custom_fields}{Domain} $_->{custom_fields}{RouterIP}" for @{$json->{results}};

CodePudding user response：

It sounds like you want to "slice" the data. You could buffer in code, or collect unique values later. Let's modify what you started with, and make some tweaks:

n.b. No need to quote my $sitedomain = "$mdata->{custom_fields}->{Domain}";. The content of the JSON is already a string, and forcing Perl to make another string by interpolating it is unnecessary.

n.b.2 JSON::XS works automatically if it's installed.

my %domains;
my %ips;

for my $mdata (@{ $json->{results} }) {
    my $sitedomain = $mdata->{custom_fields}->{Domain};
    my $routerip   = $mdata->{custom_fields}->{RouterIP};

    # Collect and count all the unique domains and IPs by storing them as hash keys
    $domains{$sitedomain}  = 1;
    $ips{$routerip}  = 1;
}

for my $key (keys %domains) {
    printf "%s %s\n", $key, $domains{$key};
    # and so on
}

If we don't know the custom fields, we can play with nested hashes to collect it all:

my %fields;

for my $mdata (@{ $json->{results} }) {
    for my $custom_field (keys %{ $mdata->{custom_fields} }) {
        $fields{$custom_field}{$mdata->{custom_fields}{$custom_field}}  = 1;
    }
}

for my $custom_field (keys %fields) {
    print "$custom_field:\n";
    for my $unique_value (keys %{ $fields{$custom_field} }){
        printf "%s - %s\n", $unique_value, $fields{$custom_field}{$unique_value};
    }
}

Example output:

RouterIP:
192.168.201.1 - 1
192.168.203.1 - 1
192.168.202.1 - 1
Domain:
site2.global.local - 1
site1.global.local - 1
site3.global.local - 1

... or something like that. Nested structures lead very quickly to messy code. You can mitigate it by dereferencing the substructures. It could also be more predictable if we work with a known list of keys e.g.

my @known_keys = qw/RouterIP Domain/;

for my $mdata (@{ $json->{results} }) {
    for my $custom_field (@known_keys) {
        if (exists $fields{$custom_field}) {
            $fields{$custom_field}{$mdata->{custom_fields}{$custom_field}}  = 1;
        }
    }
}

If the JSON file is massive you may run out of memory. For this you would need to look into a package like JSON::SL or JSON::Streaming::Reader. They're more involved to use but prevent you from needing to load the whole file into memory. There are also unix tools like jq that provide the same powers.