Is it possible to convert from JSON or YAML to XML using jq/yq/xq-CodePudding

I have managed to successfully convert an XML file to a YAML file using xq

Is it possible using the following tools jq, yq, xq, to convert from either YAML or JSON back to an XML format ?

Here is a sample of my sample JSON file:

{
  "security-settings": {
    "@xmlns": "urn:activemq:core",
    "security-setting": {
      "@match": "#",
      "permission": [
        {
          "@type": "createNonDurableQueue",
          "@roles": "admins"
        },
        {
          "@type": "deleteNonDurableQueue",
          "@roles": "admins"
        },
        {
          "@type": "manage",
          "@roles": "admins"
        }
      ]
    }
  }
}

Thank you kindely for any help or suggestion.

Additionnal Information:

The source XML I initially used is the following :

<?xml version="1.0"?>
<security-settings xmlns="urn:activemq:core">
  <security-setting match="#">
    <permission type="createNonDurableQueue" roles="admins"/>
    <permission type="deleteNonDurableQueue" roles="admins"/>
    <permission type="createDurableQueue" roles="admins"/>
    <permission type="deleteDurableQueue" roles="admins"/>
    <permission type="createAddress" roles="admins"/>
    <permission type="deleteAddress" roles="admins"/>
    <permission type="consume" roles="admins"/>
    <permission type="browse" roles="admins"/>
    <permission type="send" roles="admins"/>
    <permission type="manage" roles="admins"/>
  </security-setting>
</security-settings>

The forward conversion from XML to JSON using the command xq -yY < security-settings.xml generated the JSON output:

{
  "security-settings": {
    "@xmlns": "urn:activemq:core",
    "security-setting": {
      "@match": "#",
      "permission": [
        {
          "@type": "createNonDurableQueue",
          "@roles": "admins"
        },
        {
          "@type": "deleteNonDurableQueue",
          "@roles": "admins"
        },
        {
          "@type": "createDurableQueue",
          "@roles": "admins"
        },
        {
          "@type": "deleteDurableQueue",
          "@roles": "admins"
        },
        {
          "@type": "createAddress",
          "@roles": "admins"
        },
        {
          "@type": "deleteAddress",
          "@roles": "admins"
        },
        {
          "@type": "consume",
          "@roles": "admins"
        },
        {
          "@type": "browse",
          "@roles": "admins"
        },
        {
          "@type": "send",
          "@roles": "admins"
        },
        {
          "@type": "manage",
          "@roles": "admins"
        }
      ]
    }
  }
}

The native conversion suggested by running yq -o=xml -P json_file for the backward conversion from JSON to to XML does not generate the same result as the source XML as previously shown.

<security-settings>
  <@xmlns>urn:activemq:core</@xmlns>
  <security-setting>
    <@match>#</@match>
    <permission>
      <@type>createNonDurableQueue</@type>
      <@roles>admins</@roles>
    </permission>
    <permission>
      <@type>deleteNonDurableQueue</@type>
      <@roles>admins</@roles>
    </permission>
    <permission>
      <@type>createDurableQueue</@type>
      <@roles>admins</@roles>
    </permission>
    <permission>
      <@type>deleteDurableQueue</@type>
      <@roles>admins</@roles>
    </permission>
    <permission>
      <@type>createAddress</@type>
      <@roles>admins</@roles>
    </permission>
    <permission>
      <@type>deleteAddress</@type>
      <@roles>admins</@roles>
    </permission>
    <permission>
      <@type>consume</@type>
      <@roles>admins</@roles>
    </permission>
    <permission>
      <@type>browse</@type>
      <@roles>admins</@roles>
    </permission>
    <permission>
      <@type>send</@type>
      <@roles>admins</@roles>
    </permission>
    <permission>
      <@type>manage</@type>
      <@roles>admins</@roles>
    </permission>
  </security-setting>
</security-settings>

I am running on a Fedora 36 virtual machine and this is the yq I have installed on the box

yq --version
yq 3.0.2

yq --help
usage: yq [options] <jq filter> [input file...]

yq: Command-line YAML processor - jq wrapper for YAML documents

yq transcodes YAML documents to JSON and passes them to jq.
See https://github.com/kislyuk/yq for more information.

positional arguments:
  jq_filter
  files

options:
  -h, --help            show this help message and exit
  --yaml-output, --yml-output, -y
                        Transcode jq JSON output back into YAML and emit it
  --yaml-roundtrip, --yml-roundtrip, -Y
                        Transcode jq JSON output back into YAML and emit it. Preserve YAML tags and styles by representing them as extra items in their enclosing mappings and sequences while in JSON. This option is incompatible with jq filters that do not expect these extra items.
  --width WIDTH, -w WIDTH
                        When using --yaml-output, specify string wrap width
  --indentless-lists, --indentless
                        When using --yaml-output, indent block style lists (sequences) with 0 spaces instead of 2
  --in-place, -i        Edit files in place (no backup - use caution)
  --version             show program's version number and exit

jq - commandline JSON processor [version 1.6]

Usage:  jq [options] <jq filter> [file...]
    jq [options] --args <jq filter> [strings...]
    jq [options] --jsonargs <jq filter> [JSON_TEXTS...]

jq is a tool for processing JSON inputs, applying the given filter to
its JSON text inputs and producing the filter's results as JSON on
standard output.

The simplest filter is ., which copies jq's input to its output
unmodified (except for formatting, but note that IEEE754 is used
for number representation internally, with all that that implies).

For more advanced filters see the jq(1) manpage ("man jq")
and/or https://stedolan.github.io/jq

Example:

    $ echo '{"foo": 0}' | jq .
    {
        "foo": 0
    }

Some of the options include:
  -c               compact instead of pretty-printed output;
  -n               use `null` as the single input value;
  -e               set the exit status code based on the output;
  -s               read (slurp) all inputs into an array; apply filter to it;
  -r               output raw strings, not JSON texts;
  -R               read raw strings, not JSON texts;
  -C               colorize JSON;
  -M               monochrome (don't colorize JSON);
  -S               sort keys of objects on output;
  --tab            use tabs for indentation;
  --arg a v        set variable $a to value <v>;
  --argjson a v    set variable $a to JSON value <v>;
  --slurpfile a f  set variable $a to an array of JSON texts read from <f>;
  --rawfile a f    set variable $a to a string consisting of the contents of <f>;
  --args           remaining arguments are string arguments, not files;
  --jsonargs       remaining arguments are JSON arguments, not files;
  --               terminates argument processing;

Named arguments are also available as $ARGS.named[], while
positional arguments are available as $ARGS.positional[].

See the manpage for more options.

@ikegami

Here is the output :

echo <ele attr_name="attr_value">ele_value</ele> | xq

{
  "ele": {
    "@attr_name": "attr_value",
    "#text": "ele_value"
  }
}

echo <ele attr_name="attr_value">ele_value</ele> | xq | ./yq_linux_amd64 -o=xml -P

<ele>
  <@attr_name>attr_value</@attr_name>
  <#text>ele_value</#text>
</ele>

CodePudding user response：

First, get the version of yq from this github repository: https://github.com/mikefarah/yq/releases/download/v4.26.1/yq_linux_amd64

It provides flags/features that others might not support.

Then, one can use the following command to regenerate the XML:

./yq_linux_amd64               \
    --xml-attribute-prefix @   \
    --xml-content-name '#text' \
    --input-format yaml        \
    --output-format xml        \
    security-settings.yaml

The very same command works for the JSON inputs as well, since JSON is a subset of YAML.

CodePudding user response：

jq:

"@"     as $attr_prefix |
"#text" as $content_key |

# ">" only needs to be escaped if preceded by "]]".
# Some whitespace also need escaping, at least in attribute.
{ "&": "&amp;", "<": "&lt;", ">": "&gt;"    } as $escapes      |
{ "&": "&amp;", "<": "&lt;", "\"": "&quot;" } as $attr_escapes |

def text_to_xml:          split( "" ) | map( $escapes[.]       // . ) | join( "" );
def text_to_xml_attr_val: split( "" ) | map( $attr_escapes [.] // . ) | join( "" );

def node_to_xml:
   if type == "string" then
      text_to_xml
   else
      (
         if .attrs then
            .attrs |
            to_entries |
            map( " "   .key   "=\""   ( .value | text_to_xml_attr_val )   "\"" ) |
            join( "" )
         else
            ""
         end
      ) as $attrs |

      if .children and ( .children | length ) > 0 then
         ( .children | map( node_to_xml ) | join( "" ) ) as $children |
         "<"   .name   $attrs   ">"   $children   "</"   .name   ">"
      else
         "<"   .name   $attrs   "/>"
      end
   end
;

def fix_tree( $name ):
   type as $type |
   if $type == "array" then
      .[] | fix_tree( $name )
   elif $type == "object" then
      reduce to_entries[] as { key: $k, value: $v } (
         { name: $name, attrs: {}, children: [] };

         if $k[0:1] == $attr_prefix then
            .attrs[ $k[1:] ] = $v
         elif $k == $content_key then
            .children  = [ $v ]
         else
            .children  = [ $v | fix_tree( $k ) ]
         end
      )
   else
      { name: $name, attrs: {}, children: [ . ] }
   end
;

def fix_tree: fix_tree( "" ) | .children[];

fix_tree | node_to_xml

Demo on jqplay

It's invoked using

jq -R 'above progam' file.json >file.xml

You can also place the program in a file (say json_to_xml.jq) and use the following:

jq -Rf json_to_xml.jq file.json >file.xml

I took a two-step approach. I first convert the input to an unambiguous format, then converting this result to XML. These two steps could be merged. Here's is the result of the first conversion of the provided input:

{
  "name": "security-settings",
  "attrs": {
    "xmlns": "urn:activemq:core"
  },
  "children": [
    {
      "name": "security-setting",
      "attrs": {
        "match": "#"
      },
      "children": [
        {
          "name": "permission",
          "attrs": {
            "type": "createNonDurableQueue",
            "roles": "admins"
          },
          "children": []
        },
        {
          "name": "permission",
          "attrs": {
            "type": "deleteNonDurableQueue",
            "roles": "admins"
          },
          "children": []
        },
        {
          "name": "permission",
          "attrs": {
            "type": "manage",
            "roles": "admins"
          },
          "children": []
        }
      ]
    }
  ]
}

Note that the format into which the original XML was converted is lossy. For example, it loses the relative order of XML elements with different names. This means the output of the above program may differ from the original XML in significant ways. But there's no escaping that unless you use a JSON schema that's not lossy.