prepending / adding new entries to yaml while preserving comment-CodePudding

I would like to add entries to the beginning (or the end, it would not matter) to an existing yaml file with comments, which I would want to have preserved. Is there an elegant way to do this using ruamel.yaml?

import sys
import ruamel.yaml

root_str = """\
# block comment
# block comment
# block comment
root:
    - class:
        - subclass:
            var1: a
    - class:
        var2: b    
"""

prepend_str = """\
pre_root:
    - pre_class:
        var1: c
"""

yaml = ruamel.yaml.YAML()
yaml.indent(mapping=4, sequence=4, offset=4)
root = yaml.load(root_str)
prepend = yaml.load(prepend_str)

This obviously does not work:

new_str = {**prepend, **root}

This works for my case, but seems awfully hacky:

import io
from contextlib import redirect_stdout

with io.StringIO() as buf, redirect_stdout(buf):
    ruamel.yaml.dump(prepend, sys.stdout, Dumper=ruamel.yaml.RoundTripDumper)
    ruamel.yaml.dump(root, sys.stdout, Dumper=ruamel.yaml.RoundTripDumper)
    output = buf.getvalue()

with open("out.yaml", "w") as yaml_file:
    yaml_file.write(output)

CodePudding user response：

There are a few ways you can make this simpler. First of all you should not mix the new API ( yaml = ruamel.yaml.YAML() ) with the deprecated old API ( ruamel.yaml.dump(....). Second you can leave out the contextlib stuff and directly stream to a BytesIO (better than StringIO as YAML.dump() write an UTF-8 stream):

with io.BytesIO() as buf:
    yaml.dump(prepend, buf)
    yaml.dump(root, buf)
    output = buf.getvalue()

with open("out.yaml", "wb") as yaml_file:
    yaml_file.write(output)

sys.stdout.write(open("out.yaml").read())

which gives:

pre_root:
    - pre_class:
        var1: c
# block comment
# block comment
# block comment
root:
    - class:
        - subclass:
            var1: a
    - class:
        var2: b

You can further simplify this by directly writing to the opened file (note that in both cases it was opened "wb", because of UTF-8):

with open("out.yaml", "wb") as yaml_file:
    yaml.dump(prepend, yaml_file)
    yaml.dump(root, yaml_file)

which gives you the same file as before.

If you want to keep the block comments of root_str at the beginning, you should combine at the data level:

# This also works in case prepend has multiple keys. When prepend
# and root have keys in common, you need to do something smarter
for key in reversed(prepend):  
    root.insert(0, key, prepend[key])

yaml.dump(root, sys.stdout)

which gives:

# block comment
# block comment
# block comment
pre_root:
    - pre_class:
        var1: c
root:
    - class:
        - subclass:
            var1: a
    - class:
        var2: b

CodePudding user response：

If you just want to prepend or append YAML files, why not use strings like new_str = '\n'.join([prepend_str, root_str]).

However prepending or appending YAML can violate the unique keys. When working with an object representation, you could also merge it in.

The only way I solved it with ruamel.yaml was:

import sys
from ruamel.yaml import YAML

root_str = """\
# block comment
# block comment
# block comment
root:
    - class:
        - subclass:
            var1: a
    - class:
        var2: b    
"""

prepend_str = """\
pre_root:
    - pre_class:
        var1: c
"""
yaml = YAML()
root = yaml.load(root_str)
print("--- root: ----")
yaml.dump(root, sys.stdout)

# print(type(root))
# see methods of object:
# help(root)

prepend = yaml.load(prepend_str)
print("--- prepend: ----")
yaml.dump(prepend, sys.stdout)

def merge_into(base, mixin):
    bks = base.keys()
    mks = mixin.keys()
    if mks != bks:
        # append (without control of order)
        for k in mks:
            base[k] = mixin[k]
    else:
       print("Warning: Keys overlap! Aborted merge.")
       print("\tbase keys:", bks)
       print("\tmixin keys:", mks)

print("--- prepend merged into root: ----")
merge_into(root, prepend)
# does not work as expected
# root.insert(0, '', prepend, 'attached comment')
yaml.dump(root, sys.stdout)

However this does simply merge-in (effectively not prepend but append) and ignores any comments in the mixin.

Print output was:

--- root: ----
# block comment
# block comment
# block comment
root:
- class:
  - subclass:
      var1: a
- class:
    var2: b
--- prepend: ----
pre_root:
- pre_class:
    var1: c
--- prepend merged into root: ----
# block comment
# block comment
# block comment
root:
- class:
  - subclass:
      var1: a
- class:
    var2: b
pre_root:
- pre_class:
    var1: c