#492 Newline is stripped after second dump

[David Shay]

Since version 0.18.3, we have a strange bug using the library in our product. A minimal code that reproduces the error is the following:

import ruamel.yaml

y = ruamel.yaml.YAML()
with open("organizational_units.yaml", 'r') as file:
    ou = y.load(file)

with open("organizational_units.yaml", 'r') as file:
    content = y.load(file)

content["organizational_units"] = ou["organizational_units"]

with open("test.yaml", 'w') as file:
    y.dump(content, file)
with open("test.yaml", 'w') as file:
    y.dump(content, file)

with open("test.yaml", 'r') as file:
    assert y.load(file)

You can run it with this yaml file (organizational_units.yaml in my example):

# Organizational Unit Specification.
# blahblahblah

organizational_units:


- Name: root
  Accounts:
  - FirstAccount
  - SecondAccount

After trying to reload the file at the end of the test, it throws the following error:

ruamel.yaml.scanner.ScannerError: sequence entries are not allowed here
in "test.yaml", line 4, column 23

After the second dump, the file test.yaml looks like this:

# Organizational Unit Specification.
# blahblahblah

organizational_units: -
  Name: root
  Accounts:

  - FirstAccount
  - SecondAccount

Some newline is clearly missing here.

[Anthon van der Neut]

This is a result of how comments between keys and values are currently stored: both on

print(ou['organizational_units'].ca)
print(ou['organizational_units'][0].ca)

(and yes for ruamel.yaml internals you have a comment there, caused by the newline). If you then combine two different loaded items (or do a deepcopy), those two instances are no longer related and a second dump will not work.
See e.g. issue 410. Unfortunately this has shown not to be easy to solve and requires alternative comment attachment strategies.

[David Shay]

Thanks for the reply. Issue 410 is an old one. Strange that our issue appeared only since 0.18.3.
Following your comments, I tried this workaround that seems to work in my case:

comments = content["organizational_units"].ca.comment
content["organizational_units"] = ou["organizational_units"]
content["organizational_units"].ca.comment = comments