#339 Insufficient escaping on dictionary keys and values

[Jiri Baum]

There appears to be insufficient escaping on keys and values in dictionaries. For example:

# example 1
import ruamel.yaml
test_data = {0: '?R'}
assert ruamel.yaml.safe_load(ruamel.yaml.dump(test_data)) == test_data

or

# example 2
import ruamel.yaml
test_data = {':0': 0}
assert ruamel.yaml.safe_load(ruamel.yaml.dump(test_data)) == test_data

Expected behaviour: assertions pass (no output)

Actual behaviour: Exception is raised.

In both cases, the values can be represented in YAML in a way that ruamel.yaml can load:

# expected behaviour for example 1
import ruamel.yaml
test_data = {0: '?R'}
assert ruamel.yaml.safe_load("{0: '?R'}") == test_data

or

# example 2
import ruamel.yaml
test_data = {':0': 0}
assert ruamel.yaml.safe_load("{':0': 0}") == test_data

Use case: I would like to use hypothesis to property-test my code which uses ruamel.yaml

[Jiri Baum]

In general I would like to be able to round-trip data through YAML, without worrying whether they will be represented correctly; testing with hypothesis is just how I identified the issue.

Example values that are problematic as dictionary keys include:

• IPv6 localhost: ::1
• smileys: :-)
• programming language tokens: :=
• URL fragments: :8080

Example values that are problematic as dictionary values include:

• information annotated as uncertain or unwise: ?? not sure
• shell wildcards: ?eadme.txt
• URL fragments: ?query=xyzzy
• BASIC statements: ? "hello world!"

A practical example of these would be an Ansible inventory file including the IPv6 localhost. This is a valid inventory as far as Ansible is concerned.

import ruamel.yaml
test_yaml = """
all:
    hosts:
        '::1':
"""
test_data = ruamel.yaml.safe_load(test_yaml)
assert ruamel.yaml.safe_load(ruamel.yaml.dump(test_data)) == test_data

[Jiri Baum]

In terms of the yaml-test-suite, the relevant test cases are:

• 2EBW: Allowed characters in keys
• FBC9: Allowed characters in plain scalars

In general, ruamel.yaml should probably pass the load(dump(data)) == data test for all the test cases in the yaml-test-suite.