#43 Duplicate anchors

[Anthon van der Neut]

As seen here, and I quote:

"Anchor names act somewhat like variable assignments: at any point in the document, the parser only knows about the anchors it’s seen so far, and a second anchor with the same name takes precedence. This means that aliases cannot refer to anchors that appear later in the document."

It should be possible to have anchors 'overwrite' each other (as it is possible here), but when implementing a YAML which does so ruamel gives me an duplicate anchor error.

Shouldn't this be possible?

(originally posted on 2016-08-18 at 08:01:56 by gerald-coddut <gerald-coddut@bitbucket>)

[Anthon van der Neut]

You are right about anchors not needing to be unique but not because of the link you post (which is irrelevant as it has nothing to do, nor refers to the YAML specification and arguably says the second or every second, with the same name takes precedence, whereas it is the last one that does take precedence). The relevant information is here:

"When composing a representation graph from serialized events, an alias node refers to the most recent node in the serialization having the specified anchor. Therefore, anchors need not be unique within a serialization."

This is a feature of PyYAML that ruamel.yaml inherited: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=515634

BTW Your second link does get me to some page with a captcha that doesn't let me proceed and your image doesn't show any code that fails. To expediate solving of a bug at least include self-contained example code (as code, never as a screenshot) that generates the error.

In this case it is clear to me what goes wrong, but what isn't is what code generates the same anchors for different objects? In Python using ruamel.yaml or (PyYAML) this is AFAIK actually impossible to do.

(originally posted on 2016-08-18 at 08:50:28)

[Anthon van der Neut]

None
(originally posted on 2016-08-18 at 08:51:02)

[Anthon van der Neut]

Ah you're right about the non-relevant information I opted, sorry about that. You're also right about showing you some code instead of that rather useless screenshot. Let me correct myself.

I'm loading my YAML from a file which look somewhat like:

#!yaml

Some Colors: &some_colors
 color_primary: &color_primary "#112233FF"
 color_secondary: &color_secondary "#445566FF"

Element: &element
 color: *color_primary

Overwrite some colors: &overwrite_colors
 color_primary: &color_primary "#000000FF"

Another element: &another_element
 color: *color_primary

I'm trying to parse the above file using this code:

#!python

def load_base_style_sheet():
    f = open('BaseStyleSheet.yaml')
    pp = pprint.PrettyPrinter(indent=4)

    base_sheet = ruamel.yaml.load(f)
    pp.pprint(base_sheet)


load_base_style_sheet()

Which gives me the error with the following stack trace:

#!python

Traceback (most recent call last):
  File "StyleSheetParser.py", line 21, in <module>
    load_base_style_sheet()
  File "StyleSheetParser.py", line 17, in load_base_style_sheet
    base_sheet = ruamel.yaml.load(f)
  File "/../env/lib/python3.5/site-packages/ruamel/yaml/main.py", line 81, in load
    return loader.get_single_data()
  File "/../env/lib/python3.5/site-packages/ruamel/yaml/constructor.py", line 55, in get_single_data
    node = self.get_single_node()
  File "/../env/lib/python3.5/site-packages/ruamel/yaml/composer.py", line 49, in get_single_node
    document = self.compose_document()
  File "/../env/lib/python3.5/site-packages/ruamel/yaml/composer.py", line 69, in compose_document
    node = self.compose_node(None, None)
  File "/../env/lib/python3.5/site-packages/ruamel/yaml/composer.py", line 100, in compose_node
    node = self.compose_mapping_node(anchor)
  File "/../env/lib/python3.5/site-packages/ruamel/yaml/composer.py", line 159, in compose_mapping_node
    item_value = self.compose_node(node, item_key)
  File "/../env/lib/python3.5/site-packages/ruamel/yaml/composer.py", line 100, in compose_node
    node = self.compose_mapping_node(anchor)
  File "/../env/lib/python3.5/site-packages/ruamel/yaml/composer.py", line 159, in compose_mapping_node
    item_value = self.compose_node(node, item_key)
  File "/../env/lib/python3.5/site-packages/ruamel/yaml/composer.py", line 93, in compose_node
    "second occurence", event.start_mark)
ruamel.yaml.composer.ComposerError: found duplicate anchor 'color_primary'; first occurence
  in "BaseStyleSheet.yaml", line 2, column 17
second occurence
  in "BaseStyleSheet.yaml", line 9, column 17

I've obscured the path to the files up to the environment with /../

Does this help somewhat?

Also; I've created a question at SO. I'm aware that this is rude, but I'd like to ask if you could take a little peek at the question.

I'm struggling a bit with YAML. Starting to think I'm trying to make it go trough impossible hoops.

(originally posted on 2016-08-18 at 10:19:08 by gerald-coddut <gerald-coddut@bitbucket>)

[Anthon van der Neut]

Hi Gerald,

Thanks for the update and for reporting. I had already made my own (failing) example, and implemented a solution, but it is always good to test with something someone else came up with, in case I just use some corner case.

This now loads (using ruamel.yaml 0.12.3):

import sys
import ruamel.yaml

yaml_str = """\
Some Colors: &some_colors
 color_primary: &color_primary "#112233FF"
 color_secondary: &color_secondary "#445566FF"

Element: &element
 color: *color_primary

Overwrite some colors: &overwrite_colors
 color_primary: &color_primary "#000000FF"

Another element: &another_element
 color: *color_primary
"""

# import warnings
# from ruamel.yaml.error import ReusedAnchorWarning
# warnings.simplefilter("ignore", ReusedAnchorWarning)

data = ruamel.yaml.round_trip_load(yaml_str)

It now gives a warning (which can be suppressed by commenting out the 3 commented lines).

I would write your example:

def load_base_style_sheet():
    pp = pprint.PrettyPrinter(indent=4)

    with open('BaseStyleSheet.yaml') as f:
        base_sheet = ruamel.yaml.load(f)
    pp.pprint(base_sheet)

load_base_style_sheet()

to make sure the file is actually closed (and using with is better than calling f.close())

If you have questions, which are not really bugs, post on stackoverflow and tag with ruamel.yaml. Any bugs are of course "welcome" here ;-)

(originally posted on 2016-08-18 at 10:56:44)

[Anthon van der Neut]

• status set to resolved

fixed in 0.12.3 please confirm by closing this issue

(originally posted on 2016-08-18 at 10:59:02)

[Anthon van der Neut]

I've confirmed that this works as of 0.12.3, will close the issue with that reason.

Thanks for responding and resolving the issue this quickly!

(originally posted on 2016-08-18 at 11:51:37 by gerald-coddut <gerald-coddut@bitbucket>)

[Anthon van der Neut]

• status set to closed

(originally posted on 2016-08-18 at 11:51:54 by gerald-coddut <gerald-coddut@bitbucket>)

[Anthon van der Neut]

with ruamel.yaml version 0.15.34 this happens again and behaves the same way as pyaml does, again

import warnings

from ruamel.yaml import YAML
from ruamel.yaml.error import ReusedAnchorWarning

warnings.simplefilter("ignore", ReusedAnchorWarning)

yaml=YAML(typ='safe')

foo = """
foo: &foo "foo"
bar:
    x: *foo
    y: &foo "bar"
    z: *foo
"""

bar = yaml.load(foo)
print(bar)

fails with

  File "_ruamel_yaml.pyx", line 706, in _ruamel_yaml.CParser.get_single_node (ext/_ruamel_yaml.c:9612)
  File "_ruamel_yaml.pyx", line 724, in _ruamel_yaml.CParser._compose_document (ext/_ruamel_yaml.c:9922)
  File "_ruamel_yaml.pyx", line 775, in _ruamel_yaml.CParser._compose_node (ext/_ruamel_yaml.c:10814)
  File "_ruamel_yaml.pyx", line 889, in _ruamel_yaml.CParser._compose_mapping_node (ext/_ruamel_yaml.c:12609)
  File "_ruamel_yaml.pyx", line 775, in _ruamel_yaml.CParser._compose_node (ext/_ruamel_yaml.c:10814)
  File "_ruamel_yaml.pyx", line 889, in _ruamel_yaml.CParser._compose_mapping_node (ext/_ruamel_yaml.c:12609)
  File "_ruamel_yaml.pyx", line 767, in _ruamel_yaml.CParser._compose_node (ext/_ruamel_yaml.c:10678)
ruamel.yaml.composer.ComposerError: found duplicate anchor; first occurrence
  in "<unicode string>", line 2, column 6
second occurrence
  in "<unicode string>", line 5, column 8

issue on pyaml https://github.com/yaml/pyyaml/issues/100

(originally posted on 2017-11-25 at 05:23:46 by Nikky <nikky@bitbucket>)

[Anthon van der Neut]

Same issue here, plus this is using the yq command line tool.
Which sadly makes my use case with anchors unusable

(originally posted on 2019-01-09 at 17:25:48 by Clement Desiles <clement-desiles-hoppen@bitbucket>)

[Anthon van der Neut]

@NikkyAi The issue is caused by libyaml, so until that library is updated, you should do yaml = YAML(typ='safe', pure=True) or use the default (yaml = YAML()) . (Sorry, but had not seen your comment before, it is in general better to re-open a closed issue if you find it no longer is fixed. Or submit a new issue, and refer to the closed issue. Bitbucket doesn't always notify me on comments added to closed issues.

@clement-desiles-hoppen I don't know what you are trying to say, If "this" refers to ruamel.yaml then I can guarantee that it is not using yq. If this refers to the issue gerald-coddut submitted or the comment by Nikky (is that what you mean by same issue?) then I don't see how they are using yq. I would appreciate you taking your time, in making any comments you post less cryptic and/or ambiguous. That way I don't have to waste any time guessing (let alone reproducing and not even thinking about resolving) what you are writing about.

(originally posted on 2019-01-09 at 19:12:47)

[Anthon van der Neut]

Hi Anthon, sorry for the late answer, and the cryptic message.

I'm using jq (https://github.com/stedolan/jq) and a yaml wrapper arround it, called yq (https://pypi.org/project/yq/), which is a great tool to manipulate yaml data.
I wrongly supposed yq was based on ruaml.yaml, in fact it seems it's still based on PyYAML:

• https://github.com/kislyuk/yq/blob/master/setup.py#L18
• https://github.com/kislyuk/yq/issues/15

Using the gerald-coduct's example above, I've this result:

$ yq . /tmp/test.yml 
yq: Error running jq: ComposerError: found duplicate anchor 'color_primary'; first occurence
  in "/tmp/test.yml", line 2, column 17
second occurence
  in "/tmp/test.yml", line 9, column 17.

Here is the explanation, sorry for the noise!

(originally posted on 2019-03-26 at 09:26:10 by Clement Desiles <clement-desiles-hoppen@bitbucket>)