#414 YAML can't be parsed: microsecond must be in 0..999999

[Luís Ferreira]

Given the YAML files generated by dstask using ruamel.yaml, the following backtrace is generated:

Traceback (most recent call last):
  File "/pc/clone/lfqkO38kRCeSJaDqZ374rg/py_env-python3/bin/check-yaml", line 8, in <module>
    sys.exit(main())
  File "/pc/clone/lfqkO38kRCeSJaDqZ374rg/py_env-python3/lib/python3.8/site-packages/pre_commit_hooks/check_yaml.py", line 63, in main
    load_fn(f)
  File "/pc/clone/lfqkO38kRCeSJaDqZ374rg/py_env-python3/lib/python3.8/site-packages/ruamel/yaml/main.py", line 434, in load
    return constructor.get_single_data()
  File "/pc/clone/lfqkO38kRCeSJaDqZ374rg/py_env-python3/lib/python3.8/site-packages/ruamel/yaml/constructor.py", line 121, in get_single_data
    return self.construct_document(node)
  File "/pc/clone/lfqkO38kRCeSJaDqZ374rg/py_env-python3/lib/python3.8/site-packages/ruamel/yaml/constructor.py", line 131, in construct_document
    for _dummy in generator:
  File "/pc/clone/lfqkO38kRCeSJaDqZ374rg/py_env-python3/lib/python3.8/site-packages/ruamel/yaml/constructor.py", line 674, in construct_yaml_map
    value = self.construct_mapping(node)
  File "/pc/clone/lfqkO38kRCeSJaDqZ374rg/py_env-python3/lib/python3.8/site-packages/ruamel/yaml/constructor.py", line 445, in construct_mapping
    return BaseConstructor.construct_mapping(self, node, deep=deep)
  File "/pc/clone/lfqkO38kRCeSJaDqZ374rg/py_env-python3/lib/python3.8/site-packages/ruamel/yaml/constructor.py", line 261, in construct_mapping
    value = self.construct_object(value_node, deep=deep)
  File "/pc/clone/lfqkO38kRCeSJaDqZ374rg/py_env-python3/lib/python3.8/site-packages/ruamel/yaml/constructor.py", line 154, in construct_object
    data = self.construct_non_recursive_object(node)
  File "/pc/clone/lfqkO38kRCeSJaDqZ374rg/py_env-python3/lib/python3.8/site-packages/ruamel/yaml/constructor.py", line 189, in construct_non_recursive_object
    data = constructor(self, node)
  File "/pc/clone/lfqkO38kRCeSJaDqZ374rg/py_env-python3/lib/python3.8/site-packages/ruamel/yaml/constructor.py", line 573, in construct_yaml_timestamp
    return create_timestamp(**values)
  File "/pc/clone/lfqkO38kRCeSJaDqZ374rg/py_env-python3/lib/python3.8/site-packages/ruamel/yaml/util.py", line 104, in create_timestamp
    data = datetime.datetime(year, month, day, hour, minute, second, fraction)
ValueError: microsecond must be in 0..999999

Here is an example of YAML file:

summary: DW_AT_decl_line should report 32bit on the DWARF info
notes: ""
tags:

- dmd
project: saoc2021
priority: P2
delegatedto: ""
subtasks: []
dependencies: []
created: 2021-10-21T20:10:39.485461687+01:00
resolved: 2021-11-08T02:05:29.104026669Z
due: 0001-01-01T00:00:00Z

You can check the complete set of files on my task list: https://github.com/ljmf00/tasks
The logic behind this is backed by a pre-commit hook, which is as simple as try to parse the YAML files. See the source code of the hook here.

To reproduce this you can either use pre-commit run on my set of YAMLs uncommenting the check_yaml hook or simply create a loop through all files and try to parse it with ruamel-yaml loader.

See https://github.com/pre-commit/pre-commit-hooks/issues/710

[Anthon van der Neut]

pre-commit gives command not found on my system. IIRC it uses the C based safe loader (for whatever reason).

To reproduce provide a complete python program. I cannot reproduce this with your data using the following Python program:

import sys
import ruamel.yaml

yaml_str = """\
summary: DW_AT_decl_line should report 32bit on the DWARF info
notes: ""
tags:

- dmd
project: saoc2021
priority: P2
delegatedto: ""
subtasks: []
dependencies: []
created: 2021-10-21T20:10:39.485461687+01:00
resolved: 2021-11-08T02:05:29.104026669Z
due: 0001-01-01T00:00:00Z
"""

print(sys.version_info[:2])
print(ruamel.yaml.version_info)
for t, p in [('rt', None),  ('safe', None), ('safe', True)]:
    yaml = ruamel.yaml.YAML(typ=t, pure=p)
    data = yaml.load(yaml_str)
    print(data['resolved'])

gives:

(3, 8)
(0, 17, 19)
2021-11-08 02:05:29.104027
fraction 1: 485461687
fraction 2: 485462
fraction 1: 104026669
fraction 2: 104027
fraction 1: None
fraction 2: 0
2021-11-08 02:05:29.104027
fraction 1: 485461687
fraction 2: 485462
fraction 1: 104026669
fraction 2: 104027
fraction 1: None
fraction 2: 0
2021-11-08 02:05:29.104027

The fraction X: lines come from comments inserted in util.py

[Anthon van der Neut]

This seems to happen if the fraction is 9999995 and above

[Anthon van der Neut]

This should be fixed by ruamel.yaml 0.17.20, thanks for reporting