I'm following-up on this because someone mentioned in an issue ticket against yamlpath that you'd been fixing some date/datetime handling issue(s) in ruamel.yaml. While I can't find any mention of this in your change log, I'd like to try to draw your attention to this issue again, anyway. If you are making any changes to TimeStamp/AnchoredTimeStamp/etc, please remember to add Anchor handling. Unrelated: I'll open a new ticket for the TimeZone handling issue, above. I'm not sure why I tacked it onto...
Thank you! I'll incorporate this and see how it fares in my project. I haven't encountered any users attempting to anchor null values in YAML, yet; so, I'm not worried about that edge-case, just yet. Query: Is it really necessary to perform the UTC conversion against the timestamp values? This conversion has forced me to add code to "undo" it whenever users query timestamp nodes, otherwise they end up with datetime values that are time-zone naive; in other words, they don't get out what they put...
Thank you! I'll incorporate this and see how it fares in my project. I haven't encountered any users attempting to anchor null values in YAML, yet; so, I'm not worried about that edge-case, just yet. Query: Is it really necessary to perform the UTC conversion against the timestamp values? This conversion has forced me to add code to "undo" it whenever users query timestamp nodes, otherwise they end up with datetime values that are time-zone naive; in other words, they don't get out what they put...
Thank you! I'll incorporate this and see how it fares in my project. I haven't encountered any users attempting to anchor null values in YAML, yet; so, I'm not worried about that edge-case, just yet. Query: Is it really necessary to perform the UTC conversion against the timestamp values? This conversion has forced me to add code to "undo" it whenever users query timestamp nodes, otherwise they end up with datetime values that are time-zone naive; in other words, they don't get out what they put...
Anchored date and TimeStamp values lose their anchors
Absolutely true! I fight the same battles from time to time, so I know your pain. I learned very early the same lesson, so you'll find I use types as religiously as I can bear to. You can quiet a lot of the mypy noise regarding returning Any with a mypy.ini entry like: [mypy] warn_return_any = True I do the same for my project because I haven't spent enough time to zero-in on just what ruamel.yaml returns for many of its methods. :)
Most surprisingly, 0.17.6 and 0.17.7 both cause MYPY to become more strict with regard to my uses of CommentedMap and CommentedSeq. I verified by downgrading ruamel.yaml and running the same test suite; the strictness relaxed. Upgrading to 0.17.6+ has forced me to more strictly type my own variables wherever I'm interacting with CommentedMap and CommentedSeq. I'm not saying this is bad; it's just a surprising side-effect. I don't mind the stricter typing. That said, all 1,188 of my unit tests, plus...
Quick follow-up: I'm unable to use 0.17.5 because my tests won't complete with it. It looks like a couple of Type signatures are misaligned with their signatures: mypy errors: ./lib/python3.6/site-packages/ruamel/yaml/comments.py:92: error: Type signature has too few arguments ./lib/python3.6/site-packages/ruamel/yaml/tokens.py:348: error: Type signature has too few arguments I know you were having some issues getting your test suite to run. I hope the above can help in some way.
That was it! All of my unit tests are passing and the issue with Aliased entries in Unordered Sets is resolved with this workaround. Until your next release -- and I do hope the tox issue turns out to be something simple -- I am no longer blocked, thanks to your help. Please feel free to de-prioritize this issue to major. Thank you, so much!!
I forgot to mention: In the attached aliasstyle.py above, I commented out the following line so you can see the original error which led me down the path described above. # ruamel.yaml.nodes.MappingNode = MyMappingNode Just uncomment that line to get the followup error.
I've narrowed the code causing the above EmitterError down to the following example: import sys import ruamel.yaml inp = """\ --- device_defaults: &device_defaults os: unknown type: unknown platform: unknown notes: N/A devices: R1: <<: *device_defaults os: ios type: router platform: asr1k """ yaml = ruamel.yaml.YAML() if ruamel.yaml.version_info < (0, 17, 5): # pylint: disable=locally-disabled,import-outside-toplevel from yamlpath.patches.aliasstyle import MySerializer # type: ignore from yamlpath.patches.aliasstyle...
I'm lost "peeling the onion" with this; fixing one issue reveals more issues. The surface error from adding the above code is: AttributeError: 'MappingNode' object has no attribute 'style'. So, I added the following to the patch code: class MyMappingNode(ruamel.yaml.nodes.CollectionNode): __slots__ = 'merge', 'style' id = 'mapping' def __init__( self, tag, value, start_mark=None, end_mark=None, flow_style=None, comment=None, anchor=None, style=None, ): # type: (Any, Any, Any, Any, Any, Any, Any,...
Indeed, with the code above added to my project, several unit tests are now failing that had previously been passing. It looks like most are related to the new style attribute not present in other classes. I'll poke at it a bit to see if I can get everything passing again. Thanks for the head start!
Anchored Entries in Unordered Sets Cannot Round-Trip
I really cannot thank you enough for taking some time to look at this! I am really looking forward to your new comment handling approach. Back on-topic: Your suggestion indeed gets the folding to work without the Monkey-Patch and without reducing yaml.width. An alternative way for FoldedScalarString to accept pre-folded strings would be much appreciated as a later enhancement. As it is, this ticket can be closed. For this specific use-case, the replacement data is invariably coming from the same...
I really cannot thank you enough for taking some time to look at this! I am really looking forward to your new comment handling approach. Back on-topic: Your suggestion indeed gets the folding to work without the Monkey-Patch and without reducing yaml.width. This bug can be closed. An alternative way for FoldedScalarString to accept pre-folded strings would be much appreciated as a later enhancement. As it is, this ticket can be closed. For this specific use-case, the replacement data is invariably...
I had tried the .replace() trick before opting to Monkey-Patch the method, but had neither noticed .fold_pos nor considered casting the new value right into a FoldedScalarString. I will experiment with this and I'm very glad there may be a viable solution without requiring a code change to ruamel.yaml. I am concerned about the potential behavior of reducing yaml.width because some of my users require extremely long String values (not wrapped), which led to my setting yaml.width = sys.maxsize. This...
I had tried the .replace() trick before opting to Monkey-Patch the method, but had neither noticed .fold_pos nor considered casting the new value right into a FoldedScalarString. I will experiment with this and I'm very glad there may be a viable solution without requiring a code change. I am concerned about the potential behavior of reducing yaml.width because some of my users require extremely long String values (not wrapped), which led to my setting yaml.width = sys.maxsize. This is in addition...
I just copy-pasted the above example code into a fresh file on Ubuntu 20 and ran it; it still generated the doubling-up of the new-lines: $ cat yaml_dump.yaml --- key: > ENC[PKCS7,MIIBmQYJKoZIhvcNAQcDoIIBijCCAYYCAQAxggEhMIIBHQIBADAFMAACAQEw DQYJKoZIhvcNAQEBBQAEggEAB6I+mF2l3AQdkEjkLhroaRYYGa17aizmaWkJ gJfGbTSk/8FfBPsMJ6yDy5w3TJG4J5NaDmk9MtukPO/D/R+AYuIZee1xMnYj 0Q68Iyrj9RyJq59B9wS5LyMOpFtAtJvMAi/1fEFc8yzZpApZ3SskB1lmLdUA oxLMkLcQ2u0K4nzo9IcISIXu/YK4dJyvmr9AnB4XeIt6AlvAZG3eB4gRGHVj iygaCeSjP5Cy6PLiG3l5eQyN/uyirncPwe5/W5I9J28L4XXtJgWQ0KCEUyW3...
I'm very surprised that you don't see this behavior! I see this doubling-up of the new-lines on all of the following (I keep a workstation with each of these OSes and Python combinations specifically for multi-platform dev and testing): Operating Systems: OSX, Windows 10, Ubuntu 20 (LTS) Pythons: 3.6, 3.7, 3.8, 3.9 ruamel.yaml versions: 0.15.96, 0.16.12, 0.17.4 The input data is exactly as you see above with no special characters other than simple \ns. Even on Windows 10 with CRLF characters, the...
I'm very surprised that you don't see this behavior! I see this doubling-up of the new-lines on all of the following (I keep a workstation with each of these OSes and Python combinations specifically for multi-platform dev and testing): Operating Systems: OSX, Windows 10, Ubuntu 20 (LTS) Pythons: 3.6, 3.7, 3.8, 3.9 ruamel.yaml versions: 0.15.96, 0.16.12, 0.17.4 The input data is exactly as you see above with no special characters other than simple \n.s Even on Windows 10 with CRLF characters, the...
I'm very surprised that you don't see this behavior! I see this doubling-up of the new-lines on all of the following (I keep a workstation with each of these OSes and Python combinations specifically for multi-platform dev and testing): Operating Systems: OSX, Windows 10, Ubuntu 20 (LTS) Pythons: 3.6, 3.7, 3.8, 3.9 ruamel.yaml versions: 0.15.96, 0.16.12, 0.17.4 The input data is exactly as you see above with no special characters other than a simple \n. Even on Windows 10 with CRLF characters, the...
I'm sorry this ticketing system won't allow me to edit this to fix the unexpectedly indented triple-grave-accents (back-ticks) for the Actual and Desired output samples. I really didn't see those indents while I was composing this ticket.
Spurious new-lines in Folded values
Thank you! Because some platform package builders are still supporting OSes with access only to older versions of ruamel.yaml, I'll try to exclude 0.17.x up to 0.17.3 by using PEP440 syntax: install_requires=[ "ruamel.yaml>=0.15.96,!=0.17.0,!=0.17.1,!=0.17.2", ], I'll have to hold my next release pending your 0.17.3 release since this indeed blocks my project from proceeding. By chance, do you have a ballpark ETA for 0.17.3?
I'm guessing this critical bug has been ignored since 2020-06-02 because I didn't provide code to reproduce the error. So, here's the missing code example: import warnings from sys import maxsize import ruamel.yaml # type: ignore from ruamel.yaml import YAML from ruamel.yaml.parser import ParserError from ruamel.yaml.composer import ComposerError, ReusedAnchorWarning from ruamel.yaml.constructor import ConstructorError, DuplicateKeyError from ruamel.yaml.scanner import ScannerError from ruamel.yaml.scalarstring...
ruamel.yaml 0.17.2 Lost Parsing Error Specificity Available to 0.16.12 and Older
AssertionError on dump with commented, aliased map key and child