Menu ▾ ▴

#117 x3d.py production errors creating HAnim nodes in XML output

X3D4
closed
Don Brutzman
python (4)
2026-08-05
2026-06-19
Don Brutzman
No
1 Attachments
JinLOA1.py.traceLog.txt

Discussion

  • John W Carlson

    John W Carlson - 2026-06-19

    Thanks, Don. Very much appreciated. I've been avoiding the stairs to my office. I'll work on getting the laptop upstairs.

     

  • John W Carlson

    John W Carlson - 2026-06-19

    Note, typically HAnimSegment USE nodes are placed in containerField="segments" which should be deleted (per 4.1) or sorted after the HAnimHumanoid.skeleton field.

     

  • Don Brutzman

    Don Brutzman - 2026-06-22

    I have improved the sort of child nodes within HAnimHumanoid, which mostly solves the XML-output issue 1.

    For issue 2, while looking at how to properly handle generating the containerField value, realized
    a. No containerField information is applied anywhere in x3d.py so this is a missing capability (rather than an incorrect configuration).

    b. Python objects have no way to find out whether they have a parent, and indeed may have multiple parents. Thus there is insufficient information available to a given node, which is a design deficiency in the current structure of the x3d.py library.

    Fixing this design gap will require adjustments to each node's autogenerated field signatures and setter methods. Am looking to add the following properties to each node's class: parentNode linking to the parent node, and parentField that identifies the name of the parent node's field that is being set by the parent adding the node. These additions will enable proper identification of XML containerField values when special parent-child field relationships are needed. Having access to this information will also provide further expressive power to Python programmers using the x3d.py library.

     

    Last edit: Don Brutzman 2026-06-22

  • Alexander Hoffman

    Alexander Hoffman - 2026-06-23

    Hi John and Don,

    Thank you for addressing this, I have been traveling so I'm just getting back to my x3d experiments. I will update to 4.1 for future projects and I'm happy to resolve the ticket on Github when you deem fit. It seems issue 1 is resolved.

    Claude walked through the x3d.py source (4.0.65.3) and pointed to an example. Take PhysicalMaterial's XML() serializer as an example — around line 70878 it does:

      if self.baseTexture:  # output this SFNode
      result += self.baseTexture.XML(indentLevel=indentLevel+1, syntax=syntax)
  if self.emissiveTexture:  # output this SFNode
      result += self.emissiveTexture.XML(indentLevel=indentLevel+1, syntax=syntax)

    The parent is iterating its named fields — it knows this texture is baseTexture. But XML() takes no containerField parameter, so the child (ImageTexture) has no idea which slot it's sitting in. The child just writes <imagetexture ...=""> and hopes for the best. A conformant reader falls back to the default containerField (texture), which PhysicalMaterial doesn't define, and the texture silently vanishes.</imagetexture>

    The parentNode / parentField approach you're proposing is the right fix — it's more robust than just patching the serializer, since it also gives Python programmers proper parent-awareness for programmatic use. Worth noting that Vincent Marchetti built a working containerField extension back in March (https://github.com/vincentmarchetti/x3dpy_containerfield) using an XSLT stylesheet that patches the code generator. His approach and yours are complementary — his solves the serialization side, yours solves the object-model side.

    Here's a minimal PBR reproduction to complement John's HAnim examples. It's a different part of the spec hitting the exact same bug — every non-default containerField is affected:

      """
  PBR texture silently vanishes — containerField='baseTexture' never emitted.
  x3d 4.0.65.3, Python 3.12. Confirmed in X_ITE and Castle Model Viewer 5.3:
  texture renders only after manually injecting containerField='baseTexture'.
  """
  from x3d import x3d as X

  brick_texture = X.ImageTexture(url=["brick.png"])
  brick_material = X.PhysicalMaterial(
      baseColor=(1, 1, 1),
      roughness=0.8,
      metallic=0.0,   
  )
  brick_material.baseTexture = brick_texture  # correct in memory

  brick_box = X.Shape(
      appearance=X.Appearance(material=brick_material),
      geometry=X.Box(size=(2, 2, 2))
  )

  ground = X.Shape(
      appearance=X.Appearance(
          material=X.PhysicalMaterial(baseColor=(0.3, 0.6, 0.2), roughness=1.0)
      ),
      geometry=X.Box(size=(20, 0.1, 20))
  )

  scene = X.X3D(
      version="4.1",
      head=X.head(),
      Scene=X.Scene(children=[  
          X.Viewpoint(position=(5, 4, 5), orientation=(-0.2, 1, 0.1, 0.8)),
          X.DirectionalLight(direction=(-0.5, -1, -0.3)),
          X.Transform(translation=(0, 1, 0), children=[brick_box]),
          ground,
      ])
  )

  print(scene.XML())
  # ACTUAL output:
  #   <PhysicalMaterial>
  #     <ImageTexture url='"brick.png"'/>           <-- no containerField
  #   </PhysicalMaterial>
  #
  # EXPECTED:
  #   <PhysicalMaterial>
  #     <ImageTexture containerField='baseTexture' url='"brick.png"'/>
  #   </PhysicalMaterial>
  #
  # The box renders flat white. No error, no warning, validates clean.

    You're left staring at a white box wondering what you did wrong. The answer is nothing — the serializer just didn't write down which slot it put the texture in. The same pattern hits all five PBR texture slots and every HAnim skeleton/joints/segments placement.

     ▎ On the multiple-parents point: in X3D, USE is the mechanism for sharing a node across parents, and a USE reference is a distinct object in the 
  ▎ scene graph — so in practice each Python object has one parent assignment at serialization time. The edge case would be someone assigning the 
  ▎ same ImageTexture instance to two different materials' baseTexture fields without USE, which is already ill-formed X3D. So _parentField_ should
  ▎ be safe as a single value, not a list.

    Happy to help test once the parentField plumbing is in place.

    I hope you find this helpful!

    — Alexander

     

  • Don Brutzman

    Don Brutzman - 2026-06-30

    Issue 1 and issue 2 now appear to be fixed. See attached output, note corrected order of child nodes under HAnimHumanoid as well as inclusion of non-default containerField values on USE nodes.

    Release 4.0.65.5 of updated x3d.py is now available at

     

    Last edit: Don Brutzman 2026-06-30
    JinLOA1.py.traceLog.txt

  • Don Brutzman

    Don Brutzman - 2026-07-04
    • status: open --> closed
     

  • Don Brutzman

    Don Brutzman - 2026-08-02
    • status: closed --> open
     

  • Don Brutzman

    Don Brutzman - 2026-08-02

    Apologies for prematurely closing this ticket before all corrective actions were complete.

    Please expand the excerpt to provide a full test program (e.g. PhysicallyBasedMaterialTest.py or somesuch) so that this issue can be thoroughly tested. I will place the program into version control as part of X3DPSAIL python/examples so that it can serve as a regression test for long-term quality assurance (QA).

     

    Last edit: Don Brutzman 2026-08-02

  • Alexander Hoffman

    Alexander Hoffman - 2026-08-03

    Thank you for reopening it — the close was reasonable given what the ticket said at the time.

    First, a correction from my side. I posted a reproducer earlier today built against x3d.py 4.0.65.4, which is what my environment had pinned. You had already announced the fix in 4.0.65.5 on 30 June in this ticket, and you were right — I should have re-tested against the release you named before posting. Verified now across both:

    check 4.0.65.3 4.0.65.5
    HAnimJoint in skeleton gets containerField='skeleton' FAIL PASS
    HAnimSegment in segments gets containerField='segments' FAIL PASS
    DEF precedes USE in document order FAIL PASS
    ImageTexture in baseTexture gets containerField='baseTexture' FAIL PASS
    HAnimHumanoid.version='2.0' serialized FAIL FAIL

    The fix is also correctly slot-aware rather than blanket — a texture in the default texture slot still emits no containerField, which is right. The test below asserts that as an explicit control, so a later change cannot regress into emitting it everywhere.

    Output on 4.0.65.5:

    <Appearance>
  <PhysicalMaterial DEF='brick'>
    <ImageTexture DEF='btex' url='"brick.png"' containerField='baseTexture'/>
  </PhysicalMaterial>
</Appearance>

<HAnimHumanoid DEF='hanim_Test' name='Test'>
  <HAnimJoint DEF='hanim_humanoid_root' name='humanoid_root' containerField='skeleton'>
    <HAnimSegment DEF='hanim_sacrum' name='sacrum'/>
  </HAnimJoint>
  <HAnimSegment USE='hanim_sacrum' containerField='segments'/>
</HAnimHumanoid>

    PhysicallyBasedMaterialTest.py

    Named as you suggested, and written for the use you described — a long-term QA regression test in python/examples.

    • Only dependency is x3d.py (plus sys). No xmllint, no local schema copy, no network. Python 3.6+.
    • Exit status is deliberate. It exits 0 on 4.0.65.5 and 1 on 4.0.65.3. The one still-failing item is reported as [OPEN] and does not fail the run, because a QA test that is permanently red from a known-unfixed issue stops being a signal — the build is already broken, so a real regression changes nothing visible. --strict fails on open items too, if you would rather it did.
    • Items 1–3 are the lock on your fix. If any starts failing, it regressed.

    The single remaining item on 4.0.65.5 is HAnimHumanoid.version dropped from XML — set on the object and retained there (obj.version == '2.0'), absent from the serialization. It is not in the ticket text, but your own "expected output" excerpt above carries version='2.0', so I believe it belongs. Happy to open it as its own ticket if you would rather close this one.

      1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
    #!/usr/bin/env python3
"""
PhysicallyBasedMaterialTest.py

Regression test LOCKING IN the containerField fix released in x3d.py 4.0.65.5
(SourceForge x3d ticket #117 / Python-SAI issues #2, #3).

    python3 PhysicallyBasedMaterialTest.py           # 0 = no regression
    python3 PhysicallyBasedMaterialTest.py --strict  # also fail on open items

EXIT STATUS IS DELIBERATE. On 4.0.65.5 this exits 0 even though item 4 below is
still broken, because item 4 is KNOWN-OPEN upstream rather than a regression. A
QA test that is permanently red because of an unfixed issue stops being a signal
-- the build is already broken, so a real regression changes nothing visible.
Items 1-3 are the lock: if any of them starts failing, the fix has regressed and
the exit status goes to 1.

    4.0.65.3  ->  exit 1  (items 1-3 fail: the pre-fix behaviour)
    4.0.65.5  ->  exit 0  (items 1-3 pass, item 4 reported as OPEN)

STATUS: all containerField checks below PASS on 4.0.65.5 and FAIL on 4.0.65.3.
This test exists to keep it that way, which is what Don Brutzman asked for --
a program that can live in X3DPSAIL python/examples as long-term QA.

WHAT IT COVERS


  1. PhysicalMaterial + baseTexture -- the case from the June 23 comment. An
     ImageTexture in the `baseTexture` slot must serialize with
     containerField='baseTexture'. Without it the texture reparses into the
     default `texture` slot, which PhysicalMaterial does not define, and the
     surface renders untextured with no error anywhere.


  2. The default slot must NOT carry containerField. Emitting it there would be
     redundant, and canonical X3D omits an attribute equal to its default. This
     is the control: it proves the fix is slot-aware rather than blanket.


  3. HAnimHumanoid skeleton/segments -- containerField on non-default node
     fields, and DEF ordered before USE.


  4. HAnimHumanoid.version -- STILL FAILING as of 4.0.65.5. Set on the object,
     absent from the XML. Kept in this file as an open item rather than split
     into its own, so one run reports the whole picture.

Alexander Hoffman, 2026-08-03. Verified against x3d.py 4.0.65.3 (fails 1-3) and
4.0.65.5 (passes 1-3, fails 4).
"""
import sys

from x3d import x3d as X

VERSION = 2
FAILURES = []      # a REGRESSION -- the build should go red
OPEN = []          # a known-open defect -- reported, does not fail the run


def check(label, ok, expected, actual, open_issue=False):
    """open_issue=True marks a defect that is KNOWN-OPEN upstream.

    It is reported but does not fail the run. A QA regression test that is
    permanently red because of an unfixed issue stops being a signal -- the
    build is already broken, so a real regression changes nothing visible.
    Run with --strict to fail on open items too.
    """
    tag = 'OPEN' if (not ok and open_issue) else ('PASS' if ok else 'FAIL')
    print(f"  [{tag}] {label}")
    if not ok:
        print(f"         expected: {expected}")
        print(f"         actual  : {actual}")
        (OPEN if open_issue else FAILURES).append(label)


def main():
    print(f"PhysicallyBasedMaterialTest v{VERSION} -- x3d.py "
          f"{getattr(X, '__version__', '(version attribute absent)')}\n")

    # --- 1. non-default slot: baseTexture ---------------------------------
    app = X.Appearance(material=X.PhysicalMaterial(
        DEF="brick", baseTexture=X.ImageTexture(DEF="btex", url=["brick.png"])))
    xml = app.XML()
    print("  --- PhysicalMaterial with baseTexture ---")
    for line in xml.splitlines():
        print("  " + line)
    check("ImageTexture in `baseTexture` carries containerField='baseTexture'",
          "containerField='baseTexture'" in xml,
          "<ImageTexture ... containerField='baseTexture'/>",
          "no containerField -- reparses into `texture`, which "
          "PhysicalMaterial does not define; the surface renders untextured")

    # --- 2. control: the DEFAULT slot must stay bare -----------------------
    bare = X.Appearance(texture=X.ImageTexture(DEF="t2", url=["b.png"])).XML()
    check("control: default `texture` slot emits NO containerField",
          "containerField" not in bare,
          "<ImageTexture DEF='t2' url='\"b.png\"'/>", bare.strip())

    # --- 3. HAnim non-default node fields ---------------------------------
    h = X.HAnimHumanoid(
        DEF="hanim_Test", name="Test", version="2.0",
        skeleton=[X.HAnimJoint(DEF="hanim_humanoid_root", name="humanoid_root",
                               children=[X.HAnimSegment(DEF="hanim_sacrum",
                                                        name="sacrum")])],
        segments=[X.HAnimSegment(USE="hanim_sacrum")])
    hx = h.XML()
    print("\n  --- HAnimHumanoid ---")
    for line in hx.splitlines():
        print("  " + line)
    check("HAnimJoint in `skeleton` carries containerField='skeleton'",
          "containerField='skeleton'" in hx, "containerField='skeleton'",
          "absent -- the joint reparses into `children` and the skeleton "
          "stops being a skeleton")
    check("HAnimSegment in `segments` carries containerField='segments'",
          "containerField='segments'" in hx, "containerField='segments'", "absent")

    i_def, i_use = hx.find("DEF='hanim_sacrum'"), hx.find("USE='hanim_sacrum'")
    check("DEF precedes USE in document order",
          -1 < i_def < i_use, "the DEF node serializes first",
          f"DEF at {i_def}, USE at {i_use} -- USE is a forward reference")

    # --- 4. still open on 4.0.65.5 ----------------------------------------
    check("HAnimHumanoid.version='2.0' is serialized",
          "version=" in hx, "<HAnimHumanoid ... version='2.0'>",
          f"absent from XML although the object still holds "
          f"{getattr(h, 'version', None)!r}",
          open_issue=True)

    strict = "--strict" in sys.argv
    print()
    if OPEN:
        print(f"KNOWN-OPEN ({len(OPEN)}), not counted as regressions: "

              + "; ".join(OPEN))
    if FAILURES:
        print(f"REGRESSION ({len(FAILURES)}): " + "; ".join(FAILURES))
        return 1
    if OPEN and strict:
        print("--strict: failing on known-open items.")
        return 1
    print("No regressions." + (" Known-open items above." if OPEN else ""))
    return 0


if __name__ == "__main__":
    sys.exit(main())

    Also worth flagging separately, since it is broader than this ticket and still reproduces on 4.0.65.5: x3d.py binds xmlns:xsd to https://www.w3.org/2001/XMLSchema-instance, where the canonical namespace name is http:// (no s). Namespace names are compared as literal strings rather than resolved, so xsd:noNamespaceSchemaLocation stops being recognised and every document x3d.py writes fails XSD validation. Changing that one character makes the same bytes validate. Details and an isolating test are in Python-SAI issue #3.

     

Log in to post a comment.

MongoDB Logo MongoDB