Menu

Congrats on the last release + issues with STL and other random nitpicking ;)

Help
2021-02-22
2022-04-10
<< < 1 2 3 4 > >> (Page 2 of 4)
  • Pete

    Pete - 2021-10-14

    And a few more thoughts/findings.

    It came to my mind that it could make sense to have an option to export objects into separate files in binary (and why not ascii as well) format. But then what should happen with children? I suppose the separation to files should happen on top level only and children should got to the same file with the parent ... or then there should be a setting to that too.... Gets complicated. Maybe not such a good idea after all.

    In the (Wikipedia) description of the binary form it says that the header is 80-bytes. I thought that would mean a fixed field length but the exporter does not seem use 80 bytes. Only as much as is needed. -- This does not seem to be a problem to other readers though.

    The binary header does not seem to be used for any purpose, but I wonder if it would be better to use the AoI-filename instead of an object name as long as the objects are not separated to solids.

    Also the filename suggestion "Untitled" every time seems a bit unfinished as the dialog remembers at least some of the settings. Why not get the AoI-file name for the firts export (from that file) and the following times remember the last used name?

    At least 3D-Builder can not read compressed STL-files (binary or ascii) produced by AoI (Error code 0x80004005). Somehow the CAD can.

    There has often been something strange going on with the "center" function. I'm not exactly sure what it is supposed to do but one time quite recently it seemed to place the model to 0+ coordinates in all directions, which somehow complies with the original idea of STL. Howerer that is not how I understand "center". A much more usual case is that it moves the model farther away from the world center as if the translation vector had been inverted. Setting postion to 0,0,0 moves the model to where it was in the exported environment.

    Just things to consider, not necessarily very well premeditated. :)

     

    Last edit: Pete 2021-10-14
  • Luke S

    Luke S - 2021-10-22

    Did a bit of work on this today:

    • More reasonable normal checking for both ASCII and binary
    • can now read a solid name without the quotes. Not sure if this will work with multiple solids, as I don't have a proper test file.
    • Extended-ASCII support. (All printable ISO-8859-1 characters) Going beyond this will be rather more annoying.

    I also found out what the quotes were for in the first place: Apparently, NTJ was following or invented an undocumented format for including metadata. When I export to STL/ASCII I get this:
    solid "lucy"; Produced by Art of Illusion 3.2.0, Thu Oct 21 19:37:16 CDT 2021

    Please try it out with some of your problem files and see if anything sticks:

     
  • Pete

    Pete - 2021-10-23

    A quick check:

    • Reads with the end solid name,
    • The ";" for comment appears in the imported name
    • The quotes appear in the imported name, but those probably should if anybody wrote them.
    • No error list until I manually inverted one triangle.
      • Reports about edge count mismatch in that case and calls it a hole.

    Got to check with multiple objects back at work.

    Extended-ASCII support. (All printable ISO-8859-1 characters) Going beyond this will be rather more annoying.

    I came to think that it might not be necessary to worry about 4-byte characters in export. If someone exports with 4-byte characters they'd be read back as 2-bytes, skipping nulls and control characters. Looks weird but it should not mess up the format.

     

    Last edit: Pete 2021-10-23
  • Luke S

    Luke S - 2021-10-23

    The ";" for comment appears in the imported name

    From what I can tell, STL does not have a comment syntax. ; should just be treated as text. Do you have references that state otherwise?

    The quotes appear in the imported name, but those probably should if anybody wrote them.

    I thought that I had stripped them, at least if the name opens with a quote.

    Definitely not supporting 2- or 4 byte characters. ISO-8859-1 fits in a single byte. I was referring to Windows CP-1252, which has a bunch of random unicode points thrown into some of the unused deadspace in ISO-8859-1

     
  • Pete

    Pete - 2021-10-23

    From what I can tell, STL does not have a comment syntax. ; should just be treated as text. Do you have references that state otherwise?

    This is not mentioned in virtually any sources I could find but for one thing AoI exports with a comment on the first solid line and those files import just fine to other software. Here is someone (the only one I found) who seems to know it: https://www.grasshopper3d.com/forum/topics/add-tag-information-in-stl-file

    That of course applies to ASCII STL only. I tried to find free files to see if any of them would contain comments but so far only found models in binary.

    There does not seem to be a full formal description of StL available anywhere. Not even at 3D-Systems, who originally created it. All is just "common knowledge", that everybody has been copying from each others for decades. In many -- I dare say most -- cases even far more important things than ; for comment are not mentioned.

    A few years back I (as a part of a course I was taking) wrote a treatise on 3D-printing file formats and did a much more thorough search, with the same results. That time I found it mentioned that 3DS originally had plans to develop the format further, but for some reason they never did.

    I thought that I had stripped them

    Well, at least the version I got downloaded still both wrote and read quotes.

    fits in a single byte

    Aaarg... I had the image of the HEX-codes in my mind when I wrote that. What exceeds the 256 character ASCII is written with four characters as HEX ... and a character takes a byte. :D

     

    Last edit: Pete 2021-10-23
  • Luke S

    Luke S - 2021-10-24

    Thanks for the grasshopper link. That's the first I've heard of it. Wonder what the sourcing is... STL is a rather odd old beast.

    the version I got downloaded still both wrote and read quotes.

    Was the quote the first character in the <name> segment?

    @Fernando: Thanks. I was already aware of the Paul Bourke listing. The stackoverflow post is missing a bit: ASCII STL allows multiple solids, so can't guarantee that the last endsolid will be followed by the same <name> as the opening solid.

     
  • Pete

    Pete - 2021-10-24

    Was the quote the first character in the <name> segment?

    The export is just as before.

    solid "Cylinder_1"; Produced by Art of Illusion 3.2.0, Sun Oct 24 10:43:28 EEST 2021
    

    with no name after endsolid

    After Import the object name becomes "Cylinder_1";

    And I'd suggest to change the smoothing method to Nonefor the imported StL. When it is Shaded as it is by default, large flat surfaces look pretty weird, as the renderers show them as bent and smoothed. (See Fernando's screen capture before.)

     

    Last edit: Pete 2021-10-24
  • Pete

    Pete - 2021-10-25

    ...And it reads multiple objects from ASCII files and those almost come with names. What happens it that each name is cut at the first space in the name. 'C:\users\me\desktop\My part 1' becomes just 'C:\users\me\desktop\My'. Backlashes and colons are read as normal text.

    Also I still get the massive normal error list in those those imports.

     
  • Luke S

    Luke S - 2021-10-27

    RE: names with spaces: I'll take a look.

    I still get the massive normal error list in those those imports

    On manual inspection, do the errors Look like rounding errors, or are they more severe than that? Could you cut and paste a few lines of typical errors?

     
  • Pete

    Pete - 2021-10-27

    Could you cut and paste a few lines of typical errors?

    To me this looks like the exact same thing as before. This is a kind of strange as with the inverted triangle the error messages were entirely different and that one did not worry about minor normal direction mismatches.

    I don't know, I haven't had a look but it might happen that the exporting sw internally uses doubles and converts the numbers to singles at export. Then at import AoI converts them back to doubles and recalculates the normal whic then has the cumualtive error of three vertors that at some point were in singleprec.

    The error seems to be at about the 7th decimal which is far more precice that needed.

    Warning (line 7): facet normal mismatch. read: Vec3: 0.991445582, 0.0349328444, 0.125759116; calculated: Vec3: 0.9914453246636405, 0.034933365161394336, 0.1257609963427019
    Warning (line 14): facet normal mismatch. read: Vec3: 0.923846675, 0.102443438, 0.368798947; calculated: Vec3: 0.9238494672375656, 0.1024416301334135, 0.36879245423473006
    Warning (line 21): facet normal mismatch. read: Vec3: 0.923900158, 0.102409523, 0.368674365; calculated: Vec3: 0.9239012290554347, 0.10240880654085538, 0.36867188025768194
    Warning (line 28): facet normal mismatch. read: Vec3: 0.793366759, 0.162926548, 0.586535699; calculated: Vec3: 0.7933604273354327, 0.16292871649074794, 0.586543660506886
    Warning (line 35): facet normal mismatch. read: Vec3: 0.608852446, 0.212317416, 0.764342864; calculated: Vec3: 0.6088504171379824, 0.21231786031654348, 0.7643443567800509
    Warning (line 42): facet normal mismatch. read: Vec3: 0.793366759, 0.162926548, 0.586535699; calculated: Vec3: 0.7933684725848467, 0.16292597089766692, 0.5865335409978187
    
     

    Last edit: Pete 2021-10-27
  • Luke S

    Luke S - 2021-10-28

    Think I got the Quoted names sorted out.
    Also properly ignoring comments.
    Cleaning up the Facet normal checks is going to be... interesting, at best. See if this is any better.

    Spaces in names may be a bit trickier - I'm tempted to re-write the whole ASCII parser in a different style, just to make certain things easier. The tricky thing is STL is a white-space delimited format: One source seems to imply that the CR/LF/EOL stuff is optional, so we can't just read name until the end of the line?

     

    Last edit: Luke S 2021-10-28
  • Pete

    Pete - 2021-10-28

    Think I got the Quoted names sorted out. Also properly ignoring comments.

    Exports with quotes imports without and without the comment

    Cleaning up the Facet normal checks is going to be... interesting, at best. See if this is any better.

    Not really. It seems that some files might be imported without much of an error list but the file that produced the list in my last post still does that.

    I'm tempted to re-write the whole ASCII parser

    Go ahead. There is no reason the put old code into artificial life support. It has served its purpose if the the next generation can be made in a smarter way.

    Basically I'd split the code into separate files of Exporter, Importer and UI just for clarity. And probably the "root" plugin.

    One source seems to imply that the CR/LF/EOL stuff is optional, so we can't just read name until the end of the line?

    Which source would that be? That must be a misinterpretation of something else. As long as I remember ASCII STL has always been written line by line and besides the usage of ; for comments would contradict the idea of line changes being optional: The first ; would comment the rest of the file out.

     
  • Pete

    Pete - 2021-10-28

    the Facet normal checks

    I'm not sure what the original code is trying to do comparing lengths? The length has nothing to do with the direction.

    Starting at line 637:

    calcNorm = v2.minus(v1).cross(v3.minus(v1));
    double length = calcNorm.length();
    if (length > 0.0) 
        calcNorm.scale(1.0/length);
    
    if (!calcNorm.equals(norm) && Math.abs(calcNorm.minus(norm).length()) > tolerance)
        // Write the error message
    

    I'd just

    calcNorm = v2.minus(v1).cross(v3.minus(v1));
    calcNorm.normalize();
    norm.normalize();
    double projection = norm.dot(calcNorm);
    
    if (projection < 0.999)
         // The error is significant, probably not just numerical. 
         // Arccos(0.999) = 2.562°, just an easy number to check.
    if (projection < 0.0) 
        // The direction is inverted. (Either of the triangle or the normal.)
    
     

    Last edit: Pete 2021-10-28
  • Luke S

    Luke S - 2021-10-28

    The Wikipedia entry claims:

    White space (spaces, tabs, newlines) may be used anywhere in the file except within numbers or words.

    Not clearly sourced, though, so ??? I did get the feeling that the comment syntax was added on later, so some inconsistency is not unexpected.

    I still have mixed feelings about spaces in names, but I think It's doable.

     
  • Pete

    Pete - 2021-10-28

    The Wikipedia entry claims:

    Adding to that the rest of the paragraph, I'd read it that extra spaces or newlines shouldn't hurt between words and numbers. EDIT and RE-EDIT : If the solid line were split into several lines then the line starting with facet would be where the 3D content begins.

    I wonder (and this is entirely hypothetical) if that book (Burns, Marshall (1993). Automated Fabrication. Prentice Hall. ISBN 978-0-13-119462-5.), would be hard to find? That looks like it could be closest to the original of all available sources.

    comment syntax was added on later

    Possibly so. There seem to be differences how everybody has interpreted the STL syntax over the years.

     

    Last edit: Pete 2021-10-28
    • Fernando Cassia

      Fernando Cassia - 2021-11-03

      On Thu., 28 Oct. 2021, 15:51 Pete, peteihis@users.sourceforge.net wrote:

      The Wikipedia entry claims:

      Adding to that the rest of the paragraph, I'd read it that extra spaces
      or newlines shouldn't hurt between words and numbers. If the header would
      be split on several lines, then the one starting solid should mark the
      beginning of the 3D-content.

      I wonder (and this is entirely hypothetical) if that book (Burns,
      Marshall (1993). Automated Fabrication. Prentice Hall. ISBN
      978-0-13-119462-5.)
      , would be hard to find? That looks like it could be
      closest to the original of all available sources.

      $6.50 + 4.00 USD in shipping.

      https://www.ebay.com/itm/Automated-Fabrication-Improving-Productivity-in-Manufacturing-by-Burns-/184523373998

      FC

       
      • Pete

        Pete - 2021-11-06

        Ha! :D

        For $6.50 I might think about buying it just for the fun of it but this seller does not ship to this part of the World. There seem to be other sellers that might though...

        On pictures the book does look like the relic it is. -- To bring dome dignity to the book shelf :D

         
  • Luke S

    Luke S - 2021-10-28

    RE: the facet-normal checks & length:

    • The first check/use of length is an ad-hoc normalization. Vec3 now has this built in as a method, as you showed in your suggestion.
    • The second is trying to compare the magnitude of the difference to the tolerance. Dot product is probably a better answer here...

    If I do re-write this, we'll probably lose the centering function, and possibly support for positive-octant-only exports (If we ever had that - I'm sometimes confused by the options for this plugin)

     

    Last edit: Luke S 2021-10-28
  • Pete

    Pete - 2021-10-28

    probably lose the centering function and possibly support for positive-octant-only

    Well, those would be pretty easy stuff to add once the original import is right. As I said before the "center" and actually "frame" functions have not been too reliable anyway.

     

    Last edit: Pete 2021-10-28
  • Luke S

    Luke S - 2021-10-29

    Well, the dot-product approach didn't work out as well as I'd hoped. I dropped the compare-threshold to .7, and was still getting a huge number of mismatches (pretty much every facet, in fact!) I got better results with the per-component ulp comparisons, but even that is getting ridiculous. I'm wondering if we should drop the Facet check altogether? It's not like it actually changes how we import the data, and the only time where mismatched normals are actually useful are files from SolidWorks that use them to hold shader data, which we can't use.

    I think that I've got the name-with-spaces thing fixed. Please try this.

     
  • Luke S

    Luke S - 2021-10-30

    At least 3D-Builder can not read compressed STL-files (binary or ascii) produced by AoI (Error code 0x80004005). Somehow the CAD can.

    This one's been bugging me a bit.

    1. Does the plugin actually compress ASCII? I don't think this should work...
    2. Ever found a reference for compressed STL? I understand how it's being done, but I don't think it's standard. At best, these should be *.stl.gz files.
     
  • Pete

    Pete - 2021-10-30

    Well, the dot-product approach didn't work out as well as I'd hoped. I dropped the compare-threshold to .7, and was still getting a huge number of mismatches (pretty much every facet, in fact!)

    What? You must have done something wrong, that piece is a simple as it can get! I got to digging and testing the code a bit and did this:

        calcNorm = v2.minus(v1).cross(v3.minus(v1));
        calcNorm.normalize();
        norm.normalize();
        double projection = norm.dot(calcNorm);
    
        if (projection < 0.999)
            if (projection < 0.0) 
                message.write("Inverted normal on line: " + lineno + "\n");
            else
                message.write("Normal direction mismatch on line: " + lineno + "\n");
    

    And that bit works exactly as it should. Tested by editing an ASCII STL file as exporters don't really make that kind of mistakes.

    I think that I've got the name-with-spaces thing fixed. Please try this.

    Yeap. Still exported with quotes but was reading correctly without quotes and with spaces.

    Does the plugin actually compress ASCII? I don't think this should work...
    Ever found a reference for compressed STL? I understand how it's being done, but I don't think it's standard. At best, these should be *.stl.gz files.

    It works perfectly well. As you probably found out, is uses java.util.zip.GZIPOutputStream to do that and also zip tools can uncompress the file it produces.

    It seems to me that this might have been becoming a more or less standard (or better said common) option at some point of time, though any contemporary "sources" don't mention it. The old CAD opens compsessed STL as nothing was wrong and does not even bother to notify the user. In a few weeks I may have access to a more more modern CAD, I can try what that one does with it.

    *.stl.gz

    I'm not sure about that. I'd rather leave it as it is for those who know where to use it. (Ref. the old CAD).

    3D-Builder, and as it would seem, some other 3D-viewers of later making don't read the compressed files at all. Just as an example, 3D-Builder refuses to even see any file extensions other than the formats it actually can read so there is not any way to have it read the compressed file. And same with Paint 3D.

    As a general note, Nik's wiring is a bit tricky to follow at times. He may for example reuse the same variable names in different parts of the same class file and then with the "VT100-style" coding you don't necessarily recognize which one it is this time.

    For the example frame is the boolean to tell if the view should zoom to the imported objects but it is also the BFrame that makes the UI. And as it turned out the boolean frame does get it's value from the checkbox, but it is never used. Not at least in the version, I was looking at. And even funnier, the checkbox is named frameBox which happens to be the the old method name in ViewerCanvas for zooming to a bounding box.

    And again I don't really see any benefit in using the binary coded int type. It is first constructed from booleans and then deconstructed back to booleans. -- Though without the binary coding then you'd nee those booleans to be visible to the entire class. And then all of a sudden typeis a RowContainer....

    And yet, just my personal opinion but I would have done a full white space & formatting make over first before attending anything else.

     

    Last edit: Pete 2021-10-30
  • Luke S

    Luke S - 2021-10-30

    What? You must have done something wrong,

    Entirely possible. I'll re-check.

    *.stl.gz

    I'm not sure about that. I'd rather leave it as it is for those who know where to use it. (Ref. the old CAD).

    Apparently that old CAD program transparently tries GUnzip on files. Which is fine, and probably a good idea. The problem is when you're trying to interop with programs that don't do that. The only difference between a *.stl.gz and a *.stl is that it's using the file extension to explicitly signal that the file has been run through gzip, and if your application doesn't support inflating it, you can do so separately. (Note how source downloads from github and similar have *.tar.gz files... similar principle)

    I would have done a full white space & formatting make over first before attending anything else.

    I was hoping to make the changes at this time rather minimal, as I hope to make some interface upgrades in core code that would require another re-write anyway.

     
  • Pete

    Pete - 2021-10-30

    What? You must have done something wrong,

    Entirely possible. I'll re-check.

    Did you notice that it appears in two places, both the ASCII and binary importers? (I forgot at first)

    For the binary there are no line numbers, so line --> face and lineno--> faceno :)

    The problem is when you're trying to interop with programs that don't do that.

    I can see that, but then this is not Github or alike either.

    Well, currently I cannot say for sure but somehow I believe that this might have been intended as an entirely internal option of stl. I'm starting to have a faint recollection that I could have seen it years ago it the tools I used at the time but I might be imagining it. At least don't anymore remember how it worked. Should get to test but no those tool around now. Anyway this has been like that for a long time now and I would think that Nik must have gotten it from somewhere that this should be the implementation.

    Additive manufacturing industry has for the most part abandoned stl and gone to direct slicers or more expressive triangle formats. Those who have developed the new ones just bluntly say that stl is obsolete but not everybody is there yet or will be any time soon.

     

    Last edit: Pete 2021-10-30
<< < 1 2 3 4 > >> (Page 2 of 4)

Log in to post a comment.