Dear Gwyddion Community,
The reason I am contacting you for help and seeking support from the community is that I have some measurement datasets from a Bruker InSight AFP system (more details in attachment). The first file is a screenshot of its Nanoscope software and the second file is the measurement data, while the third file is a 3D image of the surface. I am having difficulties in loading the data as a raw data file in Gwyddion. The reason I need to load the data in Gwyddion is because it seems to be the only software program that allows me to process and export the data in a straightforward manner, in the format required for a third-party software (ASCII data matrix). Any information you can provide will be very much appreciated.
Thank you in advance for your time and understanding.
The file did not open at all, with an error message about missing header field. This part should be fixed – sort of. I had to extend a list of ‘Z scale’ fields we are looking for (r26140). And I am losing confidence that the names to look for actually come from a finite list. But at least it now loads your file.
The heights look all right (surprisingly), but the X coordinates are botched, mangled and all kind of wrong. This is a result of attempts to read the DT files. But it did not really work for them either. I disabled the code trying to do the ‘smart’ and ‘correct’ thing and it probably makes the coordinates OK for your file (r26141).
The file opens as a set of profiles, not an image. This is because I came to conclusion that the DT profiles can be taken at arbitrary Y coordinates (not necessarily uniform). So data would not be an image – definitely not image representable in Gwyddion. But although the steps in X and Y differ a lot, maybe the sampling is still regular in both axes. At least in reality, if not in principle. So perhaps we can import it as an image. I will have to look into it.
👍
1
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Dear David,
Thank you for your very swift response and for your detailed explanation. I really appreciate it. I got the same error myself, when trying to load the file. That is why I resorted to trying to load the file as raw data (as I previously mentioned), but that did not work as well, since I do not have enough understanding of the data structure itself.
By the way, would sharing more files help in further understanding the unknown aspects of this type of data. Please let me know if it does.
Thank you for your time.
Respectfully,
Steven
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
So… The reason why it is loaded as a set of individual profiles is simple. Each profile can have a different number of points. It is definitely nothing like image data. In principle it can also be imported as XYZ data instead of profiles. Each has its advantages and disadvantages.
Profiles lose the slow axis coordinate (Y) but preserve profile identity. They are more immediately useful and match more closely the structure of the data. But making 2D data from them is PITA.
XYZ data preserve Y but lose information which point comes from which profile. You cannot do much with XYZ data in Gwyddion directly so you probably have to Rasterise first… Sorting out exported XYZ data back to individual profiles takes some work.
I also still do not really trust the Y coordinates as we create them.
👍
1
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Dear David,
Thank you for investigating further. I really appreciate the effort. From what I can understand, it is most likely better to use profiles instead of XYZ data (at least in most situations). I also guess not much more can be done about this... If I have misunderstood or if you have any other suggestions, please let me know.
Thank you for your time.
Respectfully,
Steven
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Dear David,
Firstly, I apologize for the late response. I have been in contact with the equipment vendor and they basically confirmed what we had already understood. Thank you for the information and the explanation. If I have any further questions on the matter or need more support, I will let you know.
Thank you again for your assistance and support.
Respectfully,
Steven
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Dear Gwyddion Community,
The reason I am contacting you for help and seeking support from the community is that I have some measurement datasets from a Bruker InSight AFP system (more details in attachment). The first file is a screenshot of its Nanoscope software and the second file is the measurement data, while the third file is a 3D image of the surface. I am having difficulties in loading the data as a raw data file in Gwyddion. The reason I need to load the data in Gwyddion is because it seems to be the only software program that allows me to process and export the data in a straightforward manner, in the format required for a third-party software (ASCII data matrix). Any information you can provide will be very much appreciated.
Thank you in advance for your time and understanding.
Respectfully,
Steven
Last edit: Steven Carneiro 2024-01-17
Thanks for the report. The good news is that the file is similar to the Bruker Deep Trench mode data which we do not how to handle correctly either. Yes, that was the good news… But let's unpack it.
The file did not open at all, with an error message about missing header field. This part should be fixed – sort of. I had to extend a list of ‘Z scale’ fields we are looking for (r26140). And I am losing confidence that the names to look for actually come from a finite list. But at least it now loads your file.
The heights look all right (surprisingly), but the X coordinates are botched, mangled and all kind of wrong. This is a result of attempts to read the DT files. But it did not really work for them either. I disabled the code trying to do the ‘smart’ and ‘correct’ thing and it probably makes the coordinates OK for your file (r26141).
The file opens as a set of profiles, not an image. This is because I came to conclusion that the DT profiles can be taken at arbitrary Y coordinates (not necessarily uniform). So data would not be an image – definitely not image representable in Gwyddion. But although the steps in X and Y differ a lot, maybe the sampling is still regular in both axes. At least in reality, if not in principle. So perhaps we can import it as an image. I will have to look into it.
Dear David,
Thank you for your very swift response and for your detailed explanation. I really appreciate it. I got the same error myself, when trying to load the file. That is why I resorted to trying to load the file as raw data (as I previously mentioned), but that did not work as well, since I do not have enough understanding of the data structure itself.
By the way, would sharing more files help in further understanding the unknown aspects of this type of data. Please let me know if it does.
Thank you for your time.
Respectfully,
Steven
So… The reason why it is loaded as a set of individual profiles is simple. Each profile can have a different number of points. It is definitely nothing like image data. In principle it can also be imported as XYZ data instead of profiles. Each has its advantages and disadvantages.
Profiles lose the slow axis coordinate (Y) but preserve profile identity. They are more immediately useful and match more closely the structure of the data. But making 2D data from them is PITA.
XYZ data preserve Y but lose information which point comes from which profile. You cannot do much with XYZ data in Gwyddion directly so you probably have to Rasterise first… Sorting out exported XYZ data back to individual profiles takes some work.
I also still do not really trust the Y coordinates as we create them.
Dear David,
Thank you for investigating further. I really appreciate the effort. From what I can understand, it is most likely better to use profiles instead of XYZ data (at least in most situations). I also guess not much more can be done about this... If I have misunderstood or if you have any other suggestions, please let me know.
Thank you for your time.
Respectfully,
Steven
Importing data from your file as profiles should already work with current development snapshots of Gwyddion.
I upgraded the build systems about a week ago and there were some problems with the nightly builds after that, but they should work again now.
Dear David,
Firstly, I apologize for the late response. I have been in contact with the equipment vendor and they basically confirmed what we had already understood. Thank you for the information and the explanation. If I have any further questions on the matter or need more support, I will let you know.
Thank you again for your assistance and support.
Respectfully,
Steven