Thread: [PyMca] Batch fitting for HDF5 files in PyMca
Brought to you by:
vasole
Menu
▾
▴
pymca-users
|
From: PyMca g. p. m. list. <pym...@li...> - 2019-09-03 21:06:51
|
Hi pymca users, I have been using (for a quite long time) HDF5 files to store the micro-XRF dataset got from measurements collected in 2D scanning mode and I also use the "Fast XRF stack fitting" plugin to fit all XRF spectra since it is quite fast. As we are now saving many HDF5 files from one experiment and also in each of these files a new dataset is being stored related to a second detector, I am wondering if there is any possibility to have a batch fitting mode for HDF5 files but using the "Fast XRF stack fitting mode" option. Currently, If I use the PyMca batch fitting with many HDF5 files, the procedure is extremely slow, creating the *.dat file instead of creating the *.csv file. Is there any kind of solution to this problem or another way to face it?. Thanks in advance, Sincerely, Carlos Carlos A. Pérez, Ph.D. Researcher - XRF Beamline Coordinator CARNAÚBA Group LNLS -Brazilian Synchrotron Light Source Laboratory http://www.lnls.br Phone: +55 (19) 3517-5081 Mail address: Caixa Postal 6192 CEP 13083-970 Campinas-SP, Brazil Aviso Legal: Esta mensagem e seus anexos podem conter informações confidenciais e/ou de uso restrito. Observe atentamente seu conteúdo e considere eventual consulta ao remetente antes de copiá-la, divulgá-la ou distribuí-la. Se você recebeu esta mensagem por engano, por favor avise o remetente e apague-a imediatamente. Disclaimer: This email and its attachments may contain confidential and/or privileged information. Observe its content carefully and consider possible querying to the sender before copying, disclosing or distributing it. If you have received this email by mistake, please notify the sender and delete it immediately. |
|
From: PyMca g. p. m. list. <pym...@li...> - 2019-09-09 14:31:16
|
Hi Carlos, I am not sure to have understood where is the difficulty. If you have many files with each of them containing a stack and your problem is to avoid to use the ROI imaging GUI, you just need to call the appropriate PyMca module from the command line: python -m PyMca5.PyMcaPhysics.xrf.FastXRFLinearFit --cfg=configuration_file --weight=0 --tif=1 --concentrations=0 --refit=0 filename::dataset_path What is the structure of your HDF5 files? Armando On 03/09/2019 22:33, PyMca general purpose mailing list. wrote: > Hi pymca users, > I have been using (for a quite long time) HDF5 files to store the > micro-XRF dataset got from measurements collected in 2D scanning mode > and I also use the "Fast XRF stack fitting" plugin to fit all XRF > spectra since it is quite fast. > As we are now saving many HDF5 files from one experiment and also in > each of these files a new dataset is being stored related to a second > detector, I am wondering if there is any possibility to have a batch > fitting mode for HDF5 files but using the "Fast XRF stack fitting > mode" option. Currently, If I use the PyMca batch fitting with many > HDF5 files, the procedure is extremely slow, creating the *.dat file > instead of creating the *.csv file. > Is there any kind of solution to this problem or another way to face it?. > Thanks in advance, > Sincerely, > Carlos > > > > Carlos A. Pérez, Ph.D. > Researcher - XRF Beamline Coordinator > CARNAÚBA Group > LNLS -Brazilian Synchrotron Light Source Laboratory > http://www.lnls.br > Phone: +55 (19) 3517-5081 > Mail address: > Caixa Postal 6192 > CEP 13083-970 > Campinas-SP, Brazil > > Aviso Legal: Esta mensagem e seus anexos podem conter informações > confidenciais e/ou de uso restrito. Observe atentamente seu conteúdo e > considere eventual consulta ao remetente antes de copiá-la, divulgá-la > ou distribuí-la. Se você recebeu esta mensagem por engano, por favor > avise o remetente e apague-a imediatamente. > > Disclaimer: This email and its attachments may contain confidential > and/or privileged information. Observe its content carefully and > consider possible querying to the sender before copying, disclosing or > distributing it. If you have received this email by mistake, please > notify the sender and delete it immediately. > > > > _______________________________________________ > PyMca-users mailing list > PyM...@li... > https://lists.sourceforge.net/lists/listinfo/pymca-users |
|
From: PyMca g. p. m. list. <pym...@li...> - 2021-08-07 18:24:56
|
Thank you for your message. I am out of the office until Monday, August 5th. I may not be able to reply (quickly) whilst away. Best wishes, Julie De Weirdt Op 9 sep. 2019, om 16:31 heeft PyMca general purpose mailing list. <pym...@li...> het volgende geschreven: Hi Carlos, I am not sure to have understood where is the difficulty. If you have many files with each of them containing a stack and your problem is to avoid to use the ROI imaging GUI, you just need to call the appropriate PyMca module from the command line: python -m PyMca5.PyMcaPhysics.xrf.FastXRFLinearFit --cfg=configuration_file --weight=0 --tif=1 --concentrations=0 --refit=0 filename::dataset_path What is the structure of your HDF5 files? Armando On 03/09/2019 22:33, PyMca general purpose mailing list. wrote: Hi pymca users, I have been using (for a quite long time) HDF5 files to store the micro-XRF dataset got from measurements collected in 2D scanning mode and I also use the "Fast XRF stack fitting" plugin to fit all XRF spectra since it is quite fast. As we are now saving many HDF5 files from one experiment and also in each of these files a new dataset is being stored related to a second detector, I am wondering if there is any possibility to have a batch fitting mode for HDF5 files but using the "Fast XRF stack fitting mode" option. Currently, If I use the PyMca batch fitting with many HDF5 files, the procedure is extremely slow, creating the *.dat file instead of creating the *.csv file. Is there any kind of solution to this problem or another way to face it?. Thanks in advance, Sincerely, Carlos Carlos A. Pérez, Ph.D. Researcher - XRF Beamline Coordinator CARNAÚBA Group LNLS -Brazilian Synchrotron Light Source Laboratory http://www.lnls.br Phone: +55 (19) 3517-5081 Mail address: Caixa Postal 6192 CEP 13083-970 Campinas-SP, Brazil Aviso Legal: Esta mensagem e seus anexos podem conter informações confidenciais e/ou de uso restrito. Observe atentamente seu conteúdo e considere eventual consulta ao remetente antes de copiá-la, divulgá-la ou distribuí-la. Se você recebeu esta mensagem por engano, por favor avise o remetente e apague-a imediatamente. Disclaimer: This email and its attachments may contain confidential and/or privileged information. Observe its content carefully and consider possible querying to the sender before copying, disclosing or distributing it. If you have received this email by mistake, please notify the sender and delete it immediately. _______________________________________________ PyMca-users mailing list PyM...@li... https://lists.sourceforge.net/lists/listinfo/pymca-users _______________________________________________ PyMca-users mailing list PyM...@li... https://lists.sourceforge.net/lists/listinfo/pymca-users |
|
From: PyMca g. p. m. list. <pym...@li...> - 2019-09-11 13:57:31
Attachments:
pymca_main_window.png
|
Hi Armando, Thanks for the reply and apologies for my delayed response. Please find attached an image file showing the structure of our HDF file. There are two datasets in each HDF5 file which they are fitted with the fast XRF stack fitting procedure. I didn't test yet but calling the PyMca module from the command line should work fine for my case. I believe I have to do an iterative procedure in python for instance for this purpose, right? Carlos Carlos A. Pérez, Ph.D. Researcher - XRF Beamline Coordinator CARNAÚBA Group LNLS -Brazilian Synchrotron Light Source Laboratory http://www.lnls.br Phone: +55 (19) 3517-5081 Mail address: Caixa Postal 6192 CEP 13083-970 Campinas-SP, Brazil ________________________________ From: PyMca general purpose mailing list. <pym...@li...> Sent: Monday, September 9, 2019 11:31 AM To: pym...@li... <pym...@li...> Subject: Re: [PyMca] Batch fitting for HDF5 files in PyMca Hi Carlos, I am not sure to have understood where is the difficulty. If you have many files with each of them containing a stack and your problem is to avoid to use the ROI imaging GUI, you just need to call the appropriate PyMca module from the command line: python -m PyMca5.PyMcaPhysics.xrf.FastXRFLinearFit --cfg=configuration_file --weight=0 --tif=1 --concentrations=0 --refit=0 filename::dataset_path What is the structure of your HDF5 files? Armando On 03/09/2019 22:33, PyMca general purpose mailing list. wrote: Hi pymca users, I have been using (for a quite long time) HDF5 files to store the micro-XRF dataset got from measurements collected in 2D scanning mode and I also use the "Fast XRF stack fitting" plugin to fit all XRF spectra since it is quite fast. As we are now saving many HDF5 files from one experiment and also in each of these files a new dataset is being stored related to a second detector, I am wondering if there is any possibility to have a batch fitting mode for HDF5 files but using the "Fast XRF stack fitting mode" option. Currently, If I use the PyMca batch fitting with many HDF5 files, the procedure is extremely slow, creating the *.dat file instead of creating the *.csv file. Is there any kind of solution to this problem or another way to face it?. Thanks in advance, Sincerely, Carlos Carlos A. Pérez, Ph.D. Researcher - XRF Beamline Coordinator CARNAÚBA Group LNLS -Brazilian Synchrotron Light Source Laboratory http://www.lnls.br Phone: +55 (19) 3517-5081 Mail address: Caixa Postal 6192 CEP 13083-970 Campinas-SP, Brazil Aviso Legal: Esta mensagem e seus anexos podem conter informações confidenciais e/ou de uso restrito. Observe atentamente seu conteúdo e considere eventual consulta ao remetente antes de copiá-la, divulgá-la ou distribuí-la. Se você recebeu esta mensagem por engano, por favor avise o remetente e apague-a imediatamente. Disclaimer: This email and its attachments may contain confidential and/or privileged information. Observe its content carefully and consider possible querying to the sender before copying, disclosing or distributing it. If you have received this email by mistake, please notify the sender and delete it immediately. _______________________________________________ PyMca-users mailing list PyM...@li...<mailto:PyM...@li...> https://lists.sourceforge.net/lists/listinfo/pymca-users |
|
From: PyMca g. p. m. list. <pym...@li...> - 2021-08-07 18:29:55
|
Thank you for your message. I am out of the office until Monday, August 5th. I may not be able to reply (quickly) whilst away. Best wishes, Julie De Weirdt Op 11 sep. 2019, om 15:57 heeft PyMca general purpose mailing list. <pym...@li...> het volgende geschreven: Hi Armando, Thanks for the reply and apologies for my delayed response. Please find attached an image file showing the structure of our HDF file. There are two datasets in each HDF5 file which they are fitted with the fast XRF stack fitting procedure. I didn't test yet but calling the PyMca module from the command line should work fine for my case. I believe I have to do an iterative procedure in python for instance for this purpose, right? Carlos Carlos A. Pérez, Ph.D. Researcher - XRF Beamline Coordinator CARNAÚBA Group LNLS -Brazilian Synchrotron Light Source Laboratory http://www.lnls.br Phone: +55 (19) 3517-5081 Mail address: Caixa Postal 6192 CEP 13083-970 Campinas-SP, Brazil From: PyMca general purpose mailing list. <pym...@li...> Sent: Monday, September 9, 2019 11:31 AM To: pym...@li... <pym...@li...> Subject: Re: [PyMca] Batch fitting for HDF5 files in PyMca Hi Carlos, I am not sure to have understood where is the difficulty. If you have many files with each of them containing a stack and your problem is to avoid to use the ROI imaging GUI, you just need to call the appropriate PyMca module from the command line: python -m PyMca5.PyMcaPhysics.xrf.FastXRFLinearFit --cfg=configuration_file --weight=0 --tif=1 --concentrations=0 --refit=0 filename::dataset_path What is the structure of your HDF5 files? Armando On 03/09/2019 22:33, PyMca general purpose mailing list. wrote: Hi pymca users, I have been using (for a quite long time) HDF5 files to store the micro-XRF dataset got from measurements collected in 2D scanning mode and I also use the "Fast XRF stack fitting" plugin to fit all XRF spectra since it is quite fast. As we are now saving many HDF5 files from one experiment and also in each of these files a new dataset is being stored related to a second detector, I am wondering if there is any possibility to have a batch fitting mode for HDF5 files but using the "Fast XRF stack fitting mode" option. Currently, If I use the PyMca batch fitting with many HDF5 files, the procedure is extremely slow, creating the *.dat file instead of creating the *.csv file. Is there any kind of solution to this problem or another way to face it?. Thanks in advance, Sincerely, Carlos Carlos A. Pérez, Ph.D. Researcher - XRF Beamline Coordinator CARNAÚBA Group LNLS -Brazilian Synchrotron Light Source Laboratory http://www.lnls.br Phone: +55 (19) 3517-5081 Mail address: Caixa Postal 6192 CEP 13083-970 Campinas-SP, Brazil Aviso Legal: Esta mensagem e seus anexos podem conter informações confidenciais e/ou de uso restrito. Observe atentamente seu conteúdo e considere eventual consulta ao remetente antes de copiá-la, divulgá-la ou distribuí-la. Se você recebeu esta mensagem por engano, por favor avise o remetente e apague-a imediatamente. Disclaimer: This email and its attachments may contain confidential and/or privileged information. Observe its content carefully and consider possible querying to the sender before copying, disclosing or distributing it. If you have received this email by mistake, please notify the sender and delete it immediately. _______________________________________________ PyMca-users mailing list PyM...@li... https://lists.sourceforge.net/lists/listinfo/pymca-users _______________________________________________ PyMca-users mailing list PyM...@li... https://lists.sourceforge.net/lists/listinfo/pymca-users |
|
From: PyMca g. p. m. list. <pym...@li...> - 2019-09-11 14:36:02
|
Hi Carlos, On 11/09/2019 15:57, PyMca general purpose mailing list. wrote: > Hi Armando, > Thanks for the reply and apologies for my delayed response. Please > find attached an image file showing the structure of our HDF file. > There are two datasets in each HDF5 file which they are fitted with > the fast XRF stack fitting procedure. I didn't test yet but calling > the PyMca module from the command line should work fine for my case. I > believe I have to do an iterative procedure in python for instance for > this purpose, right? Since it is called from the command line, you can use any scripting language allowing you to execute a command. Even a shell script would work. If you know Python, you can do many more things. Basically all the work is done by the method fitMultipleSpectra of the FastXRFLinearFit module: https://github.com/vasole/pymca/blob/master/PyMca5/PyMcaPhysics/xrf/FastXRFLinearFit.py BTW, in the main ROI Imaging tool, you should be able to open first the dataset image1_001 and then open as slave the image1_002. That allows to inspect the two at the same time sharing the information. In fact the initial intention of it was to combine fluorescence and diffraction maps, but you can also use it to check other things. Armando |
|
From: PyMca g. p. m. list. <pym...@li...> - 2021-08-07 18:29:56
|
Thank you for your message. I am out of the office until Monday, August 5th. I may not be able to reply (quickly) whilst away. Best wishes, Julie De Weirdt Op 11 sep. 2019, om 16:35 heeft PyMca general purpose mailing list. <pym...@li...> het volgende geschreven: Hi Carlos, On 11/09/2019 15:57, PyMca general purpose mailing list. wrote: Hi Armando, Thanks for the reply and apologies for my delayed response. Please find attached an image file showing the structure of our HDF file. There are two datasets in each HDF5 file which they are fitted with the fast XRF stack fitting procedure. I didn't test yet but calling the PyMca module from the command line should work fine for my case. I believe I have to do an iterative procedure in python for instance for this purpose, right? Since it is called from the command line, you can use any scripting language allowing you to execute a command. Even a shell script would work. If you know Python, you can do many more things. Basically all the work is done by the method fitMultipleSpectra of the FastXRFLinearFit module: https://github.com/vasole/pymca/blob/master/PyMca5/PyMcaPhysics/xrf/FastXRFLinearFit.py BTW, in the main ROI Imaging tool, you should be able to open first the dataset image1_001 and then open as slave the image1_002. That allows to inspect the two at the same time sharing the information. In fact the initial intention of it was to combine fluorescence and diffraction maps, but you can also use it to check other things. Armando _______________________________________________ PyMca-users mailing list PyM...@li... https://lists.sourceforge.net/lists/listinfo/pymca-users |
|
From: PyMca g. p. m. list. <pym...@li...> - 2019-09-11 20:15:23
|
Hi Armando, Thanks for your valuable help. Regards, Carlos Carlos A. Pérez, Ph.D. Researcher - XRF Beamline Coordinator CARNAÚBA Group LNLS -Brazilian Synchrotron Light Source Laboratory http://www.lnls.br Phone: +55 (19) 3517-5081 Mail address: Caixa Postal 6192 CEP 13083-970 Campinas-SP, Brazil ________________________________ From: PyMca general purpose mailing list. <pym...@li...> Sent: Wednesday, September 11, 2019 11:35 AM To: pym...@li... <pym...@li...> Subject: Re: [PyMca] Batch fitting for HDF5 files in PyMca Hi Carlos, On 11/09/2019 15:57, PyMca general purpose mailing list. wrote: Hi Armando, Thanks for the reply and apologies for my delayed response. Please find attached an image file showing the structure of our HDF file. There are two datasets in each HDF5 file which they are fitted with the fast XRF stack fitting procedure. I didn't test yet but calling the PyMca module from the command line should work fine for my case. I believe I have to do an iterative procedure in python for instance for this purpose, right? Since it is called from the command line, you can use any scripting language allowing you to execute a command. Even a shell script would work. If you know Python, you can do many more things. Basically all the work is done by the method fitMultipleSpectra of the FastXRFLinearFit module: https://github.com/vasole/pymca/blob/master/PyMca5/PyMcaPhysics/xrf/FastXRFLinearFit.py BTW, in the main ROI Imaging tool, you should be able to open first the dataset image1_001 and then open as slave the image1_002. That allows to inspect the two at the same time sharing the information. In fact the initial intention of it was to combine fluorescence and diffraction maps, but you can also use it to check other things. Armando Aviso Legal: Esta mensagem e seus anexos podem conter informações confidenciais e/ou de uso restrito. Observe atentamente seu conteúdo e considere eventual consulta ao remetente antes de copiá-la, divulgá-la ou distribuí-la. Se você recebeu esta mensagem por engano, por favor avise o remetente e apague-a imediatamente. Disclaimer: This email and its attachments may contain confidential and/or privileged information. Observe its content carefully and consider possible querying to the sender before copying, disclosing or distributing it. If you have received this email by mistake, please notify the sender and delete it immediately. |
|
From: PyMca g. p. m. list. <pym...@li...> - 2021-08-07 18:29:48
|
Thank you for your message. I am out of the office until Monday, August 5th. I may not be able to reply (quickly) whilst away. Best wishes, Julie De Weirdt Op 11 sep. 2019, om 19:40 heeft PyMca general purpose mailing list. <pym...@li...> het volgende geschreven: Hi Armando, Thanks for your valuable help. Regards, Carlos Carlos A. Pérez, Ph.D. Researcher - XRF Beamline Coordinator CARNAÚBA Group LNLS -Brazilian Synchrotron Light Source Laboratory http://www.lnls.br Phone: +55 (19) 3517-5081 Mail address: Caixa Postal 6192 CEP 13083-970 Campinas-SP, Brazil From: PyMca general purpose mailing list. <pym...@li...> Sent: Wednesday, September 11, 2019 11:35 AM To: pym...@li... <pym...@li...> Subject: Re: [PyMca] Batch fitting for HDF5 files in PyMca Hi Carlos, On 11/09/2019 15:57, PyMca general purpose mailing list. wrote: Hi Armando, Thanks for the reply and apologies for my delayed response. Please find attached an image file showing the structure of our HDF file. There are two datasets in each HDF5 file which they are fitted with the fast XRF stack fitting procedure. I didn't test yet but calling the PyMca module from the command line should work fine for my case. I believe I have to do an iterative procedure in python for instance for this purpose, right? Since it is called from the command line, you can use any scripting language allowing you to execute a command. Even a shell script would work. If you know Python, you can do many more things. Basically all the work is done by the method fitMultipleSpectra of the FastXRFLinearFit module: https://github.com/vasole/pymca/blob/master/PyMca5/PyMcaPhysics/xrf/FastXRFLinearFit.py BTW, in the main ROI Imaging tool, you should be able to open first the dataset image1_001 and then open as slave the image1_002. That allows to inspect the two at the same time sharing the information. In fact the initial intention of it was to combine fluorescence and diffraction maps, but you can also use it to check other things. Armando Aviso Legal: Esta mensagem e seus anexos podem conter informações confidenciais e/ou de uso restrito. Observe atentamente seu conteúdo e considere eventual consulta ao remetente antes de copiá-la, divulgá-la ou distribuí-la. Se você recebeu esta mensagem por engano, por favor avise o remetente e apague-a imediatamente. Disclaimer: This email and its attachments may contain confidential and/or privileged information. Observe its content carefully and consider possible querying to the sender before copying, disclosing or distributing it. If you have received this email by mistake, please notify the sender and delete it immediately. _______________________________________________ PyMca-users mailing list PyM...@li... https://lists.sourceforge.net/lists/listinfo/pymca-users |
|
From: PyMca g. p. m. list. <pym...@li...> - 2021-08-07 18:24:56
|
Thank you for your message. I am out of the office until Monday, August 5th. I may not be able to reply (quickly) whilst away. Best wishes, Julie De Weirdt Op 3 sep. 2019, om 22:33 heeft PyMca general purpose mailing list. <pym...@li...> het volgende geschreven: Hi pymca users, I have been using (for a quite long time) HDF5 files to store the micro-XRF dataset got from measurements collected in 2D scanning mode and I also use the "Fast XRF stack fitting" plugin to fit all XRF spectra since it is quite fast. As we are now saving many HDF5 files from one experiment and also in each of these files a new dataset is being stored related to a second detector, I am wondering if there is any possibility to have a batch fitting mode for HDF5 files but using the "Fast XRF stack fitting mode" option. Currently, If I use the PyMca batch fitting with many HDF5 files, the procedure is extremely slow, creating the *.dat file instead of creating the *.csv file. Is there any kind of solution to this problem or another way to face it?. Thanks in advance, Sincerely, Carlos Carlos A. Pérez, Ph.D. Researcher - XRF Beamline Coordinator CARNAÚBA Group LNLS -Brazilian Synchrotron Light Source Laboratory http://www.lnls.br Phone: +55 (19) 3517-5081 Mail address: Caixa Postal 6192 CEP 13083-970 Campinas-SP, Brazil Aviso Legal: Esta mensagem e seus anexos podem conter informações confidenciais e/ou de uso restrito. Observe atentamente seu conteúdo e considere eventual consulta ao remetente antes de copiá-la, divulgá-la ou distribuí-la. Se você recebeu esta mensagem por engano, por favor avise o remetente e apague-a imediatamente. Disclaimer: This email and its attachments may contain confidential and/or privileged information. Observe its content carefully and consider possible querying to the sender before copying, disclosing or distributing it. If you have received this email by mistake, please notify the sender and delete it immediately. _______________________________________________ PyMca-users mailing list PyM...@li... https://lists.sourceforge.net/lists/listinfo/pymca-users |
×
Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.
Thanks for helping keep SourceForge clean.
X