Hi, I encountered a few bugs I find worthy to report.
1) PNG images with transparency: after being compressed had lost their transparency. Don't seem to be able to reproduce this, but perhaps other users had a similar incident.
2) Matroska (MKV): Source file (according to Windows Explorer) is 4198474 kB large and after compression (according to Windows Explorer) it's 4197426 kB (please compare to the screenshot below, which reports for the very same file) Does the GUI also report in kB ? Besides the numbers, I'm also wondering what compression results I should be able to expect.
3) During compression of Matroska (MKV) the GUI becomes breakable. What do I mean ? If during compression work I touch the GUI with the mouse pointer (to let say, move the GUI to the side of my screen), this is not possible since the GUI will freeze beyond recovery.
4) The GUI (see screenshot again) reports compression to 74% . Obviously this is a wrong calculation.
If I need to explain myself better on any of the above topics, please ask away.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
1) It is strange. Would be very helpful if you are able to systematically reproduce it, so I can try to fix it if it is my fault.
2) Compression ratio on MKV medias depends a lot, but reducing. I guess that the issue is with that file size, exceeding 4 GB. Can you please post the exact size in bytes of the original file? If that is the case, I am aware calculations will be broken, but I could try fixing them.
3) Can I have some MKV file were I can try to reproduce the problem? Unfortunatelly it is not happening to me with other files. Is it ocurring to you with other files?
4) 74% is correct, if you divide 3196 / 4269, it gives 0.74, which is 74%.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
1) I will try to do so and report back when I succeed.
2) Regretfully this specific file I have already deleted it, but I'll try again with another file. Then I'll provide sizes in bytes.
3) I'm posting a link to a downloadable movie that's released under the GPL license CC-BY-SA 3.0. Using this we can legally discuss this matter: http://www.valkaama.com/index.php?page=valkaama&l=en
Download the file: Valkaama 1080p
It's a 6,59 GB download, so it's a very good size example.
Use the Server Download option. I tried the Torrent, but there were almost no peers left, so that would take forever.
4) 74% may be a correct calculation, but the size of the optimized size is not calculated correctly. So it's a correct calculation of the percentage the optimized size would be versus the original size, IF the optimized size where calculated correctly, which it isn't. Am I making sense ? Perhaps I didn't express myself correctly, my apologies for that. If the optimized size where truly 3196 MB it would have fitted on a FAT32 USB Pendrive, which it didn't. As you probably know the FAT32 file system has a size limit of max. 4 GB.
4a) It would help if the GUI displays to the user what unit it uses for file size. I'm guessing that it is measuring in MB, with op to 3 digits after the comma. In the most user-friendly option the user could change/choose the setting for the used size unit. Being able to choose between GB, MB, kB and bytes is freedom, which I highly favor. Windows seems to favor kB as size unit, so that would make a logical default. But I can understand that an implementation of such options might be a lot of work for you, and I am not here to put any extra burden on your time. What you like to implement is your decision.
Last edit: Spotted Cow 2015-04-17
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
After pressing the Optimize all files button, the screen freezes.
I'm also noticing that the status 'pending' is still being displayed.
Only when 'pending' disappears and the actual compressing plugins start doing their work, the freeze goes away.
Also, I notice the progress bar doesn't correctly report the progress, but goes to 100% instantly.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I have identified the issue as FileOptimizer not managing properly file sizes of more than 2GB, and I am currently working on incrementing this limit, potentially up to 4 TB.
The task is just started as r236 (https://sourceforge.net/p/nikkhokkho/code/236/), but still needing some work to finish and test it. Will be available in next FileOptimizer version.
This is the root cause of percents not being calculed properly, nor original sizes, nor optimized ones. This is the reason behind optimized file being larger than original one. This task is handled by FO. It launches the plugin, and if ends successfully, it checks if there was any saving, and if so, uses the optimized version, while if not keeps the original one.
Just to be sure, can you please try with a MKV file smaller than 2 GB and just confirm it works?
As for the GUI not responding happening with "Pending" status, I would say is more an issue on Windows itself. At that point, FileOptimizer is doing a copy of the original file onto your system temp folder, it is using memory maped files, which is the fastest way of copying data, but also requires more CPU power, which can cause your system to be overloaded at that point. As soon as the copy ends, and it launches de plugins, it works properly again.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hi, Spotted Cow about the 1st point "PNG images with transparency: after being compressed had lost their transparency." I never experience that and I have optimized thousands of PNG images with FO and many of them with alpha/transparency, maybe is an "issue" with the program you use to view/edit the files that can't read correctly the alpha, I made a small IMG just for this example...
The img 1 is the original created with 'Corel Photo-Paint X3', it has 32bit because of the alpha channel.
The img 2 is after being optimized by XnViewMP with lvl 9 compression and it maintains the (standard) 32bit format.
And finally the img 3 is after being optimized by FO, the format now is 8bit+alpha (maybe non standard) and 'XnViewMP (v0.72)' (for example) can't read the alpha part correctly.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
It makes sense since the original image is a 32 bit PNG file with alpha transparency, but only 2 colors used, that it gets converted to a 8 bit image, and translate the alpha transparency to PNG transparency.
ImgEx_Opt_FO.png looks OK with gwenview, I will try more image viewers, and see if the problem is with XnViewMP (which still is beta), or some kind of incompatibility.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Yes I'm not complaining about the result file from FO I can read the alpha/transparency of that file without problems with 'Corel Photo-Paint X3' to.
BTW it's not only 2 colors, is grayscale on purpose, I make a pure 2 color (black & white) example file to show a even more aggressive example, here:...
The imgs 1 and 2 are the same as the others 32bit (RGBA format, standard for PNG I think).
The img 3 on the other hand after being optimized by FO is now only 1bit, 1bit format can only contain 2 values (0 or 1) and now it uses one value for the image parts and the other value for the alpha/transparency part, now even more programs fail to read the alpha channel correctly for example now even 'Corel Photo-Paint X3' can't read it right, but 'Corel Photo-Paint X7' still can.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
It seems that when more in-depth we go, more incompatibilities we find with PNG files.
It is strange, because there are standard PNG decoders created in fact my PNG team. Maybe those features have been added on recent versions of the decoder to improve compatibility? That will explain by XE7 does, but XE3 dont.
Will need to further investigate it. Maybe I can detect when the compatibility is changed, and make it optional.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Note that before using FO to optimize my PNG files I use "Png Optimizer" and "PNGGauntlet" and only with those two I get the same results relatively to the transparency/alpha, anyway what I mean is...
I don't think there is any problem with the way FO optimizes the PNG files, I think it's lack of implementation in the decoders of some programs, because even if it's not 100% standard those programs should be capable of reading the files without problem in these days...
And to justify that just look for other formats like JPG, MP3, FLAC, etc... all of those has "evolve" to non 100% standard to try to get more quality/compression and the majority of the programs can read them, and the same way the PNG files some of them can't! :-)
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hi, I encountered a few bugs I find worthy to report.
1) PNG images with transparency: after being compressed had lost their transparency. Don't seem to be able to reproduce this, but perhaps other users had a similar incident.
2) Matroska (MKV): Source file (according to Windows Explorer) is 4198474 kB large and after compression (according to Windows Explorer) it's 4197426 kB (please compare to the screenshot below, which reports for the very same file) Does the GUI also report in kB ? Besides the numbers, I'm also wondering what compression results I should be able to expect.
3) During compression of Matroska (MKV) the GUI becomes breakable. What do I mean ? If during compression work I touch the GUI with the mouse pointer (to let say, move the GUI to the side of my screen), this is not possible since the GUI will freeze beyond recovery.
4) The GUI (see screenshot again) reports compression to 74% . Obviously this is a wrong calculation.
If I need to explain myself better on any of the above topics, please ask away.
Thanks for your reports Spotted Cow:
1) It is strange. Would be very helpful if you are able to systematically reproduce it, so I can try to fix it if it is my fault.
2) Compression ratio on MKV medias depends a lot, but reducing. I guess that the issue is with that file size, exceeding 4 GB. Can you please post the exact size in bytes of the original file? If that is the case, I am aware calculations will be broken, but I could try fixing them.
3) Can I have some MKV file were I can try to reproduce the problem? Unfortunatelly it is not happening to me with other files. Is it ocurring to you with other files?
4) 74% is correct, if you divide 3196 / 4269, it gives 0.74, which is 74%.
1) I will try to do so and report back when I succeed.
2) Regretfully this specific file I have already deleted it, but I'll try again with another file. Then I'll provide sizes in bytes.
3) I'm posting a link to a downloadable movie that's released under the GPL license CC-BY-SA 3.0. Using this we can legally discuss this matter:
http://www.valkaama.com/index.php?page=valkaama&l=en
Download the file: Valkaama 1080p
It's a 6,59 GB download, so it's a very good size example.
Use the Server Download option. I tried the Torrent, but there were almost no peers left, so that would take forever.
Reference for the movie indeed being "open source".
It's on this Wikipedia list:
http://en.wikipedia.org/wiki/Open-source_film
4) 74% may be a correct calculation, but the size of the optimized size is not calculated correctly. So it's a correct calculation of the percentage the optimized size would be versus the original size, IF the optimized size where calculated correctly, which it isn't. Am I making sense ? Perhaps I didn't express myself correctly, my apologies for that. If the optimized size where truly 3196 MB it would have fitted on a FAT32 USB Pendrive, which it didn't. As you probably know the FAT32 file system has a size limit of max. 4 GB.
4a) It would help if the GUI displays to the user what unit it uses for file size. I'm guessing that it is measuring in MB, with op to 3 digits after the comma. In the most user-friendly option the user could change/choose the setting for the used size unit. Being able to choose between GB, MB, kB and bytes is freedom, which I highly favor. Windows seems to favor kB as size unit, so that would make a logical default. But I can understand that an implementation of such options might be a lot of work for you, and I am not here to put any extra burden on your time. What you like to implement is your decision.
Last edit: Spotted Cow 2015-04-17
Valkaama 1080p, before compressing started:
As you can see it reports a negative original size.
After pressing the Optimize all files button, the screen freezes.
I'm also noticing that the status 'pending' is still being displayed.
Only when 'pending' disappears and the actual compressing plugins start doing their work, the freeze goes away.
Also, I notice the progress bar doesn't correctly report the progress, but goes to 100% instantly.
Screenshot after compression finishes:
File size:
Before compresion, according to Windows Explorer:
Size: 6,59 GB (7.077.186.566 bytes)
Size on disk: 6,59 GB (7.077.187.584 bytes)
After compression, according to Windows Explorer:
Size: 6,59 GB (7.077.181.325 bytes)
Size on disk: 6,59 GB (7.077.183.488 bytes)
Thank you very much for your efforts Spotted Cow.
I have identified the issue as FileOptimizer not managing properly file sizes of more than 2GB, and I am currently working on incrementing this limit, potentially up to 4 TB.
The task is just started as r236 (https://sourceforge.net/p/nikkhokkho/code/236/), but still needing some work to finish and test it. Will be available in next FileOptimizer version.
This is the root cause of percents not being calculed properly, nor original sizes, nor optimized ones. This is the reason behind optimized file being larger than original one. This task is handled by FO. It launches the plugin, and if ends successfully, it checks if there was any saving, and if so, uses the optimized version, while if not keeps the original one.
Just to be sure, can you please try with a MKV file smaller than 2 GB and just confirm it works?
As for the GUI not responding happening with "Pending" status, I would say is more an issue on Windows itself. At that point, FileOptimizer is doing a copy of the original file onto your system temp folder, it is using memory maped files, which is the fastest way of copying data, but also requires more CPU power, which can cause your system to be overloaded at that point. As soon as the copy ends, and it launches de plugins, it works properly again.
Hi, Spotted Cow about the 1st point "PNG images with transparency: after being compressed had lost their transparency." I never experience that and I have optimized thousands of PNG images with FO and many of them with alpha/transparency, maybe is an "issue" with the program you use to view/edit the files that can't read correctly the alpha, I made a small IMG just for this example...
The img 1 is the original created with 'Corel Photo-Paint X3', it has 32bit because of the alpha channel.
The img 2 is after being optimized by XnViewMP with lvl 9 compression and it maintains the (standard) 32bit format.
And finally the img 3 is after being optimized by FO, the format now is 8bit+alpha (maybe non standard) and 'XnViewMP (v0.72)' (for example) can't read the alpha part correctly.
It makes sense since the original image is a 32 bit PNG file with alpha transparency, but only 2 colors used, that it gets converted to a 8 bit image, and translate the alpha transparency to PNG transparency.
ImgEx_Opt_FO.png looks OK with gwenview, I will try more image viewers, and see if the problem is with XnViewMP (which still is beta), or some kind of incompatibility.
Tried with some other programs, and found that Photoshop and Fireworks see the transparency in the optimized image with no problems.
Yes I'm not complaining about the result file from FO I can read the alpha/transparency of that file without problems with 'Corel Photo-Paint X3' to.
BTW it's not only 2 colors, is grayscale on purpose, I make a pure 2 color (black & white) example file to show a even more aggressive example, here:...
The imgs 1 and 2 are the same as the others 32bit (RGBA format, standard for PNG I think).
The img 3 on the other hand after being optimized by FO is now only 1bit, 1bit format can only contain 2 values (0 or 1) and now it uses one value for the image parts and the other value for the alpha/transparency part, now even more programs fail to read the alpha channel correctly for example now even 'Corel Photo-Paint X3' can't read it right, but 'Corel Photo-Paint X7' still can.
It seems that when more in-depth we go, more incompatibilities we find with PNG files.
It is strange, because there are standard PNG decoders created in fact my PNG team. Maybe those features have been added on recent versions of the decoder to improve compatibility? That will explain by XE7 does, but XE3 dont.
Will need to further investigate it. Maybe I can detect when the compatibility is changed, and make it optional.
Note that before using FO to optimize my PNG files I use "Png Optimizer" and "PNGGauntlet" and only with those two I get the same results relatively to the transparency/alpha, anyway what I mean is...
I don't think there is any problem with the way FO optimizes the PNG files, I think it's lack of implementation in the decoders of some programs, because even if it's not 100% standard those programs should be capable of reading the files without problem in these days...
And to justify that just look for other formats like JPG, MP3, FLAC, etc... all of those has "evolve" to non 100% standard to try to get more quality/compression and the majority of the programs can read them, and the same way the PNG files some of them can't! :-)