Activity for Steffen
-
Steffen modified a comment on discussion Discussion about CONVERTCP
Hi Martin, I updated your post to make the asterisks visible that have been interpreted as Markdown elements for italic formatting ... Have a look at the Wiki https://sourceforge.net/p/convertcp/wiki/Home/#db55 ff While CONVERTCP doesn't support globbing (i.e. searching filenames with wildcard characters), all shell scripting languages that I'm aware of do support it. Unfortunately you didn't tell what kind of Shell script you're using to call CONVERTCP. In case of Windows Batch, the command line...
-
Hi Martin, I updated your post to make the asterisks visible that have been interpreted as Markdown elements for italic formatting ... Have a look at the Wiki https://sourceforge.net/p/convertcp/wiki/Home/#db55 ff While CONVERTCP doesn't support globbing (i.e. searching filenames with wildcard characters), all shell scripting languages that I'm aware of do support it. Unfortunately you didn't tell what kind of Shell script you're using to call the CONVERTCP. In case of Windows Batch, the command...
-
Hi Steffen, searching for a solution to run convertcp with "*" in the filename. (because the filename changes every day) convertcp 65001 1252 /i "SFSF_TO_Remedy_*.csv" /o "chris1252.asc" this does not find the source file although existing.... Martin
-
Thank you so much!
-
Oh, that would be great! You may remember I once told in my review that your builds are attached to a portable compiler/IDE solution for both Windows x86 and x64. I still try to keep it alive. The script may even tell the users if you uploaded a new version. This doesn't work anymore now, and x86 builds don't even exist anymore 🤷♂️ So, yeah I think also on behalf of the users of my portable script solution I would much appreciate to get the the former build structure back. Thanks!
-
Oh, that would be great! You may remember I once told in my review that your builds are attached to a portable compiler/IDE solution for both Windows x86 and x64. I still try to keep it alive. The script may even tell the users if you uploaded a new version. This doesn't work anymore now, and x86 builds don't even exist any more 🤷♂️ So, yeah I think also on behalf of the users of my portable script solution I would much appreciate to get the the former build structure back. Thanks!
-
There are always AV engines that complain about the binaries. I don't care at all because I wrote the code and I know those are just false positives. So, you should actually blame it on the AV vendors for not being able to develop their engines to reliably distinguish between harmless programs and malware. I always upload the binaries to VirusTotal myself. Currently x86: https://www.virustotal.com/gui/file/4700bd124f7b6e01fac05cfbc511dafef84e70e741bf91b129f5c328561757c5 x64: https://www.virustotal.com/gui/file/b3c9112f31a49a3aa8de6bed89b77fc5afed2a746f9e489c7b3c9cd7396c87db...
-
No mingw-w32-bin-i686 anymore?
-
Replacing file content with the converted text UPDATE: Version 8.2 supports overwriting of the original file. Pass a minus sign along with option /o. Original text: I already wrote some examples in the "readme.txt" about how to convert a single file. You may ask why CONVERTCP doesn't support overwriting of a file with the converted content. The reason is that the tool doesn't read the entire content of the file at once. Especially for large files it's more memory-efficient to read only chunk-wise...
-
How to find the right Code Page ID and why CONVERTCP doesn't detect code pages automatically The supported code pages depend on the installed code pages on your computer. Option /l lists the installed code pages along with a short description. A lot of these IDs are not self-explanatory though. If you're familiar with .NET or HTML you may already know some MIME names that are listed on the Microsoft page https://msdn.microsoft.com/en-us/library/dd317756.aspx This table might be already helpful if...
-
I can confirm that this issue has been solved. Thank you Teodor and team!
-
Haha, yeah I recall that I wrote that piece of code. Although I can't even tell anymore where I ever published it. However, glad to hear it's still helpful. Steffen
-
You're welcome. I'm happy if the tool is helpful for you. Steffen
-
Hello Thomas, The reason is that 650001 is an invalid code page. In case you want to convert to UTF-8 use codepage 65001 (with only two zeros) or just pass "UTF-8" instead of the numeric value. Hope this helps Steffen
-
That's good news. Thanks for your feedback!
-
That's good new. Thanks for your feedback!
-
Thank you for your nice feedback! Developers need people like you who report issues. You have been a great help.
-
Fixed in version 7.2. This version also contains custom conversions from and to UTF-8. That's because Microsoft's API function didn't conform with the Unicode standard on XP and performed a CESU-8-like conversions.
-
Oh yes, that's a severity bug. I forgot to support 0 and 1 in the 7th versions. Will be fixed with the next release. Thank you!
-
Thank you for your feedback! Much appreciated.
-
Version 7.1 is supposed to fix the broken option /l on XP. Any feedback appreciated.
-
To the thread starter: If you are still interested, version 7 supports MIK. See https://sourceforge.net/projects/convertcp/files/custom%20charsets/
-
Thanks for letting me know. Unfortunately I can't test myself on XP anymore. Can you explain more specific what output you get?
-
Same to you! Let me know if there is anything I still could do to help you.
-
I'm certainly able to write a little separate tool for you that implements MIK. But to which code page do you want to convert it?
-
Every codepage that is supported on Windows has an ID, such like 855. See ... https://en.wikipedia.org/wiki/Code_page_855 ... which is not the same like MIK as I understood. CONVERTCP does not contain any character maps (translation tables from the bytes that represent a character to the related Unicode code points). Thus, only code pages that are installed on Windows are supported by CONVERTCP. Do you know whether or not MIK is installed on Windows? If it is installed it has to have a code page...
-
I'd be happy to add it to the names you may pass instead of the code page IDs. CONVERTCP relies on the installed code pages on you computer though. Can you help out with the code page ID that the MIK charset has?
-
If you want to use DirEvents in a Batch script I strongly recommend to download the example scripts along with the DirEvents binary. You can't use DirEvents in a FOR /F loop because it is buffering the records and doesn't begin to iterate. You have to work around using SET /P like that DirEvents "C:\" /s | (cmd /von /q /d /c ^"for /l %%# in (^) do set "rec="^&set /p "rec="^&(if not defined rec exit^)^&^ echo work with '!rec!' here^ ") Replace the echo line with your code. Note that the processing...
-
Home
-
About Virtual Terminal processing for version 6.3 and newer Most of the current command line utilities don't support Virtual Terminal processing yet. In this case ANSI escape sequences are not used to control the console output and their textual expressions get printed to the screen. Even if I expect that VT processing will be only barely used along with CONVERTCP, it won't hurt to enable it once that Windows 10 provides this possibility. Example using CONVERTCP v. 6.3: >nul chcp 65001 echo +ABsAWw-93;42m+JYgliCWIJZMlkyWTJZIlkiWSJZElkSWR-...
-
About Virtual Terminal processing for version 6.3 and newer Most of the current command line utilities don't support Virtual Terminal processing yet. In this case ANSI escape sequences are not used to control the console output and their textual expressions get printed to the screen. Even if I expect that VT processing will be only barely used along with CONVERTCP, it won't hurt to enable it once that Windows 10 provides this possibility. Example using CONVERTCP v. 6.3: >nul chcp 65001 echo +ABsAWw-93;42m+JYgliCWIJZMlkyWTJZIlkiWSJZElkSWR-...
-
About Virtual Terminal processing for version 6.3 and newer Most of the current command line utilities don't support Virtual Terminal processing yet. In this case ANSI escape sequences are not used to control the console output and their textual expressions get printed to the screen. Even if I expect that VT processing will be only barely used along with CONVERTCP, it won't hurt to enable it once that Windows 10 provides this possibility. Example using CONVERTCP v. 6.3: >nul chcp 65001 echo +ABsAWw-93;42m+JYgliCWIJZMlkyWTJZIlkiWSJZElkSWR-...
-
This has been implemented in release 4.23. Thank you Don! Steffen
-
Unfortunately, Feature Requests are disabled for new tickets. So I'll add it here. environment: Windows 10 Home x64 (but strongly suspected for every Windows version) what's happening: PAGE saves the output encoded in the default charset which is set for the current machine. On Windows that means, PAGE saves in an ANSI code page. Non-ASCII characters are represented as single bytes that don't meet the UTF-8 standard and thus, Python is not able to process these characters because it requires UTF-8....
-
How to verify if the conversion to another encoding was correct? Beginning with version 6.0, CONVERTCP supports option /v which influence the return value of the utility. If you pass /v, CONVERTCP verifies whether all characters from the input have been converted without having used any replacement characters or approximated ASCII characters. Only in this case CONVERTCP returns 0. If one or more characters have been found that do not match the same Unicode code point in the used code page for the...
-
Don I read the first half of your reply about ten times. I still don't get it though. First of all, what faulty example are you referring to? What did you mean when you said "faulty specification"? Is it a character out of the default ANSI encoding on a certain Windows machine? What did you repair, and how? I would really like to understand it because I'm quite interested in character encoding and problems related to it. Best regards Steffen
-
The new option /v (for "verify") makes CONVERTCP return 1 instead of 0 if at least one character has been replaced with a character that doesn't match the same Unicode code point. Besides of that it doesn't change the default behavior. That is, CONVERTCP silently replaces all characters that are invalid or that don't exist in the passed code pages with either a replacement character or with an approximated ASCII character which looks similar but has different semantics. Hopefully that will meet your...
-
Hannes, thank you very much for your feedback and suggestions! In general I decided to avoid the detection of the encoding. I explained the reasons in this Wiki post: https://sourceforge.net/p/convertcp/wiki/Home/#c4ac Actually your proposal is no encoding detection though. Even if you use it as such. Remember that ASCII characters and the related UTF-8 characters are absolutely equal and thus, you wouldn't even find any illegal characters. Also finding illegal characters might not necessarily mean...
-
Why is the output always written UTF-16-encoded? The Windows console works with an OEM code page. Redirected to a file and opened in a Windows application (such as Notepad) non-ASCII characters would be interpreted using an ANSI code page which corrupts the text that shows up in the window. UTF-16 is an encoding that supports Unicode entirely and it is the preferred encoding for Unicode on Windows. That's the reason why I picked it up as the default for TeeW.
-
Why is the output always written UTF-16-encoded? The Windows console works with a OEM code page. Redirected to a file and opened in a Windows application (such as Notepad) non-ASCII characters would be interpreted using an ANSI code page which corrupts the text that shows up in the window. UTF-16 is an encoding that supports Unicode entirely and it is the preferred encoding for Unicode on Windows. That's the reason why I picked it up as the default for TeeW.
-
UTF-8 support is a feature? Unlike other operating systems, UTF-8 is badly supported on Windows. On the other hand UTF-8 is one of the code pages you are able to set (using CHCP 65001) in order to support Unicode for the standard input. Unfortunately it's still buggy. For more information about this bug see https://www.dostips.com/forum/viewtopic.php?f=3&t=9017 TeeW works around this bug for the text read from standard input.
-
What is TeeW for? "Tee" has the same meaning as the tee command in *nixoid shells. The "W" is for Windows. The tee is the pronunciation of the letter T which illustrates how the utility works. The horizontal bar is for the standard input that gets passed through to the standard output of the console window. The vertical bar is like a bypass. That is, the input stream will be duplicated and written to one or more files in parallel.
-
Thank you very much for your feedback Saso! Steffen
-
You're welcome. Thanks again for your feedback. Much appreciated. Steffen
-
Mike Glad to hear the utility is useful for you. Thank you for your feedback! Of course it is possible to remove the BOM from UTF-encoded text files. Just use the same ID or alias for both CP_In and CP_Out. If you omit option /b then the output file will not have the leading BOM anymore. Say, your file "test.txt" is UTF-8-encoded and has a BOM. In this case the command line would be like that: convertcp 65001 65001 /i "test.txt" /o "test_2.txt" "test_2.txt" will not contain the BOM. If you have any...
-
How to find the right Code Page ID and why CONVERTCP doesn't detect code pages automatically The supported code pages depend on the installed code pages on your computer. Option /l lists the installed code pages along with a short description. A lot of these IDs are not self-explanatory though. If you're familiar with .NET or HTML you may already know some MIME names that are listed on the Microsoft page https://msdn.microsoft.com/en-us/library/dd317756.aspx This table might be already helpful if...
-
How to find the right Code Page ID and why CONVERTCP doesn't detect code pages automatically The supported code pages depend on the installed code pages on your computer. Option /l lists the installed code pages along with a short description. A lot of these IDs are not self-explanatory though. If you're familiar with .NET or HTML you may already know some MIME names that are listed on the Microsoft page https://msdn.microsoft.com/en-us/library/dd317756.aspx This table might be already helpful if...
-
Threading in CONVERTCP explained As already written CONVERTCP reads the incoming stream chunk-wise if possible. The advantage is that an already converted chunk can be written using an asynchronuous thread at the same time as the next chunk of text is read and converted. That leads to a good performance. Furthermore the memory usage is limited to the buffer size the chunks need. This size doesn't increase even not if very large files are converted. Sounds like a good concept, doesn't it? That's the...
-
Option /n explained The /n is for "no threading". But what does it mean? For a better understanding let's step back to the defaults. As already written CONVERTCP reads the incoming stream chunk-wise. The advantage is that an already converted chunk can be written using an asynchronuous thread at the same time as the next chunk of text is read and converted. That leads to a good performance. Furthermore the memory usage is limited to the buffer size the chunks need. This size doesn't increase even...
-
All developers like to get feedback. So do I. For me, positive feedback is as valuable as negative feedback. I'd like to know about your experiences with CONVERTCP, found bugs, code reviews, suggestions etc. Because you have to sign in to create a ticket or to rate the project this forum is an additional platform where you are able to give feedback, also as an anonymous guest without having a SourceForge account. As long as spam bots don't exploit this opportunity I'll keep the forum open for everyone....
-
About flushing the output stream buffer Maybe you are wondering what the /f option is for. I have to admit you won't find much about it in the "readme.txt". Flushing is basically not needed. Written data will be buffered by the file system because the physical writing to the drive is slow. That way the performance can be improved enormously. There is no drawback as long as the file is not accessed concurrently. Appending new data to the buffer by another write operation will not corrupt the data...
-
About flushing the output stream Maybe you are wondering what the /f option is for. I have to admit you won't find much about it in the "readme.txt". Flushing is basically not needed. Written data will be buffered by the file system because the physical writing to the drive is slow. That way the performance can be improved enormously. There is no drawback as long as the file is not accessed concurrently. Appending new data to the buffer by another write operation will not corrupt the data in the...
-
CONVERTCP and non-ASCII file names The source code of a Batch script has to be ASCII, or at least encoded in the default OEM code page or the code page you have set using CHCP. However you'll get trouble to write file names that contain characters that are not supported. This is a limitation of the command interpreter. It's not a limitation of CONVERTCP as you may have observed while running my latest example above. So working with a wildcard character (like the asterisk) in a loop is already a good...
-
CONVERTCP and non-ASCII file names The source code of a Batch script has to be ASCII, or at least encoded encoded in the default OEM code page or the code page you have set using CHCP. However you'll get trouble to write file names that contain characters that are not supported. This is a limitation of the command interpreter. It's not a limitation of CONVERTCP as you may have observed while running my latest example above. So working with a wildcard character (like the asterisk) in a loop is already...
-
CONVERTCP and non-ASCII file names The source code of a Batch script have to be ASCII, or at least encoded encoded in the default OEM code page or the code page you have set using CHCP. However you'll get trouble to write file names that contain characters that are not supported. This is a limitation of the command interpreter. It's not a limitation of CONVERTCP as you may have observed while running my latest example above. So working with a wildcard character (like the asterisk) in a loop is already...
-
Convert all files of a certain directory Not the built-in options make CONVERTCP flexible to use. It's rather the command-line interface. You don't need to drag every single file to a window, you also don't need to browse files or folders in a dialog window. Just use the possibilities that the Windows command line already provide. Say, you want to convert all .txt files in the current directory from your default OEM code page to UTF-8 (similar to the example above) then you can just use a FOR loop...
-
Convert all files of a certain directory Not the built-in options make CONVERTCP flexible to use. It's rather the command-line interface. You don't need to drag every single file to a window, you also don't need to browse files or folders in a dialog window. Just use the possibilities that the windows command line already provide. Say, you want to convert all .txt files in the current directory from your default OEM code page to UTF-8 (similar to the example above) then you can just use a FOR loop...
-
Convert all files of a certain directory Not the build-in options make CONVERTCP flexible to use. It's rather the command-line interface. You don't need to drag every single file to a window, you also don't need to browse files or folders in a dialog window. Just use the possibilities that the windows command line already provide. Say, you want to convert all .txt files in the current directory from your default OEM code page to UTF-8 (similar to the example above) then you can just use a FOR loop...
-
Convert all files of a certain directory Not the build-in options make CONVERTCP flexible to use. It's rather the command-line interface. You don't need to drag every single file to a window, you also don't need to browse files or folders in a dialog window. Just use the possibilities that the windows command line already provide. Say, you want to convert all .txt files in the current directory from your default ANSI code page to UTF-8 (similar to the example above) then you can just use a FOR loop...
-
Replacing file content with the converted text I already wrote some examples in the "readme.txt" about how to convert a single file. You may ask why CONVERTCP doesn't support overwriting of a file with the converted content. The reason is that the tool doesn't read the entire content of the file at once. Especially for large files it's more memory-efficient to read only chunk-wise in order to avoid running out of RAM space. This also makes that CONVERTCP shows a pretty high performance because as...
-
Home
-
That's one of the possibilities Simon. Unfortunately the forum link in my initial post is corrupted. I'll try to provide it again: http://forums.codeblocks.org/index.php/topic,22349.msg152174.html#msg152174
-
Thanks for the information Teodor. Yes I'm able to work around but it's likely that not everyone will read or understand this little hack. Furthermore if this is an issue on other platforms too then those guys are out of luck. That's the reason why I reported it for a proper tracking. Steffen
-
Code::Blocks 17.12 blurry text on high DPI monitors
-
Thank you drangon zhou! Much appreciated.
-
mingw-w32-bin-i686-20170521.7z contains files with date 2015
1
