SFK addcr have crashed with large input files due to out of memory on my windows 2003 server. Any fix and/or thoughts please. I am using the latest realse of SFK.
SFK addcr cannot handle large input files, because the whole text is loaded to memory to allow easy overwriting of the input file. Please describe exactly what you mean by "crash". It should just shown an error, but then continue on the next file.
- do you see some kind of popup instead?
- what kind of command syntax are you using?
- do you want to convert the large files as well?
Thanks for your Reply!
I'm running a cmd as:
C:\convert\sfk166.exe addcr inputfile.dat
Where inputfile.dat is a text file of size 3GB
The error message I got is: "error out of memory"
It works geat when the input file size is 7MB
Thanks for your help
What the input file size limit for SFK addcr?
Any suggestion as a work around to process a large input file?
I have a 3GB production file that needs to procces using addcr.
Appreciate your help on this matter!
The file size limit is 2 GBytes approximately. Three alternatives:
1. Use the freeware HxD Hex Editor and replace all 0d by 0d0a
using data types: hex-values. I just tried it and the tool says
it would take about halve an hour approx. Not sure if HxD supports
replacement from a batch file.
2. with SFK XE (commercial), it can be done with a command like:
sfk replace -usetmp -showtmp inputfile.dat -binary /0d// /0a/0d0a/ -yes
Explanation: in case that there are unexpected CR (0x0d) characters, they are
stripped, and then all LF (0x0a) endings are replaced by CRLF (0x0d0a).
Option -usetmp allows the use of a temporary file, which is required
to avoid the main memory limit. Should take 5 to 10 minutes on a 3 GB text.
3. write and compile a simple C program that reads the file line by line,
writing another output file line by line. type "sfk samp cpp" for details.
Thank you so much, I am working on option 1 & 3 to see if I can get it work.
What would be the procedure to get the SFK XE (commercial) ? I just want to get ready as an alterantive on this.
Thinking about it, another alternative might be
sfk filter inputfile.dat >outputfile.txt
which however requires inputfile not to contain any binaries, and lines larger than 4000 characters will be split.
XE is available under http://stahlworks.com/sfkxe.html