npipe / News: Recent posts

npipe version 1.8.12

Bug Fixes:
Fix line offset issue in progress report.
Fix multicast socket reads on Unix.
Fix bug in UCS-2 character substitution conversion when converting into a seperate buffer.

Added Functionality:
Add --convert function: Pr, a very basic RTP packet header stripper.

npipe version 1.8.11

Bug Fixes:
Fix missing makefile internal dependency for Unix builds.
Fix Scheme function (conout) crashing on Unix.
Fix --temp on Unix creating files in the current directory, now files are created in /tmp/.

Added Functionality:
Add switch: --quit-timer, stop reading input and quit program after n seconds.
Add switch: --quit-on-read, quit when data is read from next input. This is mainly useful with multiple parallel inputs.
Add switches: --bind-port and --bind-interface, to bind the local end of a socket to a specific address/interface and/or port.
Add switches: --mcast-ttl and --mcast-loop, to set the TTL and loopback setting for outgoing multicast datagrams.
Add switch: --msr/--mcast-socket-read, to join a multicast group and read from it.... read more

npipe version 1.8.10

Bug Fixes:
Fix socket accepts used with --parallel on Windows crashing or otherwise failing due to pointer mismanagement.
Fix --gm/--socket-read(-interface)-multi, reading no data due to calling accept again on an already accepted socket.

Added Functionality:
Add switches: --gms/--socket-read-multi-sync and --socket-read-interface-multi-sync, to accept and read from multiple connections on a socket, in the main thread.
Add properties to Scheme function npipe-get-prop: numactiveinputs, numtotalinputs and file-inputN-*, to return the number of active, and total inputs, and select an input by number, respectively.
Add Scheme function filename-safe/fns, to replace commonly problematic filename characters in a string.... read more

npipe version 1.8.9

Bug Fixes:
Fix input switches such as --no-input and --zero being erroneously rejected in some circumstances due to an operator precedence issue.
Fix command-line arguments starting with a $ having the $ removed and a variable expansion performed on the remainder, (non Unix).
Fix variable expansions of a string ending with a $ producing undefined results.

Added Functionality:
Add variable-expanding versions of the --pre-fill-buffer and --input-text set of switches.... read more

npipe version 1.8.8

Bug Fixes:
Fix --output-file-input-regex crashing when no input is defined.
Deal more consistently with the case of an input thread glob matching no files.
Fix too many pipes being defined with --wait set potentially overflowing an array on Windows.

Added Functionality:
Allow multiple inputs, which are by default processed sequentially.
Add switch --pr/--parallel, to process multiple inputs in parallel, asynchronously.
Add switches: --pm/--pmaximum, --pj/--pmaxiocount, to set per input/output maximum bytes to read/write, and maximum number of reads/writes, respectively.... read more

npipe version 1.8.7

Bug Fixes:
Fix the last line of the progress display being filled with random characters.
Remove a minor memory leak.

Added Functionality:
Add jump targets: quit, next_cycle and next_input.
Add buffer comparison conversion.

Other Changes:
Remove some unnecessary duplication of strings.

npipe version 1.8.6

Bug Fixes:
localtime and gmtime are not thread safe on Unix, use localtime_r and gmtime_r instead.

Added Functionality:
Add switch: --ex/--expand, to expand variables in the associated filename.
Add switch: --mex/--multi-expand, to repeatedly expand variables in the associated filename until success, incrementing an iterator variable on each attempt.
This switch is useful for creating sequentially numbered output files.
Add variable expansions: $t, $T and $c, which expand to a formatted: local time, GMT time, and the above iterator variable, respectively.... read more

npipe version 1.8.5

Bug Fixes:
Fix socket creation failure error message on Windows (character set mismatch).
Display IPv6 addresses with a port correctly.
Warn when a parent thread has inputs but no outputs, and do not process those inputs before aborting.
Fix a few minor memory leaks.

Added Functionality:
Socket input may now be bound to more than one local interface/port (with some caveats, see readme).
Add Scheme functions: timestamp, timestamp-local, mono-clock, newline-util and buffer-length, to return timestamp strings, a monotonic clock integer, newline strings, or return the length of a buffer respectively.
Add Scheme functions: concat-atoms-to-string and concat-atoms-to-string-glue, to return the concatenation of the inputs, converted to strings, (with a separator).
Add switch: --tag, to assign a unique (within a thread) identifier to an input/output/etc, for use with Scheme function npipe-get-prop.
Add Scheme function npipe-get-prop file property socknearend, to return the address of the near end of a socket.... read more

npipe version 1.8.4

Bug Fixes:
Fix --wccs/--write-console-cs and --rccs/--read-console-cs switches failing gracelessly when supplied with an invalid character set.
Fix transcoding variable replacements where the unexpanded variable reference intersects the difference in sizes of the pre and post transcoded strings.

Added Functionality:
Add switches: --preamble, --postamble and variants thereof, to set data to be read or written from/to inputs or outputs before or after all other data.
Add --text-convert function: setnv, set buffer without variable expansion.... read more

npipe version 1.8.3

Bug Fixes:
Now process a partially filled read buffer and/or any remaining conversion fragments, after a fatal read error.
Fix hex decode conversion (-chu).
Fix -B/--block-read with --no-read without --roll not reading from the input buffer.

Added Functionality:
Add character set option: //autoh, as an extension to //auto, to detect the character set of the input using heuristic methods, as well as checking for a BOM as in //auto. This uses libcharsetdetect.
Add file-flag switches: --flush-write, --flush-end, to flush the output buffers after each write and upon termination, respectively.
Add switches --si-file/--scheme-immediate-file, --sp-file/--scheme-process-file, --sprog-file/--scheme-progress-file, to extend the existing Scheme switches by reading the Scheme code from a file.... read more

npipe version 1.8.2

Bug Fixes:
Fix character device detection on Unix.
Fix determination of block device sizes on Unix.
Fix #-preceded quoted string variable replacements in command files on Unix being processed in the wrong character set.
Fix format buffer as UTF-16LE conversion (--convert fw) formatting the buffer as the locale character set on Unix.
Fix --pre-fill-buffer-cs and related switches ignoring character set options such as //bom.
Fix multi-thread socket reads broken in v1.8.1: --gm/--socket-read-multi and --socket-read-interface-multi.
Fix --eof-char falling through to file.
Fix PCRE replacement named subpattern insertions, when the named subpattern is the whole match and duplicate subpattern names are allowed.... read more

npipe version 1.8.1

Bug Fixes:
Fix usage of va_lists, particularly on some Unix platforms where vsnprintf modifies the va_list.
Fix makefile when current directory is not in PATH.
Fix progress report only being updated once per second on Unix.
Fix --console-lineless not turning off echo on Unix.
Fix --console-lineless not restoring previous console mode on termination on Unix.
Make transcoding console IO work with --console-lineless.
Fix a race condition where bad data would be parsed if a command file had its size decreased just before it was read.
Fix command files not being flocked before being read on Unix.... read more

npipe version 1.8.0

Bug Fixes:
Fix a broken debug string and pipe creation on Windows.
Fix pipe resurrection not working for outputs on Windows.
Fix more minor memory leaks in the command-line parser.
Fix a bug that occurred when attempting to grow certain buffers in size.

Other Changes:
Unix: flock files before trying to chmod them.
Pipe resurrection now works on Unix, and no longer requires that --force-pipe-create/--create-new is set.
Improve the performance/memory use of --interleave-stream slightly.

npipe version 1.8.0 pre 2

Bug Fixes:
Fix progress bar/report output to make it thread-safe.
Fix progress bar target length when using zero/unknown length inputs and the --maximum switch.
Fix Scheme variable replacement on Windows.
Fix Unix command line arguments starting with " or $ being misinterpreted.
Fix non-thread-safe use of strtok on Unix.
Get rid of unaligned comparisons.
Fix -cu (convert char to uppercase) being mapped to -cl (convert char to lowercase) on Unix.... read more

npipe version 1.8.0 pre 1

This version adds support for Unix-like platforms, such as Linux.

Bug Fixes:
Fix bug in section IO.
Fix bug in --chain.
Fix a large number of very minor bugs.

Added functionality:
Add switches: --mod/--creation-mode-bits and --chmod/--change-mode-bits to set (created) files' permission/mode bits on Unix, they have no effect on Windows.
Add switches: --tc/--text-convert and --tr/--trans, to convert text between character sets in a more flexible and robust, and less cryptic manner than the various --convert subfunctions. These correctly handle MBCS sequences split over buffer boundaries.
Add switches: --og/--output-glob and --ag/--append-glob.
Add switch: --pipe, a more generic hint that the following file is a named pipe/fifo, than: --force-pipe-create or --force-pipe-connect.
Add switch: --detach-console, to detach the process from its parent console or tty.... read more

npipe version 1.7.0

Bug Fixes:
Fix string length off-by-one bugs in format string conversion and regular expression conversion branch label/subpattern storage argument validation.
Fix using --roll with --no-read with no pre-set buffer causing a crash. Now it does the expected action of continuously reading zero bytes.
Fix conversions and --sp/--scheme-process not accepting file-flags (such as --rate-limit).
Fix the quit, next-input, etc commands coming into effect after the next cycle.... read more

npipe version 1.7.0 pre-release 1

Bug Fixes:
Fix acquiring a socket handle causing crashes on OSs which export GetAddrInfoW, due to a calling convention mix-up.
Fix npipe in rare cases not terminating due to lingering system-created worker threads.
Fix persistent sockets not listening for new connections when the existing connection was terminated gracefully.
Fix --control-code not working in unicode mode.
Fix --control-code CSI functions: s and u, only working if used in the same output. Now the saved cursor state is globally accessible.
Fix --control-code CSI function: m, not working/behaving poorly if --new-console is used, or if the current thread's input is not currently a console.
64-bit version: Fix PCRE search/replace conversion potentially failing when a mix of named and numbered subpattern buffer stores used.
Fix attempting to detect the BOM of a zero-length input causing a crash.... read more

npipe version 1.6.2

Bug Fixes:
Fix bug when using --rate-limit with --skip.
Fix overlapped reads on Windows Vista/7.
Fix line-ending conversions being broken due to an operator precedence ambiguity.

Added functionality:
Add switch: --ig/--input-seq-glob, to read sequentially from files matching one or more DOS-style file globs.
Add switch: --tg/--input-thread-glob, to read simultaneously from files matching one or more DOS-style file globs, using separate threads.
Add switches: --gm/--socket-read-multi, --socket-read-interface-multi, to read simultaneously from each connection accepted on a socket, using a separate thread.... read more

npipe version 1.6.1

Bug Fixes:
Fix base64 encode conversion not encoding the penultimate chunk before the terminator when the read limit was reached using -j/--max-reads or -m/--maximum.

Added functionality:
Add switch: --pfbf/--pre-fill-buffer-file to load a buffer with the full contents of a file.
Add switch: -/--stdin, to explicitly set the input to STDIN rather than setting it implictly by default.
Add option to base64 encode conversion to not coalesce input buffers.
Add switch: --exclusive, to open a file/device/etc with exclusive rather than read-share access. This is useful for devices such as serial ports.
Add switch: --control-code, to parse certain escape/CSI control codes when writing to consoles.... read more

npipe version 1.6.0

Bug fixes:
Fix --interleave and --interleave-stream threads continuing for a number of cycles equal to the number of their active outputs after their link terminated.
Fix --random not working on some systems.
Fix --jump falling through to --roll.
Fix file mapping/section read errors being partially ignored.
Fix unfinished conversion fragments (line split, hex dump, etc), not being flushed at the end when reaching a read limit set by -j/--max-reads or -m/--maximum.... read more

npipe version 1.5.4

Bug fixes:
Fix bug in cyclic xor conversion that occurred when one or both buffers was not a multiple of 4.

Added functionality:
Add switch: --console-lineless, console input is read immediately, rather than at the end of a line, and is not echoed as it is entered.
Add switch: --sparse, write zero buffers as sparse file sections, where possible.
Cyclic conversions now have an option to resume reading where they left off previously, for the case where the operating buffer is not an integral multiple of the read buffer.... read more

npipe version 1.5.3

Bug Fixes:
Fix a major memory leak.
Fix successful write operations not adding to the write statistics and producing no debug output.
Fix a number of bugs with the --zero switch.
Fix --interleave(-stream) sometimes crashing when dealing with errors/abrupt termination.
Fix progress/report texts not displaying read/write speeds greater than ~4GB/s properly.
Fix reads from file mapping sections trying to read past the end when a skip was used.
Fix overlapped/asynchronous reads ignoring the -m/--maximum and -j/--max-reads switches.... read more

npipe version 1.5.2

Bug Fixes:
Fix capital letters not working when specifying the buffer for regular expression subpattern storage.
-k/--convert-to-buffer did not fully reject invalid arguments.
Fix conversions which output multiple buffers per input (splits for example) outputting to the wrong buffer when the current buffer at the end of the write chain did not equal that at the input to the conversion.
Fix regular expression label branches not working properly.
Fix hex dump conversion not rejecting superfluous flags.... read more

npipe version 1.5.1

Bug Fixes:
Fix Unicode character substitution (-cY) crashing.

Other changes:
Adjust the help texts, in particular the documentation of -j/--max-reads.
Adjust code and compilation flags to compile cleanly on gcc 4.5.0.

npipe version 1.5.0

Bug Fixes:
Fix buffer pre-fill switches checking for too many further parameters and hence not working if placed at the end of the command line.
Fix certain kinds of contradictory flag definitions not being rejected.
Fix line split conversions crashing on a zero-length input and outputting a zero-length buffer after the last full line of an input buffer.

Added functionality:
Add switches: -H/--pre-fill-buffer-hex and -I/--no-read, to load a buffer with a hexadecimal encoded string, and process the contents of buffer A without reading from an input, respectively.
Add shorthand switches -f/--input-text, -E/--input-text-unicode, -x/--input-hex as an alias for --no-read and the respective --pre-buffer-fill switch into buffer A.
Add base64 encode and decode, cyclic copy and xor, pad, trim and split/merge conversions.
Implement backslash escaping of quoted strings where preceded by a $ sign.
Add switch: --thread, to launch an isolated processing chain in its own thread.
Add switch: --interleave, to launch an isolated processing chain in its own thread which outputs to the current thread.
Add switch: --chain, to output to a separate, run program which in turn outputs to an isolated processing thread.
If --wait is set, pipes which have --persistent and --force-pipe-create set, now wait for another client to connect upon being disconnected.... read more