#157 saga_cmd pj_proj does not output errors to stderr

[MacPgmr]

saga_cmd pj_proj4 does not output errors to a standard handle (stderr), so program that shells to saga_cmd can't detect them. Example:

Error: Proj4 [initialization]: projection not named
Error: executing module [Coordinate Transformation (Shapes)]

Even trying to redirect with ">" at command line fails - error message still displayed at console.

Tested with saga_2.1.0_20130517_win32.zip, but bug is also in Dec. 2012 beta.

Could someone check this? Should be easy to fix.

Thanks.

-Phil

[Volker Wichmann]

Hi,

I just tried to reproduce this on linux, but I can't. Errors are outputted to stderr, other messages to stdout. Could you please provide a complete example which allows to reproduce the problem?

[MacPgmr]

You need to test on Windows, I would guess.

Use an invalid .prj file, for example. That's how I noticed it.

saga_cmd.exe pj_proj4 2

Thanks.

-Phil

[Volker Wichmann]

Can't reproduce it on Windows 7 either:

c:\DEVELOP\LIS>saga_cmd pj_proj4 2 -CRS_PROJ4 test -SOURCE test.shp 1> stdout.log 2> stderr.log

results in:

---

stdout.log

--snip--
Parameters

Get CRS Definition from...: Proj4 Parameters
Proj4 Parameters: test
Well Known Text File:
EPSG Code: 4326
Precise Datum Conversion: no
Source: Graticule
Target: Target

---

stderr.log

Error: Proj4 >> WKT: no projection type defined
Error: executing module [Coordinate Transformation (Shapes)]

[MacPgmr]

(1) When I run your example without redirecting I get the same error message.

(2) When I run your example with redirecting, both stdout.log and stderr.log are created, but empty (0 bytes).

(3) When I run your example in a debugger (with redirection), I get this message:

"Unexpected parameter '1>stdout.log'"

Is that helpful?

I also tested other command line programs (dir, etc.) and had no problem redirecting both stdout and stderr.

Also, I'm testing with saga_2.1.0_20121211_win32.

Thanks.

-Phil

[MacPgmr]

This article is a few years old, but a couple of sentences caught my eye:

http://support.microsoft.com/default.aspx?scid=kb;en-us;190351

"Child processes that use such C run-time functions as printf() and fprintf() can behave poorly when redirected."

saga_cmd appears to call fprintf for stderr (if non-Unicode) or SG_FPrintf (Unicode), which in turn calls wxWidgets' wxVfprintf. wxVfprintf calls fprintf (see wxcrt.cpp).

Also this sentence:

"As a result, the output to the redirection pipe of a printf() call [...] is not flushed immediately and delays, sometimes-infinite delays occur."

That's the behavior I see when my app shells to saga_cmd.exe and saga_cmd has to output to stderr as a result of an error: it hangs (presumably what they mean by "sometimes-infinite delays").

Would something as simple as a fflush(stderr); fix this?

I'm running Windows 7.

One other thing: when I run saga_cmd.exe in a .bat file, I'm able to redirect stdout successfully, but the redirected stderr file that gets created is still empty.

Thanks.

-Phil

[MacPgmr]

One other note that might be helpful: I'm able to redirect successfully on my Win7 machine with SAGA 2.0.8. That should narrow down where and when the issue was introduced.

Thanks.

-Phil

[MacPgmr]

The 20130607 RC2 partially fixes this bug. However, it reveals a new bug:

Even when successful, grid_gridding 0 now outputs "Error: Grid" (that's the complete output) to stderr. This causes the program that shells to saga_cmd to fail since it detects output to stderr. Perhaps this error message has been in there for a while, but was never seen since flushing of stderr wasn't being done.

[Volker Wichmann]

Thanks for reporting, this should be fixed with commit

saga_api, parameter grid target: command line fix, refers to http://sourceforge.net/p/saga-gis/bugs/157/
http://sourceforge.net/p/saga-gis/code-0/1724/

Could you please test?

Thanks!

[MacPgmr]

I have no way of building SAGA on Windows (only Mac), so I'll have to wait until the next RC to test.

I think there might be one or more other places where fflush(stdout) is needed. Many times when my app shells out to saga_cmd.exe to retrieve the version (-v), nothing is returned (about 2 out of 3 times). Probably because the -v output is not getting flushed.

Thanks.

-Phil

[Volker Wichmann]

How is your app actually calling saga_cmd.exe? It may well be that the problems you observe are a result of the mechanism you use to call saga_cmd.exe and how the logging is handled in this case. We won't be able to help without information on how you exactly call saga_cmd.exe. It would be even better if you could provide examples which show this behaviour when saga_cmd.exe is simply called from the command line.

[MacPgmr]

The underlying code I'm using uses the Win API CreateProcess function to shell to external software - it's always worked with other external software. Since calling fflush fixed the stderr bug reported above, it seems reasonable to think that calling fflush(stdout) anywhere that output goes to stdout would be a good idea (per Microsoft's suggestion in article cited above). Note that I'm only seeing this problem now on Win XP, not on Win 7.

Also, the "Error: Grid" bug appears to be fixed.

Thanks.

-Phil

[Volker Wichmann]

I've just committed another fix, flushing stdout and stderr did not work under some circumstances (calling it from saga_cmd does not help in case the output occurs in a wxWidgets class), see http://sourceforge.net/p/saga-gis/code-0/1783/. Please report back in case this also helps with your problem, I would like to close this bug report.

[Volker Wichmann]

Is this problem still present in SAGA 2.1.1?

[Volker Wichmann]

I'm closing this as we got no further feedback and this should work correctly in latest versions.