mingw-users

[Mingw-users] vsnprintf, strange behavior (new mingw runtime?)

[David D. <dav...@rs...>]

Hello again list,

I just tried to build my application and noticed that wxwidgets was
outputting strange values during formatting operations (Date/Time, long
to string conversions, etc)  I believe I've tracked this down to
invocations of vsnprintf:

void test_vsnprintf(const char* fmt, ...)
{
  static const size_t sz = 128;
  
  char buffer[sz];
  va_list ap;
  va_start(ap, fmt);
  if (vsnprintf(buffer, sz, fmt, ap) == -1) {
    MessageBox(0, "VSNPRINTF ERROR", "error", MB_OK);
    //note this is never called.
  }
  
  MessageBox(0, buffer, "vsnprintf", MB_OK);
  va_end(ap);
}

int main(int argc, char* argv[])
{
  long lval = 10;  
  test_vsnprintf("%ld", lval);
}

produces:

---------------------------
vsnprintf
---------------------------
17221066510303242
---------------------------
OK   
---------------------------


It seems almost as if %ld is expecting a 64 bit integer.  If I cast the
lval to a long long, I get no warnings, and the program outputs "10" as
expected.  As far as I can tell, even the MSVCRT version should accept %
ld as a long argument.  

I would get around this by not using the printf family, but I can't
because libraries I depend on apparently use it.

I will be looking into this more, but any help would be appreciated.

Thanks,
-Dave

Re: [Mingw-users] vsnprintf, strange behavior (new mingw runtime?)

[David D. <dav...@rs...>]

Hello again,

I'm replying to myself.

> I will be looking into this more, but any help would be appreciated.

If I use _vsnprintf (notice the underscore) the program works as
expected.  However, short of patching every invocation of vsnprintf in
wxwidgets, I don't think this solves my problem.

Doesn't vsnprintf just call _vsnprintf?  

vsnprintf.c in the mingw runtime:

int vsnprintf (char* s, size_t n, const char* format, va_list arg)
  { return _vsnprintf ( s, n, format, arg); }

Why would the format be getting messed up?

- Dave

Re: [Mingw-users] vsnprintf, strange behavior (new mingw runtime?)

[James S. <jam...@op...>]

On Tue, 2007-09-04 at 21:59 -0400, David Daeschler wrote:
> Hello again,
> 
> I'm replying to myself.
> 
> > I will be looking into this more, but any help would be appreciated.
> 
> If I use _vsnprintf (notice the underscore) the program works as
> expected.  However, short of patching every invocation of vsnprintf in
> wxwidgets, I don't think this solves my problem.
> 
> Doesn't vsnprintf just call _vsnprintf?  

There has been a lot of discussion about this issue in the mailing list
recently.

The thread starts here:

http://www.nabble.com/Strange-snprintf-behaviour-tf4070911.html

Regards,
James.

Re: [Mingw-users] vsnprintf, strange behavior (new mingw runtime?)

[James S. <jam...@op...>]

On Wed, 2007-09-05 at 12:03 +1000, James Steward wrote:
> On Tue, 2007-09-04 at 21:59 -0400, David Daeschler wrote:
> > I'm replying to myself.

Me too.
 
> > If I use _vsnprintf (notice the underscore) the program works as
> > expected.  However, short of patching every invocation of vsnprintf in
> > wxwidgets, I don't think this solves my problem.

After grepping the wxWidgets source for vsnprintf, it doesn't appear all
that often.

see in:
include/wx/wxchar.h
src/zlib/gzio.c
src/zlib/zutils.h

There may be more, but not many.

Regards,
James.

Re: [Mingw-users] vsnprintf, strange behavior (new mingw runtime?)

[David D. <dav...@rs...>]

Hi James,

> There has been a lot of discussion about this issue in the mailing list
> recently.

Thank you for the link.  So from what I can now tell, since I'm not
explicitly asking to use the msvcrt vsnprintf, I'm using the MINGW
implementation.  Unless C99 doesn't support %ld, this now looks like a
bug in that implementation.


mingwex/gdtoa/mingw_snprintf.c:

As I'm peering into the source, I see:

			case 'l':
				if (fmt[0] == 'l')
				  {
				    fmt++;
				    len = LEN_LL;
				  }
  				else
				  len = LEN_LL;
				goto fmtloop;

I believe this should read:

			case 'l':
				if (fmt[0] == 'l')
				  {
				    fmt++;
				    len = LEN_LL;
				  }
  				else
				  len = LEN_L;
				goto fmtloop;

Note the second case as LEN_L instead of LL.

I will rebuild this and test to see if it solves my problem.

- Dave

Re: [Mingw-users] vsnprintf, strange behavior (new mingw runtime?)

[David D. <dav...@rs...>]

Hi List,

> I will rebuild this and test to see if it solves my problem.

Indeed that fix solves my problem.  If someone with commit authority can
verify what I've done, and agrees that it fixes a problem, is there a
specific way I should generate a patch, or is this post good enough?


Again, the file (this time with line information)

mingwex/gdtoa/mingw_snprintf.c: 
function: x_sprintf
line: 453

                        case 'l':
                                if (fmt[0] == 'l')
                                  {
                                    fmt++;
                                    len = LEN_LL;
                                  }
                                else
                                  len = LEN_LL;
                                goto fmtloop;

I believe this should read:

                        case 'l':
                                if (fmt[0] == 'l')
                                  {
                                    fmt++;
                                    len = LEN_LL;
                                  }
                                else
                                  len = LEN_L;
                                goto fmtloop;

Note the second case as LEN_L instead of LL.

Thanks,
- Dave

Re: [Mingw-users] vsnprintf, strange behavior (new mingw runtime?)

[Keith M. <kei...@to...>]

David Daeschler wrote:
>> I will rebuild this and test to see if it solves my problem.
>
> Indeed that fix solves my problem.  If someone with commit authority
> can verify what I've done, and agrees that it fixes a problem, is there
> a specific way I should generate a patch, ...

Patches, in `diff -u ...' format, please, to the patch tracker:
http://mingw.org/MinGWiki/index.php/SubmitPatches
http://sourceforge.net/tracker/?func=add&group_id=2435&atid=302435

> ... or is this post good enough?

For such a small change, you might think so.  However, by the time I, or
whoever else may wish to pick it up, get around to it, it's likely to have
been forgotten where the patch is, so `no', it isn't really.

Regards,
Keith.

Re: [Mingw-users] vsnprintf, strange behavior (new mingw runtime?)

[David D. <dav...@rs...>]

Hi Keith,

> Patches, in `diff -u ...' format, please, to the patch tracker

Patch submitted.

http://sourceforge.net/tracker/index.php?func=detail&aid=1788440&group_id=2435&atid=302435

- Dave

Re: [Mingw-users] vsnprintf, strange behavior (new mingw runtime?)

[David D. <dav...@rs...>]

Hi Greg,

> As I interpret 7.19.6.1/8, that's writing a byte that it
> shouldn't be touching.

Is there an online reference for the C standard?  It might help others
to find problems like this.  For example everywhere I go says:

"n - Nothing printed. The argument must be a pointer to a signed int,
where the number of characters written so far is stored."

Which would explain your stack getting trampled.

- Dave

Re: [Mingw-users] vsnprintf, strange behavior (new mingw runtime?)

[<Joe...@pd...>]

Hi David,

>Is there an online reference for the C standard?  It might help others
>to find problems like this.

Not the C standard but still useful: "The Open Group Base =
Specifications"
http://www.opengroup.org/onlinepubs/009695399/toc.htm =20

Documentation for snprintf reads:

The length modifiers and their meanings are:

hh
    Specifies that a following d, i, o, u, x, or X conversion specifier =
applies to a signed char or unsigned char argument (the argument will =
have been promoted according to the integer promotions, but its value =
shall be converted to signed char or unsigned char before printing); or =
that a following n conversion specifier applies to a pointer to a signed =
char argument.


   J=F6rg

Re: [Mingw-users] vsnprintf, strange behavior (new mingw runtime?)

[Keith M. <kei...@to...>]

J=F6rg Richter wrote:
>> Is there an online reference for the C standard?  It might help othe=
rs
>> to find problems like this.
>
> Not the C standard but still useful: "The Open Group Base Specificati=
ons"
> http://www.opengroup.org/onlinepubs/009695399/toc.htm

That's actually the POSIX (IEEE 1003.1-2001) specification; treat it wi=
th
caution, in the MinGW context, for Woe32 is not POSIX.  Where this conf=
licts
with MSDN, then expect MSDN documented behaviour with MinGW.

Finding an online copy of the *final* ISO-C99 Standard is not likely to=
 be
easy; the ISO usually want you to part with a significant wad of cash, =
in
exchange for it.  However, Google points to this Technical Committee Dr=
aft:
http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1124.pdf

Regards,
Keith.=

Re: [Mingw-users] vsnprintf, strange behavior (new mingw runtime?)

[David D. <dav...@rs...>]

Hi Keith,

> However, Google points to this Technical Committee Draft:
> http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1124.pdf

Thank you for the link.  The c99 draft that you linked also includes the
verbiage:

hh   Specifies that a following d, i, o, u, x, or X conversion specifier
applies to a signed char or unsigned char argument (the argument will
have been promoted according to the integer promotions, but its value
shall be converted to signed char or unsigned char before printing); or
that a following n conversion specifier applies to a pointer to a signed
char argument.

> Where this conflicts with MSDN, then expect MSDN documented behaviour
> with MinGW.

So for clarification, we are speaking of the sprintf implementation
inside of mingwex/gdtoa/mingw_snprintf.c that is now (as far as I can
tell) used as default for the mingw-runtime 3.13.  Should this new
x_sprintf support this because it is in the draft?  Or is this a case
with being compatible with MS?

Thank you,
- Dave

Re: [Mingw-users] vsnprintf, strange behavior (new mingw runtime?)

[Keith M. <kei...@to...>]

David Daeschler wrote, quoting me:
>> Where this conflicts with MSDN, then expect MSDN documented behaviour
>> with MinGW.
>
> So for clarification, we are speaking of the sprintf implementation
> inside of mingwex/gdtoa/mingw_snprintf.c that is now (as far as I can
> tell) used as default for the mingw-runtime 3.13.

In its original context, that was a general comment concerning the
applicability of IEEE 1003.1-2001 (POSIX) to MinGW.

> Should this new x_sprintf support this because it is in the draft?
> Or is this a case with being compatible with MS?

In the specific case of snprintf, the general comment doesn't apply,
for MSVCRT doesn't provide it; it *does* provide _snprintf, but that
conforms neither to ISO-C99 nor to IEEE 1003.1-2001.  Until about a
week ago, libmingwex.a provided an implementation of snprintf, which
was a simple redirect to _snprintf; we discussed that at length, and
agreed that it was broken -- libmingwex.a is supposed to provide C99
or POSIX conforming *extensions* to the MSVCRT standard, and this
non-conforming redirection of snprintf didn't fit with that goal.

On 25-Aug, to correct the anomaly, I posted an updated mingw-runtime:
http://downloads.sourceforge.net/mingw/mingw-runtime-3.13-20070825-1-src.tar.gz
http://downloads.sourceforge.net/mingw/mingw-runtime-3.13-20070825-1.tar.gz

This makes snprintf default to the mingwex/gdtoa/mingw_snprintf.c
implementation, with ISO-C99 conformance as its goal; if you use this,
or any later version of mingw-runtime, then this is the behaviour you
should expect.  If you need Microsnot's (broken) behaviour, then you
should be using _snprintf anyway.

Regards,
Keith.

Re: [Mingw-users] vsnprintf, strange behavior (new mingw runtime?)

[David D. <dav...@rs...>]

Hi Keith,

> This makes snprintf default to the mingwex/gdtoa/mingw_snprintf.c
> implementation, with ISO-C99 conformance as its goal; 

Okay, that answers my question then.  

> If you need Microsnot's (broken) behaviour, then you
> should be using _snprintf anyway.

Nope, I don't want that.  All I wanted to know was if the version in
mingwex/gdtoa/mingw_snprintf.c was supposed to conform to c99, so that I
know if it is ok to fix it to handle the hh case that Greg Chicares
found.

I guess my question now is, Greg, do you want to have a whack at this
one?  If so, it might be best if you patch your local copy first with
the other fix in
http://sourceforge.net/tracker/index.php?func=detail&aid=1788440&group_id=2435&atid=302435

If you can't do it, I can try to take a look, but time is going to be
getting tight as I'm getting married next week.

- Dave

Re: [Mingw-users] vsnprintf, strange behavior (new mingw runtime?)

[David D. <dav...@rs...>]

Hi Greg/List,

> I won't be able to do that any time soon; it's yours if you want it,
> as far as I'm concerned.

I've made the changes to mingwex/gdtoa/mingw_snprintf.c to allow %hh to
work correctly.  I followed the formatting, flow, and style of code that
I saw in mingw_snprintf.c.  I also found a few other bugs in the code
when I was in there.

I realized that the mingw-runtime 3.13 version of the code has a
snprintf.c inside mingwex/stdio that overrides the snprintf function
exported from mingw_snprintf.c.  So Greg, you need to use the CVS
version of the runtime to test my changes, or you need to delete/comment
out mingwex/stdio/snprintf.c.  This took me a while to figure out why my
changes weren't working.

What I changed:

2007-09-06  David C. Daeschler <dav...@rs...>
* mingw_sprintf.c (x_sprintf):
  l (ell) length specifier being treated as ll (ell ell) in all cases
  
  hh was not recognized:
    (hh Specifies that a following d, i, o, u, x, or X conversion
    specifier applies to a signed char or unsigned char argument (the
    argument will have been promoted according to the integer
    promotions, but its value shall be converted to signed char or
    unsigned char before printing); or that a following n conversion
    specifier applies to a pointer to a signed char argument.)

  fix bug in output of characters written to a short when using %hn
  (switch/case fallthrough, the LEN_S case was falling through to
   the LEN_LL case which would cause corruption)

I will be updating my previous patch that only took care of the first
problem.  patch is here:

http://sourceforge.net/tracker/index.php?func=detail&aid=1788440&group_id=2435&atid=302435

New output of Greg's test:

$ MINGW32_NT-5.1_i686/test.exe
%d output:  '11' ('11' is expected)
%hhn output: 2   (2 is expected)
guard 0:     55  (55 is expected)
guard 2:     77  (77 is expected)
len:         2   (2 is expected)

Re: [Mingw-users] vsnprintf, strange behavior (new mingw runtime?)

[Egon A. <po...@ta...>]

David Daeschler wrote:
> Hi Greg/List,
> 
>> I won't be able to do that any time soon; it's yours if you want it,
>> as far as I'm concerned.
> 
> I've made the changes to mingwex/gdtoa/mingw_snprintf.c to allow %hh to
> work correctly.  I followed the formatting, flow, and style of code that
> I saw in mingw_snprintf.c.  I also found a few other bugs in the code
> when I was in there.
> 
> I realized that the mingw-runtime 3.13 version of the code has a
> snprintf.c inside mingwex/stdio that overrides the snprintf function
> exported from mingw_snprintf.c.  So Greg, you need to use the CVS
> version of the runtime to test my changes, or you need to delete/comment
> out mingwex/stdio/snprintf.c.  This took me a while to figure out why my
> changes weren't working.
> 
> What I changed:
> 
> 2007-09-06  David C. Daeschler <dav...@rs...>
> * mingw_sprintf.c (x_sprintf):
>   l (ell) length specifier being treated as ll (ell ell) in all cases
>   
>   hh was not recognized:
>     (hh Specifies that a following d, i, o, u, x, or X conversion
>     specifier applies to a signed char or unsigned char argument (the
>     argument will have been promoted according to the integer
>     promotions, but its value shall be converted to signed char or
>     unsigned char before printing); or that a following n conversion
>     specifier applies to a pointer to a signed char argument.)
> 
>   fix bug in output of characters written to a short when using %hn
>   (switch/case fallthrough, the LEN_S case was falling through to
>    the LEN_LL case which would cause corruption)
> 
> I will be updating my previous patch that only took care of the first
> problem.  patch is here:
> 
> http://sourceforge.net/tracker/index.php?func=detail&aid=1788440&group_id=2435&atid=302435
> 
> New output of Greg's test:
> 
> $ MINGW32_NT-5.1_i686/test.exe
> %d output:  '11' ('11' is expected)
> %hhn output: 2   (2 is expected)
> guard 0:     55  (55 is expected)
> guard 2:     77  (77 is expected)
> len:         2   (2 is expected)
> 

I've tried to use the newest mingwex I could find
http://downloads.sourceforge.net/mingw/mingw-runtime-3.13-20070825-1.tar.gz
It works well for most of what I use, but it does still not support the 
length modifier 'z' as used in "%zd" for printing values of type size_t 
or ssize_t

Best regards
Egon Andersen

Re: [Mingw-users] vsnprintf, strange behavior (new mingw runtime?)

[Greg C. <chi...@co...>]

On 2007-09-05 17:09Z, David Daeschler wrote:

[[v]snprintf() with "%hhn"]

> I guess my question now is, Greg, do you want to have a whack at this
> one?

I won't be able to do that any time soon; it's yours if you want it,
as far as I'm concerned.

Re: [Mingw-users] vsnprintf, strange behavior (new mingw runtime?)

[Greg C. <chi...@co...>]

On 2007-09-05 13:48Z, David Daeschler wrote:
> 
>> As I interpret 7.19.6.1/8, that's writing a byte that it
>> shouldn't be touching.
> 
> Is there an online reference for the C standard?  It might help others
> to find problems like this.  For example everywhere I go says:
> 
> "n - Nothing printed. The argument must be a pointer to a signed int,
> where the number of characters written so far is stored."

Quoting from the 2005-05-06 committee draft [7.19.6.1/8], to which Keith
provided a link, and which seems to match the corresponding paragraph in
the final ISO/IEC 9899, 2nd ed., 1999-12-01, exactly:

"n The argument shall be a pointer to signed integer into which is written the
number of characters written to the output stream so far by this call to
fprintf. No argument is converted, but one is consumed. If the conversion
specification includes any flags, a field width, or a precision, the behavior is
undefined."

The crucial difference is that it points to a "signed integer" instead of
a "signed int": 6.2.5/4 says

"There are five standard signed integer types, designated as signed char, short
int, int, ...".

Re: [Mingw-users] vsnprintf, strange behavior (new mingw runtime?)

[Greg C. <chi...@co...>]

I confirm that "%ld" isn't behaving as I'd expect....

On 2007-09-05 02:36Z, David Daeschler wrote:
> 
> Again, the file (this time with line information)

The line numbers seem slightly different in HEAD:

http://sources.redhat.com/cgi-bin/cvsweb.cgi/src/winsup/mingw/mingwex/gdtoa/mingw_snprintf.c?annotate=1.2&cvsroot=src

> mingwex/gdtoa/mingw_snprintf.c: 
> function: x_sprintf
> line: 453
> 
>                         case 'l':
>                                 if (fmt[0] == 'l')
>                                   {
>                                     fmt++;
>                                     len = LEN_LL;
>                                   }
>                                 else
>                                   len = LEN_LL;

Your suggestion to change the last line this way

-                                   len = LEN_LL;
+                                   len = LEN_L;

looks correct to me.

Since we're inspecting the "l" versus "ll" code, I thought I'd
take a look at "h" versus "hh" as well, and I think there's a
problem there:

/tmp[0]$cat mingw-snprintf-test-hhn.cpp
#include <iostream>
#include <ostream>
#include <stdio.h> // snprintf()

int main()
{
    char buf[1000];

    signed char sc0 = 55;
    signed char sc1 = 66;
    signed char sc2 = 77;

    // This should modify only the second of the three variables.
    int len = snprintf(buf, 20, "%d%hhn", 11, &sc1);

    std::cout << "%d output:  '" << buf      <<  "' ('11' is expected)\n";
    std::cout << "%hhn output: " << (int)sc1 << "   (2 is expected)\n";
    std::cout << "guard 0:     " << (int)sc0 <<  "  (55 is expected)\n";
    std::cout << "guard 2:     " << (int)sc2 <<  "  (77 is expected)\n";
    std::cout << "len:         " << len      << "   (2 is expected)\n";

    std::cout << std::endl;
}

/tmp[0]$g++ -mno-cygwin -ggdb mingw-snprintf-test-hhn.cpp
/tmp[0]$./a
%d output:  '11' ('11' is expected)
%hhn output: 2   (2 is expected)
guard 0:     0  (55 is expected)
guard 2:     77  (77 is expected)
len:         2   (2 is expected)

As I interpret 7.19.6.1/8, that's writing a byte that it
shouldn't be touching.