Menu

UTF-8 BOM

pepone
2011-02-16
2013-05-28
  • pepone

    pepone - 2011-02-16

    with mcpp-2.7.2 if you use utf-8 encoding with BOM , could cause problems, i have create a patch that skip BOM, You can test that bug parsing a UTF-8 encode file that include a #define foo 1 in the first line.

    diff -c -r -N ../mcpp-2.7.2/src/system.c ./src/system.c
    *** ../mcpp-2.7.2/src/system.c  2008-11-26 10:53:51.000000000 +0100
    -- ./src/system.c      2011-02-16 15:30:23.984069926 +0100
    ***************
    *** 3534,3539 ****
    -- 3534,3565 ---
          FILEINFO *      file;
          const char *    too_many_include_nest =
                  "More than %.0s%ld nesting of #include";    /* _F_ _W4_ */
    +            
    +     //
    +     // When encoding is UTF-8, skip BOM if present.
    +     //
    +     if(mbchar == UTF8 && fp != NULL && ftell(fp) == 0)
    +     {
    +         const unsigned char UTF8_BOM = {0xEF, 0xBB, 0xBF};
    +         unsigned char FILE_HEAD = {0, 0, 0};
    +         int i;
    +         for(i = 0; i < 3; ++i)
    +         {
    +             FILE_HEAD_ = getc(fp);
    +             if(FILE_HEAD != UTF8_BOM)
    +             {
    +                 if(FILE_HEAD == (unsigned char)EOF)
    +                 {
    +                     i-;
    +                 }
    +                 for(; i >= 0; -i)
    +                 {
    +                     ungetc(FILE_HEAD, fp);
    +                 }
    +                 break;
    +             }
    +         }
    +     }
     
          filename = set_fname( filename);    /* Search or append to fnamelist  */
          fullname = set_fname( fullname);    /* Search or append to fnamelist  */_

     
  • pepone

    pepone - 2011-02-16

    There is a problem with the submit form, and the content of the patch is changed. I have submitted a bug report to sourceforge, see https://sourceforge.net/apps/trac/sourceforge/ticket/17679 , the content of the patch is attached to the bug report as mcpp-2.7.2.patch.

     

Log in to post a comment.

Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.