Re: [naviserver-devel] Towards NaviServer 4.99.24

[Gustaf N. <ne...@wu...>]

Dear David,

the automated shortening for the invalid strings is now committed.

https://bitbucket.org/naviserver/naviserver/commits/51f101928be6d27efe5ab78d7d9a9693026791c1

I'll try to make rc2 soon.

all the best

-gn

On 30.05.22 21:01, Gustaf Neumann wrote:
>
> i agree, this might be a lot of logging, especially when the decoded 
> strings are long.
>
> Not sure, what the best approach of this is
> - an configuration parameter LogInvalidUTF8warnings to deactivate 
> these messages (default true)
> - an configuration parameter NrInvalidUTF8warnings to limit these 
> messages (default 99999999)
> - truncate the invalid UTF8 sequence in the log entry to shorten the 
> messages (when longer than e.g. 20 bytes)
> - try to identify the first invalid UTF8 sequence in the string and 
> report just this (even shorter log message).
>
> The last option would be probably the best, but requires an C.API change.
>
> Other options?
>
> -gn
>
>
>
> On 30.05.22 10:41, David Osborne wrote:
>> Thanks Gustaf - the errorCode will be very handy to trap these 
>> encoding errors.
>>
>> We're using the per-server "formfallbackcharset" without issue at the 
>> moment. Working well.
>>
>> One problem we encountered is the Warnings logged when invalid UTF-8 
>> is encountered.
>> Some of the POSTs causing encoding issues were very large. This was 
>> filling logs quite quickly (I think it's possible to have "maxpost" 
>> bytes of data written to the log). Didn't want to disable all 
>> Warnings, so we patched NaviServer to log just a Warning of invalid 
>> UTF-8 but not include the data itself. But there may be a better way 
>> of dealing with this.
>>
>> Regards,
>> David
>>
>>
>> On Sat, 28 May 2022 at 18:45, Gustaf Neumann <ne...@wu...> wrote:
>>
>>     Dear all,
>>
>>     The latest commits
>>     - added the "multipart/form-data" handlingfor fallback charsets,
>>     - provides an error code for invalid UTF-8 and
>>     - adds  support for the "_charset_" field for default charsets
>>     (see RFC
>>     7578, section 4.6).
>>
>>     Also, the regression test got several more tests.
>>
>>     I have still a bug report for ns_connchan (which i could not
>>     reproduce
>>     so far),
>>     if i find something to fix the next days, this will go as well
>>     into the next
>>     release, otherwise we are ready for rc2.
>>
>>     all the best
>>
>>
>>
>> _______________________________________________
>> naviserver-devel mailing list
>> nav...@li...
>> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
> -- 
> Univ.Prof. Dr. Gustaf Neumann
> Head of the Institute of Information Systems and New Media
> of Vienna University of Economics and Business
> Program Director of MSc "Information Systems"
>
>
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel

-- 
Univ.Prof. Dr. Gustaf Neumann
Head of the Institute of Information Systems and New Media
of Vienna University of Economics and Business
Program Director of MSc "Information Systems"