#19 unicode error processing /var/log/secure

open-accepted
None
5
2007-03-28
2007-03-28
No

Running fairly generic Fedora Core 6, fully updated.

[root@wume2 ~]# rpm -q fail2ban
fail2ban-0.6.2-3.fc6

fail2ban dies at startup with:

2007-03-28 06:30:16,348 ERROR: Type: 'UnicodeEncodeError'
Value: ('latin-1', u'Mar 26 10:34:24 wume2 sshd[2544]: pam_unix(sshd:auth): bad username [\ufeffroot]\n', 69, 70, 'ordinal not in range(256)')
TB: [('/usr/bin/fail2ban', 48, '?', 'fail2ban.main()'), ('/usr/share/fail2ban/fail2ban.py', 486, 'main', 'e = element[1].getFailures()'), ('/usr/share/fail2ban/logreader/logreader.py', 141, 'getFailures', "line = line.decode('utf-8').encode('latin-1')")]

The relevant line in my /var/log/secure is
Mar 26 10:34:24 wume2 sshd[2544]: pam_unix(sshd:auth): bad username [root]

but copying and pasting here won't capture the unicode, so here is od output:

% grep username /var/log/secure | od -a
0000000 M a r sp 2 6 sp 1 0 : 3 4 : 2 4 sp
0000020 w u m e 2 sp s s h d [ 2 5 4 4 ]
0000040 : sp p a m _ u n i x ( s s h d :
0000060 a u t h ) : sp b a d sp u s e r n
0000100 a m e sp [ o ; ? r o o t ] nl M a

-- Brian

Discussion

  • Cyril Jaquier

    Cyril Jaquier - 2007-03-28

    Logged In: YES
    user_id=933467
    Originator: NO

    Hi Brian,

    Could you try this patch?

    Thank you

    Cyril

    Index: /home/cyril/workspace/fail2ban-stable/logreader/logreader.py

    --- /home/cyril/workspace/fail2ban-stable/logreader/logreader.py (revision 567)
    +++ /home/cyril/workspace/fail2ban-stable/logreader/logreader.py (working copy)
    @@ -137,8 +137,8 @@
    lastLine = None
    for line in logFile:
    try:
    - # Try to convert UTF-8 string to Latin-1
    - line = line.decode('utf-8').encode('latin-1')
    + # Decode line to UTF-8
    + line = line.decode('utf-8')
    except UnicodeDecodeError:
    pass
    if not self.hasTime(line):

     
  • Cyril Jaquier

    Cyril Jaquier - 2007-03-28
    • assigned_to: nobody --> lostcontrol
    • status: open --> open-accepted
     
  • Brian Davison

    Brian Davison - 2007-03-29

    Logged In: YES
    user_id=1139464
    Originator: YES

    Hi Cyril,

    Thanks for the suggestion. It didn't work, but suggested another variation which might. When I applied your patch, I got other errors. Here are the relevant parts from fail2ban -vv:

    2007-03-28 19:47:39,360 WARNING: SSH: ReBan 207.44.198.118
    2007-03-28 19:47:39,360 DEBUG: iptables -I fail2ban-SSH 1 -s u'207.44.198.118' -j DROP
    iptables v1.3.5: host/network `u207.44.198.118' not found
    Try `iptables -h' or 'iptables --help' for more information.
    2007-03-28 19:47:39,666 ERROR: 'iptables -I fail2ban-SSH 1 -s u'207.44.198.118' -j DROP' returned 512
    2007-03-28 19:47:39,667 ERROR: Fail2Ban got an unhandled exception and died.
    2007-03-28 19:47:39,668 ERROR: Type: 'ExternalError'
    Value: ('Big Oops happened: situation is out of control. Something is wrong with your setup. Please check your settings',)
    TB: [('/usr/bin/fail2ban', 48, '?', 'fail2ban.main()'), ('/usr/share/fail2ban/fail2ban.py', 549, 'main', 'raise ExternalError("Big Oops happened: situation is out of " +')]

    However, changing the line from

    line = line.decode('utf-8')

    to

    line = line

    was sufficient to eliminate all of my errors and run properly. However, I don't know what the ramifications of this change are (and obviously I don't know python since I substituted an identity rather than rewrite anything else).

    -- Brian

     

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:





No, thanks