#190 Invalid CodePage Issues can crash the format reader

[Adam Tauno Williams]

'Binary' data in an input record that was not iconv'd or some such can kill the read-through-format process even if discarding bad records is enabled - this is because it errors in the logger when recording the record discard. Doh! Discarded records should (a) not be logged and (b) if stored anywhere be base64 encoded since we already know something is wrong with them.

Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/OpenGroupware-0.1.49rc52-py2.6.egg/coils/logic/workflow/services/process.py", line 592, in work
    action_name, description = self.run_stanza( action_uuid )
  File "/usr/lib/python2.6/site-packages/OpenGroupware-0.1.49rc52-py2.6.egg/coils/logic/workflow/services/process.py", line 281, in run_stanza
    parameters=stanza.get('params'))
  File "/usr/lib/python2.6/site-packages/OpenGroupware-0.1.49rc52-py2.6.egg/coils/core/context.py", line 405, in run_command
    command.run()
  File "/usr/lib/python2.6/site-packages/OpenGroupware-0.1.49rc52-py2.6.egg/coils/core/logic/action.py", line 186, in run
    self.do_action()
  File "/usr/lib/python2.6/site-packages/OpenGroupware-0.1.49rc52-py2.6.egg/coils/logic/workflow/actions/format/read.py", line 35, in do_action
    format.process_in(self.rfile, self.wfile)
  File "/usr/lib/python2.6/site-packages/OpenGroupware-0.1.49rc52-py2.6.egg/coils/logic/workflow/formats/simple_fixed_field_format.py", line 102, in process_in
    self.log.warn('Record format exception on record {0}: {1}'.format( self.in_counter, unicode( record ) ) )
UnicodeDecodeError: 'ascii' codec can't decode byte 0xa0 in position 12: ordinal not in range(128)