Menu ▾ ▴

#6 Huffman dictionary encoding issue

open-fixed
nobody
None
5
2011-09-17
2011-09-17
Paul McGaley
No

In SingleTreeDictionaryEntry the dictionary cfg file is read using new StreamReader(fileStream) http://msdn.microsoft.com/en-us/library/yhfzs7at.aspx which assumes UTF-8 encoding but the bundled "Huffman Dictionary GBR.cfg" file is not UTF-8 encoded. This results in accented characters etc. coming out as garbage (particularly noticeable on Irish EPG). At least one other person has encountered this issue, see thread https://sourceforge.net/projects/epgcollector/forums/forum/1125946/topic/4075540

I have verified that transcoding the dictionary to UTF-8 fixes the issue. Transcoded file attached.

I didn't look through other dictionaries/classes to determine if this is limited to this scenario or if other dictionaries have the same problem.

Discussion

  • Paul McGaley

    Paul McGaley - 2011-09-17

    OpenTV GBR dictionary - UTF-8 encoded

     
    Huffman Dictionary GBR.cfg

  • Steve Bickell

    Steve Bickell - 2011-09-17
    • status: open --> open-fixed
     

  • Steve Bickell

    Steve Bickell - 2011-09-17

    Relevant files changed to UTF-8 in v3.2

     
MongoDB Logo MongoDB