[sqlmap-users] BUG - Arabic unicode problem

[Sherif El-D. <arc...@gm...>]

The site has Arabic text.

sqlmap version: 1.0-dev (r4595)
Python version: 2.6.5
Operating system: posix
Command line: ./sqlmap.py -u ********************** --crawl 4 --form
--random-agent --batch
Technique: None
Back-end DBMS: None (identified)
Traceback (most recent call last):
  File "/pentest/database/sqlmap/lib/core/threads.py", line 123, in
runThreads
    threadFunction()
  File "/pentest/database/sqlmap/lib/utils/crawler.py", line 75, in
crawlThread
    soup = BeautifulSoup(content)
  File "/pentest/database/sqlmap/extra/beautifulsoup/beautifulsoup.py",
line 1519, in __init__
    BeautifulStoneSoup.__init__(self, *args, **kwargs)
  File "/pentest/database/sqlmap/extra/beautifulsoup/beautifulsoup.py",
line 1144, in __init__
    self._feed(isHTML=isHTML)
  File "/pentest/database/sqlmap/extra/beautifulsoup/beautifulsoup.py",
line 1186, in _feed
    SGMLParser.feed(self, markup)
  File "/usr/lib/python2.6/sgmllib.py", line 104, in feed
    self.goahead(0)
  File "/usr/lib/python2.6/sgmllib.py", line 143, in goahead
    k = self.parse_endtag(i)
  File "/usr/lib/python2.6/sgmllib.py", line 320, in parse_endtag
    self.finish_endtag(tag)
  File "/usr/lib/python2.6/sgmllib.py", line 358, in finish_endtag
    method = getattr(self, 'end_' + tag)
UnicodeEncodeError: 'ascii' codec can't encode characters in position
586-587: ordinal not in range(128)