#1319 Error in duplicate removal code

[Huji Lee]

In revision 7461, Alex had added a line to wikipedia.py like this (currently, line 5607):

self._rights[index] = list(set(self._rights[index]))

This is trying to remove duplicates in the _rights dictionary by creating a hash and then converting it back to a list.

I have created a bot to work in TranslateWiki and I get this error when ever the bot tries to save a page on the wiki:
Traceback (most recent call last):
File "dictation.py", line 180, in <module>
main()
File "dictation.py", line 174, in main
bot.run()
File "dictation.py", line 61, in run
self.treat(page)
File "dictation.py", line 83, in treat
if not self.save(text, page, self.summary):
File "dictation.py", line 121, in save
minorEdit=minorEdit, botflag=botflag)
File "/home/hojjat/bot/wikipedia.py", line 1682, in put
sysop = self._getActionUser(action = 'edit', restriction = self.editRestriction, sysop = sysop)
File "/home/hojjat/bot/wikipedia.py", line 1555, in _getActionUser
self.site().forceLogin(sysop = sysop)
File "/home/hojjat/bot/wikipedia.py", line 4922, in forceLogin
if not self.loggedInAs(sysop = sysop):
File "/home/hojjat/bot/wikipedia.py", line 4914, in loggedInAs
self._load(sysop = sysop)
File "/home/hojjat/bot/wikipedia.py", line 5994, in _load
self._getUserData(text, sysop = sysop, force = force)
File "/home/hojjat/bot/wikipedia.py", line 5608, in _getUserData
self._rights[index] = dict.fromkeys(self._rights[index]).keys()
TypeError: unhashable type: 'dict'

The last line suggests that the duplicate removal code doesn't work with dictionaries. I tried substituting that line with this:

self._rights[index] = dict.fromkeys(self._rights[index]).keys()

But still got the same error message.

I guess this has something to do with the fact that TranslateWiki uses custom user permissions (other than MediaWiki default). I have attached the family file I created for TranslateWiki for reference.

This is my python version info:

Pywikipedia [svn+ssh] huji@trunk/pywikipedia (r9287, 2011/06/04, 10:13:18)
Python 2.7.1+ (r271:86832, Apr 11 2011, 18:05:24)
[GCC 4.5.2]
config-settings:
use_api = True
use_api_login = True
unicode test: ok

[Huji Lee]

Family defintion for TranslateWiki

[Huji Lee]

Please disregard the first few words about who added that line. I relied on VIewVC's annotate function, but that lines isn't coming from revision 7461, apparently.

[Bináris]

Once a type is unhashable, it can't be applied as a dict key either, so the second try had to fail.
If removing doubles is not vital here, you may use this instead as a workaround:

try:
&nbsp;&nbsp;self._rights[index] = list(set(self._rights[index]))
except TypeError:
&nbsp;&nbsp;pass

This will prevent the bot of stopping, but leaves duplicates.

[Bináris]

Well, I thought nbsb was working here, so indent them naturally.

[Bináris]

I can' t imagine how all the "appends" and "extends" work in the previous lines if it is a dict rather than a list.
Could you write in a new line 5604 "print self._rights[index]" and dump here the result? The attached family file has nothing to say about rights.

[Huji Lee]

>> Could you write in a new line 5604 "print self._rights[index]" and dump here the result?

Surely can, and I wonder why I didn't try that myself!

[{u'*': u'*', u'implicit': u''}, {u'*': u'user', u'implicit': u''}, u'bot', u'translator', u'bot', u'autoconfirmed', u'nominornewtalk', u'autopatrol', u'suppressredirect', u'apihighlimits', u'writeapi', u'skipcaptcha', u'move', u'move-subpages', u'move-rootuserpages', u'editinterface', u'translate', u'deletedhistory', u'createaccount', u'read', u'edit', u'minoredit', u'createpage', u'createtalk', u'upload', u'reupload', u'reupload-shared', u'purge', u'lqt-split', u'lqt-merge', u'lqt-react', u'webchat', 'read', 'createaccount', 'edit', 'upload', 'createpage', 'createtalk', 'move', 'upload']

Seems like the first two elements of the list are dicts, which is why I got that error message. I'm going to trace back to see where these dicts are being added. In the meantime, do you think it would be a correct fix to make sure each element of the dict is added as an element in the list? In other words, if I change this:

...{u'*': u'user', u'implicit': u''}, u'bot', u'translator'...

to this:

u'*', u'user', u'implicit', u'bot', u'translator'

is that okay?

>> except TypeError:

I think duplicate removal is mandatory and except is not the best workaround. I'm going to try to fix the bug instead.

[Huji Lee]

I fixed the bug in revision 9288. I don't have permissoin on this tracker, so please close the bug accordingly.