Just one suggestion before making any conclusion out of the feature weights.
Whichever classs has high frequency of the specific word you are talking about, take that as the positive class and run the experiement in binary mode(i.e. 2 class mode). Then see the feature weights if they make sense.

In CRF the feature weights are most of the times intuitive but that is not necessary as
1. they are a result of collective weight assignment. So, a feature in isolation may not have direct effect.
2. there can be bug(s) in features/dataset loading/processing :)

Also, in multi-class classification, it is not clear as how to interpret the feature weights.

Has anybody else seen such kind of thing. Would like to know kind of experiment and the possible reasons for such kind of behaviour.


On 9/19/06, crf-users-request@lists.sourceforge.net < crf-users-request@lists.sourceforge.net> wrote:
Send Crf-users mailing list submissions to

To subscribe or unsubscribe via the World Wide Web, visit
or, via email, send a message with subject or body 'help' to

You can reach the person managing the list at

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Crf-users digest..."

Today's Topics:

   1. Negative weights for highly frequent words. (Roger P. Menezes)


Message: 1
Date: Mon, 18 Sep 2006 12:45:49 +0530
From: "Roger P. Menezes" <rogerm@yahoo-inc.com>
Subject: [Crf-users] Negative weights for highly frequent words.
To: crf-users@lists.sourceforge.net
Message-ID: <450E47A5.9020104@yahoo-inc.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed

Hi All,

We are working on a page segmentation task. To be more specific, it
deals with resume segmentation. The idea is to segment a given resume
into different sections like PERSONAL_DETAILS, EXPERIENCE, EDUCATION,
SKILLS, etc. We are using a naive model with each label having a single
state in the CRF. Labels correspond to our sections (experience,
education, skills etc.). Each line in the document is an observation.

We trained the CRF for the above task and tried analysing the
WordFeatures. Everything works fine but certain highly indicative
features (words in this case) had high negative weights like -5., -3.
etc. Words which are present 90-95% in the specific LABEL have negative
weights for that label. Don't know why this happens. In fact, it helps
to neutralize (reduce them to insignificant wts) these weights and then
carry out our test results.

I'm unable to identify if there's a pattern on what kind of words are
shown with such negative weights. Not all indicative (frequent) words
suffer this. Has anybody seen this kind of thing happening? Would
provide more details as and when I start identifying things.



Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys -- and earn cash


Crf-users mailing list

End of Crf-users Digest, Vol 3, Issue 1