clucene-developers — CLucene development-related discussion.

[CLucene-dev] MultiLevelSkipListReader bug

[Šplíchal J. <spl...@to...>]

Hello,

 

I found a and corrected bug in MultiLevelSkipListReader::init method. 

This bug is serious but quite simple to correct so it would be great to merge the fix

to master. I uploaded the fix to the highlighter branch as it is necessary for the

new position based highlighting feature.

 

Jiri

 

--

Jiří Šplíchal

TOVEK, spol. s r.o.

spl...@to... <mailto:spl...@to...> 

+420 606671930

 

[CLucene-dev] MultipleTermPositions::TermPositionsQueue memory leak fix

[Šplíchal J. <spl...@to...>]

Hello,

 

I created a new branch with a simple fix of a memory leak caused by constructor

of MultipleTermPositions::TermPositionsQueue.

 

Jiri

 

Re: [CLucene-dev] CLucene-developers Digest, Vol 57, Issue 14

[muhammad i. <m.i...@gm...>]

Hello Jiri,

I am grateful for you. and I want to thank you for your work the
SpanHighlightScorer is really amazing to me and it get me out of many
problems I faced.

Thank you

From: ?pl?chal Ji?? <spl...@to...>

>
> Hello Mohammad,
>
>
>
> the new queryscorer is able to rewrite the highlighting queries itself. You
> have to allow it while creating it:
>
>
>
>    // Create token stream from the text you want to be highlighted.
> fieldName is name of the field that is being highlighted
>
>    // only the relevant part of the query is used.
>
>    StringReader reader( text );
>
>    CachingTokenFilter tokenStream( analyzer.tokenStream( fieldName, &reader
> ), true );
>
>
>
>    // set bAutoRewriteQueries=true to allow rewriting highlighting queries
> and if you use only standard query objects
>
>   // use the default extractor
>
>    SpanHighlightScorer spanScorer( true );
>
>
>
>    // initialize the query scorer with the ORIGINAL query ? NOT  REWRITTEN
> one. This query
>
>    // should contain the WildCardQuery, not the ConstantScoreQuery!
>
>    spanScorer.init( query, fieldName, &tokenStream );
>
>
>
>    // reset the token stream as it has been used by the scorer and we will
> need it in the highlighter too
>
>    tokenStream.reset();
>
>
>
>    // Using the hilghlighter is the same as with the basis Highlight
> scorer:
>
>    // Instantiate it with formatter and scorer, set fragmenter and call one
> of its methos, ie. getBestFragments
>
>    SimpleFragmenter fragmenter( fragmentSize );
>
>    Highlighter highlighter( &formatter, &spanScorer );
>
>    highlighter.setTextFragmenter( &fragmenter );
>
>    TCHAR * hilited = highlighter.getBestFragments( &tokenStream, text,
> maxFragments, _T( "..." ));
>
>
>
> The query will be rewritten within the call spanScorer.init(?) Typically,
> your documents do not contain as many different words
>
> that would match your wildcard query as the whole index and so the
> probability that your query matches more than MAX CLAUSE
>
> different words is much less.
>
>
>
> Is it possible that your wildcard matches more than MAX CLAUSE different
> word in one document?
>
>
>
> Jiri
>
> Mohammad Ismael

Re: [CLucene-dev] Compile CLucene using mingw under windows

[Ahmed S. <ci7...@gm...>]

I'm using CLucene from git, just forget about qt.
i have comiled clucene (using mingw)  on another computer and it
compile correctly, but on my computer i get this problem.

Ahmed

2011/1/14, Ben van Klinken <bva...@gm...>:
> Perhaps asking Qt might be better? If they have made changes to
> CLucene then they should know what's wrong
>
> Ben
>
> On Wed, Jan 12, 2011 at 7:56 PM, Ahmed Saidi <ci7...@gm...> wrote:
>> Yes, i want to use the unicode version.
>> When installing Qt SDK, it install mingw, any way i have just
>> installed the last mingw version from their website but stell have the
>> same problem.
>>
>> Ahmed
>>
>> 2011/1/12, Ben van Klinken <bva...@gm...>:
>>> ok, good. But i suspect you want to build in unicode mode?
>>>
>>> looks pretty good to me - tchar and _T isn't being touched - so
>>> default tchar.h behaviour should be that TCHAR is defined as wchar_t
>>> and _T as L ## x
>>>
>>> but reading back your mail - what do you mean the 'qt version'? I'm
>>> not aware of what QT did with clucene?
>>>
>>> ben
>>>
>>> On Tue, Jan 11, 2011 at 9:00 PM, Ahmed Saidi <ci7...@gm...> wrote:
>>>> here is _clucene-config.h:
>>>> http://pastebin.com/Yb8mNkiK
>>>> and clucene-config.h:
>>>> http://pastebin.com/YuSnMpm9
>>>>
>>>> i think that ming has a tchar.h, because cmake print this:
>>>> Looking for C++ include tchar.h - found
>>>>
>>>> if i enable acsii mode it compile correctly
>>>>
>>>> 2011/1/10 Ben van Klinken <bva...@gm...>
>>>>>
>>>>> I don't have a windows machine to try this out, so i'm guessing a
>>>>> bit... i did get it working a while back... but you know how these
>>>>> things go
>>>>>
>>>>> looks like there's something wrong with the _T symbol...
>>>>>
>>>>>
>>>>> The line:
>>>>> const TCHAR* _LUCENE_BLANK_STRING=_T("");
>>>>>
>>>>> _T("") should be returning actually L"". There are some things that
>>>>> those clucene-config.h file do if it think it needs to. I'm not sure
>>>>> if ming has tchar.h (which normally defines _T on windows)?
>>>>>
>>>>> can you give me your src/shared/CLucene/_clucene-config.h,
>>>>> clucene-config.h in your build directory. i'm looking for what it does
>>>>> with the _T symbol.. also let me know if there is a tchar.h in the
>>>>> mingw include path
>>>>>
>>>>>
>>>>> Cheers
>>>>> ben
>>>>>
>>>>>
>>>>> On Thu, Jan 6, 2011 at 8:37 PM, Ahmed Saidi <ci7...@gm...> wrote:
>>>>> > Here is what cmake print out:
>>>>> > http://pastebin.com/4KPZeFbB
>>>>> > mingw32-make:
>>>>> > http://pastebin.com/JFX5XzJX
>>>>> >
>>>>> > I am using the mingw version distributed along with qt 4.7
>>>>> >
>>>>> > Ahmed
>>>>> > 2011/1/6 Ben van Klinken <bva...@gm...>
>>>>> >>
>>>>> >> did you configure using cmake? what did cmake print out? turn on
>>>>> >> verbose (run cmake using -DCMAKE_VERBOSE_MAKEFILE=ON), or use the
>>>>> >> cmake gui. so that we can see what parameters are being passed in.
>>>>> >> looks to me like ming is being compiled in ascii mode.
>>>>> >>
>>>>> >> ben
>>>>> >>
>>>>> >> On Tue, Jan 4, 2011 at 7:28 PM, Ahmed nusha <ci7...@gm...>
>>>>> >> wrote:
>>>>> >> > I'm trying to compile clucene under windows and i get the
>>>>> >> > following
>>>>> >> > errors:
>>>>> >> >
>>>>> >> > C:\clucene_mingw-build>mingw32-make
>>>>> >> > [  0%] Building CXX object
>>>>> >> > src/shared/CMakeFiles/clucene-shared.dir/CLucene/SharedHeader.obj
>>>>> >> > C:\clucene\src\shared\CLucene\SharedHeader.cpp:16: error: cannot
>>>>> >> > convert
>>>>> >> > 'const char*' to 'const TCHAR*' in initialization
>>>>> >> > mingw32-make[2]: ***
>>>>> >> > [src/shared/CMakeFiles/clucene-shared.dir/CLucene/SharedHeader.obj]
>>>>> >> > Error 1
>>>>> >> > mingw32-make[1]: ***
>>>>> >> > [src/shared/CMakeFiles/clucene-shared.dir/all]
>>>>> >> > Error 2
>>>>> >> > mingw32-make: *** [all] Error 2
>>>>> >> >
>>>>> >> > what is the problem?
>>>>> >> >
>>>>> >> > Ahmed.
>>>>> >> >
>>>>> >> > Itamar Syn-Hershko <itamar@...> writes:
>>>>> >> >
>>>>> >> >>
>>>>> >> >> It should compile with MingW. What compile errors do you get?
>>>>> >> >>
>>>>> >> >> Itamar.
>>>>> >> >>
>>>>> >> >> On 1/8/2010 10:17 PM, Ahmed wrote:
>>>>> >> >> > hi,
>>>>> >> >> >
>>>>> >> >> > Can compile CLucene 2.3.2 under windows using MingW ?
>>>>> >> >> > I tried that and i get a lot of errors.
>>>>> >> >> >
>>>>> >> >> > Ahmed
>>>>> >> >> >
>>>>> >> >> >
>>>>> >> >> >
>>>>> >> >> >
>>>>> >> >> >
>>>>> >> >> > ------------------------------------------------------------------------------
>>>>> >> >> > The Palm PDK Hot Apps Program offers developers who use the
>>>>> >> >> > Plug-In Development Kit to bring their C/C++ apps to Palm for a
>>>>> >> >> > share
>>>>> >> >> > of $1 Million in cash or HP Products. Visit us here for more
>>>>> >> >> > details:
>>>>> >> >> > http://p.sf.net/sfu/dev2dev-palm
>>>>> >> >> > _______________________________________________
>>>>> >> >> > CLucene-developers mailing list
>>>>> >> >> > CLucene-developers@...
>>>>> >> >> > https://lists.sourceforge.net/lists/listinfo/clucene-developers
>>>>> >> >> >
>>>>> >> >> >
>>>>> >> >> >
>>>>> >> >>
>>>>> >> >>
>>>>> >> >>
>>>>> >> >>
>>>>> >> >> ------------------------------------------------------------------------------
>>>>> >> >> The Palm PDK Hot Apps Program offers developers who use the
>>>>> >> >> Plug-In Development Kit to bring their C/C++ apps to Palm for a
>>>>> >> >> share
>>>>> >> >> of $1 Million in cash or HP Products. Visit us here for more
>>>>> >> >> details:
>>>>> >> >> http://p.sf.net/sfu/dev2dev-palm
>>>>> >> >>
>>>>> >> >
>>>>> >> >
>>>>> >> >
>>>>> >> >
>>>>> >> > ------------------------------------------------------------------------------
>>>>> >> > Learn how Oracle Real Application Clusters (RAC) One Node allows
>>>>> >> > customers
>>>>> >> > to consolidate database storage, standardize their database
>>>>> >> > environment,
>>>>> >> > and,
>>>>> >> > should the need arise, upgrade to a full multi-node Oracle RAC
>>>>> >> > database
>>>>> >> > without downtime or disruption
>>>>> >> > http://p.sf.net/sfu/oracle-sfdevnl
>>>>> >> > _______________________________________________
>>>>> >> > CLucene-developers mailing list
>>>>> >> > CLu...@li...
>>>>> >> > https://lists.sourceforge.net/lists/listinfo/clucene-developers
>>>>> >> >
>>>>> >> >
>>>>> >>
>>>>> >>
>>>>> >>
>>>>> >> ------------------------------------------------------------------------------
>>>>> >> Learn how Oracle Real Application Clusters (RAC) One Node allows
>>>>> >> customers
>>>>> >> to consolidate database storage, standardize their database
>>>>> >> environment,
>>>>> >> and,
>>>>> >> should the need arise, upgrade to a full multi-node Oracle RAC
>>>>> >> database
>>>>> >> without downtime or disruption
>>>>> >> http://p.sf.net/sfu/oracle-sfdevnl
>>>>> >> _______________________________________________
>>>>> >> CLucene-developers mailing list
>>>>> >> CLu...@li...
>>>>> >> https://lists.sourceforge.net/lists/listinfo/clucene-developers
>>>>> >
>>>>> >
>>>>> >
>>>>> > ------------------------------------------------------------------------------
>>>>> > Learn how Oracle Real Application Clusters (RAC) One Node allows
>>>>> > customers
>>>>> > to consolidate database storage, standardize their database
>>>>> > environment,
>>>>> > and,
>>>>> > should the need arise, upgrade to a full multi-node Oracle RAC
>>>>> > database
>>>>> > without downtime or disruption
>>>>> > http://p.sf.net/sfu/oracle-sfdevnl
>>>>> > _______________________________________________
>>>>> > CLucene-developers mailing list
>>>>> > CLu...@li...
>>>>> > https://lists.sourceforge.net/lists/listinfo/clucene-developers
>>>>> >
>>>>> >
>>>>>
>>>>>
>>>>> ------------------------------------------------------------------------------
>>>>> Gaining the trust of online customers is vital for the success of any
>>>>> company
>>>>> that requires sensitive data to be transmitted over the Web.   Learn
>>>>> how
>>>>> to
>>>>> best implement a security strategy that keeps consumers' information
>>>>> secure
>>>>> and instills the confidence they need to proceed with transactions.
>>>>> http://p.sf.net/sfu/oracle-sfdevnl
>>>>> _______________________________________________
>>>>> CLucene-developers mailing list
>>>>> CLu...@li...
>>>>> https://lists.sourceforge.net/lists/listinfo/clucene-developers
>>>>
>>>>
>>>> ------------------------------------------------------------------------------
>>>> Gaining the trust of online customers is vital for the success of any
>>>> company
>>>> that requires sensitive data to be transmitted over the Web.   Learn how
>>>> to
>>>> best implement a security strategy that keeps consumers' information
>>>> secure
>>>> and instills the confidence they need to proceed with transactions.
>>>> http://p.sf.net/sfu/oracle-sfdevnl
>>>> _______________________________________________
>>>> CLucene-developers mailing list
>>>> CLu...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/clucene-developers
>>>>
>>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Protect Your Site and Customers from Malware Attacks
>>> Learn about various malware tactics and how to avoid them. Understand
>>> malware threats, the impact they can have on your business, and how you
>>> can protect your company and customers by using code signing.
>>> http://p.sf.net/sfu/oracle-sfdevnl
>>> _______________________________________________
>>> CLucene-developers mailing list
>>> CLu...@li...
>>> https://lists.sourceforge.net/lists/listinfo/clucene-developers
>>>
>>
>> --
>> Envoyé avec mon mobile
>>
>> ------------------------------------------------------------------------------
>> Protect Your Site and Customers from Malware Attacks
>> Learn about various malware tactics and how to avoid them. Understand
>> malware threats, the impact they can have on your business, and how you
>> can protect your company and customers by using code signing.
>> http://p.sf.net/sfu/oracle-sfdevnl
>> _______________________________________________
>> CLucene-developers mailing list
>> CLu...@li...
>> https://lists.sourceforge.net/lists/listinfo/clucene-developers
>>
>
> ------------------------------------------------------------------------------
> Protect Your Site and Customers from Malware Attacks
> Learn about various malware tactics and how to avoid them. Understand
> malware threats, the impact they can have on your business, and how you
> can protect your company and customers by using code signing.
> http://p.sf.net/sfu/oracle-sfdevnl
> _______________________________________________
> CLucene-developers mailing list
> CLu...@li...
> https://lists.sourceforge.net/lists/listinfo/clucene-developers
>

-- 
Envoyé avec mon mobile

Re: [CLucene-dev] CLucene-developers Digest, Vol 57, Issue 9

[Šplíchal J. <spl...@to...>]

Hello Mohammad,

 

the new queryscorer is able to rewrite the highlighting queries itself. You have to allow it while creating it:

 

    // Create token stream from the text you want to be highlighted. fieldName is name of the field that is being highlighted

    // only the relevant part of the query is used.

    StringReader reader( text );

    CachingTokenFilter tokenStream( analyzer.tokenStream( fieldName, &reader ), true );

    

    // set bAutoRewriteQueries=true to allow rewriting highlighting queries and if you use only standard query objects

   // use the default extractor

    SpanHighlightScorer spanScorer( true );

 

    // initialize the query scorer with the ORIGINAL query – NOT  REWRITTEN one. This query 

    // should contain the WildCardQuery, not the ConstantScoreQuery!

    spanScorer.init( query, fieldName, &tokenStream );

 

    // reset the token stream as it has been used by the scorer and we will need it in the highlighter too

    tokenStream.reset();

 

    // Using the hilghlighter is the same as with the basis Highlight scorer:

    // Instantiate it with formatter and scorer, set fragmenter and call one of its methos, ie. getBestFragments

    SimpleFragmenter fragmenter( fragmentSize );

    Highlighter highlighter( &formatter, &spanScorer ); 

    highlighter.setTextFragmenter( &fragmenter );

    TCHAR * hilited = highlighter.getBestFragments( &tokenStream, text, maxFragments, _T( "..." ));

 

The query will be rewritten within the call spanScorer.init(…) Typically, your documents do not contain as many different words

that would match your wildcard query as the whole index and so the probability that your query matches more than MAX CLAUSE

different words is much less. 

 

Is it possible that your wildcard matches more than MAX CLAUSE different word in one document?

 

Jiri

 

 

From: muhammad ismael [mailto:m.i...@gm...] 
Sent: Monday, January 10, 2011 4:21 PM
To: clu...@li...
Subject: Re: [CLucene-dev] CLucene-developers Digest, Vol 57, Issue 9

 

Hello Jiri,

I am sorry , it seems that i should read more in GIT. I got the SpanHighlightScorer but how can i use it to highlight terms without warring about the maxClauseCount ?

thank you

Mohammad Ismael

Re: [CLucene-dev] Compile CLucene using mingw under windows

[Ben v. K. <bva...@gm...>]

Perhaps asking Qt might be better? If they have made changes to
CLucene then they should know what's wrong

Ben

On Wed, Jan 12, 2011 at 7:56 PM, Ahmed Saidi <ci7...@gm...> wrote:
> Yes, i want to use the unicode version.
> When installing Qt SDK, it install mingw, any way i have just
> installed the last mingw version from their website but stell have the
> same problem.
>
> Ahmed
>
> 2011/1/12, Ben van Klinken <bva...@gm...>:
>> ok, good. But i suspect you want to build in unicode mode?
>>
>> looks pretty good to me - tchar and _T isn't being touched - so
>> default tchar.h behaviour should be that TCHAR is defined as wchar_t
>> and _T as L ## x
>>
>> but reading back your mail - what do you mean the 'qt version'? I'm
>> not aware of what QT did with clucene?
>>
>> ben
>>
>> On Tue, Jan 11, 2011 at 9:00 PM, Ahmed Saidi <ci7...@gm...> wrote:
>>> here is _clucene-config.h:
>>> http://pastebin.com/Yb8mNkiK
>>> and clucene-config.h:
>>> http://pastebin.com/YuSnMpm9
>>>
>>> i think that ming has a tchar.h, because cmake print this:
>>> Looking for C++ include tchar.h - found
>>>
>>> if i enable acsii mode it compile correctly
>>>
>>> 2011/1/10 Ben van Klinken <bva...@gm...>
>>>>
>>>> I don't have a windows machine to try this out, so i'm guessing a
>>>> bit... i did get it working a while back... but you know how these
>>>> things go
>>>>
>>>> looks like there's something wrong with the _T symbol...
>>>>
>>>>
>>>> The line:
>>>> const TCHAR* _LUCENE_BLANK_STRING=_T("");
>>>>
>>>> _T("") should be returning actually L"". There are some things that
>>>> those clucene-config.h file do if it think it needs to. I'm not sure
>>>> if ming has tchar.h (which normally defines _T on windows)?
>>>>
>>>> can you give me your src/shared/CLucene/_clucene-config.h,
>>>> clucene-config.h in your build directory. i'm looking for what it does
>>>> with the _T symbol.. also let me know if there is a tchar.h in the
>>>> mingw include path
>>>>
>>>>
>>>> Cheers
>>>> ben
>>>>
>>>>
>>>> On Thu, Jan 6, 2011 at 8:37 PM, Ahmed Saidi <ci7...@gm...> wrote:
>>>> > Here is what cmake print out:
>>>> > http://pastebin.com/4KPZeFbB
>>>> > mingw32-make:
>>>> > http://pastebin.com/JFX5XzJX
>>>> >
>>>> > I am using the mingw version distributed along with qt 4.7
>>>> >
>>>> > Ahmed
>>>> > 2011/1/6 Ben van Klinken <bva...@gm...>
>>>> >>
>>>> >> did you configure using cmake? what did cmake print out? turn on
>>>> >> verbose (run cmake using -DCMAKE_VERBOSE_MAKEFILE=ON), or use the
>>>> >> cmake gui. so that we can see what parameters are being passed in.
>>>> >> looks to me like ming is being compiled in ascii mode.
>>>> >>
>>>> >> ben
>>>> >>
>>>> >> On Tue, Jan 4, 2011 at 7:28 PM, Ahmed nusha <ci7...@gm...>
>>>> >> wrote:
>>>> >> > I'm trying to compile clucene under windows and i get the following
>>>> >> > errors:
>>>> >> >
>>>> >> > C:\clucene_mingw-build>mingw32-make
>>>> >> > [  0%] Building CXX object
>>>> >> > src/shared/CMakeFiles/clucene-shared.dir/CLucene/SharedHeader.obj
>>>> >> > C:\clucene\src\shared\CLucene\SharedHeader.cpp:16: error: cannot
>>>> >> > convert
>>>> >> > 'const char*' to 'const TCHAR*' in initialization
>>>> >> > mingw32-make[2]: ***
>>>> >> > [src/shared/CMakeFiles/clucene-shared.dir/CLucene/SharedHeader.obj]
>>>> >> > Error 1
>>>> >> > mingw32-make[1]: *** [src/shared/CMakeFiles/clucene-shared.dir/all]
>>>> >> > Error 2
>>>> >> > mingw32-make: *** [all] Error 2
>>>> >> >
>>>> >> > what is the problem?
>>>> >> >
>>>> >> > Ahmed.
>>>> >> >
>>>> >> > Itamar Syn-Hershko <itamar@...> writes:
>>>> >> >
>>>> >> >>
>>>> >> >> It should compile with MingW. What compile errors do you get?
>>>> >> >>
>>>> >> >> Itamar.
>>>> >> >>
>>>> >> >> On 1/8/2010 10:17 PM, Ahmed wrote:
>>>> >> >> > hi,
>>>> >> >> >
>>>> >> >> > Can compile CLucene 2.3.2 under windows using MingW ?
>>>> >> >> > I tried that and i get a lot of errors.
>>>> >> >> >
>>>> >> >> > Ahmed
>>>> >> >> >
>>>> >> >> >
>>>> >> >> >
>>>> >> >> >
>>>> >> >> >
>>>> >> >> > ------------------------------------------------------------------------------
>>>> >> >> > The Palm PDK Hot Apps Program offers developers who use the
>>>> >> >> > Plug-In Development Kit to bring their C/C++ apps to Palm for a
>>>> >> >> > share
>>>> >> >> > of $1 Million in cash or HP Products. Visit us here for more
>>>> >> >> > details:
>>>> >> >> > http://p.sf.net/sfu/dev2dev-palm
>>>> >> >> > _______________________________________________
>>>> >> >> > CLucene-developers mailing list
>>>> >> >> > CLucene-developers@...
>>>> >> >> > https://lists.sourceforge.net/lists/listinfo/clucene-developers
>>>> >> >> >
>>>> >> >> >
>>>> >> >> >
>>>> >> >>
>>>> >> >>
>>>> >> >>
>>>> >> >>
>>>> >> >> ------------------------------------------------------------------------------
>>>> >> >> The Palm PDK Hot Apps Program offers developers who use the
>>>> >> >> Plug-In Development Kit to bring their C/C++ apps to Palm for a
>>>> >> >> share
>>>> >> >> of $1 Million in cash or HP Products. Visit us here for more
>>>> >> >> details:
>>>> >> >> http://p.sf.net/sfu/dev2dev-palm
>>>> >> >>
>>>> >> >
>>>> >> >
>>>> >> >
>>>> >> >
>>>> >> > ------------------------------------------------------------------------------
>>>> >> > Learn how Oracle Real Application Clusters (RAC) One Node allows
>>>> >> > customers
>>>> >> > to consolidate database storage, standardize their database
>>>> >> > environment,
>>>> >> > and,
>>>> >> > should the need arise, upgrade to a full multi-node Oracle RAC
>>>> >> > database
>>>> >> > without downtime or disruption
>>>> >> > http://p.sf.net/sfu/oracle-sfdevnl
>>>> >> > _______________________________________________
>>>> >> > CLucene-developers mailing list
>>>> >> > CLu...@li...
>>>> >> > https://lists.sourceforge.net/lists/listinfo/clucene-developers
>>>> >> >
>>>> >> >
>>>> >>
>>>> >>
>>>> >>
>>>> >> ------------------------------------------------------------------------------
>>>> >> Learn how Oracle Real Application Clusters (RAC) One Node allows
>>>> >> customers
>>>> >> to consolidate database storage, standardize their database
>>>> >> environment,
>>>> >> and,
>>>> >> should the need arise, upgrade to a full multi-node Oracle RAC
>>>> >> database
>>>> >> without downtime or disruption
>>>> >> http://p.sf.net/sfu/oracle-sfdevnl
>>>> >> _______________________________________________
>>>> >> CLucene-developers mailing list
>>>> >> CLu...@li...
>>>> >> https://lists.sourceforge.net/lists/listinfo/clucene-developers
>>>> >
>>>> >
>>>> >
>>>> > ------------------------------------------------------------------------------
>>>> > Learn how Oracle Real Application Clusters (RAC) One Node allows
>>>> > customers
>>>> > to consolidate database storage, standardize their database
>>>> > environment,
>>>> > and,
>>>> > should the need arise, upgrade to a full multi-node Oracle RAC database
>>>> > without downtime or disruption
>>>> > http://p.sf.net/sfu/oracle-sfdevnl
>>>> > _______________________________________________
>>>> > CLucene-developers mailing list
>>>> > CLu...@li...
>>>> > https://lists.sourceforge.net/lists/listinfo/clucene-developers
>>>> >
>>>> >
>>>>
>>>>
>>>> ------------------------------------------------------------------------------
>>>> Gaining the trust of online customers is vital for the success of any
>>>> company
>>>> that requires sensitive data to be transmitted over the Web.   Learn how
>>>> to
>>>> best implement a security strategy that keeps consumers' information
>>>> secure
>>>> and instills the confidence they need to proceed with transactions.
>>>> http://p.sf.net/sfu/oracle-sfdevnl
>>>> _______________________________________________
>>>> CLucene-developers mailing list
>>>> CLu...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/clucene-developers
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Gaining the trust of online customers is vital for the success of any
>>> company
>>> that requires sensitive data to be transmitted over the Web.   Learn how
>>> to
>>> best implement a security strategy that keeps consumers' information
>>> secure
>>> and instills the confidence they need to proceed with transactions.
>>> http://p.sf.net/sfu/oracle-sfdevnl
>>> _______________________________________________
>>> CLucene-developers mailing list
>>> CLu...@li...
>>> https://lists.sourceforge.net/lists/listinfo/clucene-developers
>>>
>>>
>>
>> ------------------------------------------------------------------------------
>> Protect Your Site and Customers from Malware Attacks
>> Learn about various malware tactics and how to avoid them. Understand
>> malware threats, the impact they can have on your business, and how you
>> can protect your company and customers by using code signing.
>> http://p.sf.net/sfu/oracle-sfdevnl
>> _______________________________________________
>> CLucene-developers mailing list
>> CLu...@li...
>> https://lists.sourceforge.net/lists/listinfo/clucene-developers
>>
>
> --
> Envoyé avec mon mobile
>
> ------------------------------------------------------------------------------
> Protect Your Site and Customers from Malware Attacks
> Learn about various malware tactics and how to avoid them. Understand
> malware threats, the impact they can have on your business, and how you
> can protect your company and customers by using code signing.
> http://p.sf.net/sfu/oracle-sfdevnl
> _______________________________________________
> CLucene-developers mailing list
> CLu...@li...
> https://lists.sourceforge.net/lists/listinfo/clucene-developers
>

Re: [CLucene-dev] Compile CLucene using mingw under windows

[Ahmed S. <ci7...@gm...>]

Yes, i want to use the unicode version.
When installing Qt SDK, it install mingw, any way i have just
installed the last mingw version from their website but stell have the
same problem.

Ahmed

2011/1/12, Ben van Klinken <bva...@gm...>:
> ok, good. But i suspect you want to build in unicode mode?
>
> looks pretty good to me - tchar and _T isn't being touched - so
> default tchar.h behaviour should be that TCHAR is defined as wchar_t
> and _T as L ## x
>
> but reading back your mail - what do you mean the 'qt version'? I'm
> not aware of what QT did with clucene?
>
> ben
>
> On Tue, Jan 11, 2011 at 9:00 PM, Ahmed Saidi <ci7...@gm...> wrote:
>> here is _clucene-config.h:
>> http://pastebin.com/Yb8mNkiK
>> and clucene-config.h:
>> http://pastebin.com/YuSnMpm9
>>
>> i think that ming has a tchar.h, because cmake print this:
>> Looking for C++ include tchar.h - found
>>
>> if i enable acsii mode it compile correctly
>>
>> 2011/1/10 Ben van Klinken <bva...@gm...>
>>>
>>> I don't have a windows machine to try this out, so i'm guessing a
>>> bit... i did get it working a while back... but you know how these
>>> things go
>>>
>>> looks like there's something wrong with the _T symbol...
>>>
>>>
>>> The line:
>>> const TCHAR* _LUCENE_BLANK_STRING=_T("");
>>>
>>> _T("") should be returning actually L"". There are some things that
>>> those clucene-config.h file do if it think it needs to. I'm not sure
>>> if ming has tchar.h (which normally defines _T on windows)?
>>>
>>> can you give me your src/shared/CLucene/_clucene-config.h,
>>> clucene-config.h in your build directory. i'm looking for what it does
>>> with the _T symbol.. also let me know if there is a tchar.h in the
>>> mingw include path
>>>
>>>
>>> Cheers
>>> ben
>>>
>>>
>>> On Thu, Jan 6, 2011 at 8:37 PM, Ahmed Saidi <ci7...@gm...> wrote:
>>> > Here is what cmake print out:
>>> > http://pastebin.com/4KPZeFbB
>>> > mingw32-make:
>>> > http://pastebin.com/JFX5XzJX
>>> >
>>> > I am using the mingw version distributed along with qt 4.7
>>> >
>>> > Ahmed
>>> > 2011/1/6 Ben van Klinken <bva...@gm...>
>>> >>
>>> >> did you configure using cmake? what did cmake print out? turn on
>>> >> verbose (run cmake using -DCMAKE_VERBOSE_MAKEFILE=ON), or use the
>>> >> cmake gui. so that we can see what parameters are being passed in.
>>> >> looks to me like ming is being compiled in ascii mode.
>>> >>
>>> >> ben
>>> >>
>>> >> On Tue, Jan 4, 2011 at 7:28 PM, Ahmed nusha <ci7...@gm...>
>>> >> wrote:
>>> >> > I'm trying to compile clucene under windows and i get the following
>>> >> > errors:
>>> >> >
>>> >> > C:\clucene_mingw-build>mingw32-make
>>> >> > [  0%] Building CXX object
>>> >> > src/shared/CMakeFiles/clucene-shared.dir/CLucene/SharedHeader.obj
>>> >> > C:\clucene\src\shared\CLucene\SharedHeader.cpp:16: error: cannot
>>> >> > convert
>>> >> > 'const char*' to 'const TCHAR*' in initialization
>>> >> > mingw32-make[2]: ***
>>> >> > [src/shared/CMakeFiles/clucene-shared.dir/CLucene/SharedHeader.obj]
>>> >> > Error 1
>>> >> > mingw32-make[1]: *** [src/shared/CMakeFiles/clucene-shared.dir/all]
>>> >> > Error 2
>>> >> > mingw32-make: *** [all] Error 2
>>> >> >
>>> >> > what is the problem?
>>> >> >
>>> >> > Ahmed.
>>> >> >
>>> >> > Itamar Syn-Hershko <itamar@...> writes:
>>> >> >
>>> >> >>
>>> >> >> It should compile with MingW. What compile errors do you get?
>>> >> >>
>>> >> >> Itamar.
>>> >> >>
>>> >> >> On 1/8/2010 10:17 PM, Ahmed wrote:
>>> >> >> > hi,
>>> >> >> >
>>> >> >> > Can compile CLucene 2.3.2 under windows using MingW ?
>>> >> >> > I tried that and i get a lot of errors.
>>> >> >> >
>>> >> >> > Ahmed
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> > ------------------------------------------------------------------------------
>>> >> >> > The Palm PDK Hot Apps Program offers developers who use the
>>> >> >> > Plug-In Development Kit to bring their C/C++ apps to Palm for a
>>> >> >> > share
>>> >> >> > of $1 Million in cash or HP Products. Visit us here for more
>>> >> >> > details:
>>> >> >> > http://p.sf.net/sfu/dev2dev-palm
>>> >> >> > _______________________________________________
>>> >> >> > CLucene-developers mailing list
>>> >> >> > CLucene-developers@...
>>> >> >> > https://lists.sourceforge.net/lists/listinfo/clucene-developers
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >>
>>> >> >>
>>> >> >>
>>> >> >>
>>> >> >> ------------------------------------------------------------------------------
>>> >> >> The Palm PDK Hot Apps Program offers developers who use the
>>> >> >> Plug-In Development Kit to bring their C/C++ apps to Palm for a
>>> >> >> share
>>> >> >> of $1 Million in cash or HP Products. Visit us here for more
>>> >> >> details:
>>> >> >> http://p.sf.net/sfu/dev2dev-palm
>>> >> >>
>>> >> >
>>> >> >
>>> >> >
>>> >> >
>>> >> > ------------------------------------------------------------------------------
>>> >> > Learn how Oracle Real Application Clusters (RAC) One Node allows
>>> >> > customers
>>> >> > to consolidate database storage, standardize their database
>>> >> > environment,
>>> >> > and,
>>> >> > should the need arise, upgrade to a full multi-node Oracle RAC
>>> >> > database
>>> >> > without downtime or disruption
>>> >> > http://p.sf.net/sfu/oracle-sfdevnl
>>> >> > _______________________________________________
>>> >> > CLucene-developers mailing list
>>> >> > CLu...@li...
>>> >> > https://lists.sourceforge.net/lists/listinfo/clucene-developers
>>> >> >
>>> >> >
>>> >>
>>> >>
>>> >>
>>> >> ------------------------------------------------------------------------------
>>> >> Learn how Oracle Real Application Clusters (RAC) One Node allows
>>> >> customers
>>> >> to consolidate database storage, standardize their database
>>> >> environment,
>>> >> and,
>>> >> should the need arise, upgrade to a full multi-node Oracle RAC
>>> >> database
>>> >> without downtime or disruption
>>> >> http://p.sf.net/sfu/oracle-sfdevnl
>>> >> _______________________________________________
>>> >> CLucene-developers mailing list
>>> >> CLu...@li...
>>> >> https://lists.sourceforge.net/lists/listinfo/clucene-developers
>>> >
>>> >
>>> >
>>> > ------------------------------------------------------------------------------
>>> > Learn how Oracle Real Application Clusters (RAC) One Node allows
>>> > customers
>>> > to consolidate database storage, standardize their database
>>> > environment,
>>> > and,
>>> > should the need arise, upgrade to a full multi-node Oracle RAC database
>>> > without downtime or disruption
>>> > http://p.sf.net/sfu/oracle-sfdevnl
>>> > _______________________________________________
>>> > CLucene-developers mailing list
>>> > CLu...@li...
>>> > https://lists.sourceforge.net/lists/listinfo/clucene-developers
>>> >
>>> >
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Gaining the trust of online customers is vital for the success of any
>>> company
>>> that requires sensitive data to be transmitted over the Web.   Learn how
>>> to
>>> best implement a security strategy that keeps consumers' information
>>> secure
>>> and instills the confidence they need to proceed with transactions.
>>> http://p.sf.net/sfu/oracle-sfdevnl
>>> _______________________________________________
>>> CLucene-developers mailing list
>>> CLu...@li...
>>> https://lists.sourceforge.net/lists/listinfo/clucene-developers
>>
>>
>> ------------------------------------------------------------------------------
>> Gaining the trust of online customers is vital for the success of any
>> company
>> that requires sensitive data to be transmitted over the Web.   Learn how
>> to
>> best implement a security strategy that keeps consumers' information
>> secure
>> and instills the confidence they need to proceed with transactions.
>> http://p.sf.net/sfu/oracle-sfdevnl
>> _______________________________________________
>> CLucene-developers mailing list
>> CLu...@li...
>> https://lists.sourceforge.net/lists/listinfo/clucene-developers
>>
>>
>
> ------------------------------------------------------------------------------
> Protect Your Site and Customers from Malware Attacks
> Learn about various malware tactics and how to avoid them. Understand
> malware threats, the impact they can have on your business, and how you
> can protect your company and customers by using code signing.
> http://p.sf.net/sfu/oracle-sfdevnl
> _______________________________________________
> CLucene-developers mailing list
> CLu...@li...
> https://lists.sourceforge.net/lists/listinfo/clucene-developers
>

-- 
Envoyé avec mon mobile

Re: [CLucene-dev] Compile CLucene using mingw under windows

[Ben v. K. <bva...@gm...>]

ok, good. But i suspect you want to build in unicode mode?

looks pretty good to me - tchar and _T isn't being touched - so
default tchar.h behaviour should be that TCHAR is defined as wchar_t
and _T as L ## x

but reading back your mail - what do you mean the 'qt version'? I'm
not aware of what QT did with clucene?

ben

On Tue, Jan 11, 2011 at 9:00 PM, Ahmed Saidi <ci7...@gm...> wrote:
> here is _clucene-config.h:
> http://pastebin.com/Yb8mNkiK
> and clucene-config.h:
> http://pastebin.com/YuSnMpm9
>
> i think that ming has a tchar.h, because cmake print this:
> Looking for C++ include tchar.h - found
>
> if i enable acsii mode it compile correctly
>
> 2011/1/10 Ben van Klinken <bva...@gm...>
>>
>> I don't have a windows machine to try this out, so i'm guessing a
>> bit... i did get it working a while back... but you know how these
>> things go
>>
>> looks like there's something wrong with the _T symbol...
>>
>>
>> The line:
>> const TCHAR* _LUCENE_BLANK_STRING=_T("");
>>
>> _T("") should be returning actually L"". There are some things that
>> those clucene-config.h file do if it think it needs to. I'm not sure
>> if ming has tchar.h (which normally defines _T on windows)?
>>
>> can you give me your src/shared/CLucene/_clucene-config.h,
>> clucene-config.h in your build directory. i'm looking for what it does
>> with the _T symbol.. also let me know if there is a tchar.h in the
>> mingw include path
>>
>>
>> Cheers
>> ben
>>
>>
>> On Thu, Jan 6, 2011 at 8:37 PM, Ahmed Saidi <ci7...@gm...> wrote:
>> > Here is what cmake print out:
>> > http://pastebin.com/4KPZeFbB
>> > mingw32-make:
>> > http://pastebin.com/JFX5XzJX
>> >
>> > I am using the mingw version distributed along with qt 4.7
>> >
>> > Ahmed
>> > 2011/1/6 Ben van Klinken <bva...@gm...>
>> >>
>> >> did you configure using cmake? what did cmake print out? turn on
>> >> verbose (run cmake using -DCMAKE_VERBOSE_MAKEFILE=ON), or use the
>> >> cmake gui. so that we can see what parameters are being passed in.
>> >> looks to me like ming is being compiled in ascii mode.
>> >>
>> >> ben
>> >>
>> >> On Tue, Jan 4, 2011 at 7:28 PM, Ahmed nusha <ci7...@gm...> wrote:
>> >> > I'm trying to compile clucene under windows and i get the following
>> >> > errors:
>> >> >
>> >> > C:\clucene_mingw-build>mingw32-make
>> >> > [  0%] Building CXX object
>> >> > src/shared/CMakeFiles/clucene-shared.dir/CLucene/SharedHeader.obj
>> >> > C:\clucene\src\shared\CLucene\SharedHeader.cpp:16: error: cannot
>> >> > convert
>> >> > 'const char*' to 'const TCHAR*' in initialization
>> >> > mingw32-make[2]: ***
>> >> > [src/shared/CMakeFiles/clucene-shared.dir/CLucene/SharedHeader.obj]
>> >> > Error 1
>> >> > mingw32-make[1]: *** [src/shared/CMakeFiles/clucene-shared.dir/all]
>> >> > Error 2
>> >> > mingw32-make: *** [all] Error 2
>> >> >
>> >> > what is the problem?
>> >> >
>> >> > Ahmed.
>> >> >
>> >> > Itamar Syn-Hershko <itamar@...> writes:
>> >> >
>> >> >>
>> >> >> It should compile with MingW. What compile errors do you get?
>> >> >>
>> >> >> Itamar.
>> >> >>
>> >> >> On 1/8/2010 10:17 PM, Ahmed wrote:
>> >> >> > hi,
>> >> >> >
>> >> >> > Can compile CLucene 2.3.2 under windows using MingW ?
>> >> >> > I tried that and i get a lot of errors.
>> >> >> >
>> >> >> > Ahmed
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > ------------------------------------------------------------------------------
>> >> >> > The Palm PDK Hot Apps Program offers developers who use the
>> >> >> > Plug-In Development Kit to bring their C/C++ apps to Palm for a
>> >> >> > share
>> >> >> > of $1 Million in cash or HP Products. Visit us here for more
>> >> >> > details:
>> >> >> > http://p.sf.net/sfu/dev2dev-palm
>> >> >> > _______________________________________________
>> >> >> > CLucene-developers mailing list
>> >> >> > CLucene-developers@...
>> >> >> > https://lists.sourceforge.net/lists/listinfo/clucene-developers
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >>
>> >> >>
>> >> >>
>> >> >>
>> >> >> ------------------------------------------------------------------------------
>> >> >> The Palm PDK Hot Apps Program offers developers who use the
>> >> >> Plug-In Development Kit to bring their C/C++ apps to Palm for a
>> >> >> share
>> >> >> of $1 Million in cash or HP Products. Visit us here for more
>> >> >> details:
>> >> >> http://p.sf.net/sfu/dev2dev-palm
>> >> >>
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > ------------------------------------------------------------------------------
>> >> > Learn how Oracle Real Application Clusters (RAC) One Node allows
>> >> > customers
>> >> > to consolidate database storage, standardize their database
>> >> > environment,
>> >> > and,
>> >> > should the need arise, upgrade to a full multi-node Oracle RAC
>> >> > database
>> >> > without downtime or disruption
>> >> > http://p.sf.net/sfu/oracle-sfdevnl
>> >> > _______________________________________________
>> >> > CLucene-developers mailing list
>> >> > CLu...@li...
>> >> > https://lists.sourceforge.net/lists/listinfo/clucene-developers
>> >> >
>> >> >
>> >>
>> >>
>> >>
>> >> ------------------------------------------------------------------------------
>> >> Learn how Oracle Real Application Clusters (RAC) One Node allows
>> >> customers
>> >> to consolidate database storage, standardize their database
>> >> environment,
>> >> and,
>> >> should the need arise, upgrade to a full multi-node Oracle RAC database
>> >> without downtime or disruption
>> >> http://p.sf.net/sfu/oracle-sfdevnl
>> >> _______________________________________________
>> >> CLucene-developers mailing list
>> >> CLu...@li...
>> >> https://lists.sourceforge.net/lists/listinfo/clucene-developers
>> >
>> >
>> >
>> > ------------------------------------------------------------------------------
>> > Learn how Oracle Real Application Clusters (RAC) One Node allows
>> > customers
>> > to consolidate database storage, standardize their database environment,
>> > and,
>> > should the need arise, upgrade to a full multi-node Oracle RAC database
>> > without downtime or disruption
>> > http://p.sf.net/sfu/oracle-sfdevnl
>> > _______________________________________________
>> > CLucene-developers mailing list
>> > CLu...@li...
>> > https://lists.sourceforge.net/lists/listinfo/clucene-developers
>> >
>> >
>>
>>
>> ------------------------------------------------------------------------------
>> Gaining the trust of online customers is vital for the success of any
>> company
>> that requires sensitive data to be transmitted over the Web.   Learn how
>> to
>> best implement a security strategy that keeps consumers' information
>> secure
>> and instills the confidence they need to proceed with transactions.
>> http://p.sf.net/sfu/oracle-sfdevnl
>> _______________________________________________
>> CLucene-developers mailing list
>> CLu...@li...
>> https://lists.sourceforge.net/lists/listinfo/clucene-developers
>
>
> ------------------------------------------------------------------------------
> Gaining the trust of online customers is vital for the success of any
> company
> that requires sensitive data to be transmitted over the Web.   Learn how to
> best implement a security strategy that keeps consumers' information secure
> and instills the confidence they need to proceed with transactions.
> http://p.sf.net/sfu/oracle-sfdevnl
> _______________________________________________
> CLucene-developers mailing list
> CLu...@li...
> https://lists.sourceforge.net/lists/listinfo/clucene-developers
>
>

Re: [CLucene-dev] Compile CLucene using mingw under windows

[Ahmed S. <ci7...@gm...>]

here is _clucene-config.h:
http://pastebin.com/Yb8mNkiK
and clucene-config.h:
http://pastebin.com/YuSnMpm9

i think that ming has a tchar.h, because cmake print this:
Looking for C++ include tchar.h - found

if i enable acsii mode it compile correctly

2011/1/10 Ben van Klinken <bva...@gm...>

> I don't have a windows machine to try this out, so i'm guessing a
> bit... i did get it working a while back... but you know how these
> things go
>
> looks like there's something wrong with the _T symbol...
>
>
> The line:
> const TCHAR* _LUCENE_BLANK_STRING=_T("");
>
> _T("") should be returning actually L"". There are some things that
> those clucene-config.h file do if it think it needs to. I'm not sure
> if ming has tchar.h (which normally defines _T on windows)?
>
> can you give me your src/shared/CLucene/_clucene-config.h,
> clucene-config.h in your build directory. i'm looking for what it does
> with the _T symbol.. also let me know if there is a tchar.h in the
> mingw include path
>
>
> Cheers
> ben
>
>
> On Thu, Jan 6, 2011 at 8:37 PM, Ahmed Saidi <ci7...@gm...> wrote:
> > Here is what cmake print out:
> > http://pastebin.com/4KPZeFbB
> > mingw32-make:
> > http://pastebin.com/JFX5XzJX
> >
> > I am using the mingw version distributed along with qt 4.7
> >
> > Ahmed
> > 2011/1/6 Ben van Klinken <bva...@gm...>
> >>
> >> did you configure using cmake? what did cmake print out? turn on
> >> verbose (run cmake using -DCMAKE_VERBOSE_MAKEFILE=ON), or use the
> >> cmake gui. so that we can see what parameters are being passed in.
> >> looks to me like ming is being compiled in ascii mode.
> >>
> >> ben
> >>
> >> On Tue, Jan 4, 2011 at 7:28 PM, Ahmed nusha <ci7...@gm...> wrote:
> >> > I'm trying to compile clucene under windows and i get the following
> >> > errors:
> >> >
> >> > C:\clucene_mingw-build>mingw32-make
> >> > [  0%] Building CXX object
> >> > src/shared/CMakeFiles/clucene-shared.dir/CLucene/SharedHeader.obj
> >> > C:\clucene\src\shared\CLucene\SharedHeader.cpp:16: error: cannot
> convert
> >> > 'const char*' to 'const TCHAR*' in initialization
> >> > mingw32-make[2]: ***
> >> > [src/shared/CMakeFiles/clucene-shared.dir/CLucene/SharedHeader.obj]
> >> > Error 1
> >> > mingw32-make[1]: *** [src/shared/CMakeFiles/clucene-shared.dir/all]
> >> > Error 2
> >> > mingw32-make: *** [all] Error 2
> >> >
> >> > what is the problem?
> >> >
> >> > Ahmed.
> >> >
> >> > Itamar Syn-Hershko <itamar@...> writes:
> >> >
> >> >>
> >> >> It should compile with MingW. What compile errors do you get?
> >> >>
> >> >> Itamar.
> >> >>
> >> >> On 1/8/2010 10:17 PM, Ahmed wrote:
> >> >> > hi,
> >> >> >
> >> >> > Can compile CLucene 2.3.2 under windows using MingW ?
> >> >> > I tried that and i get a lot of errors.
> >> >> >
> >> >> > Ahmed
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> ------------------------------------------------------------------------------
> >> >> > The Palm PDK Hot Apps Program offers developers who use the
> >> >> > Plug-In Development Kit to bring their C/C++ apps to Palm for a
> share
> >> >> > of $1 Million in cash or HP Products. Visit us here for more
> details:
> >> >> > http://p.sf.net/sfu/dev2dev-palm
> >> >> > _______________________________________________
> >> >> > CLucene-developers mailing list
> >> >> > CLucene-developers@...
> >> >> > https://lists.sourceforge.net/lists/listinfo/clucene-developers
> >> >> >
> >> >> >
> >> >> >
> >> >>
> >> >>
> >> >>
> >> >>
> ------------------------------------------------------------------------------
> >> >> The Palm PDK Hot Apps Program offers developers who use the
> >> >> Plug-In Development Kit to bring their C/C++ apps to Palm for a share
> >> >> of $1 Million in cash or HP Products. Visit us here for more details:
> >> >> http://p.sf.net/sfu/dev2dev-palm
> >> >>
> >> >
> >> >
> >> >
> >> >
> ------------------------------------------------------------------------------
> >> > Learn how Oracle Real Application Clusters (RAC) One Node allows
> >> > customers
> >> > to consolidate database storage, standardize their database
> environment,
> >> > and,
> >> > should the need arise, upgrade to a full multi-node Oracle RAC
> database
> >> > without downtime or disruption
> >> > http://p.sf.net/sfu/oracle-sfdevnl
> >> > _______________________________________________
> >> > CLucene-developers mailing list
> >> > CLu...@li...
> >> > https://lists.sourceforge.net/lists/listinfo/clucene-developers
> >> >
> >> >
> >>
> >>
> >>
> ------------------------------------------------------------------------------
> >> Learn how Oracle Real Application Clusters (RAC) One Node allows
> customers
> >> to consolidate database storage, standardize their database environment,
> >> and,
> >> should the need arise, upgrade to a full multi-node Oracle RAC database
> >> without downtime or disruption
> >> http://p.sf.net/sfu/oracle-sfdevnl
> >> _______________________________________________
> >> CLucene-developers mailing list
> >> CLu...@li...
> >> https://lists.sourceforge.net/lists/listinfo/clucene-developers
> >
> >
> >
> ------------------------------------------------------------------------------
> > Learn how Oracle Real Application Clusters (RAC) One Node allows
> customers
> > to consolidate database storage, standardize their database environment,
> > and,
> > should the need arise, upgrade to a full multi-node Oracle RAC database
> > without downtime or disruption
> > http://p.sf.net/sfu/oracle-sfdevnl
> > _______________________________________________
> > CLucene-developers mailing list
> > CLu...@li...
> > https://lists.sourceforge.net/lists/listinfo/clucene-developers
> >
> >
>
>
> ------------------------------------------------------------------------------
> Gaining the trust of online customers is vital for the success of any
> company
> that requires sensitive data to be transmitted over the Web.   Learn how to
> best implement a security strategy that keeps consumers' information secure
> and instills the confidence they need to proceed with transactions.
> http://p.sf.net/sfu/oracle-sfdevnl
> _______________________________________________
> CLucene-developers mailing list
> CLu...@li...
> https://lists.sourceforge.net/lists/listinfo/clucene-developers
>

Re: [CLucene-dev] CLucene-developers Digest, Vol 57, Issue 9

[muhammad i. <m.i...@gm...>]

Hello Jiri,

I am sorry , it seems that i should read more in GIT. I got the
SpanHighlightScorer but how can i use it to highlight terms without warring
about the maxClauseCount ?

thank you

Mohammad Ismael

Re: [CLucene-dev] CLucene-developers Digest, Vol 57, Issue 9

[muhammad i. <m.i...@gm...>]

Hello Jiri,


> for highlighting the important question is how big is ONE document and not
> the size of the whole data.
>
> (At least I assume the 6GB is not the size of one document).
>
> the document size does not exceed 100000 character .

>
>
> The class SpanHighlightScorer is in the contribs-lib package at the same
> place as all other highlighter
>
> classes. But you cannot highlight a document based on a filter so to
> highlight it you have to
>
> have the original WildcardQuery. This query would rewrite only against the
> one document that is being
>
> highlighted and this probably will not generate so much clauses.
>

Excuse me , I can not find any file called SpanHighlightScorer in the
highlighter branch or in the master branch.

Mohammad Ismael

Re: [CLucene-dev] Compile CLucene using mingw under windows

[Ben v. K. <bva...@gm...>]

I don't have a windows machine to try this out, so i'm guessing a
bit... i did get it working a while back... but you know how these
things go

looks like there's something wrong with the _T symbol...


The line:
const TCHAR* _LUCENE_BLANK_STRING=_T("");

_T("") should be returning actually L"". There are some things that
those clucene-config.h file do if it think it needs to. I'm not sure
if ming has tchar.h (which normally defines _T on windows)?

can you give me your src/shared/CLucene/_clucene-config.h,
clucene-config.h in your build directory. i'm looking for what it does
with the _T symbol.. also let me know if there is a tchar.h in the
mingw include path


Cheers
ben


On Thu, Jan 6, 2011 at 8:37 PM, Ahmed Saidi <ci7...@gm...> wrote:
> Here is what cmake print out:
> http://pastebin.com/4KPZeFbB
> mingw32-make:
> http://pastebin.com/JFX5XzJX
>
> I am using the mingw version distributed along with qt 4.7
>
> Ahmed
> 2011/1/6 Ben van Klinken <bva...@gm...>
>>
>> did you configure using cmake? what did cmake print out? turn on
>> verbose (run cmake using -DCMAKE_VERBOSE_MAKEFILE=ON), or use the
>> cmake gui. so that we can see what parameters are being passed in.
>> looks to me like ming is being compiled in ascii mode.
>>
>> ben
>>
>> On Tue, Jan 4, 2011 at 7:28 PM, Ahmed nusha <ci7...@gm...> wrote:
>> > I'm trying to compile clucene under windows and i get the following
>> > errors:
>> >
>> > C:\clucene_mingw-build>mingw32-make
>> > [  0%] Building CXX object
>> > src/shared/CMakeFiles/clucene-shared.dir/CLucene/SharedHeader.obj
>> > C:\clucene\src\shared\CLucene\SharedHeader.cpp:16: error: cannot convert
>> > 'const char*' to 'const TCHAR*' in initialization
>> > mingw32-make[2]: ***
>> > [src/shared/CMakeFiles/clucene-shared.dir/CLucene/SharedHeader.obj]
>> > Error 1
>> > mingw32-make[1]: *** [src/shared/CMakeFiles/clucene-shared.dir/all]
>> > Error 2
>> > mingw32-make: *** [all] Error 2
>> >
>> > what is the problem?
>> >
>> > Ahmed.
>> >
>> > Itamar Syn-Hershko <itamar@...> writes:
>> >
>> >>
>> >> It should compile with MingW. What compile errors do you get?
>> >>
>> >> Itamar.
>> >>
>> >> On 1/8/2010 10:17 PM, Ahmed wrote:
>> >> > hi,
>> >> >
>> >> > Can compile CLucene 2.3.2 under windows using MingW ?
>> >> > I tried that and i get a lot of errors.
>> >> >
>> >> > Ahmed
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > ------------------------------------------------------------------------------
>> >> > The Palm PDK Hot Apps Program offers developers who use the
>> >> > Plug-In Development Kit to bring their C/C++ apps to Palm for a share
>> >> > of $1 Million in cash or HP Products. Visit us here for more details:
>> >> > http://p.sf.net/sfu/dev2dev-palm
>> >> > _______________________________________________
>> >> > CLucene-developers mailing list
>> >> > CLucene-developers@...
>> >> > https://lists.sourceforge.net/lists/listinfo/clucene-developers
>> >> >
>> >> >
>> >> >
>> >>
>> >>
>> >>
>> >> ------------------------------------------------------------------------------
>> >> The Palm PDK Hot Apps Program offers developers who use the
>> >> Plug-In Development Kit to bring their C/C++ apps to Palm for a share
>> >> of $1 Million in cash or HP Products. Visit us here for more details:
>> >> http://p.sf.net/sfu/dev2dev-palm
>> >>
>> >
>> >
>> >
>> > ------------------------------------------------------------------------------
>> > Learn how Oracle Real Application Clusters (RAC) One Node allows
>> > customers
>> > to consolidate database storage, standardize their database environment,
>> > and,
>> > should the need arise, upgrade to a full multi-node Oracle RAC database
>> > without downtime or disruption
>> > http://p.sf.net/sfu/oracle-sfdevnl
>> > _______________________________________________
>> > CLucene-developers mailing list
>> > CLu...@li...
>> > https://lists.sourceforge.net/lists/listinfo/clucene-developers
>> >
>> >
>>
>>
>> ------------------------------------------------------------------------------
>> Learn how Oracle Real Application Clusters (RAC) One Node allows customers
>> to consolidate database storage, standardize their database environment,
>> and,
>> should the need arise, upgrade to a full multi-node Oracle RAC database
>> without downtime or disruption
>> http://p.sf.net/sfu/oracle-sfdevnl
>> _______________________________________________
>> CLucene-developers mailing list
>> CLu...@li...
>> https://lists.sourceforge.net/lists/listinfo/clucene-developers
>
>
> ------------------------------------------------------------------------------
> Learn how Oracle Real Application Clusters (RAC) One Node allows customers
> to consolidate database storage, standardize their database environment,
> and,
> should the need arise, upgrade to a full multi-node Oracle RAC database
> without downtime or disruption
> http://p.sf.net/sfu/oracle-sfdevnl
> _______________________________________________
> CLucene-developers mailing list
> CLu...@li...
> https://lists.sourceforge.net/lists/listinfo/clucene-developers
>
>

Re: [CLucene-dev] CLucene-developers Digest, Vol 57, Issue 5

[Ahmed S. <ci7...@gm...>]

it's a macro:

#define QSTRING_TO_TCHAR(s) (const wchar_t*) s.utf16()

it's up to qt to make a clean up(maybe)

2011/1/9 Ben van Klinken <bva...@gm...>

> oh it's a qt wrapper? i have no idea how that works, sorry.
>
> just an idea, but where do you clean up the result from QSTRING_TO_TCHAR?
>
> ben
>
> On Fri, Jan 7, 2011 at 5:57 PM, Ahmed Saidi <ci7...@gm...> wrote:
> > Hello Muhammad,
> > What analyzer are you using?
> > It seem to me that your code is based on that application
> > "clucene_qt", well i think that you are an old version of that
> > application, it has several bugs concerning this problem, and the
> > ArabicAnalyzer is not well implemented nor optimized, you should use
> > the 0.6 or 0.8 version of this application.
> >
> > Ahmed
> >
> > 2011/1/6, muhammad ismael <m.i...@gm...>:
> >> Hello ben,
> >> the files are more than 6 GB.
> >> I am sorry i was remember to add this function but i forgot and here it
> is
> >>
> >> Document* IndexEngine::fileDocument(const QString &id, const QString
> &bookid
> >> , const QString &text)
> >>
> >> {
> >>
> >>     // make a new, empty document
> >>
> >>     Document* doc = _CLNEW Document();
> >>
> >>     ///page ID
> >>
> >>     doc->add( *_CLNEW Field(_T("id"), QSTRING_TO_TCHAR(id) ,
> >>
> >>                             Field::STORE_YES | Field::INDEX_UNTOKENIZED)
> );
> >>
> >>     doc->add( *_CLNEW Field(_T("bookid"), QSTRING_TO_TCHAR(bookid) ,
> >>
> >>                             Field::STORE_YES | Field::INDEX_UNTOKENIZED
> ) );
> >>
> >>     doc->add( *_CLNEW Field(_T("text"), QSTRING_TO_TCHAR(text),
> >>
> >>                             Field::STORE_NO | Field::INDEX_TOKENIZED) );
> >>
> >>     return doc;
> >>
> >> }
> >>
> >> and also i tried to remove adding documents and the size did not
> increased,
> >> which means that leaks are in addDocument() I am trying to debug it but
> i am
> >> lost.
> >>
> >> sounds pretty high. how big are the files? could you be leaking memory
> in
> >>> the 'fileDocument' function?
> >>>
> >>> as a test, try not actually adding the document
> >>>
> >>> ben
> >>>
> >>> On Thu, Jan 6, 2011 at 7:43 AM, muhammad ismael <m.i...@gm...
> >>> >wrote:
> >>>
> >>> > Hello,
> >>> > I am trying to index large files as follows
> >>> >
> >>> > for(int j = 0; (j < pagesIds.count())&& !m_stop ; j++)
> >>> >
> >>> >                 {
> >>> >
> >>> >                     pagesText =
> >>> m_DbManager->getBookPage(m_booksIds.at(i), pagesIds.at(j)).toUtf8();
> >>> >
> >>> >                     if(!pagesText.isEmpty())
> >>> >
> >>> >                     {
> >>> >
> >>> >                         Document* doc =
> >>> fileDocument(QString::number(pagesIds.at(j)),
> >>> QString::number(m_booksIds.at(i)), pagesText);
> >>> >
> >>> >                         writer->addDocument(doc);
> >>> >
> >>> >                         _CLDELETE(doc);
> >>> >
> >>> >                     }
> >>> >
> >>> >                 }
> >>> >
> >>> >
> >>> > but when the number of files exceeds 5000 files the application usage
> of
> >>> my
> >>> > computer ram is 2 GB
> >>> > I tried to debug and i found that this happens in
> >>> >
> >>> > IndexWriter::addDocument(Document*)
> >>> >
> >>> > i tried to set
> >>> > IndexWriter->setMergeFactor(5);
> >>> > and also
> >>> > IndexWriter->setRAMBufferSizeMB(10);
> >>> > I know the default ram usage should be 16 MB but i just tried.
> >>> >
> >>> > I am working on master branch and i merged with it memory_leaks
> branch.
> >>> > is am i missing something?
> >>>
> >>
> >> Mohammad Ismael
> >>
> >
> > --
> > Envoyé avec mon mobile
> >
> >
> ------------------------------------------------------------------------------
> > Gaining the trust of online customers is vital for the success of any
> company
> > that requires sensitive data to be transmitted over the Web.   Learn how
> to
> > best implement a security strategy that keeps consumers' information
> secure
> > and instills the confidence they need to proceed with transactions.
> > http://p.sf.net/sfu/oracle-sfdevnl
> > _______________________________________________
> > CLucene-developers mailing list
> > CLu...@li...
> > https://lists.sourceforge.net/lists/listinfo/clucene-developers
> >
>
>
> ------------------------------------------------------------------------------
> Gaining the trust of online customers is vital for the success of any
> company
> that requires sensitive data to be transmitted over the Web.   Learn how to
> best implement a security strategy that keeps consumers' information secure
> and instills the confidence they need to proceed with transactions.
> http://p.sf.net/sfu/oracle-sfdevnl
> _______________________________________________
> CLucene-developers mailing list
> CLu...@li...
> https://lists.sourceforge.net/lists/listinfo/clucene-developers
>

Re: [CLucene-dev] contrib: SpellChecker

[Freiholz M. <M.F...@ca...>]

Hi Veit,

here are my changes, due to the ANSI compiler errors. The code no longer uses std::wstring now.

I attached you a .zip file which contains three different methods to apply the changes.

- The files..
- A patch file
- A commit file

I'm still don't know why the suggestSimilar-function doesn't return the TCHAR** correct, when the IDE is not attached. But i'm looking forward to find the problem :)

Greetings
Manuel

________________________________________
From: Freiholz Manuel [M.F...@ca...]
Sent: Friday, January 07, 2011 11:38 PM
To: clu...@li...
Subject: Re: [CLucene-dev] contrib: SpellChecker

Hi Veit

I removed the usage of wstring and wstreams. I also will have a look on suggestSimilar function. The error only occurs in release build for me.

I will try to upload the patch tomorrow.

Greetings
Manuel

Sent from Samsung Galaxy Android TabletVeit Jahns <nun...@go...> hat geschrieben:

2011/1/5 Freiholz Manuel <M.F...@ca...>:
> Hi,
>
> ok i wil take a look at it.

Thanks! I changed your files a little bit, so that it compiles on
Ubuntu. But the test spellcheckersearch fails, because the  the method
suggestSimilar() returns no result (NULL pointer). Any idea?

Veit

------------------------------------------------------------------------------
Gaining the trust of online customers is vital for the success of any company
that requires sensitive data to be transmitted over the Web.   Learn how to
best implement a security strategy that keeps consumers' information secure
and instills the confidence they need to proceed with transactions.
http://p.sf.net/sfu/oracle-sfdevnl
_______________________________________________
CLucene-developers mailing list
CLu...@li...
https://lists.sourceforge.net/lists/listinfo/clucene-developers

------------------------------------------------------------------------------
Gaining the trust of online customers is vital for the success of any company
that requires sensitive data to be transmitted over the Web.   Learn how to
best implement a security strategy that keeps consumers' information secure
and instills the confidence they need to proceed with transactions.
http://p.sf.net/sfu/oracle-sfdevnl
_______________________________________________
CLucene-developers mailing list
CLu...@li...
https://lists.sourceforge.net/lists/listinfo/clucene-developers

Re: [CLucene-dev] CLucene-developers Digest, Vol 57, Issue 5

[Ben v. K. <bva...@gm...>]

oh it's a qt wrapper? i have no idea how that works, sorry.

just an idea, but where do you clean up the result from QSTRING_TO_TCHAR?

ben

On Fri, Jan 7, 2011 at 5:57 PM, Ahmed Saidi <ci7...@gm...> wrote:
> Hello Muhammad,
> What analyzer are you using?
> It seem to me that your code is based on that application
> "clucene_qt", well i think that you are an old version of that
> application, it has several bugs concerning this problem, and the
> ArabicAnalyzer is not well implemented nor optimized, you should use
> the 0.6 or 0.8 version of this application.
>
> Ahmed
>
> 2011/1/6, muhammad ismael <m.i...@gm...>:
>> Hello ben,
>> the files are more than 6 GB.
>> I am sorry i was remember to add this function but i forgot and here it is
>>
>> Document* IndexEngine::fileDocument(const QString &id, const QString &bookid
>> , const QString &text)
>>
>> {
>>
>>     // make a new, empty document
>>
>>     Document* doc = _CLNEW Document();
>>
>>     ///page ID
>>
>>     doc->add( *_CLNEW Field(_T("id"), QSTRING_TO_TCHAR(id) ,
>>
>>                             Field::STORE_YES | Field::INDEX_UNTOKENIZED) );
>>
>>     doc->add( *_CLNEW Field(_T("bookid"), QSTRING_TO_TCHAR(bookid) ,
>>
>>                             Field::STORE_YES | Field::INDEX_UNTOKENIZED ) );
>>
>>     doc->add( *_CLNEW Field(_T("text"), QSTRING_TO_TCHAR(text),
>>
>>                             Field::STORE_NO | Field::INDEX_TOKENIZED) );
>>
>>     return doc;
>>
>> }
>>
>> and also i tried to remove adding documents and the size did not increased,
>> which means that leaks are in addDocument() I am trying to debug it but i am
>> lost.
>>
>> sounds pretty high. how big are the files? could you be leaking memory in
>>> the 'fileDocument' function?
>>>
>>> as a test, try not actually adding the document
>>>
>>> ben
>>>
>>> On Thu, Jan 6, 2011 at 7:43 AM, muhammad ismael <m.i...@gm...
>>> >wrote:
>>>
>>> > Hello,
>>> > I am trying to index large files as follows
>>> >
>>> > for(int j = 0; (j < pagesIds.count())&& !m_stop ; j++)
>>> >
>>> >                 {
>>> >
>>> >                     pagesText =
>>> m_DbManager->getBookPage(m_booksIds.at(i), pagesIds.at(j)).toUtf8();
>>> >
>>> >                     if(!pagesText.isEmpty())
>>> >
>>> >                     {
>>> >
>>> >                         Document* doc =
>>> fileDocument(QString::number(pagesIds.at(j)),
>>> QString::number(m_booksIds.at(i)), pagesText);
>>> >
>>> >                         writer->addDocument(doc);
>>> >
>>> >                         _CLDELETE(doc);
>>> >
>>> >                     }
>>> >
>>> >                 }
>>> >
>>> >
>>> > but when the number of files exceeds 5000 files the application usage of
>>> my
>>> > computer ram is 2 GB
>>> > I tried to debug and i found that this happens in
>>> >
>>> > IndexWriter::addDocument(Document*)
>>> >
>>> > i tried to set
>>> > IndexWriter->setMergeFactor(5);
>>> > and also
>>> > IndexWriter->setRAMBufferSizeMB(10);
>>> > I know the default ram usage should be 16 MB but i just tried.
>>> >
>>> > I am working on master branch and i merged with it memory_leaks branch.
>>> > is am i missing something?
>>>
>>
>> Mohammad Ismael
>>
>
> --
> Envoyé avec mon mobile
>
> ------------------------------------------------------------------------------
> Gaining the trust of online customers is vital for the success of any company
> that requires sensitive data to be transmitted over the Web.   Learn how to
> best implement a security strategy that keeps consumers' information secure
> and instills the confidence they need to proceed with transactions.
> http://p.sf.net/sfu/oracle-sfdevnl
> _______________________________________________
> CLucene-developers mailing list
> CLu...@li...
> https://lists.sourceforge.net/lists/listinfo/clucene-developers
>

Re: [CLucene-dev] CLucene-developers Digest, Vol 57, Issue 6

[Šplíchal J. <spl...@to...>]

Hello Mohammad,

 

for highlighting the important question is how big is ONE document and not the size of the whole data. 

(At least I assume the 6GB is not the size of one document).

 

The class SpanHighlightScorer is in the contribs-lib package at the same place as all other highlighter

classes. But you cannot highlight a document based on a filter so to highlight it you have to 

have the original WildcardQuery. This query would rewrite only against the one document that is being 

highlighted and this probably will not generate so much clauses.

 

Jiri

 

 

 

From: muhammad ismael [mailto:m.i...@gm...] 
Sent: Saturday, January 08, 2011 4:31 PM
To: clu...@li...
Subject: Re: [CLucene-dev] CLucene-developers Digest, Vol 57, Issue 6

 

Hello Jiri

I have documents more than 6 GB. 
I checked-out the highlighter branch and i compiled it. but where is the SpanHighlightScorer ?

after sub-classing QueryParser and using wildcardFilter I can get results now but i can not highlight it.
How can i highlight these results?

Mohammad Ismael

Re: [CLucene-dev] CLucene-developers Digest, Vol 57, Issue 6

[muhammad i. <m.i...@gm...>]

 Hello Jiri

I have documents more than 6 GB.
I checked-out the highlighter branch and i compiled it. but where is the
SpanHighlightScorer ?

after sub-classing QueryParser and using wildcardFilter I can get results
now but i can not highlight it.
How can i highlight these results?

Mohammad Ismael

Re: [CLucene-dev] contrib: SpellChecker

[Freiholz M. <M.F...@ca...>]

Hi Veit

I removed the usage of wstring and wstreams. I also will have a look on suggestSimilar function. The error only occurs in release build for me.

I will try to upload the patch tomorrow.

Greetings
Manuel

Sent from Samsung Galaxy Android TabletVeit Jahns <nun...@go...> hat geschrieben:

2011/1/5 Freiholz Manuel <M.F...@ca...>:
> Hi,
>
> ok i wil take a look at it.

Thanks! I changed your files a little bit, so that it compiles on
Ubuntu. But the test spellcheckersearch fails, because the  the method
suggestSimilar() returns no result (NULL pointer). Any idea?

Veit

------------------------------------------------------------------------------
Gaining the trust of online customers is vital for the success of any company
that requires sensitive data to be transmitted over the Web.   Learn how to
best implement a security strategy that keeps consumers' information secure
and instills the confidence they need to proceed with transactions.
http://p.sf.net/sfu/oracle-sfdevnl
_______________________________________________
CLucene-developers mailing list
CLu...@li...
https://lists.sourceforge.net/lists/listinfo/clucene-developers

Re: [CLucene-dev] contrib: SpellChecker

[Veit J. <nun...@go...>]

2011/1/5 Freiholz Manuel <M.F...@ca...>:
> Hi,
>
> ok i wil take a look at it.

Thanks! I changed your files a little bit, so that it compiles on
Ubuntu. But the test spellcheckersearch fails, because the  the method
suggestSimilar() returns no result (NULL pointer). Any idea?

Veit

Re: [CLucene-dev] CLucene-developers Digest, Vol 57, Issue 5

[Ahmed S. <ci7...@gm...>]

Hello Muhammad,
What analyzer are you using?
It seem to me that your code is based on that application
"clucene_qt", well i think that you are an old version of that
application, it has several bugs concerning this problem, and the
ArabicAnalyzer is not well implemented nor optimized, you should use
the 0.6 or 0.8 version of this application.

Ahmed

2011/1/6, muhammad ismael <m.i...@gm...>:
> Hello ben,
> the files are more than 6 GB.
> I am sorry i was remember to add this function but i forgot and here it is
>
> Document* IndexEngine::fileDocument(const QString &id, const QString &bookid
> , const QString &text)
>
> {
>
>     // make a new, empty document
>
>     Document* doc = _CLNEW Document();
>
>     ///page ID
>
>     doc->add( *_CLNEW Field(_T("id"), QSTRING_TO_TCHAR(id) ,
>
>                             Field::STORE_YES | Field::INDEX_UNTOKENIZED) );
>
>     doc->add( *_CLNEW Field(_T("bookid"), QSTRING_TO_TCHAR(bookid) ,
>
>                             Field::STORE_YES | Field::INDEX_UNTOKENIZED ) );
>
>     doc->add( *_CLNEW Field(_T("text"), QSTRING_TO_TCHAR(text),
>
>                             Field::STORE_NO | Field::INDEX_TOKENIZED) );
>
>     return doc;
>
> }
>
> and also i tried to remove adding documents and the size did not increased,
> which means that leaks are in addDocument() I am trying to debug it but i am
> lost.
>
> sounds pretty high. how big are the files? could you be leaking memory in
>> the 'fileDocument' function?
>>
>> as a test, try not actually adding the document
>>
>> ben
>>
>> On Thu, Jan 6, 2011 at 7:43 AM, muhammad ismael <m.i...@gm...
>> >wrote:
>>
>> > Hello,
>> > I am trying to index large files as follows
>> >
>> > for(int j = 0; (j < pagesIds.count())&& !m_stop ; j++)
>> >
>> >                 {
>> >
>> >                     pagesText =
>> m_DbManager->getBookPage(m_booksIds.at(i), pagesIds.at(j)).toUtf8();
>> >
>> >                     if(!pagesText.isEmpty())
>> >
>> >                     {
>> >
>> >                         Document* doc =
>> fileDocument(QString::number(pagesIds.at(j)),
>> QString::number(m_booksIds.at(i)), pagesText);
>> >
>> >                         writer->addDocument(doc);
>> >
>> >                         _CLDELETE(doc);
>> >
>> >                     }
>> >
>> >                 }
>> >
>> >
>> > but when the number of files exceeds 5000 files the application usage of
>> my
>> > computer ram is 2 GB
>> > I tried to debug and i found that this happens in
>> >
>> > IndexWriter::addDocument(Document*)
>> >
>> > i tried to set
>> > IndexWriter->setMergeFactor(5);
>> > and also
>> > IndexWriter->setRAMBufferSizeMB(10);
>> > I know the default ram usage should be 16 MB but i just tried.
>> >
>> > I am working on master branch and i merged with it memory_leaks branch.
>> > is am i missing something?
>>
>
> Mohammad Ismael
>

-- 
Envoyé avec mon mobile

Re: [CLucene-dev] contrib: SpellChecker

[Veit J. <nun...@go...>]

2011/1/5 Veit Jahns <nun...@go...>:
> It work on Windows (I will check this also on other platforms too),
> but I saw that you used wstring/wstringstream, e.g., in SpellCheckerC.
> Thus, it works only, if the ASCII mode is not enabled. With ASCII mode
> enabled it fails to compile. You have to use either tstring (then a
> tstring replacement for Non-Windows plattform is needed) or directly
> TCHAR.

Maybe another option is to use the class CL_NS(util)::StringBuffer here.

Veit

Re: [CLucene-dev] CLucene-developers Digest, Vol 57, Issue 5

[muhammad i. <m.i...@gm...>]

Hello ben,
the files are more than 6 GB.
I am sorry i was remember to add this function but i forgot and here it is

Document* IndexEngine::fileDocument(const QString &id, const QString &bookid
, const QString &text)

{

    // make a new, empty document

    Document* doc = _CLNEW Document();

    ///page ID

    doc->add( *_CLNEW Field(_T("id"), QSTRING_TO_TCHAR(id) ,

                            Field::STORE_YES | Field::INDEX_UNTOKENIZED) );

    doc->add( *_CLNEW Field(_T("bookid"), QSTRING_TO_TCHAR(bookid) ,

                            Field::STORE_YES | Field::INDEX_UNTOKENIZED ) );

    doc->add( *_CLNEW Field(_T("text"), QSTRING_TO_TCHAR(text),

                            Field::STORE_NO | Field::INDEX_TOKENIZED) );

    return doc;

}

and also i tried to remove adding documents and the size did not increased,
which means that leaks are in addDocument() I am trying to debug it but i am
lost.

sounds pretty high. how big are the files? could you be leaking memory in
> the 'fileDocument' function?
>
> as a test, try not actually adding the document
>
> ben
>
> On Thu, Jan 6, 2011 at 7:43 AM, muhammad ismael <m.i...@gm...
> >wrote:
>
> > Hello,
> > I am trying to index large files as follows
> >
> > for(int j = 0; (j < pagesIds.count())&& !m_stop ; j++)
> >
> >                 {
> >
> >                     pagesText =
> m_DbManager->getBookPage(m_booksIds.at(i), pagesIds.at(j)).toUtf8();
> >
> >                     if(!pagesText.isEmpty())
> >
> >                     {
> >
> >                         Document* doc =
> fileDocument(QString::number(pagesIds.at(j)),
> QString::number(m_booksIds.at(i)), pagesText);
> >
> >                         writer->addDocument(doc);
> >
> >                         _CLDELETE(doc);
> >
> >                     }
> >
> >                 }
> >
> >
> > but when the number of files exceeds 5000 files the application usage of
> my
> > computer ram is 2 GB
> > I tried to debug and i found that this happens in
> >
> > IndexWriter::addDocument(Document*)
> >
> > i tried to set
> > IndexWriter->setMergeFactor(5);
> > and also
> > IndexWriter->setRAMBufferSizeMB(10);
> > I know the default ram usage should be 16 MB but i just tried.
> >
> > I am working on master branch and i merged with it memory_leaks branch.
> > is am i missing something?
>

Mohammad Ismael

Re: [CLucene-dev] Compile CLucene using mingw under windows

[Ahmed S. <ci7...@gm...>]

Here is what cmake print out:
http://pastebin.com/4KPZeFbB
mingw32-make:
http://pastebin.com/JFX5XzJX

I am using the mingw version distributed along with qt 4.7

Ahmed
2011/1/6 Ben van Klinken <bva...@gm...>

> did you configure using cmake? what did cmake print out? turn on
> verbose (run cmake using -DCMAKE_VERBOSE_MAKEFILE=ON), or use the
> cmake gui. so that we can see what parameters are being passed in.
> looks to me like ming is being compiled in ascii mode.
>
> ben
>
> On Tue, Jan 4, 2011 at 7:28 PM, Ahmed nusha <ci7...@gm...> wrote:
> > I'm trying to compile clucene under windows and i get the following
> errors:
> >
> > C:\clucene_mingw-build>mingw32-make
> > [  0%] Building CXX object
> > src/shared/CMakeFiles/clucene-shared.dir/CLucene/SharedHeader.obj
> > C:\clucene\src\shared\CLucene\SharedHeader.cpp:16: error: cannot convert
> > 'const char*' to 'const TCHAR*' in initialization
> > mingw32-make[2]: ***
> > [src/shared/CMakeFiles/clucene-shared.dir/CLucene/SharedHeader.obj] Error
> 1
> > mingw32-make[1]: *** [src/shared/CMakeFiles/clucene-shared.dir/all] Error
> 2
> > mingw32-make: *** [all] Error 2
> >
> > what is the problem?
> >
> > Ahmed.
> >
> > Itamar Syn-Hershko <itamar@...> writes:
> >
> >>
> >> It should compile with MingW. What compile errors do you get?
> >>
> >> Itamar.
> >>
> >> On 1/8/2010 10:17 PM, Ahmed wrote:
> >> > hi,
> >> >
> >> > Can compile CLucene 2.3.2 under windows using MingW ?
> >> > I tried that and i get a lot of errors.
> >> >
> >> > Ahmed
> >> >
> >> >
> >> >
> >> >
> ------------------------------------------------------------------------------
> >> > The Palm PDK Hot Apps Program offers developers who use the
> >> > Plug-In Development Kit to bring their C/C++ apps to Palm for a share
> >> > of $1 Million in cash or HP Products. Visit us here for more details:
> >> > http://p.sf.net/sfu/dev2dev-palm
> >> > _______________________________________________
> >> > CLucene-developers mailing list
> >> > CLucene-developers@...
> >> > https://lists.sourceforge.net/lists/listinfo/clucene-developers
> >> >
> >> >
> >> >
> >>
> >>
> >>
> ------------------------------------------------------------------------------
> >> The Palm PDK Hot Apps Program offers developers who use the
> >> Plug-In Development Kit to bring their C/C++ apps to Palm for a share
> >> of $1 Million in cash or HP Products. Visit us here for more details:
> >> http://p.sf.net/sfu/dev2dev-palm
> >>
> >
> >
> >
> ------------------------------------------------------------------------------
> > Learn how Oracle Real Application Clusters (RAC) One Node allows
> customers
> > to consolidate database storage, standardize their database environment,
> > and,
> > should the need arise, upgrade to a full multi-node Oracle RAC database
> > without downtime or disruption
> > http://p.sf.net/sfu/oracle-sfdevnl
> > _______________________________________________
> > CLucene-developers mailing list
> > CLu...@li...
> > https://lists.sourceforge.net/lists/listinfo/clucene-developers
> >
> >
>
>
> ------------------------------------------------------------------------------
> Learn how Oracle Real Application Clusters (RAC) One Node allows customers
> to consolidate database storage, standardize their database environment,
> and,
> should the need arise, upgrade to a full multi-node Oracle RAC database
> without downtime or disruption
> http://p.sf.net/sfu/oracle-sfdevnl
> _______________________________________________
> CLucene-developers mailing list
> CLu...@li...
> https://lists.sourceforge.net/lists/listinfo/clucene-developers
>

Re: [CLucene-dev] CLucene-developers Digest, Vol 57, Issue 4

[Šplíchal J. <spl...@to...>]

Hi, 

 

ad 1) the highlighter branch is there, did you pull the newest version?

 

ad 2) the basic QueryScorer calls extractTerms() method on the supplied query, so you can use it only with queries that implement this method.

All primitive queries do implement it, Wildcard query is not a primitive query and so you have to call its rewrite method and use the rewritten query.

 

ad 3) the new implementation – SpanHighlightScorer – is able to rewrite the queries itself because it makes (if necessary) its own memoryindex 

containing just the highlighted document. The queries are rewritten using this small index. For a wildcard query this normally leads to a much smaller 

number of matching terms and so you avoid the max clause problem. (But of course, if your Wildcard query matches more than maxTerms different terms

in the highlighted text, than you have to increase this number) – How big are your documents?

 

Jiri

 

 

From: muhammad ismael [mailto:m.i...@gm...] 
Sent: Wednesday, January 05, 2011 7:00 PM
To: clu...@li...
Subject: Re: [CLucene-dev] CLucene-developers Digest, Vol 57, Issue 4

 

	
	please could you write how you instantiate the highlighter? Which scorer do you use?


         m_highlightQuery = m_pMainQuery->clone(); 

        Query * q = m_highlightQuery->rewrite(m_pReader);
        m_queryScorer = new QueryScorer(q);
        m_highlighter = new Highlighter(m_queryScorer);

 

	
	Are you using the highlighter version from the highlighter branch?


No i am on master branch and i can not find highlighter branch  , i merged the wildcardQuery_fix and memory_leaks branches to master.

	
	I would try to find the place, where the exception is thrown.
	
	
	
	But still, there is a problem along with the ConstantScoreQuery. Both ways to highlight a document
	
	that are available now will not work because:
	
	1) If you use the "basic" highlighting - means you use the CL_NS2(search,highlight)::QueryScorer - than this scorer calls the extractTerms() method which is not
	
	implemented in ConstantScoreQuery  as this query uses filters to match documents.
	
	2 the "exact" highlighting - means you use the CL_NS2(search,highlight)::SpanHighlightScorer - also does not highlight ConstantScoreQueries for the same reason.

is this means that i can not use WildcardQuery with highlighter without setting the maximum clauses?
 

	But, you could use this scorer with the original WildcardQuery and allow rewriting queries in place. Then if your documents are not too long and
	
	the single documents does not contain  more the the max clause limit different matching terms, then this would work.
	
	
	
	If you want to use the new highlighter scorer then I could send you piece of code, how to call it.
	
	

Note :-
I can not use setMaxClauses because my documents are too long and if i use setMaxClauses i always get too many clauses exception 
 

Mohammad Ismael

 

Re: [CLucene-dev] Compile CLucene using mingw under windows

[Ben v. K. <bva...@gm...>]

did you configure using cmake? what did cmake print out? turn on
verbose (run cmake using -DCMAKE_VERBOSE_MAKEFILE=ON), or use the
cmake gui. so that we can see what parameters are being passed in.
looks to me like ming is being compiled in ascii mode.

ben

On Tue, Jan 4, 2011 at 7:28 PM, Ahmed nusha <ci7...@gm...> wrote:
> I'm trying to compile clucene under windows and i get the following errors:
>
> C:\clucene_mingw-build>mingw32-make
> [  0%] Building CXX object
> src/shared/CMakeFiles/clucene-shared.dir/CLucene/SharedHeader.obj
> C:\clucene\src\shared\CLucene\SharedHeader.cpp:16: error: cannot convert
> 'const char*' to 'const TCHAR*' in initialization
> mingw32-make[2]: ***
> [src/shared/CMakeFiles/clucene-shared.dir/CLucene/SharedHeader.obj] Error 1
> mingw32-make[1]: *** [src/shared/CMakeFiles/clucene-shared.dir/all] Error 2
> mingw32-make: *** [all] Error 2
>
> what is the problem?
>
> Ahmed.
>
> Itamar Syn-Hershko <itamar@...> writes:
>
>>
>> It should compile with MingW. What compile errors do you get?
>>
>> Itamar.
>>
>> On 1/8/2010 10:17 PM, Ahmed wrote:
>> > hi,
>> >
>> > Can compile CLucene 2.3.2 under windows using MingW ?
>> > I tried that and i get a lot of errors.
>> >
>> > Ahmed
>> >
>> >
>> >
>> > ------------------------------------------------------------------------------
>> > The Palm PDK Hot Apps Program offers developers who use the
>> > Plug-In Development Kit to bring their C/C++ apps to Palm for a share
>> > of $1 Million in cash or HP Products. Visit us here for more details:
>> > http://p.sf.net/sfu/dev2dev-palm
>> > _______________________________________________
>> > CLucene-developers mailing list
>> > CLucene-developers@...
>> > https://lists.sourceforge.net/lists/listinfo/clucene-developers
>> >
>> >
>> >
>>
>>
>> ------------------------------------------------------------------------------
>> The Palm PDK Hot Apps Program offers developers who use the
>> Plug-In Development Kit to bring their C/C++ apps to Palm for a share
>> of $1 Million in cash or HP Products. Visit us here for more details:
>> http://p.sf.net/sfu/dev2dev-palm
>>
>
>
> ------------------------------------------------------------------------------
> Learn how Oracle Real Application Clusters (RAC) One Node allows customers
> to consolidate database storage, standardize their database environment,
> and,
> should the need arise, upgrade to a full multi-node Oracle RAC database
> without downtime or disruption
> http://p.sf.net/sfu/oracle-sfdevnl
> _______________________________________________
> CLucene-developers mailing list
> CLu...@li...
> https://lists.sourceforge.net/lists/listinfo/clucene-developers
>
>