opentrep-svn Mailing List for Open Travel Request Parser (Page 4)
Status: Beta
Brought to you by:
denis_arnaud
Menu
▾
▴
opentrep-svn — Subversion Activity Notification List
You can subscribe to this list here.
| 2009 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
(52) |
Aug
(19) |
Sep
(4) |
Oct
(10) |
Nov
(2) |
Dec
(4) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2010 |
Jan
(3) |
Feb
|
Mar
(3) |
Apr
|
May
|
Jun
(1) |
Jul
(2) |
Aug
(1) |
Sep
(9) |
Oct
|
Nov
(1) |
Dec
|
| 2011 |
Jan
|
Feb
(8) |
Mar
|
Apr
|
May
|
Jun
|
Jul
(2) |
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
|
From: <den...@us...> - 2009-07-24 05:35:01
|
Revision: 164
http://opentrep.svn.sourceforge.net/opentrep/?rev=164&view=rev
Author: denis_arnaud
Date: 2009-07-24 05:34:50 +0000 (Fri, 24 Jul 2009)
Log Message:
-----------
[DB] Suppressed the classical_name2 column of the ref_place_names table.
Modified Paths:
--------------
trunk/opentrep/db/data/ref_place_names.csv
trunk/opentrep/db/maintenance/create_and_fill_mysql_db.sh
trunk/opentrep/db/maintenance/tables/ref_city.sql
trunk/opentrep/db/maintenance/tables/ref_db.sql
trunk/opentrep/opentrep/bom/ResultHolder.cpp
trunk/opentrep/opentrep/command/DBManager.cpp
trunk/opentrep/opentrep/command/IndexBuilder.cpp
trunk/opentrep/opentrep/dbadaptor/DbaPlace.cpp
trunk/opentrep/test/xapian/Makefile
Added Paths:
-----------
trunk/opentrep/db/maintenance/export_ref_data.sh
trunk/opentrep/db/maintenance/tables/export_ref_place_details.sql
trunk/opentrep/db/maintenance/tables/export_ref_place_names.sql
trunk/opentrep/db/maintenance/uptade_db_01.sh
trunk/opentrep/test/xapian/string_simple_search.cpp
Property Changed:
----------------
trunk/opentrep/test/xapian/
Modified: trunk/opentrep/db/data/ref_place_names.csv
===================================================================
--- trunk/opentrep/db/data/ref_place_names.csv 2009-07-23 17:47:31 UTC (rev 163)
+++ trunk/opentrep/db/data/ref_place_names.csv 2009-07-24 05:34:50 UTC (rev 164)
@@ -1,11127 +1,11127 @@
-concat('en,', code, ',', ticketing_name, ',', teleticketing_name, ',', extended_name)
-en,cyz,cauayan,cauayan,cauayan/ph
-en,cza,chichen itza,chichen itza,chichen itza/mx
-en,czb,cruz alta,cruz alta,cruz alta/rs/br:carlos ruhl
-en,czc,copper center,copper center,copper center/ak/us
-en,cze,coro,coro,coro/ve
-en,czf,cape romanzof,cape romanzof,cape romanzof/ak/us
-en,czh,corozal,corozal,corozal/bz
-en,czj,corazon de jesus,corazon de jesu,corazon de jesus/pa
-en,czk,cascade locks,cascade locks,cascade locks/or/us:cascade l
-en,czl,constantine,constantine,constantine/dz:ain el bey
-en,czm,cozumel,cozumel,cozumel/mx
-en,czn,chisana,chisana,chisana/ak/us:chisana field
-en,czo,chistochina,chistochina,chistochina/ak/us
-en,czp,cape pole,cape pole,cape pole/ak/us
-en,czs,cruzeiro do sul,cruzeiro do sul,cruzeiro do sul/ac/br:campo
-en,czt,carrizo springs,carrizo springs,carrizo springs/tx/us
-en,czu,corozal,corozal,corozal/co
-en,czw,czestochowa,czestochowa,czestochowa/pl
-en,czx,changzhou,changzhou,changzhou/cn
-en,czy,cluny,cluny,cluny/ql/au
-en,czz,campo,campo,campo/ca/us
-en,daa,fort belvoir,fort belvoir,fort belvoir/va/us:davison aaf
-en,dab,daytona beach,daytona beach,daytona beach/fl/us:regional
-en,dac,dhaka,dhaka,dhaka/bd:zia international
-en,dad,da nang,da nang,da nang/vn
-en,dae,daparizo,daparizo,daparizo/in
-en,daf,daup,daup,daup/pg
-en,dag,daggett,daggett,daggett/ca/us:barstow daggett
-en,dah,dathina,dathina,dathina/ye
-en,dai,darjeeling,darjeeling,darjeeling/in
-en,daj,dauan island,dauan island,dauan island/ql/au
-en,dak,dakhla,dakhla,dakhla oasis/eg:dakhla
-en,dal,dallas dal,dallas dal,dallas/tx/us:love field
-en,dam,damascus,damascus,damascus/sy:intl
-en,dan,danville,danville,danville/va/us:municipal
-en,dao,dabo,dabo,dabo/pg
-en,dap,darchula,darchula,darchula/np
-en,dar,dar es salaam,dar es salaam,dar es salaam/tz:intl
-en,das,great bear lake,great bear lake,great bear lake/nt/ca
-en,dat,datong,datong,datong/cn
-en,dau,daru,daru,daru/pg
-en,dav,david,david,david/pa:enrique malek
-en,dax,daxian,daxian,daxian/cn
-en,day,dayton day,dayton day,dayton/oh/us:james cox dayton
-en,daz,darwaz,darwaz,darwaz/af
-en,dba,dalbandin,dalbandin,dalbandin/pk
-en,dbb,dabaa alalamain,dabaa alalamain,dabaa city/eg:alalamain intl
-en,dbd,dhanbad,dhanbad,dhanbad/in
-en,dbm,debra marcos,debra marcos,debra marcos/et
-en,dbn,dublin,dublin,dublin/ga/us:municipal
-en,dbo,dubbo,dubbo,dubbo/ns/au
-en,dbp,debepare,debepare,debepare/pg
-en,dbq,dubuque,dubuque,dubuque/ia/us:dubuque mnpl
-en,dbs,dubois,dubois,dubois/id/us
-en,dbt,debra tabor,debra tabor,debra tabor/et
-en,dbu,dambula,dambula,dambula/lk:dambula oya tank
-en,dbv,dubrovnik,dubrovnik,dubrovnik/hr
-en,dby,dalby,dalby,dalby/ql/au
-en,dca,washington dca,washington dca,washington/dc/us:r reagan nat
-en,dcf,dominica dcf,dominica dcf,dominica/dm:cane field
-en,dci,decimomannu,decimomannu,decimomannu/it:r decimomannu
-en,dck,dahl creek,dahl creek,dahl creek/ak/us:dahl creek
-en,dcm,castres,castres,castres/fr:mazamet
-en,dcn,derby dcn,derby dcn,derby/wa/au:curtin
-en,dcr,decatur,decatur,decatur/in/us:decatur hi way
-en,dct,duncan town,duncan town,duncan town/bs
-en,dcu,decatur,decatur,decatur/al/us:pyor
-en,ddc,dodge city,dodge city,dodge city/ks/us:mncpl
-en,ddg,dandong,dandong,dandong/cn
-en,ddi,daydream is,daydream is,daydream is/ql/au
-en,ddm,dodoima,dodoima,dodoima/pg
-en,ddn,delta downs,delta downs,delta downs/ql/au
-en,ddp,dorado,dorado,dorado/pr:dorado beach
-en,ddu,dadu,dadu,dadu/pk
-en,bfs,belfast intl,belfast intl,belfast/gb:international
-en,bft,beaufort,beaufort,beaufort/sc/us:county
-en,bfu,bengbu,bengbu,bengbu/cn
-en,bfv,buri ram,buri ram,buri ram/th
-en,bfw,sidi belabbes,sidi belabbes,sidi belabbes/dz:sidi belabbes
-en,bfx,bafoussam,bafoussam,bafoussam/cm
-en,bga,bucaramanga,bucaramanga,bucaramanga/co:palo negro
-en,bgb,booue,booue,booue/ga
-en,bgc,braganca,braganca,braganca/pt
-en,bgd,borger,borger,borger/tx/us
-en,bge,bainbridge,bainbridge,bainbridge/ga/us:decatur cnty
-en,bgf,bangui,bangui,bangui/cf
-en,bgg,bongouanou,bongouanou,bongouanou/ci
-en,bgh,boghe,boghe,boghe/mr:abbaye
-en,bgi,bridgetown,bridgetown,bridgetown/bb:grantley adams
-en,bgj,borgarfjordur eys,borgarfjordur e,borgarfjordur eystri/is
-en,bgk,big creek,big creek,big creek/bz
-en,bgl,baglung,baglung,baglung/np
-en,bgm,binghamton,binghamton,binghamton/ny/us
-en,bgn,brueggen,brueggen,brueggen/de:r a f
-en,bgo,bergen,bergen,bergen/no:flesland
-en,bgp,bongo,bongo,bongo/ga
-en,bgq,big lake,big lake,big lake/ak/us
-en,bgr,bangor,bangor,bangor/me/us:international
-en,bgs,big spring afb,big spring afb,big spring/tx/us:webb afb
-en,bgt,bagdad,bagdad,bagdad/az/us
-en,bgu,bangassou,bangassou,bangassou/cf
-en,bgv,bento goncalves,bento goncalves,bento goncalves/rs/br
-en,bgw,baghdad bgw,baghdad bgw,baghdad/iq:baghdad intl
-en,bgx,bage,bage,bage/rs/br
-en,bgy,milan bgy,milan bgy,milan/it:orio al serio
-en,bgz,braga,braga,braga/pt
-en,bha,bahia de caraquez,bahia de caraqu,bahia de caraquez/ec
-en,bhb,bar harbor,bar harbor,bar harbor/me/us
-en,bhc,bhurban,bhurban,bhurban/pk:heliport
-en,bhd,belfast city,belfast city,belfast/gb:belfast city
-en,bhe,blenheim,blenheim,blenheim/nz
-en,bhf,bahia cupica,bahia cupica,bahia cupica/co
-en,bhg,brus laguna,brus laguna,brus laguna/hn
-en,bhh,bisha,bisha,bisha/sa
-en,bhi,bahia blanca,bahia blanca,bahia blanca/ba/ar:comandante
-en,bhj,bhuj,bhuj,bhuj/in:rudra mata
-en,bhk,bukhara,bukhara,bukhara/uz
-en,bhl,bahia angeles,bahia angeles,bahia angeles/mx
-en,bhm,birmingham,birmingham,birmingham/al/us
-en,bhn,beihan,beihan,beihan/ye
-en,bho,bhopal,bhopal,bhopal/in
-en,bhp,bhojpur,bhojpur,bhojpur/np
-en,bhq,broken hill,broken hill,broken hill/ns/au
-en,bhr,bharatpur,bharatpur,bharatpur/np
-en,bhs,bathurst,bathurst,bathurst/ns/au:raglan
-en,bht,brighton downs,brighton downs,brighton downs/ql/au
-en,bhu,bhavnagar,bhavnagar,bhavnagar/in
-en,bhv,bahawalpur,bahawalpur,bahawalpur/pk
-en,bhw,sargodha bhw,sargodha bhw,sargodha/pk:bhagatanwala
-en,bhx,birmingham,birmingham,birmingham/gb:international
-en,bhy,beihai,beihai,beihai/cn
-en,bhz,belo horizonte,belo horizonte,belo horizonte/mg/br
-en,bia,bastia,bastia,bastia/fr:poretta
-en,bib,baidoa,baidoa,baidoa/so
-en,bic,big creek,big creek,big creek/ak/us
-en,bid,block island,block island,block island/ri/us
-en,bie,beatrice,beatrice,beatrice/ne/us
-en,bif,el paso biggs aaf,el paso biggs a,el paso/tx/us:biggs aaf
-en,big,big delta,big delta,big delta/ak/us:intermediate
-en,bih,bishop,bishop,bishop/ca/us
-en,bii,bikini atoll,bikini atoll,bikini atoll/mh:enyu airfield
-en,bij,biliau,biliau,biliau/pg
-en,bik,biak,biak,biak/id:mokmer
-en,bil,billings,billings,billings/mt/us
-en,bim,bimini int,bimini int,bimini/bs:international
-en,bin,bamiyan,bamiyan,bamiyan/af
-en,bio,bilbao,bilbao,bilbao/es
-en,dea,dera ghazi khan,dera ghazi khan,dera ghazi khan/pk
-en,deb,debrecen,debrecen,debrecen/hu
-en,isd,iscuande,iscuande,iscuande/co
-en,ise,isparta,isparta,isparta/tr
-en,isg,ishigaki,ishigaki,ishigaki/jp
-en,ish,ischia,ischia,ischia/it
-en,isi,isisford,isisford,isisford/ql/au
-en,isj,isla mujeres,isla mujeres,isla mujeres/mx
-en,isk,nasik,nasik,nasik/in:gandhinagar apt
-en,isl,isabel pass,isabel pass,isabel pass/ak/us
-en,ism,kissimmee,kissimmee,kissimmee/fl/us:municipal
-en,isn,williston,williston,williston/nd/us:sloulin field
-en,iso,kinston,kinston,kinston/nc/us:stallings field
-en,isp,islip,islip,islip/ny/us:long island macar
-en,isq,manistique,manistique,manistique/mi/us:schoolcraft
-en,iss,wiscasset,wiscasset,wiscasset/me/us
-en,ist,istanbul,istanbul,istanbul/tr:ataturk
-en,isu,sulaymaniyah,sulaymaniyah,sulaymaniyah/iq:international
-en,isw,wisconsin rapids,wisconsin rapid,wisconsin rapids/wi/us:alexan
-en,ita,itacoatiara,itacoatiara,itacoatiara/am/br
-en,itb,itaituba,itaituba,itaituba/pa/br
-en,ite,itubera,itubera,itubera/ba/br
-en,ith,ithaca,ithaca,ithaca/ny/us:tompkins county
-en,iti,itambacuri,itambacuri,itambacuri/mg/br
-en,itj,itajai,itajai,itajai/sc/br:off-
-en,itk,itokama,itokama,itokama/pg
-en,itm,osaka itami apt,osaka itami apt,osaka itami apt/jp:itami
-en,itn,itabuna,itabuna,itabuna/ba/br
-en,ito,hilo,hilo,hilo/hi/us:hilo international
-en,itp,itaperuna,itaperuna,itaperuna/rj/br:itaperuna
-en,itq,itaqui,itaqui,itaqui/rs/br
-en,itr,itumbiara,itumbiara,itumbiara/go/br:itumbiara
-en,iue,niue island,niue island,niue island/nu:hanan
-en,iul,ilu,ilu,ilu/id
-en,ium,summit lake,summit lake,summit lake/bc/ca
-en,ius,inus,inus,inus/pg
-en,iva,ambanja,ambanja,ambanja/mg
-en,ivc,invercargill,invercargill,invercargill/nz
-en,ivg,ivangrad,ivangrad,ivangrad/me
-en,ivh,ivishak,ivishak,ivishak/ak/us
-en,ivl,ivalo,ivalo,ivalo/fi
-en,ivo,chivolo,chivolo,chivolo/co
-en,ivr,inverell,inverell,inverell/ns/au
-en,ivw,inverway,inverway,inverway/nt/au
-en,iwa,ivanova,ivanova,ivanova/ru
-en,iwd,ironwood,ironwood,ironwood/mi/us:gogebic county
-en,iwj,iwami,iwami,iwami/jp:iwami aiport
-en,iwo,iwo jima vol,iwo jima vol,iwo jima vol/jp:iwo jima base
-en,iws,houston west,houston west,houston/tx/us:west houston
-en,ixa,agartala,agartala,agartala/in:singerbhil
-en,ixb,bagdogra,bagdogra,bagdogra/in
-en,ixc,chandigarh,chandigarh,chandigarh/in
-en,ixd,allahabad,allahabad,allahabad/in:bamrauli
-en,ixe,mangalore,mangalore,mangalore/in:bajpe
-en,ixg,belgaum,belgaum,belgaum/in:sambre
-en,ixh,kailashahar,kailashahar,kailashahar/in
-en,ixi,lilabari,lilabari,lilabari/in
-en,ixj,jammu,jammu,jammu/in:satwari
-en,ixk,keshod,keshod,keshod/in
-en,ixl,leh,leh,leh/in
-en,ixm,madurai,madurai,madurai/in
-en,ixn,khowai,khowai,khowai/in
-en,ixp,pathankot,pathankot,pathankot/in
-en,ixq,kamalpur,kamalpur,kamalpur/in
-en,ixr,ranchi,ranchi,ranchi/in
-en,ixs,silchar,silchar,silchar/in:kumbhirgram
-en,ixt,pasighat,pasighat,pasighat/in
-en,ixu,aurangabad,aurangabad,aurangabad/in:chikkalthana
-en,ixv,along,along,along/in
-en,ixw,jamshedpur,jamshedpur,jamshedpur/in:sonari
-en,ixy,kandla,kandla,kandla/in
-en,ixz,port blair,port blair,port blair/in
-en,iyk,inyokern,inyokern,inyokern/ca/us:kern county
-en,izm,izmir,izmir,izmir/tr
-en,izo,izumo,izumo,izumo/jp
-en,izt,ixtepec,ixtepec,ixtepec/mx
-en,jaa,jalalabad,jalalabad,jalalabad/af
-en,cew,crestview,crestview,crestview/fl/us:bob sikes
-en,cex,chena hot springs,chena hot sprin,chena hot springs/ak/us
-en,hoy,hoy island,hoy island,hoy island/gb
-en,hpa,ha'apai,ha'apai,ha'apai/to:salote pilolevu
-en,hpb,hooper bay,hooper bay,hooper bay/ak/us
-en,hpe,hope vale,hope vale,hope vale/ql/au
-en,hph,haiphong,haiphong,haiphong/vn:catbi
-en,hpn,westchester count,westchester cou,westchester county/ny/us:westc
-en,hpp,poipet,poipet,poipet/kh
-en,hpr,pretoria hpr,pretoria hpr,pretoria/za:central hpr
-en,hpt,hampton,hampton,hampton/ia/us:municipal
-en,hpv,kauai island hpv,kauai island hp,kauai island/hi/us:princeville
-en,hpy,baytown,baytown,baytown/tx/us
-en,hqm,aberdeen,aberdeen,aberdeen/wa/us
-en,hra,mansehra,mansehra,mansehra/pk
-en,hrb,harbin,harbin,harbin/cn
-en,hrc,zhairem,zhairem,zhairem/kz
-en,hrd,harstad,harstad,harstad/no:harstad
-en,hre,harare,harare,harare/zw
-en,hrg,hurghada,hurghada,hurghada/eg
-en,hrj,chaurjhari,chaurjhari,chaurjhari/np
-en,hrk,kharkov,kharkov,kharkov/ua
-en,hrl,harlingen,harlingen,harlingen/tx/us:valley intl
-en,hrm,hassi r'mel,hassi r'mel,hassi r'mel/dz:tilrempt
-en,hrn,heron island,heron island,heron island/ql/au:heliport
-en,hro,harrison,harrison,harrison/ar/us:boone county
-en,hrr,herrera,herrera,herrera/co
-en,hrs,harrismith,harrismith,harrismith/za:harrismith
-en,hrt,harrogate,harrogate,harrogate/gb:linton on ouse
-en,hry,henbury,henbury,henbury/nt/au
-en,hrz,horizontina,horizontina,horizontina/rs/br
-en,hsb,harrisburg,harrisburg,harrisburg/il/us:raleigh
-en,hsc,shaoguan,shaoguan,shaoguan/cn
-en,hsg,saga,saga,saga/jp:saga airport
-en,hsh,las vegas hsh,las vegas hsh,las vegas/nv/us:henderson sky
-en,hsi,hastings,hastings,hastings/ne/us
-en,hsk,huesca,huesca,huesca/es
-en,hsl,huslia,huslia,huslia/ak/us
-en,hsm,horsham,horsham,horsham/vi/au
-en,hsn,zhoushan,zhoushan,zhoushan/cn
-en,hsp,hot springs,hot springs,hot springs/va/us:ingalls fld
-en,hss,hissar,hissar,hissar/in
-en,hst,homestead,homestead,homestead/fl/us:afb
-en,hsv,huntsvil decatur,huntsvil decatu,huntsville/al/us:intl apt
-en,hsz,hsinchu,hsinchu,hsinchu/tw
-en,hta,chita,chita,chita/ru
-en,htb,terre de bas,terre de bas,terre de bas/gp
-en,htf,hatfield,hatfield,hatfield/gb
-en,htg,hatanga,hatanga,hatanga/ru
-en,hth,hawthorne,hawthorne,hawthorne/nv/us
-en,hti,hamilton island,hamilton island,hamilton island/ql/au
-en,htl,houghton,houghton,houghton/mi/us:roscommon cnty
-en,htm,khatgal,khatgal,khatgal/mn
-en,htn,hotan,hotan,hotan/cn
-en,hto,east hampton,east hampton,east hampton/ny/us
-en,htr,hateruma,hateruma,hateruma/jp
-en,hts,huntington,huntington,huntington/wv/us:tri state
-en,htu,hopetoun,hopetoun,hopetoun/vi/au
-en,htv,huntsville,huntsville,huntsville/tx/us
-en,htw,chesapeake,chesapeake,huntington/oh/us
-en,hty,hatay,hatay,hatay/tr:hatay
-en,htz,hato corozal,hato corozal,hato corozal/co
-en,hua,huntsville aaf,huntsville aaf,huntsville/al/us:redstone aaf
-en,hub,humbert river,humbert river,humbert river/nt/au
-en,huc,humacao arpt,humacao arpt,humacao/pr:humacao arpt
-en,hud,humboldt,humboldt,humboldt/ia/us
-en,hue,humera,humera,humera/et
-en,huf,terre haute,terre haute,terre haute/in/us:hulman field
-en,hug,huehuetenango,huehuetenango,huehuetenango/gt
-en,huh,huahine,huahine,huahine/pf:huahine
-en,hui,hue,hue,hue/vn:phu bai
-en,huj,hugo,hugo,hugo/ok/us
-en,huk,hukuntsi,hukuntsi,hukuntsi/bw
-en,hul,houlton,houlton,houlton/me/us:international
-en,hum,houma,houma,houma/la/us:terrebonne
-en,hun,hualien,hualien,hualien/tw
-en,huq,houn,houn,houn/ly
-en,gtc,green turtle,green turtle,green turtle/bs
-en,gte,groote eylandt,groote eylandt,groote eylandt/nt/au:alyangula
-en,gtf,great falls intl,great falls int,great falls/mt/us:intl
-en,gtg,grantsburg,grantsburg,grantsburg/wi/us:municipal
-en,gti,guettin,guettin,guettin/de
-en,gtk,sungei tekai,sungei tekai,sungei tekai/my
-en,gtn,mount cook gtn,mount cook gtn,mount cook/nz:glentanner
-en,gto,gorontalo,gorontalo,gorontalo/id:tolotio
-en,gtp,grants pass,grants pass,grants pass/or/us
-en,gtr,columbus gtr,columbus gtr,columbus/ms/us:golden triangle
-en,gts,granites,granites,granites/nt/au
-en,gtt,georgetown,georgetown,georgetown/ql/au
-en,gtw,zlin,zlin,zlin/cz:holesov
-en,gty,gettysburg,gettysburg,gettysburg/pa/us
-en,gua,guatemala city,guatemala city,guatemala city/gt:la aurora
-en,gub,guerrero negro,guerrero negro,guerrero negro/mx
-en,guc,gunnison,gunnison,gunnison/co/us
-en,gud,goundam,goundam,goundam/ml
-en,gue,guriaso,guriaso,guriaso/pg
-en,guf,gulf shores,gulf shores,gulf shores/al/us:edwards
-en,gug,guari,guari,guari/pg
-en,guh,gunnedah,gunnedah,gunnedah/ns/au
-en,gui,guiria,guiria,guiria/ve
-en,guj,guaratingueta,guaratingueta,guaratingueta/sp/br
-en,gul,goulburn,goulburn,goulburn/ns/au
-en,gum,guam won pat intl,guam won pat in,guam/gu:a.b won pat intl
-en,gun,montgomery gu afb,montgomery gu a,montgomery/al/us:gunter afb
-en,guo,gualaco,gualaco,gualaco/hn
-en,gup,gallup,gallup,gallup/nm/us:senator clark
-en,guq,guanare,guanare,guanare/ve
-en,gur,alotau,alotau,alotau/pg:gurney
-en,gus,peru,peru,peru/in/us:grissom afb
-en,gut,guetersloh,guetersloh,guetersloh/de
-en,guu,grundarfjordur,grundarfjordur,grundarfjordur/is
-en,guv,mougulu,mougulu,mougulu/pg
-en,guw,atyrau,atyrau,atyrau/kz
-en,gux,guna,guna,guna/in
-en,guy,guymon,guymon,guymon/ok/us
-en,guz,guarapari,guarapari,guarapari/es/br
-en,gva,geneva,geneva,geneva/ch:geneva intl
-en,gve,gordonsville,gordonsville,gordonsville/va/us:municipal
-en,gvi,green river,green river,green river/pg
-en,gvl,gainesville,gainesville,gainesville/ga/us:lee gilmer
-en,gvn,sovetskaya gavan,sovetskaya gava,sovetskaya gavan/ru:sovetskaya
-en,gvp,greenvale,greenvale,greenvale/ql/au
-en,gvr,governador valada,governador vala,governador valada/mg/br
-en,gvt,greenville,greenville,greenville/tx/us:majors field
-en,gvw,grandview,grandview,grandview/mo/us:richards gebau
-en,gvx,gavle,gavle,gavle/se:sandviken
-en,gwa,gwa,gwa,gwa/mm
-en,gwd,gwadar,gwadar,gwadar/pk
-en,gwe,gweru,gweru,gweru/zw
-en,gwl,gwalior,gwalior,gwalior/in
-en,gwn,gnarowein,gnarowein,gnarowein/pg
-en,gwo,greenwood,greenwood,greenwood/ms/us:leflore
-en,gws,glenwood springs,glenwood spring,glenwood springs/co/us
-en,gwt,westerland,westerland,westerland/de:westerland sylt
-en,gwv,glendale,glendale,glendale/wv/us
-en,gwy,galway,galway,galway/ie:carnmore
-en,gxf,seiyun,seiyun,seiyun/ye
-en,gxg,negage,negage,negage/ao
-en,gxh,mildenhall naf,mildenhall naf,mildenhall/gb:naf
-en,gxq,coyhaique,coyhaique,coyhaique/cl:ten vidal
-en,gxx,yagoua,yagoua,yagoua/cm
-en,gxy,greeley,greeley,greeley/co/us:weld county
-en,gya,guayaramerin,guayaramerin,guayaramerin/bo
-en,gyd,baku intl,baku intl,baku/az:heydar aliyev intl
-en,gye,guayaquil,guayaquil,guayaquil/ec:jose joaquin de o
-en,gyi,gisenyi,gisenyi,gisenyi/rw
-en,gyl,argyle,argyle,argyle/wa/au
-en,gym,guaymas,guaymas,guaymas/mx:gen jose m yanez
-en,gyn,goiania,goiania,goiania/go/br:santa genoveva
-en,gyp,gympie,gympie,gympie/ql/au
-en,dec,decatur,decatur,decatur/il/us:decatur apt
-en,ded,dehra dun,dehra dun,dehra dun/in
-en,def,dezful,dezful,dezful/ir:dezful
-en,deh,decorah,decorah,decorah/ia/us:municipal
-en,dei,denis island,denis island,denis island/sc
-en,del,delhi,delhi,delhi/in:indira gandhi intl
-en,dem,dembidollo,dembidollo,dembidollo/et
-en,den,denver,denver,denver/co/us:denver intl
-en,deo,dearborn,dearborn,dearborn/mi/us:hyatt regency
-en,dep,deparizo,deparizo,deparizo/in
-en,der,derim,derim,derim/pg
-en,des,desroches,desroches,desroches/sc
-en,det,detroit city,detroit city,detroit/mi/us:detroit city
-en,dez,deirezzor,deirezzor,deirezzor/sy:al
-en,dfi,defiance,defiance,defiance/oh/us:memorial
-en,dfp,drumduff,drumduff,drumduff/ql/au
-en,dfw,dallas fort worth,dallas fort wor,dallas/tx/us:dallas ft worth
-en,dga,dangriga,dangriga,dangriga/bz
-en,dgb,danger bay,danger bay,danger bay/ak/us
-en,dgc,degahbur,degahbur,degahbur/et
-en,dgd,dalgaranga,dalgaranga,dalgaranga/wa/au
-en,dge,mudgee,mudgee,mudgee/ns/au
-en,dgf,douglas lake,douglas lake,douglas lake/bc/ca
-en,dgg,daugo,daugo,daugo/pg
-en,dgk,dugong,dugong,dugong/mz
-en,dgl,douglas municipal,douglas municip,douglas/az/us:municipal
-en,dgm,dongguan,dongguan,dongguan/cn
-en,dgn,dahlgren,dahlgren,dahlgren/va/us:naf
-en,dgo,durango,durango,durango/mx:guadalupe victoria
-en,dgp,daugavpils,daugavpils,daugavpils/lv
-en,dgr,dargaville,dargaville,dargaville/nz
-en,dgt,dumaguete,dumaguete,dumaguete/ph
-en,dgu,dedougou,dedougou,dedougou/bf
-en,dgw,douglas,douglas,douglas/wy/us:converse county
-en,dha,dhahran,dhahran,dhahran/sa
-en,dhd,durham downs,durham downs,durham downs/ql/au
-en,dhf,abu dhabi al dhaf,abu dhabi al dh,abu dhabi/ae:al dhafra milit
-en,dhi,dhangarhi,dhangarhi,dhangarhi/np
-en,dhl,dhala,dhala,dhala/ye
-en,dhm,dharamsala,dharamsala,dharamsala/in:gaggal airport
-en,dhn,dothan,dothan,dothan/al/us:regional apt
-en,dhr,den helder,den helder,den helder/nl:de kooy
-en,dht,dalhart,dalhart,dalhart/tx/us
-en,dib,dibrugarh,dibrugarh,dibrugarh/in:chabua
-en,dic,dili,dili,dili/cd
-en,die,antsiranana,antsiranana,antsiranana/mg:antsiranana
-en,dig,diqing,diqing,diqing/cn:diqing
-en,dij,dijon,dijon,dijon/fr
-en,dik,dickinson,dickinson,dickinson/nd/us
-en,dil,dili,dili,dili/tl:comoro
-en,dim,dimbokro,dimbokro,dimbokro/ci
-en,din,dien bien phu,dien bien phu,dien bien/vn:dien bien airport
-en,dio,diomede island,diomede island,diomede island/ak/us
-en,dip,diapaga,diapaga,diapaga/bf
-en,diq,divinopolis,divinopolis,divinopolis/mg/br
-en,dir,dire dawa,dire dawa,dire dawa/et:aba tenna d yilma
-en,dis,loubomo,loubomo,loubomo/cg
-en,diu,diu,diu,diu/in
-en,div,divo,divo,divo/ci
-en,diw,dickwella mawella,dickwella mawel,dickwella/lk:mawella lagoon
-en,diy,diyarbakir,diyarbakir,diyarbakir/tr
-en,dja,djougou,djougou,djougou/bj
-en,djb,jambi,jambi,jambi/id:sultan taha syarifudn
-en,dje,djerba,djerba,djerba/tn:melita
-en,djg,djanet,djanet,djanet/dz:inedbirenne
-en,djj,jayapura,jayapura,jayapura/id:sentani
-en,djm,djambala,djambala,djambala/cg
-en,djn,delta junction,delta junction,delta junction/ak/us
-en,djo,daloa,daloa,daloa/ci
-en,dju,djupivogur,djupivogur,djupivogur/is
-en,dki,dunk island,dunk island,dunk island/ql/au
-en,dkk,dunkirk,dunkirk,dunkirk/ny/us
-en,dkr,dakar,dakar,dakar/sn:yoff
-en,dks,dikson,dikson,dikson/ru
-en,anu,antigua,antigua,antigua/ag:vc bird intl
-en,anv,anvik,anvik,anvik/ak/us
-en,anw,ainsworth,ainsworth,ainsworth/ne/us
-en,cey,murray,murray,murray/ky/us:calloway county
-en,cez,cortez,cortez,cortez/co/us:montezuma county
-en,cfa,coffee point,coffee point,coffee point/ak/us
-en,cfb,cabo frio,cabo frio,cabo f/rj/br:cabo frio airport
-en,cfc,cacador,cacador,cacador/sc/br
-en,cfd,bryan,bryan,bryan/tx/us:coulter field
-en,cfe,clermont ferrand,clermont ferran,clermont ferrand/fr:aulnat
-en,cff,cafunfo,cafunfo,cafunfo/ao
-en,cfg,cienfuegos,cienfuegos,cienfuegos/cu
-en,cfh,clifton hills,clifton hills,clifton hills/sa/au
-en,cfi,camfield,camfield,camfield/nt/au
-en,cfk,chlef,chlef,chlef/dz:abou bakr belkaid
-en,cfn,donegal,donegal,donegal/ie
-en,cfo,confreza,confreza,confreza/mt/br
-en,cfp,carpentaria downs,carpentaria dow,carpentaria downs/ql/au
-en,cfq,creston,creston,creston/bc/ca
-en,cfr,caen,caen,caen/fr:carpiquet
-en,cfs,coffs harbour,coffs harbour,coffs harbour/ns/au
-en,cft,clifton,clifton,clifton/az/us:morenci
-en,cfu,kerkyra/i kapodi,kerkyra/i kapod,kerkyra/gr:i kapodistrais
-en,cfv,coffeyville,coffeyville,coffeyville/ks/us:municipal
-en,cga,craig,craig,craig/ak/us:craig spb
-en,cgb,cuiaba,cuiaba,cuiaba/mt/br:m rondon
-en,cgc,cape gloucester,cape gloucester,cape gloucester/pg
-en,cgd,changde,changde,changde/cn
-en,cge,cambridge,cambridge,cambridge/md/us
-en,cgf,cleveland cgf,cleveland cgf,cleveland/oh/us:cuyahoga
-en,cgg,casiguran,casiguran,casiguran/ph
-en,cgh,sao paulo cgh,sao paulo cgh,sao paulo/sp/br:congonhas
-en,cgi,cape girardeau,cape girardeau,cape girardeau/mo/us
-en,cgj,chingola,chingola,chingola/zm
-en,cgk,jakarta intl,jakarta intl,jakarta/id:soekarno hatta intl
-en,cgm,camiguin,camiguin,camiguin/ph:mambajao
-en,cgn,cologne/bonn,cologne/bonn,cologne/de:koeln/bonn
-en,cgo,zhengzhou,zhengzhou,zhengzhou/cn
-en,cgp,chittagong,chittagong,chittagong/bd:patenga
-en,cgq,changchun,changchun,changchun/cn
-en,cgr,campo grande,campo grande,campo grande/ms/br:intl
-en,cgs,college park,college park,college park/md/us
-en,cgt,chinguitti,chinguitti,chinguitti/mr
-en,cgu,ciudad guayana,ciudad guayana,ciudad guayana/ve
-en,cgv,caiguna,caiguna,caiguna/wa/au
-en,cgx,chicago cgx,chicago cgx,chicago/il/us:merrill c meigs
-en,cgy,cagayan de oro,cagayan de oro,cagayan de oro/ph:lumbia
-en,cgz,casa grande,casa grande,casa grande/az/us:municipal
-en,cha,chattanooga,chattanooga,chattanooga/tn/us:lovell field
-en,chb,chilas,chilas,chilas/pk
-en,chc,christchurch,christchurch,christchurch/nz
-en,che,caherciveen,caherciveen,caherciveen/ie:reenroe
-en,chf,jinhae,jinhae,jinhae/kr
-en,chg,chaoyang,chaoyang,chaoyang/cn:chaoyang airport
-en,chh,chachapoyas,chachapoyas,chachapoyas/pe
-en,chi,chicago,chicago,chicago/il/us
-en,chj,chipinge,chipinge,chipinge/zw
-en,chk,chickasha,chickasha,chickasha/ok/us:municipal
-en,chl,challis,challis,challis/id/us
-en,chm,chimbote,chimbote,chimbote/pe
-en,chn,jeonju,jeonju,jeonju/kr
-en,cho,charlottesville,charlottesville,charlottesville/va/us:albemarl
-en,chp,circle hot spring,circle hot spri,circle hot spring/ak/us
-en,chq,chania,chania,chania/gr:souda
-en,chr,chateauroux,chateauroux,chateauroux/fr
-en,chs,charleston,charleston,charleston/sc/us
-en,cht,chatham island,chatham island,chatham island/nz:karewa
-en,chu,chuathbaluk,chuathbaluk,chuathbaluk/ak/us
-en,chv,chaves,chaves,chaves/pt
-en,chw,jiuquan,jiuquan,jiuquan/cn
-en,chx,changuinola,changuinola,changuinola/pa
-en,chy,choiseul bay,choiseul bay,choiseul bay/sb
-en,fkl,franklin,franklin,franklin/pa/us:chess lambertin
-en,fkn,franklin,franklin,franklin/va/us:municipal
-en,efw,jefferson,jefferson,jefferson/ia/us:municipal
-en,ega,engati,engati,engati/pg
-en,egc,bergerac,bergerac,bergerac/fr:roumanieres
-en,ege,vail eagle,vail eagle,vail eagle/co/us:eagle county
-en,egi,valparaiso duke f,valparaiso duke,valparaiso/fl/us:duke field
-en,egl,neghelli,neghelli,neghelli/et
-en,egm,sege,sege,sege/sb
-en,egn,geneina,geneina,geneina/sd
-en,ego,belgorod,belgorod,belgorod/ru
-en,egp,eagle pass,eagle pass,eagle pass/tx/us:maverick co
-en,egs,egilsstadir,egilsstadir,egilsstadir/is
-en,egv,eagle river,eagle river,eagle river/wi/us
-en,egx,egegik,egegik,egegik/ak/us
-en,ehd,pseudo city code,pseudo city cod,pseudo city code/zz
-en,ehl,el bolson,el bolson,el bolson/rn/ar
-en,ehm,cape newenham,cape newenham,cape newenham/ak/us
-en,eht,east hartford,east hartford,east hartford/ct/us:rentschler
-en,eia,eia,eia,eia/pg:popondetta
-en,eib,eisenach,eisenach,eisenach/de
-en,eie,eniseysk,eniseysk,eniseysk/ru
-en,eih,einasleigh,einasleigh,einasleigh/ql/au
-en,eik,eisk,eisk,eisk/ru
-en,eil,fairbanks afb,fairbanks afb,fairbanks/ak/us:eielson afb
-en,ein,eindhoven,eindhoven,eindhoven/nl
-en,eis,beef island,beef island,beef island/vg
-en,eiy,ein yahav,ein yahav,ein yahav/il
-en,eja,barrancabermeja,barrancabermeja,barrancabermeja/co:variguies
-en,ejh,wedjh,wedjh,wedjh/sa
-en,ejt,mili atoll,mili atoll,mili atoll/mh:enijet airport
-en,eka,eureka,eureka,eureka/ca/us:murray field
-en,ekb,ekibastuz,ekibastuz,ekibastuz/kz
-en,ekd,elkedra,elkedra,elkedra/nt/au
-en,eke,ekereku,ekereku,ekereku/gy
-en,eki,elkhart,elkhart,elkhart/in/us:elkhart mnpl
-en,ekn,elkins,elkins,elkins/wv/us
-en,eko,elko,elko,elko/nv/us
-en,eks,shakhtersk,shakhtersk,shakhtersk/ru:shakhtersk
-en,ekt,eskilstuna,eskilstuna,eskilstuna/se
-en,ekx,elizabethtown,elizabethtown,elizabethtown/ky/us
-en,ela,eagle lake,eagle lake,eagle lake/tx/us
-en,elb,el banco,el banco,el banco/co:san bernado
-en,elc,elcho island,elcho island,elcho island/nt/au
-en,eld,el dorado,el dorado,el dorado/ar/us:goodwin field
-en,ele,el real,el real,el real/pa
-en,elf,el fasher,el fasher,el fasher/sd
-en,elg,el golea,el golea,el golea/dz
-en,elh,north eleuthera,north eleuthera,north eleuthera/bs:intl
-en,eli,elim,elim,elim/ak/us
-en,elj,el recreo,el recreo,el recreo/co
-en,elk,elk city,elk city,elk city/ok/us:municipal
-en,ell,ellisras,ellisras,ellisras/za
-en,elm,elmira,elmira,elmira/ny/us
-en,eln,ellensburg,ellensburg,ellensburg/wa/us:bowers field
-en,elo,eldorado,eldorado,eldorado/mi/ar
-en,elp,el paso intl,el paso intl,el paso/tx/us:el paso intl
-en,elq,gassim,gassim,gassim/sa
-en,elr,elelim indonesia,elelim,elelim/id
-en,els,east london,east london,east london/za:east london
-en,elt,tour sinai city,tour sinai city,tour sinai city/eg
-en,elu,el oued,el oued,el oued/dz:guemar
-en,elv,elfin cove,elfin cove,elfin cove/ak/us:elfin cove s
-en,elw,ellamar,ellamar,ellamar/ak/us
-en,elx,el tigre,el tigre,el tigre/ve
-en,ely,ely,ely,ely/nv/us:yelland
-en,elz,wellsville,wellsville,wellsville/ny/us:municipal
-en,ema,east midlands,east midlands,nottingham/gb:east midlands
-en,emb,san francisco emb,san francisco e,san francisco/ca/us:embarkader
-en,emd,emerald,emerald,emerald/ql/au
-en,eme,emden,emden,emden/de
-en,emg,empangeni,empangeni,empangeni/za
-en,emi,emirau,emirau,emirau/pg
-en,emk,emmonak,emmonak,emmonak/ak/us
-en,eml,emmen,emmen,emmen/ch:emmen
-en,emm,kemerer,kemerer,kemerer/wy/us
-en,gog,gobabis,gobabis,gobabis/na
-en,goh,nuuk,nuuk,nuuk/gl
-en,goi,goa,goa,goa/in:dabolim
-en,goj,nizhniy novgorod,nizhniy novgoro,nizhniy novgorod/ru
-en,gok,guthrie,guthrie,guthrie/ok/us
-en,gol,gold beach,gold beach,gold beach/or/us:state
-en,gom,goma,goma,goma/cd
-en,gon,groton,groton,groton/ct/us:groton new london
-en,goo,goondiwindi,goondiwindi,goondiwindi/ql/au
-en,gop,gorakhpur,gorakhpur,gorakhpur/in
-en,goq,golmud,golmud,golmud/cn
-en,gor,gore,gore,gore/et
-en,gos,gosford,gosford,gosford/ns/au
-en,got,gothenburg got,gothenburg got,gothenburg/se:landvetter
-en,gou,garoua,garoua,garoua/cm
-en,gov,gove,gove,gove/nt/au:nhulunbuy
-en,goy,gal oya,gal oya,gal oya/lk:amparai
-en,goz,gorna orechovitsa,gorna orechovit,gorna orechovitsa/bg
-en,gpa,patras,patras,patras/gr:araxos airport
-en,gpb,guarapuava,guarapuava,guarapuava/pr/br:tancredo
-en,gpi,guapi,guapi,guapi/co
-en,gpl,guapiles,guapiles,guapiles/cr
-en,gpn,garden point,garden point,garden point/nt/au
-en,gpo,general pico,general pico,general pico/lp/ar
-en,gps,galapagos is,galapagos is,galapagos is/ec:baltra
-en,gpt,gulfport,gulfport,gulfport/ms/us:biloxi regional
-en,gpz,grand rapids,grand rapids,grand rapids/mn/us
-en,gqj,machrihanish,machrihanish,machrihanish/gb:raf station
-en,gqq,galion,galion,galion/oh/us
-en,gra,gamarra,gamarra,gamarra/co
-en,grb,green bay,green bay,green bay/wi/us:austin field
-en,grc,grand cess,grand cess,grand cess/lr
-en,grd,greenwood,greenwood,greenwood/sc/us
-en,gre,greenville,greenville,greenville/il/us:municipal
-en,grf,tacoma gray aaf,tacoma gray aaf,tacoma/wa/us:gray aaf
-en,grg,gardez,gardez,gardez/af
-en,grh,garuahi,garuahi,garuahi/pg
-en,gri,grand island,grand island,grand island/ne/us
-en,grj,george,george,george/za
-en,grk,killeen gray aaf,killeen gray aa,killeen/tx/us:gray aaf
-en,grl,garasa,garasa,garasa/pg
-en,grm,grand marais,grand marais,grand marais/mn/us:devils trck
-en,grn,gordon,gordon,gordon/ne/us
-en,gro,girona,girona,girona/es:costa brava
-en,grp,gurupi,gurupi,gurupi/go/br
-en,grq,groningen,groningen,groningen/nl:eelde
-en,grr,grand rapids,grand rapids,grand rapids/mi/us:gerald r fo
-en,grs,grosseto,grosseto,grosseto/it:baccarini
-en,grt,gujrat,gujrat,gujrat/pk
-en,gru,sao paulo gru,sao paulo gru,sao paulo/sp/br:guarulhos intl
-en,grv,groznyj,groznyj,groznyj/ru
-en,grw,graciosa island,graciosa island,graciosa island/pt
-en,grx,granada,granada,granada/es
-en,gry,grimsey,grimsey,grimsey/is
-en,grz,graz,graz,graz/at:thalerhof
-en,gsa,long pasia,long pasia,long pasia/my
-en,gsb,goldsboro,goldsboro,goldsboro/nc/us:seymour johns
-en,gsc,gascoyne junction,gascoyne juncti,gascoyne junction/wa/au
-en,gse,gothenburg gse,gothenburg gse,gothenburg gse/se:saeve
-en,gsh,goshen,goshen,goshen/in/us
-en,gsi,guadalcanal,guadalcanal,guadalcanal/sb
-en,gsl,taltheilei,taltheilei,taltheilei/nt/ca
-en,gsm,gheshm,gheshm,gheshm/ir
-en,gsn,mount gunson,mount gunson,mount gunson/sa/au
-en,gso,greensboro,greensboro,greensboro/high point/nc/us:pi
-en,gsp,greenvl spartanbg,greenvl spartan,greenville/sc/us:spartanburg
-en,gsq,shark alowainat,shark alowainat,shark alowainat/eg
-en,gsr,gardo,gardo,gardo/so
-en,gss,sabi sabi,sabi sabi,sabi sabi/za
-en,gst,gustavus airport,gustavus airpor,gustavus/ak/us:gustavus
-en,gsu,gedaref,gedaref,gedaref/sd
-en,gsy,grimsby,grimsby,grimsby/gb:binbrook
-en,gta,gatokae,gatokae,gatokae/sb:aerodrom
-en,gtb,genting,genting,genting/my
-en,anx,andenes,andenes,andenes/no
-en,any,anthony,anthony,anthony/ks/us
-en,anz,angus downs,angus downs,angus downs/nt/au
-en,aoa,aroa,aroa,aroa/pg
-en,aob,annanberg,annanberg,annanberg/pg
-en,aoc,altenburg,altenburg,altenburg/de:altenburg nobitz
-en,aod,abou deia,abou deia,abou deia/td
-en,aoe,anadolu ubiv apt,anadolu ubiv ap,eskisehir/tr:anadolu universit
-en,aog,anshan,anshan,anshan/cn
-en,aoh,lima,lima,lima/oh/us:allen county
-en,aoi,ancona,ancona,ancona/it:falconara
-en,aoj,aomori,aomori,aomori/jp
-en,aok,karpathos,karpathos,karpathos/gr
-en,aol,paso de los libre,paso de los lib,paso de los libres/cr/ar
-en,aon,arona,arona,arona/pg
-en,aoo,altoona/martinsbg,altoona,altoona/pa/us
-en,aor,alor setar,alor setar,alor setar/my
-en,aos,amook,amook,amook/ak/us
-en,aot,aosta,aosta,aosta/it:corrado gex airport
-en,aou,attopeu,attopeu,attopeu/la
-en,apa,denver arapahoe,denver arapahoe,denver/co/us:arapahoe co
-en,apb,apolo,apolo,apolo/bo
-en,apc,napa,napa,napa/ca/us:napa county
-en,ape,san juan aposento,san juan aposen,san juan aposento/pe
-en,apf,naples,naples,naples/fl/us
-en,apg,aberdeen,aberdeen,aberdeen/md/us:phillips aaf
-en,aph,bowling green,bowling green,bowling green/va/us:camp a p h
-en,api,apiay,apiay,apiay/co
-en,apk,apataki,apataki,apataki/pf
-en,apl,nampula,nampula,nampula/mz
-en,apn,alpena,alpena,alpena/mi/us:county regional
-en,apo,apartado,apartado,apartado/co
-en,app,asapa,asapa,asapa/pg
-en,apq,arapiraca,arapiraca,arapiraca/al/br
-en,apr,april river,april river,april river/pg
-en,aps,anapolis,anapolis,anapolis/go/br
-en,apt,jasper,jasper,jasper/tn/us:marion county
-en,apu,apucarana,apucarana,apucarana/pr/br
-en,apv,apple valley,apple valley,apple valley/ca/us
-en,apw,apia faleolo,apia faleolo,apia/ws:faleolo
-en,apx,arapongas,arapongas,arapongas/pr/br
-en,apy,alto parnaiba,alto parnaiba,alto parnaiba/ma/br
-en,apz,zapala,zapala,zapala/ne/ar
-en,aqa,araraquara,araraquara,araraquara/sp/br
-en,aqb,quiche,quiche,quiche/gt:quiche
-en,aqg,anqing,anqing,anqing/cn
-en,aqi,qaisumah,qaisumah,qaisumah/sa
-en,aqj,aqaba,aqaba,aqaba/jo:king hussein intl
-en,aqm,ariquemes,ariquemes,ariquemes/ro/br
-en,aqp,arequipa,arequipa,arequipa/pe:rodriguez ballon
-en,aqs,saqani,saqani,saqani/fj
-en,aqy,alyeska,alyeska,alyeska/ak/us
-en,ara,new iberia,new iberia,new iberia/la/us:acadiana rgnl
-en,arb,ann arbor,ann arbor,ann arbor/mi/us:municipal
-en,arc,arctic village,arctic village,arctic village/ak/us
-en,ard,alor island,alor island,alor island/id
-en,are,arecibo,arecibo,arecibo/pr
-en,arf,acaricuara,acaricuara,acaricuara/co
-en,arg,walnut ridge,walnut ridge,walnut ridge/ar/us
-en,arh,arkhangelsk,arkhangelsk,arkhangelsk/ru
-en,ari,arica,arica,arica/cl:chacalluta
-en,arj,arso,arso,arso/id
-en,ark,arusha,arusha,arusha/tz
-en,arl,arly,arly,arly/bf
-en,arm,armidale,armidale,armidale/ns/au
-en,arn,stockholm arlanda,stockholm arlan,stockholm/se:arlanda
-en,aro,arboletas,arboletas,arboletas/co
-en,arp,aragip,aragip,aragip/pg
-en,arq,arauquita,arauquita,arauquita/co
-en,arr,alto rio senguerr,alto rio sengue,alto rio senguerr/cb/ar
-en,ars,aragarcas,aragarcas,aragarcas/go/br
-en,art,watertown,watertown,watertown/ny/us
-en,aru,aracatuba,aracatuba,aracatuba/sp/br
-en,hus,hughes,hughes,hughes/ak/us:municipal
-en,hut,hutchinson,hutchinson,hutchinson/ks/us
-en,huu,huanuco,huanuco,huanuco/pe
-en,fkq,fak fak,fak fak,fak fak/id
-en,fks,fukushima,fukushima,fukushima/jp:airport
-en,fla,florencia,florencia,florencia/co:capitolio
-en,flb,floriano,floriano,floriano/pi/br:cangapara
-en,flc,falls creek,falls creek,falls creek/vi/au
-en,fld,fond du lac,fond du lac,fond du lac/wi/us
-en,fle,petersburg aaf,petersburg aaf,petersburg/va/us:fort lee aaf
-en,flf,flensburg,flensburg,flensburg/de:schaferhaus
-en,flg,flagstaff,flagstaff,flagstaff/az/us:pulliam field
-en,flh,flotta,flotta,flotta/gb
-en,fli,flateyri,flateyri,flateyri/is
-en,flj,falls bay,falls bay,falls bay/ak/us
-en,fll,ft lauderdale,ft lauderdale,ft lauderdale/fl/us:fll intl
-en,flm,filadelfia,filadelfia,filadelfia/py
-en,fln,florianopolis,florianopolis,florianopolis/sc/br:hercilio l
-en,flo,florence,florence,florence/sc/us
-en,flp,flippin,flippin,flippin/ar/us
-en,flr,florence,florence,florence/it:peretola
-en,fls,flinders island,flinders island,flinders island/ts/au
-en,flt,flat,flat,flat/ak/us
-en,flu,new york flushing,new york flushi,new york/ny/us:flushing helipo
-en,flv,fort leavenworth,fort leavenwort,fort leavenworth/ks/us:sherman
-en,flw,flores island/san,flores island,flores island/pt:santa cruz
-en,flx,fallon municipal,fallon municipa,fallon/nv/us:municipal
-en,fly,finley,finley,finley/ns/au
-en,fma,formosa,formosa,formosa/fo/ar:el pucu
-en,fmc,five mile,five mile,five mile/ak/us
-en,fme,fort meade,fort meade,fort meade/md/us:tipton aaf
-en,fmg,flamingo,flamingo,flamingo/cr
-en,fmh,falmouth,falmouth,falmouth/ma/us:otis afb
-en,fmi,kalemie,kalemie,kalemie/cd
-en,fmm,memmingen,memmingen,memmingen/de:allgaeu
-en,fmn,farmington,farmington,farmington/nm/us:mnpl
-en,fmo,muenster,muenster,muenster/de
-en,fms,fort madison,fort madison,fort madison/ia/us:municipal
-en,fmu,florence mnpl,florence mnpl,florence/or/us:municipal
-en,fmy,fort myers fmy,fort myers fmy,fort myers/fl/us:page field
-en,fna,freetown fna,freetown fna,freetown/sl:lungi intl
-en,fnb,neubrandenburg,neubrandenburg,neubrandenburg/de
-en,fnc,madeira,madeira,madeira/pt:madeira
-en,fne,fane,fane,fane/pg
-en,fng,fada ngourma,fada ngourma,fada ngourma/bf
-en,fnh,fincha,fincha,fincha/et
-en,fni,nimes,nimes,nimes/fr:garons
-en,fnj,pyongyang,pyongyang,pyongyang/kp:sunan
-en,fnk,fin creek,fin creek,fin creek/ak/us
-en,fnl,ft collins love,ft collins love,fort collins love/co/us:mncpl
-en,fnr,funter bay,funter bay,funter bay/ak/us:spb
-en,fnt,flint,flint,flint/mi/us:bishop
-en,foa,foula,foula,foula/gb
-en,fob,fort bragg,fort bragg,fort bragg/ca/us
-en,foc,fuzhou,fuzhou,fuzhou/cn
-en,fod,fort dodge,fort dodge,fort dodge/ia/us
-en,foe,topeka forbes,topeka forbes,topeka forbes/ks/us:forbes afb
-en,fog,foggia,foggia,foggia/it:gino lisa
-en,fok,westhampton,westhampton,westhampton/ny/us:suffolk cnty
-en,fom,foumban,foumban,foumban/cm
-en,fon,fortuna,fortuna,fortuna/cr:fortuna airport
-en,foo,numfoor,numfoor,numfoor/id
-en,fop,forest park,forest park,forest park/ga/us:morris aaf
-en,for,fortaleza,fortaleza,fortaleza/ce/br:pinto martins
-en,fos,forrest,forrest,forrest/wa/au
-en,fot,forster,forster,forster/ns/au
-en,fou,fougamou,fougamou,fougamou/ga
-en,fox,fox,fox,fox/ak/us
-en,foy,foya,foya,foya/lr
-en,fpo,freeport,freeport,freeport/bs:grand bahama intl
-en,fpr,fort pierce,fort pierce,fort pierce/fl/us:st lucie cnt
-en,fpy,perry,perry,perry/fl/us:perry foley
-en,bcw,benguera island,benguera island,benguera island/mz
-en,bcx,beloreck,beloreck,beloreck/ru
-en,bcy,bulchi,bulchi,bulchi/et
-en,lar,laramie,laramie,laramie/wy/us:general brees
-en,las,las vegas las,las vegas las,las vegas/nv/us:mccarran intl
-en,lat,la uribe,la uribe,la uribe/co
-en,lau,lamu,lamu,lamu/ke
-en,lav,lalomalava,lalomalava,lalomalava/ws
-en,law,lawton,lawton,lawton/ok/us:lawton mnpl
-en,lax,los angeles,los angeles,los angeles/ca/us:intl,los angeles
-en,lay,ladysmith,ladysmith,ladysmith/za
-en,laz,bom jesus da lapa,bom jesus da la,bom jesus da lapa/ba/br
-en,lba,leeds,leeds,leeds/gb:leeds bradford
-en,lbb,lubbock intl,lubbock intl,lubbock/tx/us:lubbock intl
-en,lbc,luebeck,luebeck,hamburg/de:luebeck
-en,lbd,khudzhand,khudzhand,khudzhand/tj
-en,lbe,latrobe,latrobe,latrobe/pa/us:westmoreland cnt
-en,lbf,north platte,north platte,north platte/ne/us:lee bird fl
-en,lbg,paris le bourget,paris le bourge,paris le bourget/fr:le bourget
-en,lbh,sydney palm beach,sydney palm bea,sydney/ns/au:palm beach spb
-en,lbi,albi,albi,albi/fr:le sequestre
-en,lbj,labuan bajo,labuan bajo,labuan bajo/id:mutiara
-en,lbk,liboi,liboi,liboi/ke
-en,lbl,liberal,liberal,liberal/ks/us:municipal
-en,lbm,luabo,luabo,luabo/mz
-en,lbn,lake baringo,lake baringo,lake baringo/ke
-en,lbo,lusambo,lusambo,lusambo/cd
-en,lbp,long banga,long banga,long banga/my:airfield
-en,lbq,lambarene,lambarene,lambarene/ga
-en,lbr,labrea,labrea,labrea/am/br
-en,lbs,labasa,labasa,labasa/fj
-en,lbt,lumberton,lumberton,lumberton/nc/us
-en,lbu,labuan,labuan,labuan/my
-en,lbv,libreville,libreville,libreville/ga:leon m ba
-en,lbw,long bawan,long bawan,long bawan/id
-en,lbx,lubang,lubang,lubang/ph
-en,lby,la baule,la baule,la baule/fr:montoir
-en,lbz,lukapa,lukapa,lukapa/ao
-en,lca,larnaca,larnaca,larnaca/cy
-en,lcb,pontes e lacerda,pontes e lacerd,pontes e lacerda/mt/br
-en,lcc,lecce,lecce,lecce/it:galatina
-en,lcd,louis trichardt,louis trichardt,louis trichardt/za
-en,lce,la ceiba,la ceiba,la ceiba/hn:goloson intl
-en,lcf,rio dulce,rio dulce,rio dulce/gt:las vegas airport
-en,lcg,la coruna,la coruna,la coruna/es
-en,lch,lake charles,lake charles,lake charles/la/us:mnpl
-en,lci,laconia,laconia,laconia/nh/us:municipal
-en,lcj,lodz,lodz,lodz/pl:lodz lublinek airport
-en,lck,columbus lck,columbus lck,columbus/oh/us:rickenbacker
-en,lcl,la coloma,la coloma,la coloma/cu
-en,lcm,la cumbre,la cumbre,la cumbre/cd/ar
-en,lcn,balcanoona,balcanoona,balcanoona/sa/au
-en,lco,lague,lague,lague/cg
-en,lcp,loncopue,loncopue,loncopue/ne/ar
-en,lcr,la chorrera,la chorrera,la chorrera/co
-en,lcs,las canas,las canas,las canas/cr
-en,lcv,lucca,lucca,lucca/it
-en,lcx,longyan,longyan,longyan/cn:liancheng apt
-en,lcy,london city,london city,london/gb:london city
-en,lda,malda,malda,malda/in
-en,ldb,londrina,londrina,londrina/pr/br
-en,ldc,lindeman island,lindeman island,lindeman island/ql/au
-en,lde,lourdes tarbes,lourdes tarbes,lourdes/fr:lourdes tarbes intl
-en,ldg,leshukonskoye,leshukonskoye,leshukonskoye/ru:leshukonskoye
-en,ldh,lord howe island,lord howe islan,lord howe island/ns/au
-en,ldi,lindi,lindi,lindi/tz:kikwetu
-en,ldj,linden,linden,linden/nj/us
-en,ldk,lidkoping,lidkoping,lidkoping/se:hovby
-en,ldm,ludington,ludington,ludington/mi/us:mason county
-en,ldn,lamidanda,lamidanda,lamidanda/np
-en,ldo,ladouanie,ladouanie,ladouanie/sr
-en,ldr,lodar,lodar,lodar/ye
-en,lds,fictitious,fictitious,fictitious/zz
-en,ldu,lahad datu,lahad datu,lahad datu/my
-en,ldv,landivisiau,landivisiau,landivisiau/fr
-en,ldw,lansdowne,lansdowne,lansdowne/wa/au
-en,ldx,st laurent du mar,st laurent du m,st laurent du maroni/gf
-en,lyt,lady elliot islan,lady elliot isl,lady elliot islan/ql/au
-en,lyu,ely,ely,ely/mn/us
-en,lyx,lydd ashford,lydd ashford,lydd/gb:london ashford
-en,lza,luiza,luiza,luiza/cd
-en,lzc,lazaro cardenas,lazaro cardenas,lazaro cardenas/mx
-en,lzd,lanzhou lzd,lanzhou lzd,lanzhou/cn:lanzhoudong
-en,lzh,liuzhou,liuzhou,liuzhou/cn
-en,lzi,luozi,luozi,luozi/cd
-en,lzm,luzamba,luzamba,luzamba/ao
-en,lzn,nangan,nangan,nangan/cn
-en,lzo,luzhou,luzhou,luzhou/cn
-en,lzr,lizard island,lizard island,lizard island/ql/au
-en,lzy,lin zhi,lin zhi,lin zhi/cn
-en,maa,chennai,chennai,chennai/in
-en,mab,maraba,maraba,maraba/pa/br
-en,mac,macon smart,macon smart,macon/ga/us:smart
-en,mad,madrid barajas,madrid barajas,madrid/es:barajas
-en,mae,madera,madera,madera/ca/us
-en,maf,midland odessa rg,midland odessa,midland/tx/us:odessa
-en,mag,madang,madang,madang/pg
-en,mah,menorca,menorca,menorca/es
-en,mai,mangochi,mangochi,mangochi/mw
-en,maj,majuro,majuro,majuro/mh:amata kabua intl
-en,mak,malakal,malakal,malakal/sd
-en,mal,mangole,mangole,mangole/id
-en,mam,matamoros,matamoros,matamoros/mx
-en,man,manchester int,manchester int,manchester/gb:manchester intl, manchester, manchester, manchester, man, man
-en,mao,manaus,manaus,manaus/am/br:eduardo gomes
-en,map,mamai,mamai,mamai/pg
-en,maq,mae sot,mae sot,mae sot/th
-en,mar,maracaibo,maracaibo,maracaibo/ve:la chinita
-en,mas,manus island,manus island,manus island/pg:momote
-en,mat,matadi,matadi,matadi/cd
-en,mau,maupiti,maupiti,maupiti/pf
-en,mav,maloelap island,maloelap island,maloelap island/mh
-en,maw,malden,malden,malden/mo/us
-en,max,matam,matam,matam/sn
-en,may,mangrove cay,mangrove cay,mangrove cay/bs
-en,maz,mayaguez de hosto,mayaguez de hos,mayaguez/pr:de hostos
-en,mba,mombasa,mombasa,mombasa/ke:moi intl
-en,mbb,marble bar,marble bar,marble bar/wa/au
-en,mbc,mbigou,mbigou,mbigou/ga
-en,mbd,mmabatho,mmabatho,mmabatho/za:intl
-en,mbe,monbetsu,monbetsu,monbetsu/jp
-en,mbf,mount buffalo,mount buffalo,mount buffalo/vi/au
-en,mbg,mobridge,mobridge,mobridge/sd/us
-en,mbh,maryborough,maryborough,maryborough/ql/au
-en,mbi,mbeya,mbeya,mbeya/tz
-en,mbj,montego bay,montego bay,montego bay/jm:sangster intl
-en,mbk,matupa,matupa,matupa/mt/br
-en,mbl,manistee,manistee,manistee/mi/us:blacker
-en,mbm,mkambati,mkambati,mkambati/za
-en,mbn,mt barnett,mt barnett,mt barnett/wa/au
-en,mbo,mamburao,mamburao,mamburao/ph
-en,mbp,moyobamba,moyobamba,moyobamba/pe
-en,mbq,mbarara,mbarara,mbarara/ug
-en,mbr,mbout,mbout,mbout/mr
-en,mbs,saginaw baycity,saginaw baycity,saginaw/mi/us:tri city
-en,mbt,masbate,masbate,masbate/ph
-en,mbu,mbambanakira,mbambanakira,mbambanakira/sb
-en,mbv,masa,masa,masa/pg
-en,mbw,moorabbin,moorabbin,moorabbin/vi/au
-en,mbx,maribor,maribor,maribor/si
-en,mby,moberly,moberly,moberly/mo/us
-en,mbz,maues,maues,maues/am/br
-en,mca,macenta,macenta,macenta/gn
-en,mcb,mccomb,mccomb,mccomb/ms/us:pike county
-en,mcc,sacramento mcc,sacramento mcc,sacramento/ca/us:mcclellan afb
-en,mcd,mackinac island,mackinac island,mackinac island/mi/us
-en,mce,macready regional,macready region,merced/ca/us:macready regional
-en,mcf,tampa afb,tampa afb,tampa/fl/us:mac dill afb
-en,mcg,mcgrath,mcgrath,mcgrath/ak/us
-en,mch,machala,machala,machala/ec
-en,mci,kansas city int,kansas city int,kansas city/mo/us:intl
-en,mcj,maicao,maicao,maicao/co
-en,mck,mccook,mccook,mccook/ne/us
-en,huv,hudiksvall,hudiksvall,hudiksvall/se
-en,hux,huatulco,huatulco,huatulco/mx
-en,huy,humberside,humberside,humberside/gb:humberside
-en,huz,huizhou,huizhou,huizhou/cn
-en,hva,analalava,analalava,analalava/mg
-en,hvb,hervey bay,hervey bay,hervey bay/ql/au
-en,hvd,khovd,khovd,khovd/mn:khovd
-en,hve,hanksville,hanksville,hanksville/ut/us:intermediate
-en,hvg,honningsvag,honningsvag,honningsvag/no:valan
-en,hvk,holmavik,holmavik,holmavik/is
-en,hvm,hvammstangi,hvammstangi,hvammstangi/is
-en,hvn,new haven,new haven,new haven/ct/us
-en,hvr,havre,havre,havre/mt/us:city county
-en,hvs,hartsville,hartsville,hartsville/sc/us:municipal
-en,hwa,hawabango,hawabango,hawabango/pg
-en,hwd,hayward,hayward,hayward/ca/us:air terminal
-en,hwi,hawk inlet,hawk inlet,hawk inlet/ak/us:spb
-en,hwk,hawker,hawker,hawker/sa/au:wilpena pound
-en,hwn,hwange nat park,hwange nat park,hwange nat park/zw
-en,hwo,hollywood,hollywood,hollywood/fl/us:north perry
-en,hxx,hay,hay,hay/ns/au
-en,hya,hyannis,hyannis,hyannis/ma/us:barnstable
-en,hyc,high wycombe,high wycombe,high wycombe/gb
-en,hyd,shamshabad,shamshabad,shamshabad/in:rajiv gandhi int
-en,hyf,hayfields,hayfields,hayfields/pg
-en,hyg,hydaburg,hydaburg,hydaburg/ak/us:spb
-en,hyl,hollis,hollis,hollis/ak/us:spb
-en,hyn,huangyan,huangyan,huangyan/cn
-en,hyr,hayward,hayward,hayward/wi/us:municipal
-en,hys,hays,hays,hays/ks/us:municipal
-en,hyv,hyvinkaa,hyvinkaa,hyvinkaa/fi:hyvinkaa
-en,hzb,hazebrouck,hazebrouck,hazebrouck/fr:merville calonne
-en,hzg,hanzhong,hanzhong,hanzhong/cn
-en,hzh,liping,liping,liping city/cn:liping
-en,hzk,husavik,husavik,husavik/is
-en,hzl,hazleton,hazleton,hazleton/pa/us
-en,hzp,fort mackay,fort mackay,fort mackay/ab/ca:horizon
-en,hzv,hazyview,hazyview,hazyview/za
-en,iaa,igarka,igarka,igarka/ru
-en,iab,wichita afb,wichita afb,wichita/ks/us:mcconnell afb
-en,iad,washington dulles,washington dull,washington/dc/us:dulles intl
-en,iag,niagara falls,niagara falls,niagara falls/ny/us:intl
-en,iah,houston-iah,houston-iah,houston/tx/us:g.bush intercont
-en,iam,in amenas,in amenas,in amenas/dz
-en,ian,kiana,kiana,kiana/ak/us:bob barker mem
-en,iaq,bahregan,bahregan,bahregan/ir
-en,iar,yaroslavl,yaroslavl,yaroslavl/ru
-en,ias,iasi,iasi,iasi/ro
-en,iat,iata traffic serv,iata traffic se,iata traffic serv/zz
-en,iau,iaura,iaura,iaura/pg
-en,iba,ibadan,ibadan,ibadan/ng
-en,ibe,ibague,ibague,ibague/co
-en,ibi,iboki,iboki,iboki/pg
-en,ibl,indigo bay lodge,indigo bay lodg,indigo bay lodge/mz
-en,ibo,ibo,ibo,ibo/mz
-en,ibp,iberia,iberia,iberia/pe
-en,ibz,ibiza,ibiza,ibiza/es
-en,ica,icabaru,icabaru,icabaru/ve
-en,ici,cicia,cicia,cicia/fj
-en,ick,nieuw nickerie,nieuw nickerie,nieuw nickerie/sr
-en,icl,clarinda,clarinda,clarinda/ia/us:municipal
-en,icn,seoul incheon int,seoul incheon i,seoul/kr:incheon international
-en,ico,sicogon island,sicogon island,sicogon island/ph
-en,icr,nicaro,nicaro,nicaro/cu
-en,ics,cascade,cascade,cascade/id/us
-en,ict,wichita mid cont,wichita mid con,wichita/ks/us:mid continent
-en,icy,icy bay,icy bay,icy bay/ak/us
-en,ida,idaho falls,idaho falls,idaho falls/id/us:fanning fld
-en,idb,idre,idre,idre/se
-en,idc,iladachilonzuene,iladachilonzuen,ila da chilonzuene/mz
-en,idf,idiofa,idiofa,idiofa/cd
-en,idg,ida grove,ida grove,ida grove/ia/us:municipal
-en,arv,minocqua,minocqua,minocqua/wi/us:noble f lee
-en,arw,arad,arad,arad/ro
-en,arx,asbury park,asbury park,asbury park/nj/us
-en,bcz,bickerton island,bickerton islan,bickerton island/nt/au
-en,bda,bermuda,bermuda,bermuda/bm:bermuda intl
-en,bdb,bundaberg,bundaberg,bundaberg/ql/au
-en,bdc,barra do corda,barra do corda,barra do corda/ma/br
-en,bdd,badu island,badu island,badu island/ql/au
-en,bde,baudette,baudette,baudette/mn/us
-en,bdf,bradford,bradford,bradford/il/us:rinkenberger
-en,bdg,blanding,blanding,blanding/ut/us
-en,bdh,bandar lengeh,bandar lengeh,bandar lengeh/ir
-en,bdi,bird island,bird island,bird island/sc
-en,bdj,banjarmasin,banjarmasin,banjarmasin/id:sjamsudin noor
-en,bdk,bondoukou,bondoukou,bondoukou/ci
-en,bdl,hartford/sprngfld,hartford/sprngf,hartford/ct/us:bradley intl
-en,bdm,bandirma,bandirma,bandirma/tr
-en,bdn,badin,badin,badin/pk:talhar
-en,bdo,bandung,bandung,bandung/id:husein sastranegara
-en,bdp,bhadrapur,bhadrapur,bhadrapur/np
-en,bdq,vadodara,vadodara,vadodara/in
-en,bdr,bridgeport,bridgeport,bridgeport/ct/us:i sikorsky
-en,bds,brindisi,brindisi,brindisi/it:papola casale
-en,bdt,gbadolite,gbadolite,gbadolite/cd
-en,bdu,bardufoss,bardufoss,bardufoss/no
-en,bdv,moba,moba,moba/cd
-en,bdw,bedford downs,bedford downs,bedford downs/wa/au
-en,bdx,broadus,broadus,broadus/mt/us
-en,bdy,bandon,bandon,bandon/or/us:state
-en,bdz,baindoung,baindoung,baindoung/pg
-en,bea,bereina,bereina,bereina/pg
-en,beb,benbecula,benbecula,benbecula/gb
-en,bec,wichita beech,wichita beech,wichita beech/ks/us:beech
-en,bed,bedford hanscom,bedford hanscom,bedford ha/ma/us:hanscom field
-en,bee,beagle bay,beagle bay,beagle bay/wa/au
-en,bef,bluefields,bluefields,bluefields/ni
-en,beg,belgrade,belgrade,belgrade/rs:beograd
-en,beh,benthnhbr/stjosep,benton,benton harbor/mi/us:ross field
-en,bei,beica,beica,beica/et
-en,bej,berau,berau,berau/id
-en,bek,bareli,bareli,bareli/in
-en,bel,belem,belem,belem/pa/br:val de cans
-en,bem,bossembele,bossembele,bossembele/cf
-en,ben,benghazi,benghazi,benghazi/ly:benina intl
-en,beo,newcastle beo,newcastle beo,newcastle/ns/au:belmont
-en,bep,bellary,bellary,bellary/in
-en,beq,bury st edmunds,bury st edmunds,bury st edmunds/gb:honington
-en,ber,berlin,berlin,berlin/de
-en,bes,brest,brest,brest/fr:guipavas
-en,bet,bethel airport,bethel airport,bethel/ak/us:bethel airport
-en,beu,bedourie,bedourie,bedourie/ql/au
-en,bev,beer sheba,beer sheba,beer sheba/il
-en,bew,beira,beira,beira/mz
-en,bex,benson,benson,benson/gb:raf station
-en,bey,beirut,beirut,beirut/lb:international
-en,bez,beru,beru,beru/ki
-en,bfa,bahia negra,bahia negra,bahia negra/py:bahia negra
-en,bfb,blue fox bay,blue fox bay,blue fox bay/ak/us
-en,bfc,bloomfield,bloomfield,bloomfield/ql/au
-en,bfd,bradford,bradford,bradford/pa/us
-en,bfe,bielefeld,bielefeld,bielefeld/de:bielefeld
-en,bff,scottsbluff,scottsbluff,scottsbluff/ne/us:scotts bluff
-en,bfg,bullfrog basin,bullfrog basin,bullfrog basin/ut/us
-en,bfh,curitiba,curitiba,curitiba/pr/br:bacacheri
-en,bfi,seattle bfi,seattle bfi,seattle/wa/us:boeing fld intl
-en,bfj,ba,ba,ba/fj:ba
-en,bfk,denver,denver,denver/co/us:buckley angb
-en,bfl,bakersfield,bakersfield,bakersfield/ca/us:meadows fld
-en,bfm,mobile aerospace,mobile aerospac,mobile/al/us:mob aerospace
-en,bfn,bloemfontein,bloemfontein,bloemfontein/za:intl
-en,bfo,buffalo range,buffalo range,buffalo range/zw
-en,bfp,beaver falls,beaver falls,beaver falls/pa/us
-en,mqt,sawyer intl,sawyer intl,marquette/mi/us:sawyer intl
-en,mqu,mariquita,mariquita,mariquita/co
-en,mqv,mostaganem,mostaganem,mostaganem/dz:mostaganem
-en,bza,bonanza,bonanza,bonanza/ni:san pedro
-en,bzb,bazaruto island,bazaruto island,bazaruto island/mz
-en,bzc,buzios,buzios,buzios/rj/br
-en,bzd,balranald,balranald,balranald/ns/au
-en,bze,belize city,belize city,belize/bz:p.s.w. goldson intl
-en,bzf,benton field,benton field,redding/ca/us:benton field
-en,bzg,bydgoszcz,bydgoszcz,bydgoszcz/pl
-en,bzh,bumi hills,bumi hills,bumi hills/zw:airfield
-en,bzi,balikesir,balikesir,balikesir/tr
-en,bzk,briansk,briansk,briansk/ru
-en,bzl,barisal,barisal,barisal/bd
-en,bzm,bergen op zoom,bergen op zoom,bergen op zoom/nl:woensdrecht
-en,bzn,bozeman,bozeman,bozeman/mt/us:gallatin field
-en,bzo,bolzano bozen,bolzano bozen,bolzano bozen/it:bolzano bozen
-en,bzp,bizant,bizant,bizant/ql/au
-en,bzr,beziers,beziers,beziers/fr:vias
-en,bzs,washington bzs,washington bzs,washington/dc/us:buzzards pt s
-en,bzt,brazoria,brazoria,brazoria/tx/us:hinkles ferry
-en,bzu,buta,buta,buta/cd
-en,bzv,brazzaville,brazzaville,brazzaville/cg:maya maya
-en,bzy,beltsy,beltsy,beltsy/md
-en,bzz,brize norton,brize norton,brize norton/gb:raf station
-en,caa,catacamas,catacamas,catacamas/hn
-en,cab,cabinda,cabinda,cabinda/ao
-en,cac,cascavel,cascavel,cascavel/pr/br
-en,cad,cadillac,cadillac,cadillac/mi/us
-en,cae,columbia met,columbia met,columbia/sc/us:columbia met
-en,caf,carauari,carauari,carauari/am/br
-en,cag,cagliari,cagliari,cagliari/it:elmas
-en,cah,ca mau,ca mau,ca mau/vn
-en,cai,cairo,cairo,cairo/eg:cairo intl
-en,caj,canaima,canaima,canaima/ve
-en,cak,canton akron,canton akron,canton akron/oh/us:akron
-en,cal,campbeltown,campbeltown,campbeltown/gb:machrihanish
-en,cam,camiri,camiri,camiri/bo
-en,can,guangzhou,guangzhou,guangzhou/cn:baiyun
-en,cao,clayton,clayton,clayton/nm/us
-en,cap,cap haitien,cap haitien,cap haitien/ht
-en,caq,caucasia,caucasia,caucasia/co
-en,car,caribou,caribou,caribou/me/us:municipal
-en,cas,casablanca,casablanca,casablanca/ma:anfa
-en,cat,cat island,cat island,cat island/bs
-en,cau,caruaru,caruaru,caruaru/pe/br
-en,cav,cazombo,cazombo,cazombo/ao
-en,caw,campos,campos,campos/rj/br:bartolomeu lisand
-en,cax,carlisle,carlisle,carlisle/gb
-en,cay,cayenne,cayenne,cayenne/gf:rochambeau
-en,caz,cobar,cobar,cobar/ns/au
-en,cba,corner bay,corner bay,corner bay/ak/us
-en,cbb,cochabamba,cochabamba,cochabamba/bo:j wilsterman
-en,cbc,cherrabun,cherrabun,cherrabun/wa/au
-en,cbd,car nicobar,car nicobar,car nicobar/in
-en,cbe,cumberland,cumberland,cumberland/md/us:wiley ford
-en,cbf,council bluffs,council bluffs,council bluffs/ia/us:municipal
-en,cbg,cambridge,cambridge,cambridge/gb
-en,cbh,bechar,bechar,bechar/dz:leger
-en,cbi,cape barren,cape barren,cape barren/ts/au
-en,cbj,cabo rojo,cabo rojo,cabo rojo/do
-en,cbk,colby,colby,colby/ks/us:municipal
-en,cbl,ciudad bolivar,ciudad bolivar,ciudad bolivar/ve
-en,cbm,columbus afb,columbus afb,columbus/ms/us:columbus afb
-en,cbn,cirebon,cirebon,cirebon/id:penggung
-en,cbo,cotabato,cotabato,cotabato/ph:awang
-en,cbp,coimbra,coimbra,coimbra/pt
-en,cbq,calabar,calabar,calabar/ng
-en,cbr,canberra,canberra,canberra/ac/au
-en,cbs,cabimas,cabimas,cabimas/ve:oro negro
-en,cbt,catumbela,catumbela,catumbela/ao
-en,cbu,cottbus,cottbus,cottbus/de:cottbus apt
-en,cbv,coban,coban,coban/gt
-en,cbw,campo mourao,campo mourao,campo mourao/pr/br
-en,cbx,condobolin,condobolin,condobolin/ns/au
-en,cby,canobie,canobie,canobie/ql/au
-en,cbz,cabin creek,cabin creek,cabin creek/ak/us
-en,dkv,docker river,docker river,docker river/nt/au
-en,dla,douala,douala,douala/cm
-en,dlb,dalbertis,dalbertis,dalbertis/pg
-en,dlc,dalian,dalian,dalian/cn
-en,dld,geilo,geilo,geilo/no:dagali airport
-en,dle,dole,dole,dole/fr:tavaux
-en,dlf,del rio afb,del rio afb,del rio/tx/us:laughlin afb
-en,dlg,dillingham,dillingham,dillingham/ak/us:municipal
-en,dlh,duluth int,duluth int,duluth/mn/us:duluth intl
-en,dli,dalat,dalat,dalat/vn:lienkhang
-en,dlk,dulkaninna,dulkaninna,dulkaninna/sa/au
-en,dll,dillon,dillon,dillon/sc/us
-en,dlm,dalaman,dalaman,dalaman/tr
-en,dln,dillon,dillon,dillon/mt/us
-en,dlo,dolomi,dolomi,dolomi/ak/us
-en,dls,the dalles,the dalles,the dalles/or/us
-en,dlu,dali,dali,dali/cn:dali
-en,dlv,delissaville,delissaville,delissaville/nt/au
-en,dly,dillons bay,dillons bay,dillons bay/vu
-en,dlz,dalanzadgad,dalanzadgad,dalanzadgad/mn
-en,dma,tucson afb,tucson afb,tucson/az/us:davis monthan afb
-en,dmb,zhambyl,zhambyl,zhambyl/kz
-en,dmd,doomadgee,doomadgee,doomadgee/ql/au
-en,dme,moscow dme,moscow dme,moscow/ru:domodedovo
-en,dmk,bkk don mueang,bkk don mueang,bangkok/th:don mueang intl
-en,dmm,dammam,dammam,dammam/sa:king fahad intl
-en,dmn,deming,deming,deming/nm/us
-en,dmo,sedalia,sedalia,sedalia/mo/us
-en,dmr,dhamar,dhamar,dhamar/ye
-en,dmt,diamantino,diamantino,diamantino/mt/br
-en,dmu,dimapur,dimapur,dimapur/in
-en,dna,okinawa kad afb,okinawa kad afb,okinawa/jp:kadena afb
-en,dnb,dunbar,dunbar,dunbar/ql/au
-en,dnc,danane,danane,danane/ci
-en,dnd,dundee,dundee,dundee/gb
-en,dne,dallas north arpt,dallas north ar,dallas/tx/us:dallas north
-en,dnf,derna,derna,derna/ly:martuba
-en,dng,doogan airport,doogan airport,doogan/wa/au:doogan airport
-en,dnh,dunhuang,dunhuang,dunhuang/cn
-en,dni,wad medani,wad medani,wad medani/sd
-en,dnk,dnepropetrovsk,dnepropetrovsk,dnepropetrovsk/ua
-en,dnl,augusta daniel fd,augusta daniel,augusta daniel/ga/us:daniel
-en,dnm,denham,denham,denham/wa/au
-en,dnn,dalton,dalton,dalton/ga/us:municipal
-en,dno,dianopolis,dianopolis,dianopolis/to/br
-en,dnp,dang,dang,dang/np
-en,dnq,deniliquin,deniliquin,deniliquin/ns/au
-en,dnr,dinard,dinard,dinard/fr:pleurtuit
-en,dns,denison,denison,denison/ia/us:municipal
-en,dnt,stana laguna dnt,stana laguna dn,santa ana/ca/us:downtown hpt
-en,dnu,dinangat,dinangat,dinangat/pg
-en,dnv,danville,danville,danville/il/us:vermilion cnty
-en,dnx,dinder,dinder,dinder/sd:galegu
-en,dnz,denizli,denizli,denizli/tr:cardak
-en,doa,doany,doany,doany/mg
-en,dob,dobo aru,dobo aru,dobo/id:airport
-en,doc,dornoch,dornoch,dornoch/gb
-en,dod,dodoma,dodoma,dodoma/tz
-en,doe,djoemoe,djoemoe,djoemoe/sr
-en,dof,dora bay,dora bay,dora bay/ak/us
-en,dog,dongola,dongola,dongola/sd
-en,doh,doha,doha,doha/qa
-en,doi,doini,doini,doini/pg
-en,dok,donetsk,donetsk,donetsk/ua
-en,dol,deauville,deauville,deauville/fr:st gatien
-en,dom,dominica dom,dominica dom,dominica/dm:melville hall
-en,don,dos lagunas,dos lagunas,dos lagunas/gt
-en,doo,dorobisoro,dorobisoro,dorobisoro/pg
-en,dop,dolpa,dolpa,dolpa/np
-en,dor,dori,dori,dori/bf
-en,dos,dios,dios,dios/pg
-en,dou,dourados,dourados,dourados/ms/br
-en,dov,dover cheswold,dover cheswold,dover cheswold/de/us:dover afb
-en,dox,dongara,dongara,dongara/wa/au
-en,doy,dongying,dongying,dongying/cn:dongying
-en,dpa,chicago dpa,chicago dpa,chicago/il/us:dupage county
-en,dpe,dieppe,dieppe,dieppe/fr
-en,ary,ararat,ararat,ararat/vi/au
-en,arz,n'zeto,n'zeto,n'zeto/ao
-en,asa,assab,assab,assab/er
-en,asb,ashgabad,ashgabad,ashgabad/tm:ashgabad
-en,asc,ascension,ascension,ascension/bo
-en,asd,andros town,andros town,andros town/bs
-en,ase,aspen,aspen,aspen/co/us
-en,asf,astrakhan,astrakhan,astrakhan/ru
-en,asg,ashburton,ashburton,ashburton/nz
-en,ash,nashua,nashua,nashua/nh/us:boire field
-en,asi,georgetown,georgetown,georgetown/sh:wideawake field
-en,asj,amami o shima,amami o shima,amami o shima/jp
-en,ask,yamoussoukro,yamoussoukro,yamoussoukro/ci
-en,asl,marshall,marshall,marshall/tx/us:harrison county
-en,asm,asmara,asmara,asmara/er:asmara international
-en,asn,talladega,talladega,talladega/al/us
-en,aso,asosa,asosa,asosa/et
-en,asp,alice springs,alice springs,alice springs/nt/au
-en,asq,austin,austin,austin/nv/us
-en,asr,kayseri,kayseri,kayseri/tr
-en,ast,astoria,astoria,astoria/or/us
-en,asu,asuncion,asuncion,asuncion/py:silvio pettirossi
-en,asv,amboseli,amboseli,amboseli/ke
-en,asw,aswan,aswan,aswan/eg
-en,asx,ashland,ashland,ashland/wi/us
-en,asy,ashley,ashley,ashley/nd/us
-en,asz,asirim,asirim,asirim/pg
-en,ata,anta,anta,anta/pe
-en,atb,atbara,atbara,atbara/sd
-en,atc,arthur's town,arthur's town,arthur's town/bs
-en,atd,atoifi,atoifi,atoifi/sb
-en,ate,antlers,antlers,antlers/ok/us
-en,atf,ambato,ambato,ambato/ec:chachoan
-e...
[truncated message content] |
|
From: <den...@us...> - 2009-07-23 17:47:43
|
Revision: 163
http://opentrep.svn.sourceforge.net/opentrep/?rev=163&view=rev
Author: denis_arnaud
Date: 2009-07-23 17:47:31 +0000 (Thu, 23 Jul 2009)
Log Message:
-----------
[Dev] Better protected the wrappers (e.g., Python) against unexpected exceptions.
Modified Paths:
--------------
trunk/opentrep/opentrep/batches/indexer.cpp
trunk/opentrep/opentrep/batches/searcher.cpp
trunk/opentrep/opentrep/bom/StringMatcher.cpp
trunk/opentrep/opentrep/python/pyopentrep.cpp
trunk/opentrep/opentrep/service/OPENTREP_Service.cpp
Modified: trunk/opentrep/opentrep/batches/indexer.cpp
===================================================================
--- trunk/opentrep/opentrep/batches/indexer.cpp 2009-07-23 09:54:10 UTC (rev 162)
+++ trunk/opentrep/opentrep/batches/indexer.cpp 2009-07-23 17:47:31 UTC (rev 163)
@@ -70,7 +70,7 @@
("passwd,p",
boost::program_options::value< std::string >(&ioDBPasswd)->default_value(K_OPENTREP_DEFAULT_DB_PASSWD),
"SQL database hostname (e.g., opentrep)")
- ("host,h",
+ ("host,H",
boost::program_options::value< std::string >(&ioDBHost)->default_value(K_OPENTREP_DEFAULT_DB_HOST),
"SQL database hostname (e.g., localhost)")
("port,P",
Modified: trunk/opentrep/opentrep/batches/searcher.cpp
===================================================================
--- trunk/opentrep/opentrep/batches/searcher.cpp 2009-07-23 09:54:10 UTC (rev 162)
+++ trunk/opentrep/opentrep/batches/searcher.cpp 2009-07-23 17:47:31 UTC (rev 163)
@@ -126,9 +126,6 @@
("error,e",
boost::program_options::value< unsigned short >(&ioSpellingErrorDistance)->default_value(K_OPENTREP_DEFAULT_SPELLING_ERROR_DISTANCE),
"Spelling error distance (e.g., 3)")
- ("query,q",
- boost::program_options::value< WordList_T >(&lWordList)->multitoken(),
- "Traval query word list (e.g. sna francicso rio de janero lso anglese reykyavki")
("database,d",
boost::program_options::value< std::string >(&ioDatabaseFilepath)->default_value(K_OPENTREP_DEFAULT_DATABSE_FILEPATH),
"Xapian database filepath (e.g., /tmp/opentrep/traveldb)")
@@ -141,7 +138,7 @@
("passwd,p",
boost::program_options::value< std::string >(&ioDBPasswd)->default_value(K_OPENTREP_DEFAULT_DB_PASSWD),
"SQL database hostname (e.g., opentrep)")
- ("host,h",
+ ("host,H",
boost::program_options::value< std::string >(&ioDBHost)->default_value(K_OPENTREP_DEFAULT_DB_HOST),
"SQL database hostname (e.g., localhost)")
("port,P",
@@ -150,6 +147,9 @@
("dbname,m",
boost::program_options::value< std::string >(&ioDBDBName)->default_value(K_OPENTREP_DEFAULT_DB_DBNAME),
"SQL database name (e.g., opentrep)")
+ ("query,q",
+ boost::program_options::value< WordList_T >(&lWordList)->multitoken(),
+ "Travel query word list (e.g. sna francicso rio de janero lso anglese reykyavki), which sould be located at the end of the command line (otherwise, the other options would be interpreted as part of that travel query word list)")
;
// Hidden options, will be allowed both on command line and
Modified: trunk/opentrep/opentrep/bom/StringMatcher.cpp
===================================================================
--- trunk/opentrep/opentrep/bom/StringMatcher.cpp 2009-07-23 09:54:10 UTC (rev 162)
+++ trunk/opentrep/opentrep/bom/StringMatcher.cpp 2009-07-23 17:47:31 UTC (rev 163)
@@ -114,7 +114,7 @@
// ///////////////////////////////////////////////////////////////////
void checkAndAlterIfNeeded (TravelQuery_T& ioSuggestedString,
const TravelQuery_T& iOriginalString,
- const NbOfErrors_T& iEditDistance,
+ const NbOfErrors_T& iMaxEditDistance,
const Xapian::Database& iDatabase) {
/**
@@ -126,33 +126,50 @@
/**
Get a spell-corrected suggestion for the reduced original string.
+ <br>Limit the edit distance to the given maximal one.
*/
+ NbOfErrors_T lCalculatedEditDistance =
+ calculateEditDistance (lOriginalStringCopy);
+
+ lCalculatedEditDistance = std::min (lCalculatedEditDistance,
+ iMaxEditDistance);
+
std::string lSuggestionForReducedOriginalString =
- iDatabase.get_spelling_suggestion (lOriginalStringCopy, iEditDistance);
+ iDatabase.get_spelling_suggestion (lOriginalStringCopy,
+ lCalculatedEditDistance);
+
+ /**
+ Note that if the suggestion on the reduced-original string is
+ empty, it normally means that the reduced-original string is
+ correct (in spelling terms), as the non-reduced original string
+ yielded a suggestion.
+ */
+ if (lSuggestionForReducedOriginalString.empty() == true) {
+ lSuggestionForReducedOriginalString = lOriginalStringCopy;
+ }
+
+ if (lSuggestionForReducedOriginalString == ioSuggestedString) {
- if (lSuggestionForReducedOriginalString.empty() == true
- || lSuggestionForReducedOriginalString == ioSuggestedString) {
+ // DEBUG
+ OPENTREP_LOG_DEBUG ("The suggestion (`" << ioSuggestedString
+ << "') for `" << iOriginalString
+ << "', with an edit distance/error of "
+ << lCalculatedEditDistance
+ << " over " << iMaxEditDistance << " allowable"
+ << ", is the same as the suggestion for the reduced "
+ << "original string (`" << lOriginalStringCopy
+ << "') -> discarded.");
+
/**
The suggestion on the reduced-original string is either empty
or the same as the suggestion on the original string. Either
way, the suggestion is not valid. It must be discarded
(emptied).
- <br>Note that if the suggestion on the reduced-original
- string is empty, it normally means that the reduced-original
- string is correct (in spelling terms), as the non-reduced
- original string yielded a suggestion.
<br>There may be side effects with the edit distance/error,
though, but it has been unobserved, as of now.
*/
ioSuggestedString = "";
- // DEBUG
- OPENTREP_LOG_DEBUG ("The suggestion (`" << ioSuggestedString
- << "') for `" << iOriginalString
- << "' is the same as the suggestion for the reduced "
- << "original string (`" << lOriginalStringCopy
- << "') -> discarded.");
-
return;
}
@@ -165,24 +182,44 @@
/**
Get a spell-corrected suggestion for the reduced original string.
+ <br>Limit the edit distance to the given maximal one.
*/
+ lCalculatedEditDistance = calculateEditDistance (lOriginalStringCopy);
+
+ lCalculatedEditDistance = std::min (lCalculatedEditDistance,
+ iMaxEditDistance);
+
lSuggestionForReducedOriginalString =
- iDatabase.get_spelling_suggestion (lOriginalStringCopy, iEditDistance);
+ iDatabase.get_spelling_suggestion (lOriginalStringCopy,
+ lCalculatedEditDistance);
- if (lSuggestionForReducedOriginalString.empty() == true
- || lSuggestionForReducedOriginalString == ioSuggestedString) {
- /**
- See the remark above.
- */
- ioSuggestedString = "";
+ /**
+ Note that if the suggestion on the reduced-original string is
+ empty, it normally means that the reduced-original string is
+ correct (in spelling terms), as the non-reduced original string
+ yielded a suggestion.
+ */
+ if (lSuggestionForReducedOriginalString.empty() == true) {
+ lSuggestionForReducedOriginalString = lOriginalStringCopy;
+ }
+
+ if (lSuggestionForReducedOriginalString == ioSuggestedString) {
// DEBUG
OPENTREP_LOG_DEBUG ("The suggestion (`" << ioSuggestedString
<< "') for `" << iOriginalString
- << "' is the same as the suggestion for the reduced "
+ << "', with an edit distance/error of "
+ << lCalculatedEditDistance
+ << " over " << iMaxEditDistance << " allowable"
+ << ", is the same as the suggestion for the reduced "
<< "original string (`" << lOriginalStringCopy
<< "') -> discarded.");
+ /**
+ See the remark above.
+ */
+ ioSuggestedString = "";
+
return;
}
}
@@ -389,7 +426,7 @@
will be otherwise/rather recognised in another step.
*/
checkAndAlterIfNeeded (lFullWordCorrectedString, lOriginalQueryString,
- lCalculatedEditDistance, iDatabase);
+ ioMaxEditDistance, iDatabase);
/**
Since there is still no match, we search on the string
Modified: trunk/opentrep/opentrep/python/pyopentrep.cpp
===================================================================
--- trunk/opentrep/opentrep/python/pyopentrep.cpp 2009-07-23 09:54:10 UTC (rev 162)
+++ trunk/opentrep/opentrep/python/pyopentrep.cpp 2009-07-23 17:47:31 UTC (rev 163)
@@ -85,10 +85,17 @@
// DEBUG
*_logOutputStream << oStr.str() << std::endl;
- } catch (const std::exception& error) {
- *_logOutputStream << "Exception: " << error.what() << std::endl;
- }
+ } catch (const RootException& eOpenTrepError) {
+ *_logOutputStream << "OpenTrep error: " << eOpenTrepError.what()
+ << std::endl;
+ } catch (const std::exception& eStdError) {
+ *_logOutputStream << "Error: " << eStdError.what() << std::endl;
+
+ } catch (...) {
+ *_logOutputStream << "Unknown error" << std::endl;
+ }
+
return oStr.str();
}
@@ -145,9 +152,16 @@
// DEBUG
*_logOutputStream << "Python wrapper initialised" << std::endl;
- } catch (const std::exception& error) {
- *_logOutputStream << "Exception: " << error.what() << std::endl;
- }
+ } catch (const RootException& eOpenTrepError) {
+ *_logOutputStream << "OpenTrep error: " << eOpenTrepError.what()
+ << std::endl;
+
+ } catch (const std::exception& eStdError) {
+ *_logOutputStream << "Error: " << eStdError.what() << std::endl;
+
+ } catch (...) {
+ *_logOutputStream << "Unknown error" << std::endl;
+ }
return isEverythingOK;
}
Modified: trunk/opentrep/opentrep/service/OPENTREP_Service.cpp
===================================================================
--- trunk/opentrep/opentrep/service/OPENTREP_Service.cpp 2009-07-23 09:54:10 UTC (rev 162)
+++ trunk/opentrep/opentrep/service/OPENTREP_Service.cpp 2009-07-23 17:47:31 UTC (rev 163)
@@ -5,6 +5,7 @@
#include <cassert>
// Boost
#include <boost/date_time/gregorian/gregorian.hpp>
+#include <boost/date_time/posix_time/ptime.hpp>
// OpenTrep
#include <opentrep/basic/BasConst_OPENTREP_Service.hpp>
#include <opentrep/basic/BasChronometer.hpp>
@@ -147,11 +148,16 @@
OPENTREP_ServiceContext& lOPENTREP_ServiceContext= *_opentrepServiceContext;
// Get the date-time for the present time
- // boost::gregorian::date lNowDateTime = boost::gregorian::now.date();
+ boost::posix_time::ptime lNowDateTime =
+ boost::posix_time::second_clock::local_time();
+ boost::gregorian::date lNowDate = lNowDateTime.date();
// DEBUG
- // OPENTREP_LOG_DEBUG (lNowDateTime << " - Match query '" << iTravelQuery
- // << " ' on Xapian database (index)");
+ OPENTREP_LOG_DEBUG (std::endl
+ << "==================================================="
+ << std::endl
+ << lNowDateTime << " - Match query '" << iTravelQuery
+ << "' on Xapian database (index)");
// Check that the travel request is not empty
if (iTravelQuery.empty() == true) {
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <den...@us...> - 2009-07-23 09:54:14
|
Revision: 162
http://opentrep.svn.sourceforge.net/opentrep/?rev=162&view=rev
Author: denis_arnaud
Date: 2009-07-23 09:54:10 +0000 (Thu, 23 Jul 2009)
Log Message:
-----------
Fixed the matching bug (until some other is discovered...).
Modified Paths:
--------------
trunk/opentrep/opentrep/bom/Document.cpp
trunk/opentrep/opentrep/bom/ResultHolder.cpp
trunk/opentrep/opentrep/bom/StringMatcher.cpp
trunk/opentrep/opentrep/service/OPENTREP_Service.cpp
Modified: trunk/opentrep/opentrep/bom/Document.cpp
===================================================================
--- trunk/opentrep/opentrep/bom/Document.cpp 2009-07-23 06:20:10 UTC (rev 161)
+++ trunk/opentrep/opentrep/bom/Document.cpp 2009-07-23 09:54:10 UTC (rev 162)
@@ -23,9 +23,9 @@
// //////////////////////////////////////////////////////////////////////
std::string Document::describeKey() const {
std::ostringstream oStr;
- oStr << "`" << describeShortKey() << "´";
+ oStr << "`" << describeShortKey() << "'";
if (_correctedQueryString.empty() == false) {
- oStr << " (corrected into `" << _correctedQueryString << "´)";
+ oStr << " (corrected into `" << _correctedQueryString << "')";
}
return oStr.str();
}
Modified: trunk/opentrep/opentrep/bom/ResultHolder.cpp
===================================================================
--- trunk/opentrep/opentrep/bom/ResultHolder.cpp 2009-07-23 06:20:10 UTC (rev 161)
+++ trunk/opentrep/opentrep/bom/ResultHolder.cpp 2009-07-23 09:54:10 UTC (rev 162)
@@ -4,6 +4,7 @@
// C
#include <cassert>
#include <sstream>
+#include <limits>
// Xapian
#include <xapian.h>
// OpenTREP
@@ -89,8 +90,14 @@
<br>And so on until the maximum of the edit distance/error
becomes greater than the maximal allowable distance/error.
reached.
+
+ <br>NOTE: that feature is de-activated, as it seems it does
+ not bring any added value. To re-activate it, just initialise
+ the lMaxEditDistance to 0, instead of to the positive infinite.
*/
- NbOfErrors_T lMaxEditDistance = 0;
+ // NbOfErrors_T lMaxEditDistance = 0;
+ NbOfErrors_T lMaxEditDistance = std::numeric_limits<NbOfErrors_T>::max();
+
bool hasReachedMaximalAllowableEditDistance = false;
bool shouldStop = false;
while (shouldStop == false) {
Modified: trunk/opentrep/opentrep/bom/StringMatcher.cpp
===================================================================
--- trunk/opentrep/opentrep/bom/StringMatcher.cpp 2009-07-23 06:20:10 UTC (rev 161)
+++ trunk/opentrep/opentrep/bom/StringMatcher.cpp 2009-07-23 09:54:10 UTC (rev 162)
@@ -31,7 +31,7 @@
on a 4-letter word, while an edit distance of 3 will be allowed on an
11-letter word. */
// //////////////////////////////////////////////////////////////////////
- static unsigned int calculateEditDistance (const std::string& iPhrase) {
+ static unsigned int calculateEditDistance (const TravelQuery_T& iPhrase) {
EditDistance_T oEditDistance = 2;
const EditDistance_T lQueryStringSize = iPhrase.size();
@@ -87,13 +87,113 @@
}
}
+ /**
+ That algorithm checks whether the first (resp. the last) word of
+ the original string "belongs" to the spell-corrected suggestion
+ (i.e., "recognised" by the Xapian database/index).
+
+ <br>For instance, let assume that the original string is 'nce
+ petropavlosk' and that the spell-corrected suggestion (for the
+ full original string) is 'petropavlosk', meaning that an edit
+ distance/error of 3 letters ('nce') has been allowed.
+
+ <br>We check whether the left-reduced original string, namely
+ 'petropavlosk', returns the same suggestion as the non-reduced
+ original string, namely 'nce petropavlosk'. In that case, both
+ the reduced and non-reduced original strings yield the same
+ spell-corrected suggestions, namely 'petropavlosk'. Hence, the
+ spell-corrected suggestion (namely 'petropavlosk') should not be
+ taken into account, i.e., the spell-corrected suggestion should
+ be empty.
+
+ <br>In other words, 'nce' does not belong to the spell-corrected
+ suggestion 'petropavlosk'. So, 'petropavlosk' is not a good
+ suggestion at that stage (because if accepted, it would enduce
+ the removal of 'nce' from the remaining string).
+ */
// ///////////////////////////////////////////////////////////////////
+ void checkAndAlterIfNeeded (TravelQuery_T& ioSuggestedString,
+ const TravelQuery_T& iOriginalString,
+ const NbOfErrors_T& iEditDistance,
+ const Xapian::Database& iDatabase) {
+
+ /**
+ Store a copy of the suggested string, as it will me altered by
+ the below method.
+ */
+ TravelQuery_T lOriginalStringCopy (iOriginalString);
+ StringMatcher::removeFurthestLeftWord (lOriginalStringCopy);
+
+ /**
+ Get a spell-corrected suggestion for the reduced original string.
+ */
+ std::string lSuggestionForReducedOriginalString =
+ iDatabase.get_spelling_suggestion (lOriginalStringCopy, iEditDistance);
+
+ if (lSuggestionForReducedOriginalString.empty() == true
+ || lSuggestionForReducedOriginalString == ioSuggestedString) {
+ /**
+ The suggestion on the reduced-original string is either empty
+ or the same as the suggestion on the original string. Either
+ way, the suggestion is not valid. It must be discarded
+ (emptied).
+ <br>Note that if the suggestion on the reduced-original
+ string is empty, it normally means that the reduced-original
+ string is correct (in spelling terms), as the non-reduced
+ original string yielded a suggestion.
+ <br>There may be side effects with the edit distance/error,
+ though, but it has been unobserved, as of now.
+ */
+ ioSuggestedString = "";
+
+ // DEBUG
+ OPENTREP_LOG_DEBUG ("The suggestion (`" << ioSuggestedString
+ << "') for `" << iOriginalString
+ << "' is the same as the suggestion for the reduced "
+ << "original string (`" << lOriginalStringCopy
+ << "') -> discarded.");
+
+ return;
+ }
+
+ /**
+ Store a copy of the suggested string, as it will me altered by
+ the below method.
+ */
+ lOriginalStringCopy = iOriginalString;
+ StringMatcher::removeFurthestRightWord (lOriginalStringCopy);
+
+ /**
+ Get a spell-corrected suggestion for the reduced original string.
+ */
+ lSuggestionForReducedOriginalString =
+ iDatabase.get_spelling_suggestion (lOriginalStringCopy, iEditDistance);
+
+ if (lSuggestionForReducedOriginalString.empty() == true
+ || lSuggestionForReducedOriginalString == ioSuggestedString) {
+ /**
+ See the remark above.
+ */
+ ioSuggestedString = "";
+
+ // DEBUG
+ OPENTREP_LOG_DEBUG ("The suggestion (`" << ioSuggestedString
+ << "') for `" << iOriginalString
+ << "' is the same as the suggestion for the reduced "
+ << "original string (`" << lOriginalStringCopy
+ << "') -> discarded.");
+
+ return;
+ }
+ }
+
+ // ///////////////////////////////////////////////////////////////////
std::string StringMatcher::
searchString (Xapian::MSet& ioMatchingSet,
- const std::string& iSearchString,
+ const TravelQuery_T& iSearchString,
NbOfErrors_T& ioMaxEditDistance,
bool& ioHasReachedMaximalAllowableEditDistance,
- const Xapian::Database& ioDatabase) {
+ const Xapian::Database& iDatabase) {
NbOfErrors_T lMaxEditDistance = std::numeric_limits<EditDistance_T>::min();
std::string oMatchedString;
@@ -136,7 +236,7 @@
// Build the query object
Xapian::QueryParser lQueryParser;
- lQueryParser.set_database (ioDatabase);
+ lQueryParser.set_database (iDatabase);
/**
As explained in http://www.xapian.org/docs/queryparser.html,
Xapian::Query::OP_ADJ is better than Xapian::Query::OP_PHRASE,
@@ -152,7 +252,7 @@
*/
// Start an enquire session
- Xapian::Enquire enquire (ioDatabase);
+ Xapian::Enquire enquire (iDatabase);
/**
The Xapian::QueryParser::parse_query() method aggregates all
@@ -212,7 +312,7 @@
*/
WordList_T lCorrectedWordList;
createCorrectedWordList (lOriginalWordList, lCorrectedWordList,
- ioDatabase);
+ iDatabase);
const std::string lCorrectedQueryString =
WordHolder::createStringFromWordList (lCorrectedWordList);
@@ -280,11 +380,18 @@
lCalculatedEditDistance = std::min (lCalculatedEditDistance,
ioMaxEditDistance);
- const std::string lFullWordCorrectedString =
- ioDatabase.get_spelling_suggestion (lOriginalQueryString,
- lCalculatedEditDistance);
-
+ std::string lFullWordCorrectedString =
+ iDatabase.get_spelling_suggestion (lOriginalQueryString,
+ lCalculatedEditDistance);
+
/**
+ Check that the suggestion does not encompass extra words, which
+ will be otherwise/rather recognised in another step.
+ */
+ checkAndAlterIfNeeded (lFullWordCorrectedString, lOriginalQueryString,
+ lCalculatedEditDistance, iDatabase);
+
+ /**
Since there is still no match, we search on the string
corrected as a whole.
<br>As, with the above example, the full corrected string is
Modified: trunk/opentrep/opentrep/service/OPENTREP_Service.cpp
===================================================================
--- trunk/opentrep/opentrep/service/OPENTREP_Service.cpp 2009-07-23 06:20:10 UTC (rev 161)
+++ trunk/opentrep/opentrep/service/OPENTREP_Service.cpp 2009-07-23 09:54:10 UTC (rev 162)
@@ -3,6 +3,8 @@
// //////////////////////////////////////////////////////////////////////
// C
#include <cassert>
+// Boost
+#include <boost/date_time/gregorian/gregorian.hpp>
// OpenTrep
#include <opentrep/basic/BasConst_OPENTREP_Service.hpp>
#include <opentrep/basic/BasChronometer.hpp>
@@ -144,6 +146,13 @@
assert (_opentrepServiceContext != NULL);
OPENTREP_ServiceContext& lOPENTREP_ServiceContext= *_opentrepServiceContext;
+ // Get the date-time for the present time
+ // boost::gregorian::date lNowDateTime = boost::gregorian::now.date();
+
+ // DEBUG
+ // OPENTREP_LOG_DEBUG (lNowDateTime << " - Match query '" << iTravelQuery
+ // << " ' on Xapian database (index)");
+
// Check that the travel request is not empty
if (iTravelQuery.empty() == true) {
OPENTREP_LOG_ERROR ("The travel request is empty.");
@@ -174,10 +183,8 @@
// DEBUG
OPENTREP_LOG_DEBUG ("Match query on Xapian database (index): "
- << lRequestInterpreterMeasure << " - "
- << lOPENTREP_ServiceContext.display());
+ << lRequestInterpreterMeasure);
-
} catch (const std::exception& error) {
OPENTREP_LOG_ERROR ("Exception: " << error.what());
throw InterpreteTravelRequestException();
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <den...@us...> - 2009-07-23 06:20:13
|
Revision: 161
http://opentrep.svn.sourceforge.net/opentrep/?rev=161&view=rev
Author: denis_arnaud
Date: 2009-07-23 06:20:10 +0000 (Thu, 23 Jul 2009)
Log Message:
-----------
[DB] Corrected Tokyo (now matching with TYO).
Modified Paths:
--------------
trunk/opentrep/db/data/ref_place_names.csv
Modified: trunk/opentrep/db/data/ref_place_names.csv
===================================================================
--- trunk/opentrep/db/data/ref_place_names.csv 2009-07-23 05:13:30 UTC (rev 160)
+++ trunk/opentrep/db/data/ref_place_names.csv 2009-07-23 06:20:10 UTC (rev 161)
@@ -917,7 +917,7 @@
en,mak,malakal,malakal,malakal/sd
en,mal,mangole,mangole,mangole/id
en,mam,matamoros,matamoros,matamoros/mx
-en,man,manchester int,manchester int,manchester/gb:manchester intl
+en,man,manchester int,manchester int,manchester/gb:manchester intl, manchester, manchester, manchester, man, man
en,mao,manaus,manaus,manaus/am/br:eduardo gomes
en,map,mamai,mamai,mamai/pg
en,maq,mae sot,mae sot,mae sot/th
@@ -6452,7 +6452,7 @@
en,nrq,n'riquinha,n'riquinha,n'riquinha/ao
en,nrr,roosevelt roads,roosevelt roads,roosevelt roads/pr:nas
en,nrs,imperial beach,imperial beach,imperial beach/ca/us:nas
-en,nrt,tokyo narita,tokyo narita,tokyo/jp:narita
+en,nrt,tokyo narita,tokyo narita,tokyo/jp:narita,tokyo
en,nrv,guam uscg shore s,guam uscg shore,guam/gu:uscg shore station
en,nry,newry,newry,newry/nt/au
en,nsa,noosa,noosa,noosa/ql/au
@@ -6986,7 +6986,7 @@
en,tyl,talara,talara,talara/pe
en,tym,staniel cay,staniel cay,staniel cay/bs
en,tyn,taiyuan,taiyuan,taiyuan/cn
-en,tyo,tokyo,tokyo,tokyo/jp
+en,tyo,tokyo,tokyo,tokyo/jp,tokyo,tokyo
en,typ,tobermorey,tobermorey,tobermorey/nt/au
en,tyr,tyler,tyler,tyler/tx/us:pounds field
en,tys,knoxville,knoxville,knoxville/tn/us:mc ghee tyson
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <den...@us...> - 2009-07-23 05:13:33
|
Revision: 160
http://opentrep.svn.sourceforge.net/opentrep/?rev=160&view=rev
Author: denis_arnaud
Date: 2009-07-23 05:13:30 +0000 (Thu, 23 Jul 2009)
Log Message:
-----------
[Dev] Second attempt to fix a matching bug.
Modified Paths:
--------------
trunk/opentrep/db/data/ref_place_names.csv
trunk/opentrep/opentrep/bom/ResultHolder.cpp
trunk/opentrep/opentrep/bom/ResultHolder.hpp
trunk/opentrep/opentrep/bom/StringMatcher.cpp
trunk/opentrep/opentrep/bom/StringMatcher.hpp
Modified: trunk/opentrep/db/data/ref_place_names.csv
===================================================================
--- trunk/opentrep/db/data/ref_place_names.csv 2009-07-23 04:35:16 UTC (rev 159)
+++ trunk/opentrep/db/data/ref_place_names.csv 2009-07-23 05:13:30 UTC (rev 160)
@@ -3550,7 +3550,7 @@
en,cdd,cauquira,cauquira,cauquira/hn:cauquira airport
en,cde,caledonia,caledonia,caledonia/pa
en,cdf,cortina d'ampezzo,cortina d'ampez,cortina d'ampezzo/it:fiames
-en,cdg,paris cdg,paris cdg,paris/fr:charles de gaulle,cdg,cdg
+en,cdg,paris cdg,paris cdg,paris/fr:charles de gaulle,cdg,cdg,charles de gaulle
en,cdh,camden,camden,camden/ar/us:harrell fld
en,cdi,cachoeiro,cachoeiro,cachoeiro de i/es/br:cachoeiro
en,cdj,conceicao do arag,conceicao do ar,conceicao do arag/pa/br
Modified: trunk/opentrep/opentrep/bom/ResultHolder.cpp
===================================================================
--- trunk/opentrep/opentrep/bom/ResultHolder.cpp 2009-07-23 04:35:16 UTC (rev 159)
+++ trunk/opentrep/opentrep/bom/ResultHolder.cpp 2009-07-23 05:13:30 UTC (rev 160)
@@ -68,30 +68,53 @@
}
// //////////////////////////////////////////////////////////////////////
- std::string ResultHolder::
- searchString (Xapian::MSet& ioMatchingSet,
- TravelQuery_T& ioPartialQueryString,
- NbOfErrors_T& ioMaxEditDistance,
- bool ioHasReachedMaximalAllowableEditDistance,
- Document& ioMatchingDocument) {
+ std::string ResultHolder::searchString (Xapian::MSet& ioMatchingSet,
+ TravelQuery_T& ioPartialQueryString,
+ Document& ioMatchingDocument) {
std::string oMatchedString;
// Catch any Xapian::Error exceptions thrown
try {
+ /**
+ The query string must first be checked, without allowing any
+ spelling errors, but by removing the furthest right word at
+ every step.
+ <br>If no match is found, the maximal allowable edit
+ distance/error becomes 1, and the process (trying to match
+ the whole sentence, then by removing the furthest right word,
+ etc.) is re-performed.
+ <br>If no match is found, the maximal allowable edit
+ distance/error becomes 2.
+ <br>And so on until the maximum of the edit distance/error
+ becomes greater than the maximal allowable distance/error.
+ reached.
+ */
+ NbOfErrors_T lMaxEditDistance = 0;
+ bool hasReachedMaximalAllowableEditDistance = false;
bool shouldStop = false;
while (shouldStop == false) {
- // Retrieve the list of documents matching the query string
+
+ // DEBUG
+ OPENTREP_LOG_DEBUG ("--------");
+ OPENTREP_LOG_DEBUG ("Current query string: `"
+ << ioPartialQueryString
+ << "', with a maximal edit distance of "
+ << lMaxEditDistance << ".");
+
+ // Retrieve the list of Xapian documents matching the query string
oMatchedString =
StringMatcher::searchString (ioMatchingSet, ioPartialQueryString,
- ioMaxEditDistance,
- ioHasReachedMaximalAllowableEditDistance,
+ lMaxEditDistance,
+ hasReachedMaximalAllowableEditDistance,
_database);
// DEBUG
- OPENTREP_LOG_DEBUG ("Current initial query string: `"
+ OPENTREP_LOG_DEBUG ("---- Current query string: `"
<< ioPartialQueryString << "' --- Kept query: `"
- << oMatchedString << "' for "
+ << oMatchedString
+ << "', with a maximal edit distance of "
+ << lMaxEditDistance << ", for "
<< ioMatchingSet.size() << " matches.");
if (ioMatchingSet.empty() == false) {
@@ -106,22 +129,15 @@
break;
}
- // Since the query, as is, yield no match, the furthest right
- // word must be removed from the query string.
- StringMatcher::removeFurthestRightWord (ioPartialQueryString);
-
+ // Allow for one more spelling error
+ ++lMaxEditDistance;
+
/**
- Stop when the resulting string gets empty.
-
- <br>Note that whether maximal allowable edit distance/error
- has been reached is not checked at that stage. That
- algorithm is performed independently for each level of
- maximal allowable edit distance/error. Only the caller
- (below) retriggers this process by changing the level of
- maximal allowable edit distance/error, until that latter be
- reached.
+ Stop when it is no longer necessary to increase the maximal
+ allowable edit distance, as it is already greater than the
+ maximum of the calculated edit distance.
*/
- if (ioPartialQueryString.empty() == true) {
+ if (hasReachedMaximalAllowableEditDistance == true) {
shouldStop = true;
}
}
@@ -141,39 +157,26 @@
// Catch any Xapian::Error exceptions thrown
try {
- bool shouldStop = false;
- NbOfErrors_T lMaxEditDistance = 0;
-
/**
- The query string must first be checked, without allowing any
- spelling errors, but by removing the furthest right word at
- every step.
- <br>If no match is found, the maximal allowable edit
- distance/error becomes 1, and the process (trying to match
- the whole sentence, then by removing the furthest right word,
- etc.) is re-performed.
- <br>If no match is found, the maximal allowable edit
- distance/error becomes 2.
- <br>And so on until the maximum of the edit distance/error
- becomes greater than the maximal allowable distance/error.
- reached.
+ A copy of the query is made, as that copy will be altered by
+ the below process, whereas a clean copy needs to be reprocessed
+ for each level of maximal edit distance/error.
+ <br>However, in case of match, the modifications on the query
+ string (lPartialQueryString) must be replicated on the
+ original one (ioPartialQueryString).
*/
+ TravelQuery_T lPartialQueryString (ioPartialQueryString);
+
+ bool shouldStop = false;
while (shouldStop == false) {
- /**
- A copy of the query is made, as that copy will be altered by
- the below process, whereas a clean copy needs to be reprocessed
- for each level of maximal edit distance/error.
- <br>However, at the end, the modifications on the query
- string (lPartialQueryString) must be replicated on the
- original one (ioPartialQueryString).
- */
- TravelQuery_T lPartialQueryString (ioPartialQueryString);
+ // DEBUG
+ OPENTREP_LOG_DEBUG ("----------------");
+ OPENTREP_LOG_DEBUG ("Current query string: `"
+ << lPartialQueryString << "'");
+
Xapian::MSet lMatchingSet;
- bool hasReachedMaximalAllowableEditDistance = false;
oMatchedString = searchString (lMatchingSet, lPartialQueryString,
- lMaxEditDistance,
- hasReachedMaximalAllowableEditDistance,
ioMatchingDocument);
if (oMatchedString.empty() == false) {
@@ -189,17 +192,28 @@
break;
}
- // Allow for one more spelling error
- ++lMaxEditDistance;
+ // Since the query, as is, yields no match, the furthest right
+ // word must be removed from the query string.
+ StringMatcher::removeFurthestRightWord (lPartialQueryString);
+
+ /**
+ Stop when the resulting string gets empty.
- /**
- Stop when it is no longer necessary to increase the maximal
- allowable edit distance, as it is already greater than the
- maximum of the calculated edit distance.
+ <br>Note that whether maximal allowable edit distance/error
+ has been reached is not checked at that stage. That
+ algorithm is performed independently for each level of
+ maximal allowable edit distance/error. Only the caller
+ (below) retriggers this process by changing the level of
+ maximal allowable edit distance/error, until that latter be
+ reached.
*/
- if (hasReachedMaximalAllowableEditDistance == true) {
- ioPartialQueryString = lPartialQueryString;
+ if (lPartialQueryString.empty() == true) {
shouldStop = true;
+
+ // DEBUG
+ OPENTREP_LOG_DEBUG ("----------------");
+ OPENTREP_LOG_DEBUG ("Still no match for current query string: `"
+ << ioPartialQueryString << "'");
}
}
@@ -221,7 +235,7 @@
bool shouldStop = false;
while (shouldStop == false) {
// DEBUG
- OPENTREP_LOG_DEBUG ("---------------------")
+ OPENTREP_LOG_DEBUG ("+++++++++++++++++++++")
OPENTREP_LOG_DEBUG ("Remaining part of the query string: `"
<< lRemainingQueryString << "'");
Modified: trunk/opentrep/opentrep/bom/ResultHolder.hpp
===================================================================
--- trunk/opentrep/opentrep/bom/ResultHolder.hpp 2009-07-23 04:35:16 UTC (rev 159)
+++ trunk/opentrep/opentrep/bom/ResultHolder.hpp 2009-07-23 05:13:30 UTC (rev 160)
@@ -48,16 +48,10 @@
/** Retrieve the document best matching the query string.
@param Xapian::MSet& The Xapian matching set. It can be empty.
@param TravelQuery_T& The partial query string.
- @param NbOfErrors_T& The maximal allowable edit distance/error.
- @param bool Whether or not the maximal allowable edit distance/error
- has become greater than the maximum of the edit distance/errors
- calculated on the phrase.
@param MatchingDocument_T& The best matching Xapian document (if found).
@return bool Whether such a best matching document has been found. */
std::string searchString (Xapian::MSet& ioMatchingSet,
TravelQuery_T& ioPartialQueryString,
- NbOfErrors_T& ioMaxEditDistance,
- bool ioHasReachedMaximalAllowableEditDistance,
Document& ioMatchingDocument);
/** Retrieve the document best matching the query string.
Modified: trunk/opentrep/opentrep/bom/StringMatcher.cpp
===================================================================
--- trunk/opentrep/opentrep/bom/StringMatcher.cpp 2009-07-23 04:35:16 UTC (rev 159)
+++ trunk/opentrep/opentrep/bom/StringMatcher.cpp 2009-07-23 05:13:30 UTC (rev 160)
@@ -92,7 +92,7 @@
searchString (Xapian::MSet& ioMatchingSet,
const std::string& iSearchString,
NbOfErrors_T& ioMaxEditDistance,
- bool ioHasReachedMaximalAllowableEditDistance,
+ bool& ioHasReachedMaximalAllowableEditDistance,
const Xapian::Database& ioDatabase) {
NbOfErrors_T lMaxEditDistance = std::numeric_limits<EditDistance_T>::min();
@@ -176,9 +176,11 @@
int nbMatches = ioMatchingSet.size();
// DEBUG
+ /*
OPENTREP_LOG_DEBUG ("Original query `" << lOriginalQueryString
<< "', i.e., `" << lOriginalQuery.get_description()
<< "' => " << nbMatches << " results found");
+ */
/**
When no match is found, we search on the corrected phrase/string
@@ -241,10 +243,12 @@
nbMatches = ioMatchingSet.size();
// DEBUG
+ /*
OPENTREP_LOG_DEBUG ("Corrected query `" << lCorrectedQueryString
<< "', i.e., `" << lCorrectedQuery.get_description()
<< "' => " << nbMatches
<< " results found on corrected string");
+ */
if (nbMatches != 0) {
/**
@@ -300,6 +304,7 @@
nbMatches = ioMatchingSet.size();
// DEBUG
+ /*
OPENTREP_LOG_DEBUG ("Query corrected as a full sentence `"
<< lFullWordCorrectedString
<< "' with an allowable maximal edit distance of "
@@ -308,6 +313,7 @@
<< ", i.e., `"<< lFullQueryCorrected.get_description()
<< "' => " << nbMatches
<< " results found on corrected full string");
+ */
if (nbMatches != 0) {
oMatchedString = lFullWordCorrectedString;
@@ -329,7 +335,7 @@
of the calculated edit distance, it becomes useless to go on
increasing the maximal allowable edit distance.
*/
- if (lMaxEditDistance <= ioMaxEditDistance) {
+ if (ioMaxEditDistance >= lMaxEditDistance) {
ioHasReachedMaximalAllowableEditDistance = true;
}
Modified: trunk/opentrep/opentrep/bom/StringMatcher.hpp
===================================================================
--- trunk/opentrep/opentrep/bom/StringMatcher.hpp 2009-07-23 04:35:16 UTC (rev 159)
+++ trunk/opentrep/opentrep/bom/StringMatcher.hpp 2009-07-23 05:13:30 UTC (rev 160)
@@ -29,7 +29,7 @@
@param Xapian::MSet& The Xapian matching set. It can be empty.
@param const std::string& The query string.
@param NbOfErrors_T& The maximal allowable edit distance/error.
- @param bool Whether or not the maximal allowable edit distance/error
+ @param bool& Whether or not the maximal allowable edit distance/error
has become greater than the maximum of the edit distance/errors
calculated on the phrase.
@param const Xapian::Database& The Xapian index/database.
@@ -38,7 +38,7 @@
static std::string searchString (Xapian::MSet&,
const std::string& iSearchString,
NbOfErrors_T& ioMaxEditDistance,
- bool ioHasReachedMaximalAllowableEditDistance,
+ bool& ioHasReachedMaximalAllowableEditDistance,
const Xapian::Database&);
/** Extract the best matching Xapian document.
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <den...@us...> - 2009-07-23 04:35:25
|
Revision: 159
http://opentrep.svn.sourceforge.net/opentrep/?rev=159&view=rev
Author: denis_arnaud
Date: 2009-07-23 04:35:16 +0000 (Thu, 23 Jul 2009)
Log Message:
-----------
[Dev] Intermediary commit while attempting to fix a matching bug.
Modified Paths:
--------------
trunk/opentrep/opentrep/bom/ResultHolder.cpp
trunk/opentrep/opentrep/bom/StringMatcher.cpp
Modified: trunk/opentrep/opentrep/bom/ResultHolder.cpp
===================================================================
--- trunk/opentrep/opentrep/bom/ResultHolder.cpp 2009-07-22 06:06:26 UTC (rev 158)
+++ trunk/opentrep/opentrep/bom/ResultHolder.cpp 2009-07-23 04:35:16 UTC (rev 159)
@@ -160,9 +160,12 @@
*/
while (shouldStop == false) {
/**
- A copy of the query is made, as it the copy will be altered by
- the above process, whereas a clean copy needs to be reprocessed
+ A copy of the query is made, as that copy will be altered by
+ the below process, whereas a clean copy needs to be reprocessed
for each level of maximal edit distance/error.
+ <br>However, at the end, the modifications on the query
+ string (lPartialQueryString) must be replicated on the
+ original one (ioPartialQueryString).
*/
TravelQuery_T lPartialQueryString (ioPartialQueryString);
@@ -174,13 +177,14 @@
ioMatchingDocument);
if (oMatchedString.empty() == false) {
- // Create the corresponding list of documents
+ // Create the corresponding document (from the Xapian MSet object)
StringMatcher::
extractBestMatchingDocumentFromMSet (lMatchingSet,
ioMatchingDocument);
// Since a result has been found, the search can be stopped
// for that part of the query.
+ ioPartialQueryString = lPartialQueryString;
shouldStop = true;
break;
}
@@ -194,6 +198,7 @@
maximum of the calculated edit distance.
*/
if (hasReachedMaximalAllowableEditDistance == true) {
+ ioPartialQueryString = lPartialQueryString;
shouldStop = true;
}
}
@@ -262,10 +267,15 @@
parsed is 'rio de janeiro'. So, the already parsed part,
namely 'sna francisco', must be subtracted from the
initial query string.
+ <br>To do that, it is not an exact subtraction. Rather,
+ the words within the already parsed part ('sna
+ francisco') are counted, and the corresponding number of
+ words are subtracted from the furthest left hand of the
+ remaining part of the string.
*/
StringMatcher::subtractParsedToRemaining (lQueryString,
lRemainingQueryString);
-
+
} else {
// DEBUG
OPENTREP_LOG_DEBUG ("==> No matching of the query string: `"
Modified: trunk/opentrep/opentrep/bom/StringMatcher.cpp
===================================================================
--- trunk/opentrep/opentrep/bom/StringMatcher.cpp 2009-07-22 06:06:26 UTC (rev 158)
+++ trunk/opentrep/opentrep/bom/StringMatcher.cpp 2009-07-23 04:35:16 UTC (rev 159)
@@ -177,7 +177,7 @@
// DEBUG
OPENTREP_LOG_DEBUG ("Original query `" << lOriginalQueryString
- << ", i.e., `" << lOriginalQuery.get_description()
+ << "', i.e., `" << lOriginalQuery.get_description()
<< "' => " << nbMatches << " results found");
/**
@@ -302,8 +302,9 @@
// DEBUG
OPENTREP_LOG_DEBUG ("Query corrected as a full sentence `"
<< lFullWordCorrectedString
- << "' with a maximal edit distance of "
- << lMaxEditDistance
+ << "' with an allowable maximal edit distance of "
+ << ioMaxEditDistance
+ << " on a potential of " << lMaxEditDistance
<< ", i.e., `"<< lFullQueryCorrected.get_description()
<< "' => " << nbMatches
<< " results found on corrected full string");
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <den...@us...> - 2009-07-22 06:06:29
|
Revision: 158
http://opentrep.svn.sourceforge.net/opentrep/?rev=158&view=rev
Author: denis_arnaud
Date: 2009-07-22 06:06:26 +0000 (Wed, 22 Jul 2009)
Log Message:
-----------
[Dev] Re-worked the algorithm for the search with maximal allowable edit distance.
Modified Paths:
--------------
trunk/opentrep/opentrep/DistanceErrorRule.hpp
trunk/opentrep/opentrep/OPENTREP_Types.hpp
trunk/opentrep/opentrep/bom/ResultHolder.cpp
trunk/opentrep/opentrep/bom/ResultHolder.hpp
trunk/opentrep/opentrep/bom/StringMatcher.cpp
trunk/opentrep/opentrep/bom/StringMatcher.hpp
Modified: trunk/opentrep/opentrep/DistanceErrorRule.hpp
===================================================================
--- trunk/opentrep/opentrep/DistanceErrorRule.hpp 2009-07-21 21:56:56 UTC (rev 157)
+++ trunk/opentrep/opentrep/DistanceErrorRule.hpp 2009-07-22 06:06:26 UTC (rev 158)
@@ -17,19 +17,6 @@
namespace OPENTREP {
- // //////////////////////////////////////////////////////////////////
- /** Number of letters in a word or phrase. */
- typedef unsigned int NbOfLetters_T;
-
- /** Number of errors. */
- typedef unsigned short NbOfErrors_T;
-
- /** Number of (distance) errors allowed for a given number of letters. */
- typedef std::map<NbOfLetters_T, NbOfErrors_T> DistanceErrorScale_T;
-
- /** Number of (distance) errors allowed for a given number of letters. */
- typedef boost::array<NbOfLetters_T, 5> DistanceErrorScaleArray_T;
-
// ////////////////////////////////////////////////////////////////////
/** Default distance error scale.
<br>Allowed error for a given number of letters:
Modified: trunk/opentrep/opentrep/OPENTREP_Types.hpp
===================================================================
--- trunk/opentrep/opentrep/OPENTREP_Types.hpp 2009-07-21 21:56:56 UTC (rev 157)
+++ trunk/opentrep/opentrep/OPENTREP_Types.hpp 2009-07-22 06:06:26 UTC (rev 158)
@@ -8,6 +8,9 @@
#include <exception>
#include <string>
#include <list>
+#include <map>
+// Boost Arry
+#include <boost/array.hpp>
namespace OPENTREP {
@@ -87,15 +90,27 @@
/** Travel search query. */
typedef std::string TravelQuery_T;
- /** Number of matching documents. */
- typedef unsigned short NbOfMatches_T;
-
/** Word, which is the atomic element of a query string. */
typedef std::string Word_T;
/** List of words. */
typedef std::list<Word_T> WordList_T;
+ /** Number of letters in a word or phrase. */
+ typedef unsigned int NbOfLetters_T;
+
+ /** Number of matching documents. */
+ typedef unsigned short NbOfMatches_T;
+
+ /** Number of errors. */
+ typedef unsigned short NbOfErrors_T;
+
+ /** Number of (distance) errors allowed for a given number of letters. */
+ typedef std::map<NbOfLetters_T, NbOfErrors_T> DistanceErrorScale_T;
+
+ /** Number of (distance) errors allowed for a given number of letters. */
+ typedef boost::array<NbOfLetters_T, 5> DistanceErrorScaleArray_T;
+
}
#endif // __OPENTREP_OPENTREP_TYPES_HPP
Modified: trunk/opentrep/opentrep/bom/ResultHolder.cpp
===================================================================
--- trunk/opentrep/opentrep/bom/ResultHolder.cpp 2009-07-21 21:56:56 UTC (rev 157)
+++ trunk/opentrep/opentrep/bom/ResultHolder.cpp 2009-07-22 06:06:26 UTC (rev 158)
@@ -68,8 +68,12 @@
}
// //////////////////////////////////////////////////////////////////////
- std::string ResultHolder::searchString (TravelQuery_T& ioPartialQueryString,
- Document& ioMatchingDocument) {
+ std::string ResultHolder::
+ searchString (Xapian::MSet& ioMatchingSet,
+ TravelQuery_T& ioPartialQueryString,
+ NbOfErrors_T& ioMaxEditDistance,
+ bool ioHasReachedMaximalAllowableEditDistance,
+ Document& ioMatchingDocument) {
std::string oMatchedString;
// Catch any Xapian::Error exceptions thrown
@@ -78,21 +82,22 @@
bool shouldStop = false;
while (shouldStop == false) {
// Retrieve the list of documents matching the query string
- Xapian::MSet lMatchingSet;
- oMatchedString = StringMatcher::searchString (lMatchingSet,
- ioPartialQueryString,
- _database);
+ oMatchedString =
+ StringMatcher::searchString (ioMatchingSet, ioPartialQueryString,
+ ioMaxEditDistance,
+ ioHasReachedMaximalAllowableEditDistance,
+ _database);
// DEBUG
OPENTREP_LOG_DEBUG ("Current initial query string: `"
<< ioPartialQueryString << "' --- Kept query: `"
<< oMatchedString << "' for "
- << lMatchingSet.size() << " matches.");
+ << ioMatchingSet.size() << " matches.");
- if (lMatchingSet.empty() == false) {
+ if (ioMatchingSet.empty() == false) {
// Create the corresponding list of documents
StringMatcher::
- extractBestMatchingDocumentFromMSet (lMatchingSet,
+ extractBestMatchingDocumentFromMSet (ioMatchingSet,
ioMatchingDocument);
// Since a result has been found, the search can be stopped
@@ -105,7 +110,17 @@
// word must be removed from the query string.
StringMatcher::removeFurthestRightWord (ioPartialQueryString);
- // Stop when the resulting string gets empty.
+ /**
+ Stop when the resulting string gets empty.
+
+ <br>Note that whether maximal allowable edit distance/error
+ has been reached is not checked at that stage. That
+ algorithm is performed independently for each level of
+ maximal allowable edit distance/error. Only the caller
+ (below) retriggers this process by changing the level of
+ maximal allowable edit distance/error, until that latter be
+ reached.
+ */
if (ioPartialQueryString.empty() == true) {
shouldStop = true;
}
@@ -119,6 +134,78 @@
}
// //////////////////////////////////////////////////////////////////////
+ std::string ResultHolder::searchString (TravelQuery_T& ioPartialQueryString,
+ Document& ioMatchingDocument) {
+ std::string oMatchedString;
+
+ // Catch any Xapian::Error exceptions thrown
+ try {
+
+ bool shouldStop = false;
+ NbOfErrors_T lMaxEditDistance = 0;
+
+ /**
+ The query string must first be checked, without allowing any
+ spelling errors, but by removing the furthest right word at
+ every step.
+ <br>If no match is found, the maximal allowable edit
+ distance/error becomes 1, and the process (trying to match
+ the whole sentence, then by removing the furthest right word,
+ etc.) is re-performed.
+ <br>If no match is found, the maximal allowable edit
+ distance/error becomes 2.
+ <br>And so on until the maximum of the edit distance/error
+ becomes greater than the maximal allowable distance/error.
+ reached.
+ */
+ while (shouldStop == false) {
+ /**
+ A copy of the query is made, as it the copy will be altered by
+ the above process, whereas a clean copy needs to be reprocessed
+ for each level of maximal edit distance/error.
+ */
+ TravelQuery_T lPartialQueryString (ioPartialQueryString);
+
+ Xapian::MSet lMatchingSet;
+ bool hasReachedMaximalAllowableEditDistance = false;
+ oMatchedString = searchString (lMatchingSet, lPartialQueryString,
+ lMaxEditDistance,
+ hasReachedMaximalAllowableEditDistance,
+ ioMatchingDocument);
+
+ if (oMatchedString.empty() == false) {
+ // Create the corresponding list of documents
+ StringMatcher::
+ extractBestMatchingDocumentFromMSet (lMatchingSet,
+ ioMatchingDocument);
+
+ // Since a result has been found, the search can be stopped
+ // for that part of the query.
+ shouldStop = true;
+ break;
+ }
+
+ // Allow for one more spelling error
+ ++lMaxEditDistance;
+
+ /**
+ Stop when it is no longer necessary to increase the maximal
+ allowable edit distance, as it is already greater than the
+ maximum of the calculated edit distance.
+ */
+ if (hasReachedMaximalAllowableEditDistance == true) {
+ shouldStop = true;
+ }
+ }
+
+ } catch (const Xapian::Error& error) {
+ OPENTREP_LOG_ERROR ("Exception: " << error.get_msg());
+ }
+
+ return oMatchedString;
+ }
+
+ // //////////////////////////////////////////////////////////////////////
void ResultHolder::searchString (DocumentList_T& ioDocumentList,
WordList_T& ioWordList) {
Modified: trunk/opentrep/opentrep/bom/ResultHolder.hpp
===================================================================
--- trunk/opentrep/opentrep/bom/ResultHolder.hpp 2009-07-21 21:56:56 UTC (rev 157)
+++ trunk/opentrep/opentrep/bom/ResultHolder.hpp 2009-07-22 06:06:26 UTC (rev 158)
@@ -46,9 +46,24 @@
private:
/** Retrieve the document best matching the query string.
+ @param Xapian::MSet& The Xapian matching set. It can be empty.
@param TravelQuery_T& The partial query string.
+ @param NbOfErrors_T& The maximal allowable edit distance/error.
+ @param bool Whether or not the maximal allowable edit distance/error
+ has become greater than the maximum of the edit distance/errors
+ calculated on the phrase.
@param MatchingDocument_T& The best matching Xapian document (if found).
@return bool Whether such a best matching document has been found. */
+ std::string searchString (Xapian::MSet& ioMatchingSet,
+ TravelQuery_T& ioPartialQueryString,
+ NbOfErrors_T& ioMaxEditDistance,
+ bool ioHasReachedMaximalAllowableEditDistance,
+ Document& ioMatchingDocument);
+
+ /** Retrieve the document best matching the query string.
+ @param TravelQuery_T& The partial query string.
+ @param MatchingDocument_T& The best matching Xapian document (if found).
+ @return bool Whether such a best matching document has been found. */
std::string searchString (TravelQuery_T& ioPartialQueryString, Document&);
Modified: trunk/opentrep/opentrep/bom/StringMatcher.cpp
===================================================================
--- trunk/opentrep/opentrep/bom/StringMatcher.cpp 2009-07-21 21:56:56 UTC (rev 157)
+++ trunk/opentrep/opentrep/bom/StringMatcher.cpp 2009-07-22 06:06:26 UTC (rev 158)
@@ -44,12 +44,9 @@
corrections. If the word is correctly spelled, it is copied as
is. Otherwise, a corrected version is stored. */
// //////////////////////////////////////////////////////////////////////
- static bool createCorrectedWordList (const WordList_T& iOriginalWordList,
+ static void createCorrectedWordList (const WordList_T& iOriginalWordList,
WordList_T& ioCorrectedWordList,
- EditDistance_T& ioMaxEditDistance,
const Xapian::Database& iDatabase) {
- bool hasReachedMaximalAllowableEditDistance = false;
- EditDistance_T lMaxEditDistance= std::numeric_limits<EditDistance_T>::min();
// Empty the target list
ioCorrectedWordList.clear();
@@ -62,16 +59,9 @@
const std::string& lOriginalWord = *itWord;
// Calculate the distance, depending on the length of the word
- EditDistance_T lCalculatedEditDistance =
+ const EditDistance_T lCalculatedEditDistance =
calculateEditDistance (lOriginalWord);
- // Store the greatest edit distance/error
- lMaxEditDistance = std::max (lMaxEditDistance, lCalculatedEditDistance);
-
- // Limit the edit distance to the given maximal one
- lCalculatedEditDistance = std::min (lCalculatedEditDistance,
- ioMaxEditDistance);
-
// Get a spelling suggestion for that word
const std::string& lSuggestedWord =
iDatabase.get_spelling_suggestion (lOriginalWord,
@@ -88,36 +78,24 @@
/*
OPENTREP_LOG_DEBUG ("Original word: `" << lOriginalWord
<< "' ==> corrected word: `"
- << lSuggestedWord
- << "'. Maximal allowable distance: "
- << ioMaxEditDistance
- << ", maximum of edit distance: "
- << lMaxEditDistance
- << ", calculated edit distance: "
- << lCalculatedEditDistance);
+ << lSuggestedWord << "'");
*/
}
- /**
- When the maximal allowable edit distance reaches the maximum
- of the calculated edit distance, it becomes useless to go on
- increasing the maximal allowable edit distance.
- */
- if (lMaxEditDistance <= ioMaxEditDistance) {
- hasReachedMaximalAllowableEditDistance = true;
- }
-
} catch (const Xapian::Error& error) {
OPENTREP_LOG_ERROR ("Exception: " << error.get_msg());
}
-
- return hasReachedMaximalAllowableEditDistance;
}
// ///////////////////////////////////////////////////////////////////
- std::string StringMatcher::searchString (Xapian::MSet& ioMatchingSet,
- const std::string& iSearchString,
- const Xapian::Database& ioDatabase) {
+ std::string StringMatcher::
+ searchString (Xapian::MSet& ioMatchingSet,
+ const std::string& iSearchString,
+ NbOfErrors_T& ioMaxEditDistance,
+ bool ioHasReachedMaximalAllowableEditDistance,
+ const Xapian::Database& ioDatabase) {
+ NbOfErrors_T lMaxEditDistance = std::numeric_limits<EditDistance_T>::min();
+
std::string oMatchedString;
// Catch any Xapian::Error exceptions thrown
@@ -207,81 +185,79 @@
(where the words have been corrected one by one).
*/
if (nbMatches != 0) {
+ /**
+ When the maximal allowable edit distance reaches the maximum
+ of the calculated edit distance, it becomes useless to go on
+ increasing the maximal allowable edit distance.
+ */
+ if (lMaxEditDistance <= ioMaxEditDistance) {
+ ioHasReachedMaximalAllowableEditDistance = true;
+ }
+
oMatchedString = lOriginalQueryString;
return oMatchedString;
}
assert (nbMatches == 0 && oMatchedString.empty() == true);
- bool shouldStop = false;
- EditDistance_T lMaxEditDistance = 0;
/**
From the clean list of original words, another list of
- spell-corrected words is built.
- <br>The original query string must first be checked, without
- allowing any spelling errors. If no match is found, the list
- of spell-corrected words must be rebuilt allowing an
- distance/error of 1. If again no match is found, the list is
- rebuilt allowing a distance/error of 2. And so on until the
- maximal allowable distance/error is reached.
+ spell-corrected words is built, where every word is
+ spell-corrected one by one, according to its own length.
+ <br>For instance, "sna" can not be corrected into "san", as
+ "sna" has a length of 3 letters only, but "francicso" will be
+ corrected into "francisco". So, "sna francicso" will be
+ corrected into "sna francisco".
*/
- while (shouldStop == false) {
- WordList_T lCorrectedWordList;
- const bool hasReachedMaximalAllowableEditDistance =
- createCorrectedWordList (lOriginalWordList, lCorrectedWordList,
- lMaxEditDistance, ioDatabase);
+ WordList_T lCorrectedWordList;
+ createCorrectedWordList (lOriginalWordList, lCorrectedWordList,
+ ioDatabase);
- const std::string lCorrectedQueryString =
- WordHolder::createStringFromWordList (lCorrectedWordList);
+ const std::string lCorrectedQueryString =
+ WordHolder::createStringFromWordList (lCorrectedWordList);
- /**
- Strangely enough (is it?), the corrected query given by the Xapian
- QueryParser corresponds to the full original string, where words
- have been corrected one by one, but considered as a single block.
- <br>With the above example, 'sna francicso' yields (wrongly)
- 'sna francisco', instead of "sna PHRASE 2 francisco", as generated
- by the following (uncommented) code.
- <br>Just to be clear, the two lines of commented code below
- are wrong. Those two commented lines are kept, just to be
- sure that nobody gets the idea to use them.
- */
- // Xapian::Query lCorrectedQuery =
- // lQueryParser.get_corrected_query_string();
- Xapian::Query lCorrectedQuery =
- lQueryParser.parse_query (lCorrectedQueryString,
- Xapian::QueryParser::FLAG_BOOLEAN
- | Xapian::QueryParser::FLAG_PHRASE
- | Xapian::QueryParser::FLAG_LOVEHATE);
+ /**
+ Strangely enough (is it?), the corrected query given by the Xapian
+ QueryParser corresponds to the full original string, where words
+ have been corrected one by one, but considered as a single block.
+ <br>With the above example, 'sna francicso' yields (wrongly)
+ 'sna francisco', instead of "sna PHRASE 2 francisco", as generated
+ by the following (uncommented) code.
+ <br>Just to be clear, the two lines of commented code below
+ are wrong. Those two commented lines are kept, just to be
+ sure that nobody gets the idea to use them.
+ */
+ // Xapian::Query lCorrectedQuery =
+ // lQueryParser.get_corrected_query_string();
+ Xapian::Query lCorrectedQuery =
+ lQueryParser.parse_query (lCorrectedQueryString,
+ Xapian::QueryParser::FLAG_BOOLEAN
+ | Xapian::QueryParser::FLAG_PHRASE
+ | Xapian::QueryParser::FLAG_LOVEHATE);
+
+ enquire.set_query (lCorrectedQuery);
+ ioMatchingSet = enquire.get_mset (0, 10);
- enquire.set_query (lCorrectedQuery);
- ioMatchingSet = enquire.get_mset (0, 10);
-
- // Display the results
- nbMatches = ioMatchingSet.size();
-
- // DEBUG
- OPENTREP_LOG_DEBUG ("Corrected query `" << lCorrectedQueryString
- << "' with maximal edit distance of "
- << lMaxEditDistance
- << ", i.e., `" << lCorrectedQuery.get_description()
- << "' => " << nbMatches
- << " results found on corrected string");
-
- if (nbMatches != 0) {
- oMatchedString = lCorrectedQueryString;
- return oMatchedString;
- }
-
- // Allow for one more spelling error
- ++lMaxEditDistance;
-
+ // Display the results
+ nbMatches = ioMatchingSet.size();
+
+ // DEBUG
+ OPENTREP_LOG_DEBUG ("Corrected query `" << lCorrectedQueryString
+ << "', i.e., `" << lCorrectedQuery.get_description()
+ << "' => " << nbMatches
+ << " results found on corrected string");
+
+ if (nbMatches != 0) {
/**
- Stop when it is no longer necessary to increase the maximal
- allowable edit distance, as it is already greater than the
- maximum of the calculated edit distance.
+ When the maximal allowable edit distance reaches the maximum
+ of the calculated edit distance, it becomes useless to go on
+ increasing the maximal allowable edit distance.
*/
- if (hasReachedMaximalAllowableEditDistance == true) {
- shouldStop = true;
+ if (lMaxEditDistance <= ioMaxEditDistance) {
+ ioHasReachedMaximalAllowableEditDistance = true;
}
+
+ oMatchedString = lCorrectedQueryString;
+ return oMatchedString;
}
assert (nbMatches == 0 && oMatchedString.empty() == true);
@@ -290,13 +266,20 @@
phrase/string. With the above example, 'sna francisco' yields the
suggestion 'san francisco'.
*/
- const EditDistance_T lCalculatedEditDistance =
+ NbOfErrors_T lCalculatedEditDistance =
calculateEditDistance (lOriginalQueryString);
+
+ // Store the greatest edit distance/error
+ lMaxEditDistance = std::max (lMaxEditDistance, lCalculatedEditDistance);
+ // Limit the edit distance to the given maximal one
+ lCalculatedEditDistance = std::min (lCalculatedEditDistance,
+ ioMaxEditDistance);
+
const std::string lFullWordCorrectedString =
ioDatabase.get_spelling_suggestion (lOriginalQueryString,
lCalculatedEditDistance);
-
+
/**
Since there is still no match, we search on the string
corrected as a whole.
@@ -309,24 +292,26 @@
Xapian::QueryParser::FLAG_BOOLEAN
| Xapian::QueryParser::FLAG_PHRASE
| Xapian::QueryParser::FLAG_LOVEHATE);
-
+
enquire.set_query (lFullQueryCorrected);
ioMatchingSet = enquire.get_mset (0, 10);
// Display the results
nbMatches = ioMatchingSet.size();
-
+
// DEBUG
OPENTREP_LOG_DEBUG ("Query corrected as a full sentence `"
- << lFullWordCorrectedString << "', i.e., `"
- << lFullQueryCorrected.get_description() << "' => "
- << nbMatches
+ << lFullWordCorrectedString
+ << "' with a maximal edit distance of "
+ << lMaxEditDistance
+ << ", i.e., `"<< lFullQueryCorrected.get_description()
+ << "' => " << nbMatches
<< " results found on corrected full string");
-
+
if (nbMatches != 0) {
oMatchedString = lFullWordCorrectedString;
}
-
+
// DEBUG
/*
const Xapian::Query& lActualQuery = enquire.get_query();
@@ -338,6 +323,15 @@
OPENTREP_LOG_ERROR ("Exception: " << error.get_msg());
}
+ /**
+ When the maximal allowable edit distance reaches the maximum
+ of the calculated edit distance, it becomes useless to go on
+ increasing the maximal allowable edit distance.
+ */
+ if (lMaxEditDistance <= ioMaxEditDistance) {
+ ioHasReachedMaximalAllowableEditDistance = true;
+ }
+
return oMatchedString;
}
Modified: trunk/opentrep/opentrep/bom/StringMatcher.hpp
===================================================================
--- trunk/opentrep/opentrep/bom/StringMatcher.hpp 2009-07-21 21:56:56 UTC (rev 157)
+++ trunk/opentrep/opentrep/bom/StringMatcher.hpp 2009-07-22 06:06:26 UTC (rev 158)
@@ -28,11 +28,17 @@
words of the search string.
@param Xapian::MSet& The Xapian matching set. It can be empty.
@param const std::string& The query string.
+ @param NbOfErrors_T& The maximal allowable edit distance/error.
+ @param bool Whether or not the maximal allowable edit distance/error
+ has become greater than the maximum of the edit distance/errors
+ calculated on the phrase.
@param const Xapian::Database& The Xapian index/database.
@return std::string The query string, potentially corrected,
which has yielded matches. */
static std::string searchString (Xapian::MSet&,
const std::string& iSearchString,
+ NbOfErrors_T& ioMaxEditDistance,
+ bool ioHasReachedMaximalAllowableEditDistance,
const Xapian::Database&);
/** Extract the best matching Xapian document.
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <den...@us...> - 2009-07-21 21:56:59
|
Revision: 157
http://opentrep.svn.sourceforge.net/opentrep/?rev=157&view=rev
Author: denis_arnaud
Date: 2009-07-21 21:56:56 +0000 (Tue, 21 Jul 2009)
Log Message:
-----------
[Dev] Worked on the edit distance calculation.
Modified Paths:
--------------
trunk/opentrep/opentrep/batches/opentrep_searcher.cfg
trunk/opentrep/opentrep/batches/searcher.cpp
trunk/opentrep/opentrep/bom/ResultHolder.cpp
trunk/opentrep/opentrep/bom/StringMatcher.cpp
Modified: trunk/opentrep/opentrep/batches/opentrep_searcher.cfg
===================================================================
--- trunk/opentrep/opentrep/batches/opentrep_searcher.cfg 2009-07-21 16:57:52 UTC (rev 156)
+++ trunk/opentrep/opentrep/batches/opentrep_searcher.cfg 2009-07-21 21:56:56 UTC (rev 157)
@@ -1,9 +1,9 @@
database=../../test/traveldb
-log=opentrep_indexer.log
+log=opentrep_searcher.log
user=opentrep
passwd=opentrep
host=localhost
port=3306
dbname=trep_opentrep
error=3
-query="sna francicso rio de janero lso angles reykyavki"
\ No newline at end of file
+query="sna francicso rio de janero lso angles reykyavki"
Modified: trunk/opentrep/opentrep/batches/searcher.cpp
===================================================================
--- trunk/opentrep/opentrep/batches/searcher.cpp 2009-07-21 16:57:52 UTC (rev 156)
+++ trunk/opentrep/opentrep/batches/searcher.cpp 2009-07-21 21:56:56 UTC (rev 157)
@@ -247,6 +247,7 @@
int main (int argc, char* argv[]) {
try {
+ /*
const OPENTREP::NbOfLetters_T lScaleArray[5] = {3, 6, 9, 14, 19};
const OPENTREP::DistanceErrorScaleArray_T lScaleBoostArray =
@@ -264,6 +265,7 @@
}
return 0;
+ */
// Travel query
OPENTREP::TravelQuery_T lTravelQuery;
@@ -316,7 +318,7 @@
lNonMatchedWordList);
std::cout << nbOfMatches << " (geographical) location(s) have been found "
- << "matching your query (`" << lTravelQuery << "\xB4). "
+ << "matching your query (`" << lTravelQuery << "'). "
<< lNonMatchedWordList.size() << " words were left unmatched."
<< std::endl;
Modified: trunk/opentrep/opentrep/bom/ResultHolder.cpp
===================================================================
--- trunk/opentrep/opentrep/bom/ResultHolder.cpp 2009-07-21 16:57:52 UTC (rev 156)
+++ trunk/opentrep/opentrep/bom/ResultHolder.cpp 2009-07-21 21:56:56 UTC (rev 157)
@@ -85,8 +85,8 @@
// DEBUG
OPENTREP_LOG_DEBUG ("Current initial query string: `"
- << ioPartialQueryString << "´ --- Kept query: `"
- << oMatchedString << "´ for "
+ << ioPartialQueryString << "' --- Kept query: `"
+ << oMatchedString << "' for "
<< lMatchingSet.size() << " matches.");
if (lMatchingSet.empty() == false) {
@@ -160,8 +160,8 @@
lMatchingDocument.notifyIfExtraMatch();
OPENTREP_LOG_DEBUG ("==> " << lNbOfMatches
<< " matches for the query string: `"
- << lMatchedString << "´ (from `"
- << lQueryString << "´)");
+ << lMatchedString << "' (from `"
+ << lQueryString << "')");
/**
Remove, from the lRemainingQueryString string, the part
Modified: trunk/opentrep/opentrep/bom/StringMatcher.cpp
===================================================================
--- trunk/opentrep/opentrep/bom/StringMatcher.cpp 2009-07-21 16:57:52 UTC (rev 156)
+++ trunk/opentrep/opentrep/bom/StringMatcher.cpp 2009-07-21 21:56:56 UTC (rev 157)
@@ -7,6 +7,7 @@
#include <istream>
#include <ostream>
#include <sstream>
+#include <limits>
#include <string>
#include <list>
#include <map>
@@ -43,9 +44,13 @@
corrections. If the word is correctly spelled, it is copied as
is. Otherwise, a corrected version is stored. */
// //////////////////////////////////////////////////////////////////////
- static void createCorrectedWordList (const WordList_T& iOriginalWordList,
+ static bool createCorrectedWordList (const WordList_T& iOriginalWordList,
WordList_T& ioCorrectedWordList,
+ EditDistance_T& ioMaxEditDistance,
const Xapian::Database& iDatabase) {
+ bool hasReachedMaximalAllowableEditDistance = false;
+ EditDistance_T lMaxEditDistance= std::numeric_limits<EditDistance_T>::min();
+
// Empty the target list
ioCorrectedWordList.clear();
@@ -55,9 +60,22 @@
for (WordList_T::const_iterator itWord = iOriginalWordList.begin();
itWord != iOriginalWordList.end(); ++itWord) {
const std::string& lOriginalWord = *itWord;
- const EditDistance_T lEditDistance= calculateEditDistance(lOriginalWord);
+
+ // Calculate the distance, depending on the length of the word
+ EditDistance_T lCalculatedEditDistance =
+ calculateEditDistance (lOriginalWord);
+
+ // Store the greatest edit distance/error
+ lMaxEditDistance = std::max (lMaxEditDistance, lCalculatedEditDistance);
+
+ // Limit the edit distance to the given maximal one
+ lCalculatedEditDistance = std::min (lCalculatedEditDistance,
+ ioMaxEditDistance);
+
+ // Get a spelling suggestion for that word
const std::string& lSuggestedWord =
- iDatabase.get_spelling_suggestion (lOriginalWord, lEditDistance);
+ iDatabase.get_spelling_suggestion (lOriginalWord,
+ lCalculatedEditDistance);
if (lSuggestedWord.empty() == true) {
ioCorrectedWordList.push_back (lOriginalWord);
@@ -68,14 +86,32 @@
// DEBUG
/*
- OPENTREP_LOG_DEBUG ("Original word: `" << lOriginalWord
- << "' ==> corrected word: `" << lSuggestedWord << "'");
+ OPENTREP_LOG_DEBUG ("Original word: `" << lOriginalWord
+ << "' ==> corrected word: `"
+ << lSuggestedWord
+ << "'. Maximal allowable distance: "
+ << ioMaxEditDistance
+ << ", maximum of edit distance: "
+ << lMaxEditDistance
+ << ", calculated edit distance: "
+ << lCalculatedEditDistance);
*/
}
+ /**
+ When the maximal allowable edit distance reaches the maximum
+ of the calculated edit distance, it becomes useless to go on
+ increasing the maximal allowable edit distance.
+ */
+ if (lMaxEditDistance <= ioMaxEditDistance) {
+ hasReachedMaximalAllowableEditDistance = true;
+ }
+
} catch (const Xapian::Error& error) {
OPENTREP_LOG_ERROR ("Exception: " << error.get_msg());
}
+
+ return hasReachedMaximalAllowableEditDistance;
}
// ///////////////////////////////////////////////////////////////////
@@ -109,43 +145,17 @@
WordHolder::tokeniseStringIntoWordList (iSearchString, lOriginalWordList);
/**
- We rebuild a clean query string from the word list. Indeed,
+ A clean query string is rebuilt from the word list. Indeed,
the original string may have contained a few separators
(e.g., '/', ';', etc.), which have been removed by the
tokeniseStringIntoWordList() method. All those separators are
thus replaced by spaces.
- For instance, the 'san francisco, ca, us' initial string would be
- replaced by 'san francisco ca us'.
+ <br>For instance, the 'san francisco, ca, us' initial string
+ would be replaced by 'san francisco ca us'.
*/
const std::string lOriginalQueryString =
WordHolder::createStringFromWordList (lOriginalWordList);
-
- WordList_T lCorrectedWordList;
- createCorrectedWordList (lOriginalWordList, lCorrectedWordList,
- ioDatabase);
-
- const std::string lCorrectedQueryString =
- WordHolder::createStringFromWordList (lCorrectedWordList);
- /**
- Try to find, if relevant, an orthographic suggestion for the whole
- phrase/string. With the above example, 'sna francisco' yields the
- suggestion 'san francisco'.
- */
- const EditDistance_T lEditDistance =
- calculateEditDistance (lOriginalQueryString);
- const std::string lFullWordCorrectedString =
- ioDatabase.get_spelling_suggestion(lOriginalQueryString, lEditDistance);
-
- // DEBUG
- /*
- OPENTREP_LOG_DEBUG ("Query string `" << lOriginalQueryString
- << "' ==> corrected query string: `"
- << lCorrectedQueryString
- << "' and correction for the full query string: `"
- << lFullWordCorrectedString << "'");
- */
-
// Build the query object
Xapian::QueryParser lQueryParser;
lQueryParser.set_database (ioDatabase);
@@ -163,58 +173,23 @@
<< "'");
*/
+ // Start an enquire session
+ Xapian::Enquire enquire (ioDatabase);
+
/**
- The Xapian::QueryParser::parse_query() method aggregates all the words
- with operators inbetween them (here, the "PHRASE" operator).
- With the above example ('sna francicso'), it yields
- "sna PHRASE 2 francicso".
+ The Xapian::QueryParser::parse_query() method aggregates all
+ the words with operators inbetween them (here, the "PHRASE"
+ operator). With the above example ('sna francicso'), it
+ yields "sna PHRASE 2 francicso".
*/
- Xapian::Query lQuery =
+ Xapian::Query lOriginalQuery =
lQueryParser.parse_query (lOriginalQueryString,
Xapian::QueryParser::FLAG_BOOLEAN
| Xapian::QueryParser::FLAG_PHRASE
- | Xapian::QueryParser::FLAG_LOVEHATE
- | Xapian::QueryParser::FLAG_SPELLING_CORRECTION);
- /**
- Strangely enough (is it?), the corrected query given by the Xapian
- QueryParser corresponds to the full original string, where words
- have been corrected one by one, but considered as a single block.
- With the above example, 'sna francicso' yields (wrongly)
- 'sna francisco', instead of "sna PHRASE 2 francisco", as generated
- by the following code.
- */
- // Xapian::Query lCorrectedQuery =
- // lQueryParser.get_corrected_query_string();
- Xapian::Query lCorrectedQuery =
- lQueryParser.parse_query (lCorrectedQueryString,
- Xapian::QueryParser::FLAG_BOOLEAN
- | Xapian::QueryParser::FLAG_PHRASE
| Xapian::QueryParser::FLAG_LOVEHATE);
- /**
- As, with the above example, the full corrected string is
- 'san francisco', it yields the query "san PHRASE 2 francisco",
- which is eventually right.
- */
- Xapian::Query lFullQueryCorrected =
- lQueryParser.parse_query (lFullWordCorrectedString,
- Xapian::QueryParser::FLAG_BOOLEAN
- | Xapian::QueryParser::FLAG_PHRASE
- | Xapian::QueryParser::FLAG_LOVEHATE);
-
- // DEBUG
- /*
- OPENTREP_LOG_DEBUG ("Query `" << lQuery.get_description()
- << "', corrected query `" << lCorrectedQuery.get_description()
- << "' and corrected for full query `"
- << lFullQueryCorrected.get_description() << "' ");
- */
-
- // Start an enquire session
- Xapian::Enquire enquire (ioDatabase);
-
// Give the query object to the enquire session
- enquire.set_query (lQuery);
+ enquire.set_query (lOriginalQuery);
// Get the top 10 results of the query
ioMatchingSet = enquire.get_mset (0, 10);
@@ -223,13 +198,60 @@
int nbMatches = ioMatchingSet.size();
// DEBUG
- // OPENTREP_LOG_DEBUG (nbMatches << " results found");
+ OPENTREP_LOG_DEBUG ("Original query `" << lOriginalQueryString
+ << ", i.e., `" << lOriginalQuery.get_description()
+ << "' => " << nbMatches << " results found");
/**
When no match is found, we search on the corrected phrase/string
(where the words have been corrected one by one).
*/
- if (nbMatches == 0) {
+ if (nbMatches != 0) {
+ oMatchedString = lOriginalQueryString;
+ return oMatchedString;
+ }
+ assert (nbMatches == 0 && oMatchedString.empty() == true);
+
+ bool shouldStop = false;
+ EditDistance_T lMaxEditDistance = 0;
+ /**
+ From the clean list of original words, another list of
+ spell-corrected words is built.
+ <br>The original query string must first be checked, without
+ allowing any spelling errors. If no match is found, the list
+ of spell-corrected words must be rebuilt allowing an
+ distance/error of 1. If again no match is found, the list is
+ rebuilt allowing a distance/error of 2. And so on until the
+ maximal allowable distance/error is reached.
+ */
+ while (shouldStop == false) {
+ WordList_T lCorrectedWordList;
+ const bool hasReachedMaximalAllowableEditDistance =
+ createCorrectedWordList (lOriginalWordList, lCorrectedWordList,
+ lMaxEditDistance, ioDatabase);
+
+ const std::string lCorrectedQueryString =
+ WordHolder::createStringFromWordList (lCorrectedWordList);
+
+ /**
+ Strangely enough (is it?), the corrected query given by the Xapian
+ QueryParser corresponds to the full original string, where words
+ have been corrected one by one, but considered as a single block.
+ <br>With the above example, 'sna francicso' yields (wrongly)
+ 'sna francisco', instead of "sna PHRASE 2 francisco", as generated
+ by the following (uncommented) code.
+ <br>Just to be clear, the two lines of commented code below
+ are wrong. Those two commented lines are kept, just to be
+ sure that nobody gets the idea to use them.
+ */
+ // Xapian::Query lCorrectedQuery =
+ // lQueryParser.get_corrected_query_string();
+ Xapian::Query lCorrectedQuery =
+ lQueryParser.parse_query (lCorrectedQueryString,
+ Xapian::QueryParser::FLAG_BOOLEAN
+ | Xapian::QueryParser::FLAG_PHRASE
+ | Xapian::QueryParser::FLAG_LOVEHATE);
+
enquire.set_query (lCorrectedQuery);
ioMatchingSet = enquire.get_mset (0, 10);
@@ -237,32 +259,71 @@
nbMatches = ioMatchingSet.size();
// DEBUG
- //OPENTREP_LOG_DEBUG(nbMatches << " results found on corrected string");
+ OPENTREP_LOG_DEBUG ("Corrected query `" << lCorrectedQueryString
+ << "' with maximal edit distance of "
+ << lMaxEditDistance
+ << ", i.e., `" << lCorrectedQuery.get_description()
+ << "' => " << nbMatches
+ << " results found on corrected string");
- } else {
- oMatchedString = lOriginalQueryString;
+ if (nbMatches != 0) {
+ oMatchedString = lCorrectedQueryString;
+ return oMatchedString;
+ }
+
+ // Allow for one more spelling error
+ ++lMaxEditDistance;
+
+ /**
+ Stop when it is no longer necessary to increase the maximal
+ allowable edit distance, as it is already greater than the
+ maximum of the calculated edit distance.
+ */
+ if (hasReachedMaximalAllowableEditDistance == true) {
+ shouldStop = true;
+ }
}
+ assert (nbMatches == 0 && oMatchedString.empty() == true);
/**
- If there is still no match, we search on the string corrected
- as a whole.
+ Try to find, if relevant, an orthographic suggestion for the whole
+ phrase/string. With the above example, 'sna francisco' yields the
+ suggestion 'san francisco'.
*/
- if (nbMatches == 0) {
- enquire.set_query (lFullQueryCorrected);
- ioMatchingSet = enquire.get_mset (0, 10);
-
- // Display the results
- nbMatches = ioMatchingSet.size();
+ const EditDistance_T lCalculatedEditDistance =
+ calculateEditDistance (lOriginalQueryString);
- // DEBUG
- // OPENTREP_LOG_DEBUG (nbMatches
- // << " results found on corrected full string");
-
- } else {
- oMatchedString = lCorrectedQueryString;
- }
+ const std::string lFullWordCorrectedString =
+ ioDatabase.get_spelling_suggestion (lOriginalQueryString,
+ lCalculatedEditDistance);
- if (nbMatches != 0 && oMatchedString.empty() == true) {
+ /**
+ Since there is still no match, we search on the string
+ corrected as a whole.
+ <br>As, with the above example, the full corrected string is
+ 'san francisco', it yields the query "san PHRASE 2
+ francisco", which is eventually right.
+ */
+ Xapian::Query lFullQueryCorrected =
+ lQueryParser.parse_query (lFullWordCorrectedString,
+ Xapian::QueryParser::FLAG_BOOLEAN
+ | Xapian::QueryParser::FLAG_PHRASE
+ | Xapian::QueryParser::FLAG_LOVEHATE);
+
+ enquire.set_query (lFullQueryCorrected);
+ ioMatchingSet = enquire.get_mset (0, 10);
+
+ // Display the results
+ nbMatches = ioMatchingSet.size();
+
+ // DEBUG
+ OPENTREP_LOG_DEBUG ("Query corrected as a full sentence `"
+ << lFullWordCorrectedString << "', i.e., `"
+ << lFullQueryCorrected.get_description() << "' => "
+ << nbMatches
+ << " results found on corrected full string");
+
+ if (nbMatches != 0) {
oMatchedString = lFullWordCorrectedString;
}
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <den...@us...> - 2009-07-21 16:58:03
|
Revision: 156
http://opentrep.svn.sourceforge.net/opentrep/?rev=156&view=rev
Author: denis_arnaud
Date: 2009-07-21 16:57:52 +0000 (Tue, 21 Jul 2009)
Log Message:
-----------
Temporary searcher batch.
Modified Paths:
--------------
trunk/opentrep/opentrep/batches/searcher.cpp
Modified: trunk/opentrep/opentrep/batches/searcher.cpp
===================================================================
--- trunk/opentrep/opentrep/batches/searcher.cpp 2009-07-21 08:45:44 UTC (rev 155)
+++ trunk/opentrep/opentrep/batches/searcher.cpp 2009-07-21 16:57:52 UTC (rev 156)
@@ -247,6 +247,24 @@
int main (int argc, char* argv[]) {
try {
+ const OPENTREP::NbOfLetters_T lScaleArray[5] = {3, 6, 9, 14, 19};
+
+ const OPENTREP::DistanceErrorScaleArray_T lScaleBoostArray =
+ { {3, 6, 9, 14, 19} };
+
+ OPENTREP::DistanceErrorRule lScale (5, lScaleArray);
+ OPENTREP::DistanceErrorRule lScaleBoost (lScaleBoostArray);
+
+ std::cout << "Standard array: " << lScale << std::endl;
+ std::cout << "Boost array: " << lScaleBoost << std::endl;
+
+ for (int idx = 0; idx != 20; ++idx) {
+ std::cout << "For " << idx << " letters => "
+ << lScale.getAllowedDistanceError(idx) << std::endl;
+ }
+
+ return 0;
+
// Travel query
OPENTREP::TravelQuery_T lTravelQuery;
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <den...@us...> - 2009-07-21 08:45:46
|
Revision: 155
http://opentrep.svn.sourceforge.net/opentrep/?rev=155&view=rev
Author: denis_arnaud
Date: 2009-07-21 08:45:44 +0000 (Tue, 21 Jul 2009)
Log Message:
-----------
[Interface] Completed the DistanceErrorRule structure. There is still to take it into account.
Modified Paths:
--------------
trunk/opentrep/opentrep/DistanceErrorRule.hpp
Modified: trunk/opentrep/opentrep/DistanceErrorRule.hpp
===================================================================
--- trunk/opentrep/opentrep/DistanceErrorRule.hpp 2009-07-21 05:54:17 UTC (rev 154)
+++ trunk/opentrep/opentrep/DistanceErrorRule.hpp 2009-07-21 08:45:44 UTC (rev 155)
@@ -9,22 +9,43 @@
#include <ostream>
#include <string>
#include <map>
+// Boost Array
+#include <boost/array.hpp>
// OpenTrep
#include <opentrep/OPENTREP_Types.hpp>
#include <opentrep/OPENTREP_Abstract.hpp>
namespace OPENTREP {
+ // //////////////////////////////////////////////////////////////////
/** Number of letters in a word or phrase. */
typedef unsigned int NbOfLetters_T;
/** Number of errors. */
typedef unsigned short NbOfErrors_T;
- /** List of names for a given (geographical) distanceerrorrule. */
+ /** Number of (distance) errors allowed for a given number of letters. */
typedef std::map<NbOfLetters_T, NbOfErrors_T> DistanceErrorScale_T;
+ /** Number of (distance) errors allowed for a given number of letters. */
+ typedef boost::array<NbOfLetters_T, 5> DistanceErrorScaleArray_T;
+ // ////////////////////////////////////////////////////////////////////
+ /** Default distance error scale.
+ <br>Allowed error for a given number of letters:
+ <ul>
+ <li>< 4 => 0</li>
+ <li>< 7 => 1</li>
+ <li>< 10 => 2</li>
+ <li>< 15 => 3</li>
+ <li>< 20 => 4</li>
+ <li>>= 20 => (nb of letters) / 5</li>
+ </ul> */
+ static const DistanceErrorScaleArray_T K_DEFAULT_ERROR_SCALE =
+ { {3, 6, 9, 14, 19} };
+
+
+ // ///////////////////////////////////////////////////////////////
/** Structure modelling a (geographical) distanceerrorrule. */
struct DistanceErrorRule : public OPENTREP_Abstract {
public:
@@ -39,6 +60,27 @@
public:
+ // ///////// Business methods ////////
+ /** Get, for a given number of letters, the corresponding allowed
+ (distance) error. */
+ NbOfErrors_T getAllowedDistanceError (const NbOfLetters_T& iNbOfLetters) const {
+ NbOfErrors_T oNbOfErrors_T = 0;
+
+ DistanceErrorScale_T::const_iterator itError =
+ _scale.lower_bound (iNbOfLetters);
+
+ if (itError != _scale.end()) {
+ oNbOfErrors_T = itError->second;
+
+ } else {
+ oNbOfErrors_T = iNbOfLetters / 4;
+ }
+
+ return oNbOfErrors_T;
+ }
+
+
+ public:
// ///////// Display methods ////////
/** Dump a structure into an output stream.
@param ostream& the output stream. */
@@ -79,6 +121,23 @@
: _scale (iScale) {
}
+ /** Constructor. */
+ DistanceErrorRule (const NbOfErrors_T iSize,
+ const NbOfLetters_T iScaleArray[]) {
+ for (NbOfErrors_T idx = 0; idx != iSize; ++idx) {
+ _scale.insert (DistanceErrorScale_T::value_type (iScaleArray[idx],
+ idx));
+ }
+ }
+
+ /** Constructor. */
+ DistanceErrorRule (const DistanceErrorScaleArray_T& iScaleArray) {
+ for (NbOfErrors_T idx = 0; idx != iScaleArray.size(); ++idx) {
+ _scale.insert (DistanceErrorScale_T::value_type (iScaleArray[idx],
+ idx));
+ }
+ }
+
/** Default Constructor. */
// DistanceErrorRule ();
/** Default copy constructor. */
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <den...@us...> - 2009-07-21 05:54:19
|
Revision: 154
http://opentrep.svn.sourceforge.net/opentrep/?rev=154&view=rev
Author: denis_arnaud
Date: 2009-07-21 05:54:17 +0000 (Tue, 21 Jul 2009)
Log Message:
-----------
[Dev] Added a structure for the scale of distance errors.
Modified Paths:
--------------
trunk/opentrep/opentrep/OPENTREP_Service.hpp
trunk/opentrep/opentrep/sources.mk
Added Paths:
-----------
trunk/opentrep/opentrep/DistanceErrorRule.hpp
Added: trunk/opentrep/opentrep/DistanceErrorRule.hpp
===================================================================
--- trunk/opentrep/opentrep/DistanceErrorRule.hpp (rev 0)
+++ trunk/opentrep/opentrep/DistanceErrorRule.hpp 2009-07-21 05:54:17 UTC (rev 154)
@@ -0,0 +1,98 @@
+#ifndef __OPENTREP_DISTANCEERRORRULE_HPP
+#define __OPENTREP_DISTANCEERRORRULE_HPP
+
+// //////////////////////////////////////////////////////////////////////
+// Import section
+// //////////////////////////////////////////////////////////////////////
+// STL
+#include <istream>
+#include <ostream>
+#include <string>
+#include <map>
+// OpenTrep
+#include <opentrep/OPENTREP_Types.hpp>
+#include <opentrep/OPENTREP_Abstract.hpp>
+
+namespace OPENTREP {
+
+ /** Number of letters in a word or phrase. */
+ typedef unsigned int NbOfLetters_T;
+
+ /** Number of errors. */
+ typedef unsigned short NbOfErrors_T;
+
+ /** List of names for a given (geographical) distanceerrorrule. */
+ typedef std::map<NbOfLetters_T, NbOfErrors_T> DistanceErrorScale_T;
+
+
+ /** Structure modelling a (geographical) distanceerrorrule. */
+ struct DistanceErrorRule : public OPENTREP_Abstract {
+ public:
+ // ///////// Getters ////////
+ /** Get the DistanceErrorRule code. */
+ const DistanceErrorScale_T& getDistanceErrorScale() const {
+ return _scale;
+ }
+
+
+ // ///////// Setters //////////
+
+
+ public:
+ // ///////// Display methods ////////
+ /** Dump a structure into an output stream.
+ @param ostream& the output stream. */
+ void toStream (std::ostream& ioOut) const {
+ ioOut << toString();
+ }
+
+ /** Read a structure from an input stream.
+ @param istream& the input stream. */
+ void fromStream (std::istream&) {
+ }
+
+ /** Get a short display of the DistanceErrorRule structure. */
+ std::string toShortString() const {
+ std::ostringstream oStr;
+ NbOfLetters_T idx = 0;
+ for (DistanceErrorScale_T::const_iterator itError = _scale.begin();
+ itError != _scale.end(); ++itError, ++idx) {
+ if (idx != 0) {
+ oStr << ", ";
+ }
+ oStr << itError->second << ": " << itError->first;
+ }
+ return oStr.str();
+ }
+
+ /** Get the serialised version of the DistanceErrorRule structure. */
+ std::string toString() const {
+ std::ostringstream oStr;
+ oStr << toShortString();
+ return oStr.str();
+ }
+
+
+ public:
+ /** Main Constructor. */
+ DistanceErrorRule (const DistanceErrorScale_T& iScale)
+ : _scale (iScale) {
+ }
+
+ /** Default Constructor. */
+ // DistanceErrorRule ();
+ /** Default copy constructor. */
+ // DistanceErrorRule (const DistanceErrorRule&);
+
+ /** Destructor. */
+ virtual ~DistanceErrorRule() {}
+
+
+ private:
+ // /////// Attributes /////////
+ /** Scale of distance errors. */
+ DistanceErrorScale_T _scale;
+ };
+
+}
+#endif // __OPENTREP_DISTANCEERRORRULE_HPP
Modified: trunk/opentrep/opentrep/OPENTREP_Service.hpp
===================================================================
--- trunk/opentrep/opentrep/OPENTREP_Service.hpp 2009-07-20 22:41:55 UTC (rev 153)
+++ trunk/opentrep/opentrep/OPENTREP_Service.hpp 2009-07-21 05:54:17 UTC (rev 154)
@@ -11,6 +11,7 @@
#include <opentrep/OPENTREP_Types.hpp>
#include <opentrep/DBParams.hpp>
#include <opentrep/Location.hpp>
+#include <opentrep/DistanceErrorRule.hpp>
namespace OPENTREP {
Modified: trunk/opentrep/opentrep/sources.mk
===================================================================
--- trunk/opentrep/opentrep/sources.mk 2009-07-20 22:41:55 UTC (rev 153)
+++ trunk/opentrep/opentrep/sources.mk 2009-07-21 05:54:17 UTC (rev 154)
@@ -3,5 +3,6 @@
$(top_srcdir)/opentrep/OPENTREP_Abstract.hpp \
$(top_srcdir)/opentrep/DBParams.hpp \
$(top_srcdir)/opentrep/Location.hpp \
+ $(top_srcdir)/opentrep/DistanceErrorRule.hpp \
$(top_srcdir)/opentrep/OPENTREP_Service.hpp
service_cc_sources =
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <den...@us...> - 2009-07-20 22:42:05
|
Revision: 153
http://opentrep.svn.sourceforge.net/opentrep/?rev=153&view=rev
Author: denis_arnaud
Date: 2009-07-20 22:41:55 +0000 (Mon, 20 Jul 2009)
Log Message:
-----------
[Test] Fixed some bugs with the tests after the interface (API) change.
Modified Paths:
--------------
trunk/opentrep/opentrep/Makefile.am
trunk/opentrep/test/IndexBuildingTestSuite.cpp
trunk/opentrep/test/python/Makefile.am
Removed Paths:
-------------
trunk/opentrep/test/python/pyopentrep.cpp
trunk/opentrep/test/python/pyopentrep.py
Modified: trunk/opentrep/opentrep/Makefile.am
===================================================================
--- trunk/opentrep/opentrep/Makefile.am 2009-07-20 22:27:02 UTC (rev 152)
+++ trunk/opentrep/opentrep/Makefile.am 2009-07-20 22:41:55 UTC (rev 153)
@@ -5,7 +5,7 @@
MAINTAINERCLEANFILES = Makefile.in
-SUBDIRS = basic bom factory dbadaptor command service core config batches
+SUBDIRS = basic bom factory dbadaptor command service core config batches python
EXTRA_DIST = config_msvc.h
Modified: trunk/opentrep/test/IndexBuildingTestSuite.cpp
===================================================================
--- trunk/opentrep/test/IndexBuildingTestSuite.cpp 2009-07-20 22:27:02 UTC (rev 152)
+++ trunk/opentrep/test/IndexBuildingTestSuite.cpp 2009-07-20 22:41:55 UTC (rev 153)
@@ -24,16 +24,24 @@
// Open and clean the log outputfile
logOutputFile.open (lLogFilename.c_str());
logOutputFile.clear();
+
+ // SQL database parameters
+ OPENTREP::DBParams lDBParams ("opentrep", "opentrep", "localhost", "3306",
+ "trep_opentrep");
// Initialise the context
- OPENTREP::OPENTREP_Service opentrepService;
- opentrepService.init (logOutputFile, lXapianDatabaseName);
+ OPENTREP::OPENTREP_Service opentrepService (logOutputFile, lDBParams,
+ lXapianDatabaseName);
// Launch a simulation
//opentrepService.buildSearchIndex();
// Query the Xapian database (index)
- opentrepService.interpretTravelRequest (lTravelQuery);
+ OPENTREP::WordList_T lNonMatchedWordList;
+ OPENTREP::LocationList_T lLocationList;
+ const OPENTREP::NbOfMatches_T nbOfMatches =
+ opentrepService.interpretTravelRequest (lTravelQuery, lLocationList,
+ lNonMatchedWordList);
// Close the Log outputFile
logOutputFile.close();
Modified: trunk/opentrep/test/python/Makefile.am
===================================================================
--- trunk/opentrep/test/python/Makefile.am 2009-07-20 22:27:02 UTC (rev 152)
+++ trunk/opentrep/test/python/Makefile.am 2009-07-20 22:41:55 UTC (rev 153)
@@ -4,7 +4,7 @@
MAINTAINERCLEANFILES = Makefile.in
# Libraries
-lib_LTLIBRARIES = libpyboost.la libpy@PACKAGE@.la
+lib_LTLIBRARIES = libpyboost.la
libpyboost_la_SOURCES = boost_python.cpp
libpyboost_la_CXXFLAGS = ${PYTHON_CFLAGS}
@@ -12,10 +12,4 @@
libpyboost_la_LDFLAGS = ${PYTHON_LIBS} ${PYTHON_ADD_LIBS} \
${BOOST_PYTHON_LIB} -version-info $(GENERIC_LIBRARY_VERSION)
-libpy@PACKAGE@_la_SOURCES = pyopentrep.cpp
-libpy@PACKAGE@_la_CXXFLAGS = ${PYTHON_CFLAGS} ${BOOST_CFLAGS}
-libpy@PACKAGE@_la_LIBADD = $(top_builddir)/@PACKAGE@/core/lib@PACKAGE@.la
-libpy@PACKAGE@_la_LDFLAGS = ${PYTHON_LIBS} ${PYTHON_ADD_LIBS} \
- ${BOOST_PYTHON_LIB} -version-info $(GENERIC_LIBRARY_VERSION)
-
-EXTRA_DIST =
+EXTRA_DIST = boost_python.py
Deleted: trunk/opentrep/test/python/pyopentrep.cpp
===================================================================
--- trunk/opentrep/test/python/pyopentrep.cpp 2009-07-20 22:27:02 UTC (rev 152)
+++ trunk/opentrep/test/python/pyopentrep.cpp 2009-07-20 22:41:55 UTC (rev 153)
@@ -1,110 +0,0 @@
-// C
-#include <cassert>
-// STL
-#include <sstream>
-#include <string>
-#include <list>
-#include <vector>
-// Boost String
-#include <boost/python.hpp>
-// OpenTREP
-#include <opentrep/OPENTREP_Service.hpp>
-
-namespace OPENTREP {
-
- struct OpenTrepSearcher {
- public:
-
- /** Wrapper around the search use case. */
- std::string search (const std::string& iTravelQuery) {
- std::ostringstream oStr;
-
- if (_opentrepService == NULL) {
- oStr << "The OpenTREP service has not been initialise, i.e., "
- << "the init() method has not been called on the "
- << "OpenTrepSearcher object. Please do so.";
- return oStr.str();
- }
- assert (_opentrepService != NULL);
-
- // Query the Xapian database (index)
- WordList_T lNonMatchedWordList;
- LocationList_T lLocationList;
- const NbOfMatches_T nbOfMatches =
- _opentrepService->interpretTravelRequest (iTravelQuery, lLocationList,
- lNonMatchedWordList);
-
- if (nbOfMatches != 0) {
- NbOfMatches_T idx = 0;
- for (LocationList_T::const_iterator itLocation = lLocationList.begin();
- itLocation != lLocationList.end(); ++itLocation, ++idx) {
- const Location& lLocation = *itLocation;
- if (idx != 0) {
- oStr << ",";
- }
- oStr << lLocation.getLocationCode();
- }
- }
-
- if (lNonMatchedWordList.empty() == false) {
- oStr << ";";
- NbOfMatches_T idx = 0;
- for (WordList_T::const_iterator itWord = lNonMatchedWordList.begin();
- itWord != lNonMatchedWordList.end(); ++itWord, ++idx) {
- const Word_T& lWord = *itWord;
- if (idx != 0) {
- oStr << ",";
- }
- oStr << lWord;
- }
- }
-
- return oStr.str();
- }
-
- public:
- /** Default constructor. */
- OpenTrepSearcher() : _opentrepService (NULL), _logOutputStream (NULL) {
- }
-
- /** Default copy constructor. */
- OpenTrepSearcher (const OpenTrepSearcher& iOpenTrepSearcher)
- : _opentrepService (iOpenTrepSearcher._opentrepService),
- _logOutputStream (iOpenTrepSearcher._logOutputStream) {
- }
-
- /** Default constructor. */
- ~OpenTrepSearcher() {
- _opentrepService = NULL;
- _logOutputStream = NULL;
- }
-
- /** Wrapper around the search use case. */
- void init (const std::string& iXapianDatabaseFilepath,
- const std::string& iDBUser, const std::string& iDBPasswd,
- const std::string& iDBHost, const std::string& iDBPort,
- const std::string& iDBDBName) {
-
- // Output stream (for logs)
- _logOutputStream = new std::ostringstream();
-
- // Initialise the context
- DBParams lDBParams (iDBUser, iDBPasswd, iDBHost, iDBPort, iDBDBName);
- _opentrepService = new OPENTREP_Service (*_logOutputStream, lDBParams,
- iXapianDatabaseFilepath);
- }
-
- private:
- /** Handle on the OpenTREP services (API). */
- OPENTREP_Service* _opentrepService;
- std::ostringstream* _logOutputStream;
- };
-
-}
-
-// /////////////////////////////////////////////////////////////
-BOOST_PYTHON_MODULE(libpyopentrep) {
- boost::python::class_<OPENTREP::OpenTrepSearcher> ("OpenTrepSearcher")
- .def ("search", &OPENTREP::OpenTrepSearcher::search)
- .def ("init", &OPENTREP::OpenTrepSearcher::init);
-}
Deleted: trunk/opentrep/test/python/pyopentrep.py
===================================================================
--- trunk/opentrep/test/python/pyopentrep.py 2009-07-20 22:27:02 UTC (rev 152)
+++ trunk/opentrep/test/python/pyopentrep.py 2009-07-20 22:41:55 UTC (rev 153)
@@ -1,14 +0,0 @@
-#!/usr/bin/env python
-
-import sys
-sys.path.append('.libs')
-import libpyopentrep
-
-mySearch = libpyopentrep.OpenTrepSearcher()
-mySearch.init('../traveldb', 'opentrep', 'opentrep', 'localhost', '3306', 'opentrep')
-a = mySearch.search('sna francicso rio de janero lso angles reykyavki')
-print a
-
-#print ', '.join(['%s: %s' % (msg, id) for (msg, id) in zip(MsgList, IDList)])
-#a = myWorld.toSimpleString().split(',')
-#print '^'.join(a[::-1])
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <den...@us...> - 2009-07-20 22:27:08
|
Revision: 152
http://opentrep.svn.sourceforge.net/opentrep/?rev=152&view=rev
Author: denis_arnaud
Date: 2009-07-20 22:27:02 +0000 (Mon, 20 Jul 2009)
Log Message:
-----------
[DB] Changed the default SQL database name from opentrep to trep_opentrep.
Modified Paths:
--------------
trunk/opentrep/db/maintenance/create_and_fill_mysql_db.sh
trunk/opentrep/db/maintenance/drop_tables_from_mysql_db.sh
trunk/opentrep/opentrep/batches/indexer.cpp
trunk/opentrep/opentrep/batches/opentrep_indexer.cfg
trunk/opentrep/opentrep/batches/opentrep_searcher.cfg
trunk/opentrep/opentrep/batches/searcher.cpp
trunk/opentrep/opentrep/python/pyopentrep.py
Modified: trunk/opentrep/db/maintenance/create_and_fill_mysql_db.sh
===================================================================
--- trunk/opentrep/db/maintenance/create_and_fill_mysql_db.sh 2009-07-20 22:11:59 UTC (rev 151)
+++ trunk/opentrep/db/maintenance/create_and_fill_mysql_db.sh 2009-07-20 22:27:02 UTC (rev 152)
@@ -3,42 +3,59 @@
# One parameter is required for this script:
# - the username
#
-# Two parameters are optional:
+# Three parameters are optional:
+# - the database name
# - the host server of the database
# - the port of the database
#
-if [ "$1" = "" -o "$1" = "-h" -o "$1" = "--help" ];
+if [ "$1" = "-h" -o "$1" = "--h" -o "$1" = "--help" ];
then
- echo "Usage: $0 <Database Username> [<Database Server Hostname> [<Database Server Port>]]"
+ echo "Usage: $0 [<Database Username> [<Database Name> [<Database Server Hostname> [<Database Server Port>]]]]"
echo ""
+ echo "Default values:"
+ echo "<Database Username> = opentrep"
+ echo "<Database Name> = trep_opentrep"
+ echo "<Database Server Hostname> = localhost"
+ echo "<Database Server Port> = 3306"
+ echo ""
exit -1
fi
##
# Database Server Hostname
-DB_HOST="localhost"
+# Database User
+DB_USER="opentrep"
+if [ "$1" != "" ];
+then
+ DB_USER="$1"
+fi
+
+# Database Password
+DB_PASSWD="${DB_USER}"
+
+# Database Name
+DB_NAME="trep_${DB_USER}"
if [ "$2" != "" ];
then
- DB_HOST="$2"
+ DB_NAME="$2"
fi
+DB_HOST="localhost"
+if [ "$3" != "" ];
+then
+ DB_HOST="$3"
+fi
+
# Database Server Port
DB_PORT="3306"
-if [ "$3" != "" ];
+if [ "$4" != "" ];
then
- DB_PORT="$3"
+ DB_PORT="$4"
fi
-# Database User
-DB_USER="$1"
-# Database Password
-DB_PASSWD="${DB_USER}"
-
-# Database Name
-DB_NAME="opentrep"
-
+##
# Check file existence
function checkSQLFile() {
if [ ! -r ${SQL_FILE} ]; then
Modified: trunk/opentrep/db/maintenance/drop_tables_from_mysql_db.sh
===================================================================
--- trunk/opentrep/db/maintenance/drop_tables_from_mysql_db.sh 2009-07-20 22:11:59 UTC (rev 151)
+++ trunk/opentrep/db/maintenance/drop_tables_from_mysql_db.sh 2009-07-20 22:27:02 UTC (rev 152)
@@ -3,42 +3,58 @@
# One parameter is required for this script:
# - the username
#
-# Two parameters are optional:
+# Three parameters are optional:
+# - the database name
# - the host server of the database
# - the port of the database
#
-if [ "$1" = "" -o "$1" = "-h" -o "$1" = "--help" ];
+if [ "$1" = "-h" -o "$1" = "--h" -o "$1" = "--help" ];
then
- echo "Usage: $0 <Database Username> [<Database Server Hostname> [<Database Server Port>]]"
+ echo "Usage: $0 [<Database Username> [<Database Name> [<Database Server Hostname> [<Database Server Port>]]]]"
echo ""
+ echo "Default values:"
+ echo "<Database Username> = opentrep"
+ echo "<Database Name> = trep_opentrep"
+ echo "<Database Server Hostname> = localhost"
+ echo "<Database Server Port> = 3306"
+ echo ""
exit -1
fi
##
# Database Server Hostname
-DB_HOST="localhost"
+# Database User
+DB_USER="opentrep"
+if [ "$1" != "" ];
+then
+ DB_USER="$1"
+fi
+
+# Database Password
+DB_PASSWD="${DB_USER}"
+
+# Database Name
+DB_NAME="trep_${DB_USER}"
if [ "$2" != "" ];
then
- DB_HOST="$2"
+ DB_NAME="$2"
fi
+DB_HOST="localhost"
+if [ "$3" != "" ];
+then
+ DB_HOST="$3"
+fi
+
# Database Server Port
DB_PORT="3306"
-if [ "$3" != "" ];
+if [ "$4" != "" ];
then
- DB_PORT="$3"
+ DB_PORT="$4"
fi
-# Database User
-DB_USER="$1"
-
-# Database Password
-DB_PASSWD="${DB_USER}"
-
-# Database Name
-DB_NAME="opentrep"
-
+##
# Drop a table
function dropTable() {
echo "The ${TABLE} table will be dropped:"
Modified: trunk/opentrep/opentrep/batches/indexer.cpp
===================================================================
--- trunk/opentrep/opentrep/batches/indexer.cpp 2009-07-20 22:11:59 UTC (rev 151)
+++ trunk/opentrep/opentrep/batches/indexer.cpp 2009-07-20 22:27:02 UTC (rev 152)
@@ -30,7 +30,7 @@
/** Default name and location for the Xapian database. */
const std::string K_OPENTREP_DEFAULT_DB_USER ("opentrep");
const std::string K_OPENTREP_DEFAULT_DB_PASSWD ("opentrep");
-const std::string K_OPENTREP_DEFAULT_DB_DBNAME ("opentrep");
+const std::string K_OPENTREP_DEFAULT_DB_DBNAME ("trep_opentrep");
const std::string K_OPENTREP_DEFAULT_DB_HOST ("localhost");
const std::string K_OPENTREP_DEFAULT_DB_PORT ("3306");
Modified: trunk/opentrep/opentrep/batches/opentrep_indexer.cfg
===================================================================
--- trunk/opentrep/opentrep/batches/opentrep_indexer.cfg 2009-07-20 22:11:59 UTC (rev 151)
+++ trunk/opentrep/opentrep/batches/opentrep_indexer.cfg 2009-07-20 22:27:02 UTC (rev 152)
@@ -4,4 +4,4 @@
passwd=opentrep
host=localhost
port=3306
-dbname=opentrep
+dbname=trep_opentrep
Modified: trunk/opentrep/opentrep/batches/opentrep_searcher.cfg
===================================================================
--- trunk/opentrep/opentrep/batches/opentrep_searcher.cfg 2009-07-20 22:11:59 UTC (rev 151)
+++ trunk/opentrep/opentrep/batches/opentrep_searcher.cfg 2009-07-20 22:27:02 UTC (rev 152)
@@ -4,6 +4,6 @@
passwd=opentrep
host=localhost
port=3306
-dbname=opentrep
+dbname=trep_opentrep
error=3
query="sna francicso rio de janero lso angles reykyavki"
\ No newline at end of file
Modified: trunk/opentrep/opentrep/batches/searcher.cpp
===================================================================
--- trunk/opentrep/opentrep/batches/searcher.cpp 2009-07-20 22:11:59 UTC (rev 151)
+++ trunk/opentrep/opentrep/batches/searcher.cpp 2009-07-20 22:27:02 UTC (rev 152)
@@ -34,7 +34,7 @@
/** Default name and location for the Xapian database. */
const std::string K_OPENTREP_DEFAULT_DB_USER ("opentrep");
const std::string K_OPENTREP_DEFAULT_DB_PASSWD ("opentrep");
-const std::string K_OPENTREP_DEFAULT_DB_DBNAME ("opentrep");
+const std::string K_OPENTREP_DEFAULT_DB_DBNAME ("trep_opentrep");
const std::string K_OPENTREP_DEFAULT_DB_HOST ("localhost");
const std::string K_OPENTREP_DEFAULT_DB_PORT ("3306");
Modified: trunk/opentrep/opentrep/python/pyopentrep.py
===================================================================
--- trunk/opentrep/opentrep/python/pyopentrep.py 2009-07-20 22:11:59 UTC (rev 151)
+++ trunk/opentrep/opentrep/python/pyopentrep.py 2009-07-20 22:27:02 UTC (rev 152)
@@ -5,7 +5,7 @@
import libpyopentrep
mySearch = libpyopentrep.OpenTrepSearcher()
-mySearch.init('../../test/traveldb', 'pyopentrep.log', 'opentrep', 'opentrep', 'localhost', '3306', 'opentrep')
+mySearch.init('../../test/traveldb', 'pyopentrep.log', 'opentrep', 'opentrep', 'localhost', '3306', 'trep_opentrep')
a = mySearch.search('sna francicso rio de janero lso angles reykyavki')
print a
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <den...@us...> - 2009-07-20 22:12:10
|
Revision: 151
http://opentrep.svn.sourceforge.net/opentrep/?rev=151&view=rev
Author: denis_arnaud
Date: 2009-07-20 22:11:59 +0000 (Mon, 20 Jul 2009)
Log Message:
-----------
[Python] Improved the robustness of the library.
Modified Paths:
--------------
trunk/opentrep/configure.ac
trunk/opentrep/opentrep/DBParams.hpp
trunk/opentrep/opentrep/OPENTREP_Types.hpp
trunk/opentrep/opentrep/command/RequestInterpreter.cpp
trunk/opentrep/opentrep/service/OPENTREP_Service.cpp
Added Paths:
-----------
trunk/opentrep/opentrep/python/Makefile.am
trunk/opentrep/opentrep/python/pyopentrep.cpp
trunk/opentrep/opentrep/python/pyopentrep.py
trunk/opentrep/opentrep/python/sources.mk
Property Changed:
----------------
trunk/opentrep/opentrep/python/
Modified: trunk/opentrep/configure.ac
===================================================================
--- trunk/opentrep/configure.ac 2009-07-20 21:12:10 UTC (rev 150)
+++ trunk/opentrep/configure.ac 2009-07-20 22:11:59 UTC (rev 151)
@@ -232,6 +232,7 @@
opentrep/config/Makefile
opentrep/core/Makefile
opentrep/batches/Makefile
+ opentrep/python/Makefile
man/Makefile
info/Makefile
doc/Makefile
Modified: trunk/opentrep/opentrep/DBParams.hpp
===================================================================
--- trunk/opentrep/opentrep/DBParams.hpp 2009-07-20 21:12:10 UTC (rev 150)
+++ trunk/opentrep/opentrep/DBParams.hpp 2009-07-20 22:11:59 UTC (rev 151)
@@ -77,6 +77,18 @@
public:
+ // ///////// Busines methods ////////
+ /** Check that all the parameters are fine. */
+ bool check () const {
+ if (_user.empty() == true || _passwd.empty() == true
+ || _host.empty() == true || _port.empty()
+ || _dbname.empty() == true) {
+ return false;
+ }
+ return true;
+ }
+
+ public:
// ///////// Display methods ////////
/** Dump a structure into an output stream.
@param ostream& the output stream. */
Modified: trunk/opentrep/opentrep/OPENTREP_Types.hpp
===================================================================
--- trunk/opentrep/opentrep/OPENTREP_Types.hpp 2009-07-20 21:12:10 UTC (rev 150)
+++ trunk/opentrep/opentrep/OPENTREP_Types.hpp 2009-07-20 22:11:59 UTC (rev 151)
@@ -30,13 +30,37 @@
class ObjectNotFoundException : public RootException {
};
- class DocumentNotFoundException : public RootException {
+ class XapianException : public RootException {
};
+
+ class DocumentNotFoundException : public XapianException {
+ };
- class SQLDatabaseConnectionImpossibleException : public RootException {
+ class XapianDatabaseFailureException : public XapianException {
};
-
+ class XapianTravelDatabaseEmptyException : public XapianException {
+ };
+
+ class SQLDatabaseException : public RootException {
+ };
+
+ class SQLDatabaseConnectionImpossibleException : public SQLDatabaseException {
+ };
+
+ class BuildIndexException : public RootException {
+ };
+
+ class InterpreteUseCaseException : public RootException {
+ };
+
+ class InterpreteTravelRequestException : public InterpreteUseCaseException {
+ };
+
+ class TravelRequestEmptyException : public InterpreteUseCaseException {
+ };
+
+
// /////////////// Log /////////////
/** Level of logs. */
namespace LOG {
Modified: trunk/opentrep/opentrep/command/RequestInterpreter.cpp
===================================================================
--- trunk/opentrep/opentrep/command/RequestInterpreter.cpp 2009-07-20 21:12:10 UTC (rev 150)
+++ trunk/opentrep/opentrep/command/RequestInterpreter.cpp 2009-07-20 22:11:59 UTC (rev 151)
@@ -115,6 +115,10 @@
WordList_T& ioWordList) {
NbOfMatches_T oNbOfMatches = 0;
+ // Sanity checks
+ assert (iTravelDatabaseName.empty() == false);
+ assert (iTravelQuery.empty() == false);
+
// Create a PlaceHolder object, to collect the matching Place objects
PlaceHolder& lPlaceHolder = FacPlaceHolder::instance().create();
@@ -146,6 +150,7 @@
} catch (const Xapian::Error& error) {
OPENTREP_LOG_ERROR ("Exception: " << error.get_msg());
+ throw XapianDatabaseFailureException();
}
// DEBUG
Property changes on: trunk/opentrep/opentrep/python
___________________________________________________________________
Added: svn:ignore
+ .deps
.libs
Makefile.in
Makefile
pyopentrep.log
Added: trunk/opentrep/opentrep/python/Makefile.am
===================================================================
--- trunk/opentrep/opentrep/python/Makefile.am (rev 0)
+++ trunk/opentrep/opentrep/python/Makefile.am 2009-07-20 22:11:59 UTC (rev 151)
@@ -0,0 +1,15 @@
+## command sub-directory
+include $(top_srcdir)/Makefile.common
+
+MAINTAINERCLEANFILES = Makefile.in
+
+# Library
+lib_LTLIBRARIES = libpy@PACKAGE@.la
+
+libpy@PACKAGE@_la_SOURCES = pyopentrep.cpp
+libpy@PACKAGE@_la_CXXFLAGS = ${PYTHON_CFLAGS} ${BOOST_CFLAGS}
+libpy@PACKAGE@_la_LIBADD = $(top_builddir)/@PACKAGE@/core/lib@PACKAGE@.la
+libpy@PACKAGE@_la_LDFLAGS = ${PYTHON_LIBS} ${PYTHON_ADD_LIBS} \
+ ${BOOST_PYTHON_LIB} -version-info $(GENERIC_LIBRARY_VERSION)
+
+EXTRA_DIST = pyopentrep.py
Added: trunk/opentrep/opentrep/python/pyopentrep.cpp
===================================================================
--- trunk/opentrep/opentrep/python/pyopentrep.cpp (rev 0)
+++ trunk/opentrep/opentrep/python/pyopentrep.cpp 2009-07-20 22:11:59 UTC (rev 151)
@@ -0,0 +1,168 @@
+// C
+#include <cassert>
+// STL
+#include <stdexcept>
+#include <fstream>
+#include <sstream>
+#include <string>
+#include <list>
+#include <vector>
+// Boost String
+#include <boost/python.hpp>
+// OpenTREP
+#include <opentrep/OPENTREP_Service.hpp>
+
+namespace OPENTREP {
+
+ struct OpenTrepSearcher {
+ public:
+
+ /** Wrapper around the search use case. */
+ std::string search (const std::string& iTravelQuery) {
+ std::ostringstream oStr;
+
+ // Sanity check
+ if (_logOutputStream == NULL) {
+ oStr << "The log filepath is not valid." << std::endl;
+ return oStr.str();
+ }
+
+ try {
+
+ // DEBUG
+ *_logOutputStream << "Python search for '" << iTravelQuery << "'"
+ << std::endl;
+
+ if (_opentrepService == NULL) {
+ oStr << "The OpenTREP service has not been initialised, i.e., "
+ << "the init() method has not been called correctly on the "
+ << "OpenTrepSearcher object. Please check that all the "
+ << "parameters are not empty and point to actual files.";
+ return oStr.str();
+ }
+ assert (_opentrepService != NULL);
+
+ // Query the Xapian database (index)
+ WordList_T lNonMatchedWordList;
+ LocationList_T lLocationList;
+ const NbOfMatches_T nbOfMatches =
+ _opentrepService->interpretTravelRequest (iTravelQuery, lLocationList,
+ lNonMatchedWordList);
+
+ // DEBUG
+ *_logOutputStream << "Python search for '" << iTravelQuery << "' gave "
+ << nbOfMatches << " matches." << std::endl;
+
+ if (nbOfMatches != 0) {
+ NbOfMatches_T idx = 0;
+ for(LocationList_T::const_iterator itLocation = lLocationList.begin();
+ itLocation != lLocationList.end(); ++itLocation, ++idx) {
+ const Location& lLocation = *itLocation;
+ if (idx != 0) {
+ oStr << ",";
+ }
+ oStr << lLocation.getLocationCode();
+ }
+ }
+
+ if (lNonMatchedWordList.empty() == false) {
+ oStr << ";";
+ NbOfMatches_T idx = 0;
+ for (WordList_T::const_iterator itWord = lNonMatchedWordList.begin();
+ itWord != lNonMatchedWordList.end(); ++itWord, ++idx) {
+ const Word_T& lWord = *itWord;
+ if (idx != 0) {
+ oStr << ",";
+ }
+ oStr << lWord;
+ }
+ }
+
+ // DEBUG
+ *_logOutputStream << "Python search for '" << iTravelQuery
+ << "' yielded:" << std::endl;
+
+ // DEBUG
+ *_logOutputStream << oStr.str() << std::endl;
+
+ } catch (const std::exception& error) {
+ *_logOutputStream << "Exception: " << error.what() << std::endl;
+ }
+
+ return oStr.str();
+ }
+
+ public:
+ /** Default constructor. */
+ OpenTrepSearcher() : _opentrepService (NULL), _logOutputStream (NULL) {
+ }
+
+ /** Default copy constructor. */
+ OpenTrepSearcher (const OpenTrepSearcher& iOpenTrepSearcher)
+ : _opentrepService (iOpenTrepSearcher._opentrepService),
+ _logOutputStream (iOpenTrepSearcher._logOutputStream) {
+ }
+
+ /** Default constructor. */
+ ~OpenTrepSearcher() {
+ _opentrepService = NULL;
+ _logOutputStream = NULL;
+ }
+
+ /** Wrapper around the search use case. */
+ bool init (const std::string& iXapianDatabaseFilepath,
+ const std::string& iLogFilepath,
+ const std::string& iDBUser, const std::string& iDBPasswd,
+ const std::string& iDBHost, const std::string& iDBPort,
+ const std::string& iDBDBName) {
+ bool isEverythingOK = true;
+
+ try {
+
+ // TODO: use Boost Filesystem to check the filepaths
+ if (iXapianDatabaseFilepath.empty() == true
+ || iLogFilepath.empty() == true) {
+ isEverythingOK = false;
+ return isEverythingOK;
+ }
+
+ // Set the log parameters
+ _logOutputStream = new std::ofstream;
+ assert (_logOutputStream != NULL);
+
+ // Open and clean the log outputfile
+ _logOutputStream->open (iLogFilepath.c_str());
+ _logOutputStream->clear();
+
+ // DEBUG
+ *_logOutputStream << "Python wrapper initialisation" << std::endl;
+
+ // Initialise the context
+ DBParams lDBParams (iDBUser, iDBPasswd, iDBHost, iDBPort, iDBDBName);
+ _opentrepService = new OPENTREP_Service (*_logOutputStream, lDBParams,
+ iXapianDatabaseFilepath);
+
+ // DEBUG
+ *_logOutputStream << "Python wrapper initialised" << std::endl;
+
+ } catch (const std::exception& error) {
+ *_logOutputStream << "Exception: " << error.what() << std::endl;
+ }
+
+ return isEverythingOK;
+ }
+
+ private:
+ /** Handle on the OpenTREP services (API). */
+ OPENTREP_Service* _opentrepService;
+ std::ofstream* _logOutputStream;
+ };
+
+}
+
+// /////////////////////////////////////////////////////////////
+BOOST_PYTHON_MODULE(libpyopentrep) {
+ boost::python::class_<OPENTREP::OpenTrepSearcher> ("OpenTrepSearcher")
+ .def ("search", &OPENTREP::OpenTrepSearcher::search)
+ .def ("init", &OPENTREP::OpenTrepSearcher::init);
+}
Added: trunk/opentrep/opentrep/python/pyopentrep.py
===================================================================
--- trunk/opentrep/opentrep/python/pyopentrep.py (rev 0)
+++ trunk/opentrep/opentrep/python/pyopentrep.py 2009-07-20 22:11:59 UTC (rev 151)
@@ -0,0 +1,11 @@
+#!/usr/bin/env python
+
+import sys
+sys.path.append('.libs')
+import libpyopentrep
+
+mySearch = libpyopentrep.OpenTrepSearcher()
+mySearch.init('../../test/traveldb', 'pyopentrep.log', 'opentrep', 'opentrep', 'localhost', '3306', 'opentrep')
+
+a = mySearch.search('sna francicso rio de janero lso angles reykyavki')
+print a
Property changes on: trunk/opentrep/opentrep/python/pyopentrep.py
___________________________________________________________________
Added: svn:executable
+ *
Added: trunk/opentrep/opentrep/python/sources.mk
===================================================================
--- trunk/opentrep/opentrep/python/sources.mk (rev 0)
+++ trunk/opentrep/opentrep/python/sources.mk 2009-07-20 22:11:59 UTC (rev 151)
@@ -0,0 +1,2 @@
+python_svc_h_sources = $(top_srcdir)/opentrep/python/pyopentrep.cpp
+python_svc_sources =
Modified: trunk/opentrep/opentrep/service/OPENTREP_Service.cpp
===================================================================
--- trunk/opentrep/opentrep/service/OPENTREP_Service.cpp 2009-07-20 21:12:10 UTC (rev 150)
+++ trunk/opentrep/opentrep/service/OPENTREP_Service.cpp 2009-07-20 22:11:59 UTC (rev 151)
@@ -56,6 +56,20 @@
// Set the log file
logInit (LOG::DEBUG, ioLogStream);
+ // Check that the Xapian travel database is not empty
+ if (iTravelDatabaseName.empty() == true) {
+ OPENTREP_LOG_ERROR ("The filepath for the Xapian travel database is "
+ << "empty.");
+ throw XapianTravelDatabaseEmptyException();
+ }
+
+ // Check that the parameters for the SQL database are not empty
+ if (iDBParams.check() == false) {
+ OPENTREP_LOG_ERROR ("At least one of the parameters for the SQL "
+ << "database is empty: " << iDBParams);
+ throw XapianTravelDatabaseEmptyException();
+ }
+
// Initialise the context
OPENTREP_ServiceContext& lOPENTREP_ServiceContext =
FacOpenTrepServiceContext::instance().create (iTravelDatabaseName);
@@ -88,26 +102,33 @@
}
assert (_opentrepServiceContext != NULL);
OPENTREP_ServiceContext& lOPENTREP_ServiceContext= *_opentrepServiceContext;
-
- // Retrieve the SOCI Session
- soci::session& lSociSession =
- lOPENTREP_ServiceContext.getSociSessionHandler();
-
- // Retrieve the Xapian database name (directorty of the index)
- const TravelDatabaseName_T& lTravelDatabaseName =
- lOPENTREP_ServiceContext.getTravelDatabaseName();
- // Delegate the index building to the dedicated command
- BasChronometer lBuildSearchIndexChronometer;
- lBuildSearchIndexChronometer.start();
- IndexBuilder::buildSearchIndex (lSociSession, lTravelDatabaseName);
- const double lBuildSearchIndexMeasure =
- lBuildSearchIndexChronometer.elapsed();
+ try {
+
+ // Retrieve the SOCI Session
+ soci::session& lSociSession =
+ lOPENTREP_ServiceContext.getSociSessionHandler();
+
+ // Retrieve the Xapian database name (directorty of the index)
+ const TravelDatabaseName_T& lTravelDatabaseName =
+ lOPENTREP_ServiceContext.getTravelDatabaseName();
+
+ // Delegate the index building to the dedicated command
+ BasChronometer lBuildSearchIndexChronometer;
+ lBuildSearchIndexChronometer.start();
+ IndexBuilder::buildSearchIndex (lSociSession, lTravelDatabaseName);
+ const double lBuildSearchIndexMeasure =
+ lBuildSearchIndexChronometer.elapsed();
+
+ // DEBUG
+ OPENTREP_LOG_DEBUG ("Build Xapian database (index): "
+ << lBuildSearchIndexMeasure << " - "
+ << lOPENTREP_ServiceContext.display());
- // DEBUG
- OPENTREP_LOG_DEBUG ("Build Xapian database (index): "
- << lBuildSearchIndexMeasure << " - "
- << lOPENTREP_ServiceContext.display());
+ } catch (const std::exception& error) {
+ OPENTREP_LOG_ERROR ("Exception: " << error.what());
+ throw BuildIndexException();
+ }
}
// //////////////////////////////////////////////////////////////////////
@@ -115,6 +136,7 @@
interpretTravelRequest (const std::string& iTravelQuery,
LocationList_T& ioLocationList,
WordList_T& ioWordList) {
+ NbOfMatches_T nbOfMatches = 0;
if (_opentrepServiceContext == NULL) {
throw NonInitialisedServiceException();
@@ -122,30 +144,45 @@
assert (_opentrepServiceContext != NULL);
OPENTREP_ServiceContext& lOPENTREP_ServiceContext= *_opentrepServiceContext;
- // Retrieve the SOCI Session
- soci::session& lSociSession =
- lOPENTREP_ServiceContext.getSociSessionHandler();
+ // Check that the travel request is not empty
+ if (iTravelQuery.empty() == true) {
+ OPENTREP_LOG_ERROR ("The travel request is empty.");
+ throw TravelRequestEmptyException();
+ }
- // Retrieve the Xapian database name (directorty of the index)
- const TravelDatabaseName_T& lTravelDatabaseName =
- lOPENTREP_ServiceContext.getTravelDatabaseName();
-
- // Delegate the query execution to the dedicated command
- BasChronometer lRequestInterpreterChronometer;
- lRequestInterpreterChronometer.start();
- const NbOfMatches_T nbOfMatches =
- RequestInterpreter::interpretTravelRequest (lSociSession,
- lTravelDatabaseName,
- iTravelQuery, ioLocationList,
- ioWordList);
- const double lRequestInterpreterMeasure =
- lRequestInterpreterChronometer.elapsed();
+ try {
+
+ // Retrieve the SOCI Session
+ soci::session& lSociSession =
+ lOPENTREP_ServiceContext.getSociSessionHandler();
+
+ // Retrieve the Xapian database name (directorty of the index)
+ const TravelDatabaseName_T& lTravelDatabaseName =
+ lOPENTREP_ServiceContext.getTravelDatabaseName();
+
+ // Delegate the query execution to the dedicated command
+ BasChronometer lRequestInterpreterChronometer;
+ lRequestInterpreterChronometer.start();
+ nbOfMatches =
+ RequestInterpreter::interpretTravelRequest (lSociSession,
+ lTravelDatabaseName,
+ iTravelQuery,
+ ioLocationList,
+ ioWordList);
+ const double lRequestInterpreterMeasure =
+ lRequestInterpreterChronometer.elapsed();
- // DEBUG
- OPENTREP_LOG_DEBUG ("Match query on Xapian database (index): "
- << lRequestInterpreterMeasure << " - "
- << lOPENTREP_ServiceContext.display());
-
+ // DEBUG
+ OPENTREP_LOG_DEBUG ("Match query on Xapian database (index): "
+ << lRequestInterpreterMeasure << " - "
+ << lOPENTREP_ServiceContext.display());
+
+
+ } catch (const std::exception& error) {
+ OPENTREP_LOG_ERROR ("Exception: " << error.what());
+ throw InterpreteTravelRequestException();
+ }
+
return nbOfMatches;
}
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <den...@us...> - 2009-07-20 21:12:17
|
Revision: 150
http://opentrep.svn.sourceforge.net/opentrep/?rev=150&view=rev
Author: denis_arnaud
Date: 2009-07-20 21:12:10 +0000 (Mon, 20 Jul 2009)
Log Message:
-----------
[Test] Fixed a typo in the db parameters.
Modified Paths:
--------------
trunk/opentrep/test/python/pyopentrep.py
Modified: trunk/opentrep/test/python/pyopentrep.py
===================================================================
--- trunk/opentrep/test/python/pyopentrep.py 2009-07-20 15:47:54 UTC (rev 149)
+++ trunk/opentrep/test/python/pyopentrep.py 2009-07-20 21:12:10 UTC (rev 150)
@@ -5,7 +5,7 @@
import libpyopentrep
mySearch = libpyopentrep.OpenTrepSearcher()
-mySearch.init('../traveldb', 'opentrep', 'opentrep', 'nceorilnx02', '3321', 'opentrep')
+mySearch.init('../traveldb', 'opentrep', 'opentrep', 'localhost', '3306', 'opentrep')
a = mySearch.search('sna francicso rio de janero lso angles reykyavki')
print a
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <den...@us...> - 2009-07-20 15:47:58
|
Revision: 149
http://opentrep.svn.sourceforge.net/opentrep/?rev=149&view=rev
Author: denis_arnaud
Date: 2009-07-20 15:47:54 +0000 (Mon, 20 Jul 2009)
Log Message:
-----------
[Test] Fixed a small compilation bug in the Python-related tests.
Modified Paths:
--------------
trunk/opentrep/test/python/pyopentrep.cpp
trunk/opentrep/test/python/pyopentrep.py
Modified: trunk/opentrep/test/python/pyopentrep.cpp
===================================================================
--- trunk/opentrep/test/python/pyopentrep.cpp 2009-07-20 15:17:31 UTC (rev 148)
+++ trunk/opentrep/test/python/pyopentrep.cpp 2009-07-20 15:47:54 UTC (rev 149)
@@ -1,4 +1,6 @@
// C
+#include <cassert>
+// STL
#include <sstream>
#include <string>
#include <list>
@@ -11,24 +13,26 @@
namespace OPENTREP {
struct OpenTrepSearcher {
-
+ public:
+
/** Wrapper around the search use case. */
- std::string search (const std::string& iTravelQuery,
- const std::string& iXapianDatabaseFilepath) {
+ std::string search (const std::string& iTravelQuery) {
std::ostringstream oStr;
+
+ if (_opentrepService == NULL) {
+ oStr << "The OpenTREP service has not been initialise, i.e., "
+ << "the init() method has not been called on the "
+ << "OpenTrepSearcher object. Please do so.";
+ return oStr.str();
+ }
+ assert (_opentrepService != NULL);
- // Output stream (for logs)
- std::ostringstream logOutputStream;
-
- // Initialise the context
- OPENTREP_Service opentrepService(logOutputStream, iXapianDatabaseFilepath);
-
// Query the Xapian database (index)
WordList_T lNonMatchedWordList;
LocationList_T lLocationList;
const NbOfMatches_T nbOfMatches =
- opentrepService.interpretTravelRequest (iTravelQuery, lLocationList,
- lNonMatchedWordList);
+ _opentrepService->interpretTravelRequest (iTravelQuery, lLocationList,
+ lNonMatchedWordList);
if (nbOfMatches != 0) {
NbOfMatches_T idx = 0;
@@ -58,6 +62,42 @@
return oStr.str();
}
+ public:
+ /** Default constructor. */
+ OpenTrepSearcher() : _opentrepService (NULL), _logOutputStream (NULL) {
+ }
+
+ /** Default copy constructor. */
+ OpenTrepSearcher (const OpenTrepSearcher& iOpenTrepSearcher)
+ : _opentrepService (iOpenTrepSearcher._opentrepService),
+ _logOutputStream (iOpenTrepSearcher._logOutputStream) {
+ }
+
+ /** Default constructor. */
+ ~OpenTrepSearcher() {
+ _opentrepService = NULL;
+ _logOutputStream = NULL;
+ }
+
+ /** Wrapper around the search use case. */
+ void init (const std::string& iXapianDatabaseFilepath,
+ const std::string& iDBUser, const std::string& iDBPasswd,
+ const std::string& iDBHost, const std::string& iDBPort,
+ const std::string& iDBDBName) {
+
+ // Output stream (for logs)
+ _logOutputStream = new std::ostringstream();
+
+ // Initialise the context
+ DBParams lDBParams (iDBUser, iDBPasswd, iDBHost, iDBPort, iDBDBName);
+ _opentrepService = new OPENTREP_Service (*_logOutputStream, lDBParams,
+ iXapianDatabaseFilepath);
+ }
+
+ private:
+ /** Handle on the OpenTREP services (API). */
+ OPENTREP_Service* _opentrepService;
+ std::ostringstream* _logOutputStream;
};
}
@@ -65,5 +105,6 @@
// /////////////////////////////////////////////////////////////
BOOST_PYTHON_MODULE(libpyopentrep) {
boost::python::class_<OPENTREP::OpenTrepSearcher> ("OpenTrepSearcher")
- .def ("search", &OPENTREP::OpenTrepSearcher::search);
+ .def ("search", &OPENTREP::OpenTrepSearcher::search)
+ .def ("init", &OPENTREP::OpenTrepSearcher::init);
}
Modified: trunk/opentrep/test/python/pyopentrep.py
===================================================================
--- trunk/opentrep/test/python/pyopentrep.py 2009-07-20 15:17:31 UTC (rev 148)
+++ trunk/opentrep/test/python/pyopentrep.py 2009-07-20 15:47:54 UTC (rev 149)
@@ -5,7 +5,8 @@
import libpyopentrep
mySearch = libpyopentrep.OpenTrepSearcher()
-a = mySearch.search('sna francicso rio de janero lso angles reykyavki','../traveldb')
+mySearch.init('../traveldb', 'opentrep', 'opentrep', 'nceorilnx02', '3321', 'opentrep')
+a = mySearch.search('sna francicso rio de janero lso angles reykyavki')
print a
#print ', '.join(['%s: %s' % (msg, id) for (msg, id) in zip(MsgList, IDList)])
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <den...@us...> - 2009-07-20 15:17:38
|
Revision: 148
http://opentrep.svn.sourceforge.net/opentrep/?rev=148&view=rev
Author: denis_arnaud
Date: 2009-07-20 15:17:31 +0000 (Mon, 20 Jul 2009)
Log Message:
-----------
[Dev] Created a DBParams structure in the interface (API), so that MySQL parameters can be set by the caller.
Modified Paths:
--------------
trunk/opentrep/opentrep/Location.hpp
trunk/opentrep/opentrep/OPENTREP_Service.hpp
trunk/opentrep/opentrep/OPENTREP_Types.hpp
trunk/opentrep/opentrep/batches/indexer.cpp
trunk/opentrep/opentrep/batches/searcher.cpp
trunk/opentrep/opentrep/bom/StringMatcher.cpp
trunk/opentrep/opentrep/command/SociSessionManager.cpp
trunk/opentrep/opentrep/command/SociSessionManager.hpp
trunk/opentrep/opentrep/service/OPENTREP_Service.cpp
trunk/opentrep/opentrep/sources.mk
Added Paths:
-----------
trunk/opentrep/opentrep/DBParams.hpp
trunk/opentrep/opentrep/batches/opentrep_indexer.cfg
trunk/opentrep/opentrep/batches/opentrep_searcher.cfg
Added: trunk/opentrep/opentrep/DBParams.hpp
===================================================================
--- trunk/opentrep/opentrep/DBParams.hpp (rev 0)
+++ trunk/opentrep/opentrep/DBParams.hpp 2009-07-20 15:17:31 UTC (rev 148)
@@ -0,0 +1,140 @@
+#ifndef __OPENTREP_DBPARAMS_HPP
+#define __OPENTREP_DBPARAMS_HPP
+
+// //////////////////////////////////////////////////////////////////////
+// Import section
+// //////////////////////////////////////////////////////////////////////
+// STL
+#include <istream>
+#include <ostream>
+#include <sstream>
+#include <string>
+// OpenTrep
+#include <opentrep/OPENTREP_Types.hpp>
+#include <opentrep/OPENTREP_Abstract.hpp>
+
+namespace OPENTREP {
+
+ /** List of names for a given (geographical) dbparams. */
+ typedef std::list<std::string> DBParamsNameList_T;
+
+
+ /** Structure modelling a (geographical) dbparams. */
+ struct DBParams : public OPENTREP_Abstract {
+ public:
+ // ///////// Getters ////////
+ /** Get the database user name. */
+ std::string getUser() const {
+ return _user;
+ }
+
+ /** Get the database user password. */
+ std::string getPassword() const {
+ return _passwd;
+ }
+
+ /** Get the database host name. */
+ std::string getHost() const {
+ return _host;
+ }
+
+ /** Get the database port number. */
+ std::string getPort() const {
+ return _port;
+ }
+
+ /** Get the database name. */
+ std::string getDBName() const {
+ return _dbname;
+ }
+
+
+ // ///////// Setters //////////
+ /** Set the database user name. */
+ void setUser (const std::string& iUser) {
+ _user = iUser;
+ }
+
+ /** Set the database password. */
+ void setPassword (const std::string& iPasswd) {
+ _passwd = iPasswd;
+ }
+
+ /** Set the database host name. */
+ void setHost (const std::string& iHost) {
+ _host = iHost;
+ }
+
+ /** Set the database port number. */
+ void setPort (const std::string& iPort) {
+ _port = iPort;
+ }
+
+ /** Set the database name. */
+ void setDBName (const std::string& iDBName) {
+ _dbname = iDBName;
+ }
+
+
+ public:
+ // ///////// Display methods ////////
+ /** Dump a structure into an output stream.
+ @param ostream& the output stream. */
+ void toStream (std::ostream& ioOut) const {
+ ioOut << toString();
+ }
+
+ /** Read a structure from an input stream.
+ @param istream& the input stream. */
+ void fromStream (std::istream&) {
+ }
+
+ /** Get a short display of the DBParams structure. */
+ std::string toShortString() const {
+ std::ostringstream oStr;
+ oStr << _dbname << "." << _user << "@" << _host << ":" << _port;
+ return oStr.str();
+ }
+
+ /** Get the serialised version of the DBParams structure. */
+ std::string toString() const {
+ std::ostringstream oStr;
+ oStr << _dbname << "." << _user << "@" << _host << ":" << _port;
+ return oStr.str();
+ }
+
+
+ public:
+ /** Main Constructor. */
+ DBParams (const std::string& iDBUser, const std::string& iDBPasswd,
+ const std::string& iDBHost, const std::string& iDBPort,
+ const std::string& iDBName)
+ : _user (iDBUser), _passwd (iDBPasswd), _host (iDBHost), _port (iDBPort),
+ _dbname (iDBName) {
+ }
+
+ /** Default Constructor. */
+ // DBParams ();
+ /** Default copy constructor. */
+ // DBParams (const DBParams&);
+
+ /** Destructor. */
+ virtual ~DBParams() {}
+
+
+ private:
+ // /////// Attributes /////////
+ /** Database user name. */
+ std::string _user;
+ /** Database user password. */
+ std::string _passwd;
+ /** Database host name. */
+ std::string _host;
+ /** Database port number. */
+ std::string _port;
+ /** Database name. */
+ std::string _dbname;
+ };
+
+}
+#endif // __OPENTREP_DBPARAMS_HPP
Modified: trunk/opentrep/opentrep/Location.hpp
===================================================================
--- trunk/opentrep/opentrep/Location.hpp 2009-07-20 14:03:18 UTC (rev 147)
+++ trunk/opentrep/opentrep/Location.hpp 2009-07-20 15:17:31 UTC (rev 148)
@@ -121,7 +121,7 @@
}
/** Set the list of (American) English names for that location. */
- void getNameList (const LocationNameList_T& iNameList) {
+ void setNameList (const LocationNameList_T& iNameList) {
_nameList = iNameList;
}
Modified: trunk/opentrep/opentrep/OPENTREP_Service.hpp
===================================================================
--- trunk/opentrep/opentrep/OPENTREP_Service.hpp 2009-07-20 14:03:18 UTC (rev 147)
+++ trunk/opentrep/opentrep/OPENTREP_Service.hpp 2009-07-20 15:17:31 UTC (rev 148)
@@ -9,6 +9,7 @@
#include <string>
// OpenTREP
#include <opentrep/OPENTREP_Types.hpp>
+#include <opentrep/DBParams.hpp>
#include <opentrep/Location.hpp>
namespace OPENTREP {
@@ -22,7 +23,7 @@
public:
// /////////// Business Methods /////////////
/** Build the Xapian database (index) on the BOM held in memory. */
- void buildSearchIndex ();
+ void buildSearchIndex();
/** Match the given string, thanks to a full-text search on the
underlying Xapian index (named "database").
@@ -39,8 +40,9 @@
// ////////// Constructors and destructors //////////
/** Constructor.
@param std::ostream& Output log stream (for instance, std::cout).
+ @param const DBParams& The SQL database parameters.
@param const std::string& Filepath of the Xapian index/database. */
- OPENTREP_Service (std::ostream& ioLogStream,
+ OPENTREP_Service (std::ostream& ioLogStream, const DBParams&,
const std::string& iXapianDatabaseFilepath);
/** Destructor. */
@@ -55,7 +57,7 @@
OPENTREP_Service (const OPENTREP_Service&);
/** Initialise. */
- void init (std::ostream& ioLogStream,
+ void init (std::ostream& ioLogStream, const DBParams&,
const std::string& iXapianDatabaseFilepath);
/** Finalise. */
Modified: trunk/opentrep/opentrep/OPENTREP_Types.hpp
===================================================================
--- trunk/opentrep/opentrep/OPENTREP_Types.hpp 2009-07-20 14:03:18 UTC (rev 147)
+++ trunk/opentrep/opentrep/OPENTREP_Types.hpp 2009-07-20 15:17:31 UTC (rev 148)
@@ -33,6 +33,9 @@
class DocumentNotFoundException : public RootException {
};
+ class SQLDatabaseConnectionImpossibleException : public RootException {
+ };
+
// /////////////// Log /////////////
/** Level of logs. */
Modified: trunk/opentrep/opentrep/batches/indexer.cpp
===================================================================
--- trunk/opentrep/opentrep/batches/indexer.cpp 2009-07-20 14:03:18 UTC (rev 147)
+++ trunk/opentrep/opentrep/batches/indexer.cpp 2009-07-20 15:17:31 UTC (rev 148)
@@ -4,14 +4,15 @@
#include <iostream>
#include <sstream>
#include <fstream>
-#include <map>
#include <vector>
+#include <string>
// Boost (Extended STL)
#include <boost/date_time/posix_time/posix_time.hpp>
#include <boost/date_time/gregorian/gregorian.hpp>
#include <boost/program_options.hpp>
-// OPENTREP
+// OpenTREP
#include <opentrep/OPENTREP_Service.hpp>
+#include <opentrep/DBParams.hpp>
#include <opentrep/config/opentrep-paths.hpp>
@@ -26,7 +27,14 @@
/** Default name and location for the Xapian database. */
const std::string K_OPENTREP_DEFAULT_DATABSE_FILEPATH("/tmp/opentrep/traveldb");
+/** Default name and location for the Xapian database. */
+const std::string K_OPENTREP_DEFAULT_DB_USER ("opentrep");
+const std::string K_OPENTREP_DEFAULT_DB_PASSWD ("opentrep");
+const std::string K_OPENTREP_DEFAULT_DB_DBNAME ("opentrep");
+const std::string K_OPENTREP_DEFAULT_DB_HOST ("localhost");
+const std::string K_OPENTREP_DEFAULT_DB_PORT ("3306");
+
// ///////// Parsing of Options & Configuration /////////
/** Early return status (so that it can be differentiated from an error). */
const int K_OPENTREP_EARLY_RETURN_STATUS = 99;
@@ -34,7 +42,10 @@
/** Read and parse the command line options. */
int readConfiguration (int argc, char* argv[],
std::string& ioDatabaseFilepath,
- std::string& ioLogFilename) {
+ std::string& ioLogFilename,
+ std::string& ioDBUser, std::string& ioDBPasswd,
+ std::string& ioDBHost, std::string& ioDBPort,
+ std::string& ioDBDBName) {
// Declare a group of options that will be allowed only on command line
boost::program_options::options_description generic ("Generic options");
@@ -42,7 +53,7 @@
("prefix", "print installation prefix")
("version,v", "print version string")
("help,h", "produce help message");
-
+
// Declare a group of options that will be allowed both on command
// line and in config file
boost::program_options::options_description config ("Configuration");
@@ -53,6 +64,21 @@
("log,l",
boost::program_options::value< std::string >(&ioLogFilename)->default_value(K_OPENTREP_DEFAULT_LOG_FILENAME),
"Filepath for the logs")
+ ("user,u",
+ boost::program_options::value< std::string >(&ioDBUser)->default_value(K_OPENTREP_DEFAULT_DB_USER),
+ "SQL database hostname (e.g., opentrep)")
+ ("passwd,p",
+ boost::program_options::value< std::string >(&ioDBPasswd)->default_value(K_OPENTREP_DEFAULT_DB_PASSWD),
+ "SQL database hostname (e.g., opentrep)")
+ ("host,h",
+ boost::program_options::value< std::string >(&ioDBHost)->default_value(K_OPENTREP_DEFAULT_DB_HOST),
+ "SQL database hostname (e.g., localhost)")
+ ("port,P",
+ boost::program_options::value< std::string >(&ioDBPort)->default_value(K_OPENTREP_DEFAULT_DB_PORT),
+ "SQL database port (e.g., 3306)")
+ ("dbname,m",
+ boost::program_options::value< std::string >(&ioDBDBName)->default_value(K_OPENTREP_DEFAULT_DB_DBNAME),
+ "SQL database name (e.g., opentrep)")
;
// Hidden options, will be allowed both on command line and
@@ -111,6 +137,31 @@
std::cout << "Log filename is: " << ioLogFilename << std::endl;
}
+ if (vm.count ("user")) {
+ ioDBUser = vm["user"].as< std::string >();
+ std::cout << "SQL database user name is: " << ioDBUser << std::endl;
+ }
+
+ if (vm.count ("passwd")) {
+ ioDBPasswd = vm["passwd"].as< std::string >();
+ // std::cout << "SQL database user password is: " << ioDBPasswd << std::endl;
+ }
+
+ if (vm.count ("host")) {
+ ioDBHost = vm["host"].as< std::string >();
+ std::cout << "SQL database host name is: " << ioDBHost << std::endl;
+ }
+
+ if (vm.count ("port")) {
+ ioDBPort = vm["port"].as< std::string >();
+ std::cout << "SQL database port number is: " << ioDBPort << std::endl;
+ }
+
+ if (vm.count ("dbname")) {
+ ioDBDBName = vm["dbname"].as< std::string >();
+ std::cout << "SQL database name is: " << ioDBDBName << std::endl;
+ }
+
return 0;
}
@@ -124,14 +175,26 @@
// Xapian database name (directory of the index)
OPENTREP::TravelDatabaseName_T lXapianDatabaseName;
-
+
+ // SQL database parameters
+ std::string lDBUser;
+ std::string lDBPasswd;
+ std::string lDBHost;
+ std::string lDBPort;
+ std::string lDBDBName;
+
// Call the command-line option parser
const int lOptionParserStatus =
- readConfiguration (argc, argv, lXapianDatabaseName, lLogFilename);
+ readConfiguration (argc, argv, lXapianDatabaseName, lLogFilename,
+ lDBUser, lDBPasswd, lDBHost, lDBPort, lDBDBName);
if (lOptionParserStatus == K_OPENTREP_EARLY_RETURN_STATUS) {
return 0;
}
+
+ // Set the database parameters
+ OPENTREP::DBParams lDBParams (lDBUser, lDBPasswd, lDBHost, lDBPort,
+ lDBDBName);
// Set the log parameters
std::ofstream logOutputFile;
@@ -145,7 +208,7 @@
<< std::endl;
// Initialise the context
- OPENTREP::OPENTREP_Service opentrepService (logOutputFile,
+ OPENTREP::OPENTREP_Service opentrepService (logOutputFile, lDBParams,
lXapianDatabaseName);
// Launch the indexation
Added: trunk/opentrep/opentrep/batches/opentrep_indexer.cfg
===================================================================
--- trunk/opentrep/opentrep/batches/opentrep_indexer.cfg (rev 0)
+++ trunk/opentrep/opentrep/batches/opentrep_indexer.cfg 2009-07-20 15:17:31 UTC (rev 148)
@@ -0,0 +1,7 @@
+database=../../test/traveldb
+log=opentrep_indexer.log
+user=opentrep
+passwd=opentrep
+host=localhost
+port=3306
+dbname=opentrep
Added: trunk/opentrep/opentrep/batches/opentrep_searcher.cfg
===================================================================
--- trunk/opentrep/opentrep/batches/opentrep_searcher.cfg (rev 0)
+++ trunk/opentrep/opentrep/batches/opentrep_searcher.cfg 2009-07-20 15:17:31 UTC (rev 148)
@@ -0,0 +1,9 @@
+database=../../test/traveldb
+log=opentrep_indexer.log
+user=opentrep
+passwd=opentrep
+host=localhost
+port=3306
+dbname=opentrep
+error=3
+query="sna francicso rio de janero lso angles reykyavki"
\ No newline at end of file
Modified: trunk/opentrep/opentrep/batches/searcher.cpp
===================================================================
--- trunk/opentrep/opentrep/batches/searcher.cpp 2009-07-20 14:03:18 UTC (rev 147)
+++ trunk/opentrep/opentrep/batches/searcher.cpp 2009-07-20 15:17:31 UTC (rev 148)
@@ -13,6 +13,7 @@
#include <boost/program_options.hpp>
// OpenTREP
#include <opentrep/OPENTREP_Service.hpp>
+#include <opentrep/DBParams.hpp>
#include <opentrep/config/opentrep-paths.hpp>
@@ -30,6 +31,13 @@
/** Default travel query string, to be seached against the Xapian database. */
const std::string K_OPENTREP_DEFAULT_QUERY_STRING ("sna francicso rio de janero lso angles reykyavki");
+/** Default name and location for the Xapian database. */
+const std::string K_OPENTREP_DEFAULT_DB_USER ("opentrep");
+const std::string K_OPENTREP_DEFAULT_DB_PASSWD ("opentrep");
+const std::string K_OPENTREP_DEFAULT_DB_DBNAME ("opentrep");
+const std::string K_OPENTREP_DEFAULT_DB_HOST ("localhost");
+const std::string K_OPENTREP_DEFAULT_DB_PORT ("3306");
+
/** Default error distance for spelling corrections. */
const unsigned short K_OPENTREP_DEFAULT_SPELLING_ERROR_DISTANCE = 3;
@@ -90,7 +98,10 @@
unsigned short& ioSpellingErrorDistance,
std::string& ioQueryString,
std::string& ioDatabaseFilepath,
- std::string& ioLogFilename) {
+ std::string& ioLogFilename,
+ std::string& ioDBUser, std::string& ioDBPasswd,
+ std::string& ioDBHost, std::string& ioDBPort,
+ std::string& ioDBDBName) {
// Initialise the travel query string, if that one is empty
if (ioQueryString.empty() == true) {
@@ -124,6 +135,21 @@
("log,l",
boost::program_options::value< std::string >(&ioLogFilename)->default_value(K_OPENTREP_DEFAULT_LOG_FILENAME),
"Filepath for the logs")
+ ("user,u",
+ boost::program_options::value< std::string >(&ioDBUser)->default_value(K_OPENTREP_DEFAULT_DB_USER),
+ "SQL database hostname (e.g., opentrep)")
+ ("passwd,p",
+ boost::program_options::value< std::string >(&ioDBPasswd)->default_value(K_OPENTREP_DEFAULT_DB_PASSWD),
+ "SQL database hostname (e.g., opentrep)")
+ ("host,h",
+ boost::program_options::value< std::string >(&ioDBHost)->default_value(K_OPENTREP_DEFAULT_DB_HOST),
+ "SQL database hostname (e.g., localhost)")
+ ("port,P",
+ boost::program_options::value< std::string >(&ioDBPort)->default_value(K_OPENTREP_DEFAULT_DB_PORT),
+ "SQL database port (e.g., 3306)")
+ ("dbname,m",
+ boost::program_options::value< std::string >(&ioDBDBName)->default_value(K_OPENTREP_DEFAULT_DB_DBNAME),
+ "SQL database name (e.g., opentrep)")
;
// Hidden options, will be allowed both on command line and
@@ -182,6 +208,31 @@
std::cout << "Log filename is: " << ioLogFilename << std::endl;
}
+ if (vm.count ("user")) {
+ ioDBUser = vm["user"].as< std::string >();
+ std::cout << "SQL database user name is: " << ioDBUser << std::endl;
+ }
+
+ if (vm.count ("passwd")) {
+ ioDBPasswd = vm["passwd"].as< std::string >();
+ // std::cout << "SQL database user password is: " << ioDBPasswd << std::endl;
+ }
+
+ if (vm.count ("host")) {
+ ioDBHost = vm["host"].as< std::string >();
+ std::cout << "SQL database host name is: " << ioDBHost << std::endl;
+ }
+
+ if (vm.count ("port")) {
+ ioDBPort = vm["port"].as< std::string >();
+ std::cout << "SQL database port number is: " << ioDBPort << std::endl;
+ }
+
+ if (vm.count ("dbname")) {
+ ioDBDBName = vm["dbname"].as< std::string >();
+ std::cout << "SQL database name is: " << ioDBDBName << std::endl;
+ }
+
std::cout << "The spelling error distance is: " << ioSpellingErrorDistance
<< std::endl;
@@ -205,18 +256,30 @@
// Xapian database name (directory of the index)
OPENTREP::TravelDatabaseName_T lXapianDatabaseName;
+ // SQL database parameters
+ std::string lDBUser;
+ std::string lDBPasswd;
+ std::string lDBHost;
+ std::string lDBPort;
+ std::string lDBDBName;
+
// Xapian spelling error distance
unsigned short lSpellingErrorDistance;
// Call the command-line option parser
const int lOptionParserStatus =
readConfiguration (argc, argv, lSpellingErrorDistance, lTravelQuery,
- lXapianDatabaseName, lLogFilename);
+ lXapianDatabaseName, lLogFilename,
+ lDBUser, lDBPasswd, lDBHost, lDBPort, lDBDBName);
if (lOptionParserStatus == K_OPENTREP_EARLY_RETURN_STATUS) {
return 0;
}
+ // Set the database parameters
+ OPENTREP::DBParams lDBParams (lDBUser, lDBPasswd, lDBHost, lDBPort,
+ lDBDBName);
+
// Set the log parameters
std::ofstream logOutputFile;
// open and clean the log outputfile
@@ -224,7 +287,7 @@
logOutputFile.clear();
// Initialise the context
- OPENTREP::OPENTREP_Service opentrepService (logOutputFile,
+ OPENTREP::OPENTREP_Service opentrepService (logOutputFile, lDBParams,
lXapianDatabaseName);
// Query the Xapian database (index)
Modified: trunk/opentrep/opentrep/bom/StringMatcher.cpp
===================================================================
--- trunk/opentrep/opentrep/bom/StringMatcher.cpp 2009-07-20 14:03:18 UTC (rev 147)
+++ trunk/opentrep/opentrep/bom/StringMatcher.cpp 2009-07-20 15:17:31 UTC (rev 148)
@@ -322,7 +322,7 @@
NbOfMatches_T idx = 1;
for ( ; itDoc != iMatchingSet.end(); ++itDoc, ++idx) {
const Xapian::percent& lPercentage = itDoc.get_percent();
- const Xapian::Document& lDocument = itDoc.get_document();
+ // const Xapian::Document& lDocument = itDoc.get_document();
// DEBUG
/*
Modified: trunk/opentrep/opentrep/command/SociSessionManager.cpp
===================================================================
--- trunk/opentrep/opentrep/command/SociSessionManager.cpp 2009-07-20 14:03:18 UTC (rev 147)
+++ trunk/opentrep/opentrep/command/SociSessionManager.cpp 2009-07-20 15:17:31 UTC (rev 148)
@@ -2,7 +2,7 @@
// Import section
// //////////////////////////////////////////////////////////////////////
// C
-#include <assert.h>
+#include <cassert>
// STL
#include <string>
#include <sstream>
@@ -10,23 +10,21 @@
#include <soci/core/soci.h>
#include <soci/backends/mysql/soci-mysql.h>
// OpenTrep
+#include <opentrep/DBParams.hpp>
#include <opentrep/command/SociSessionManager.hpp>
#include <opentrep/service/Logger.hpp>
namespace OPENTREP {
// //////////////////////////////////////////////////////////////////////
- void SociSessionManager::init (soci::session*& ioSociSession_ptr) {
+ void SociSessionManager::init (soci::session*& ioSociSession_ptr,
+ const DBParams& iDBParams) {
+
// Database parameters
- const std::string lUserName ("opentrep");
- const std::string lPassword ("opentrep");
- const std::string lDBName ("opentrep");
- const std::string lDBPort ("3306");
- const std::string lDBHost ("localhost");
std::ostringstream oStr;
- oStr << "db=" << lDBName << " user=" << lUserName
- << " password=" << lPassword << " port=" << lDBPort
- << " host=" << lDBHost;
+ oStr << "db=" << iDBParams.getDBName() << " user=" << iDBParams.getUser()
+ << " password=" << iDBParams.getPassword()
+ << " port=" << iDBParams.getPort() << " host=" << iDBParams.getHost();
const std::string lSociSessionConnectionString (oStr.str());
// Instanciate a SOCI Session: nothing is performed at that stage
@@ -40,6 +38,12 @@
} catch (std::exception const& lException) {
OPENTREP_LOG_ERROR ("Error while opening a connection to database: "
<< lException.what());
+ OPENTREP_LOG_ERROR ("Database parameters used:"
+ << " db=" << iDBParams.getDBName()
+ << " user=" << iDBParams.getUser()
+ << " port=" << iDBParams.getPort()
+ << " host=" << iDBParams.getHost());
+ throw SQLDatabaseConnectionImpossibleException();
}
}
Modified: trunk/opentrep/opentrep/command/SociSessionManager.hpp
===================================================================
--- trunk/opentrep/opentrep/command/SociSessionManager.hpp 2009-07-20 14:03:18 UTC (rev 147)
+++ trunk/opentrep/opentrep/command/SociSessionManager.hpp 2009-07-20 15:17:31 UTC (rev 148)
@@ -11,13 +11,16 @@
}
namespace OPENTREP {
+
+ // Forward declarations
+ struct DBParams;
/** Class handling the SOCI session. */
class SociSessionManager {
friend class OPENTREP_Service;
private:
/** Initialise (MySQL) database connection. */
- static void init (soci::session*&);
+ static void init (soci::session*&, const DBParams&);
/** Finalise (MySQL) database connection. */
static void finalise (soci::session*&);
Modified: trunk/opentrep/opentrep/service/OPENTREP_Service.cpp
===================================================================
--- trunk/opentrep/opentrep/service/OPENTREP_Service.cpp 2009-07-20 14:03:18 UTC (rev 147)
+++ trunk/opentrep/opentrep/service/OPENTREP_Service.cpp 2009-07-20 15:17:31 UTC (rev 148)
@@ -20,10 +20,10 @@
// //////////////////////////////////////////////////////////////////////
OPENTREP_Service::
- OPENTREP_Service (std::ostream& ioLogStream,
+ OPENTREP_Service (std::ostream& ioLogStream, const DBParams& iDBParams,
const std::string& iXapianDatabaseFilepath)
: _opentrepServiceContext (NULL) {
- init (ioLogStream, iXapianDatabaseFilepath);
+ init (ioLogStream, iDBParams, iXapianDatabaseFilepath);
}
// //////////////////////////////////////////////////////////////////////
@@ -51,6 +51,7 @@
// //////////////////////////////////////////////////////////////////////
void OPENTREP_Service::init (std::ostream& ioLogStream,
+ const DBParams& iDBParams,
const std::string& iTravelDatabaseName) {
// Set the log file
logInit (LOG::DEBUG, ioLogStream);
@@ -62,7 +63,7 @@
// Initialise the SOCI Session
soci::session* lSociSession_ptr = lOPENTREP_ServiceContext.getSociSession();
- SociSessionManager::init (lSociSession_ptr);
+ SociSessionManager::init (lSociSession_ptr, iDBParams);
assert (lSociSession_ptr != NULL);
lOPENTREP_ServiceContext.setSociSession (*lSociSession_ptr);
Modified: trunk/opentrep/opentrep/sources.mk
===================================================================
--- trunk/opentrep/opentrep/sources.mk 2009-07-20 14:03:18 UTC (rev 147)
+++ trunk/opentrep/opentrep/sources.mk 2009-07-20 15:17:31 UTC (rev 148)
@@ -1,6 +1,7 @@
service_h_sources = \
$(top_srcdir)/opentrep/OPENTREP_Types.hpp \
$(top_srcdir)/opentrep/OPENTREP_Abstract.hpp \
+ $(top_srcdir)/opentrep/DBParams.hpp \
$(top_srcdir)/opentrep/Location.hpp \
$(top_srcdir)/opentrep/OPENTREP_Service.hpp
service_cc_sources =
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <den...@us...> - 2009-07-20 14:03:25
|
Revision: 147
http://opentrep.svn.sourceforge.net/opentrep/?rev=147&view=rev
Author: denis_arnaud
Date: 2009-07-20 14:03:18 +0000 (Mon, 20 Jul 2009)
Log Message:
-----------
[Config] Changed config/ax_mysql.m4, so as to allow for more flexibility on some non-standard platforms.
Modified Paths:
--------------
trunk/opentrep/config/ax_mysql.m4
Modified: trunk/opentrep/config/ax_mysql.m4
===================================================================
--- trunk/opentrep/config/ax_mysql.m4 2009-07-20 13:22:52 UTC (rev 146)
+++ trunk/opentrep/config/ax_mysql.m4 2009-07-20 14:03:18 UTC (rev 147)
@@ -1,172 +1,163 @@
-dnl $Id: ax_mysql.m4,v 1.3 2007/06/23 01:51:22 mloskot Exp $
-dnl
-dnl @synopsis AX_LIB_MYSQL([MINIMUM-VERSION])
-dnl
-dnl This macro provides tests of availability of MySQL client library
-dnl of particular version or newer.
+dnl @synopsis AX_MYSQL
dnl
-dnl AX_LIB_MYSQL macro takes only one argument which is optional. If there is no
-dnl required version passed, then macro does not run version test.
+dnl This macro tries to find MySQL C API header and library locations.
dnl
-dnl The --with-mysql option takes one of three possible values:
-dnl no - do not check for MySQL client library
-dnl yes - do check for MySQL library in standard locations
-dnl (mysql_config should be in the PATH)
-dnl path - complete path to mysql_config utility, use this option
-dnl if mysql_config can't be found in the PATH
+dnl We define the following configure script flags:
dnl
-dnl This macro calls:
+dnl --with-mysql: Give prefix for both library and headers, and try
+dnl to guess subdirectory names for each. (e.g. Tack /lib and
+dnl /include onto given dir name, and other common schemes.)
+dnl --with-mysql-lib: Similar to --with-mysql, but for library only.
+dnl --with-mysql-include: Similar to --with-mysql, but for headers
+dnl only.
dnl
-dnl AC_SUBST(MYSQL_CFLAGS)
-dnl AC_SUBST(MYSQL_LDFLAGS)
-dnl AC_SUBST(MYSQL_VERSION)
-dnl
-dnl And sets:
-dnl
-dnl HAVE_MYSQL
-dnl
-dnl @category InstalledPackages
-dnl @category Cxx
-dnl @author Mateusz Loskot <ma...@lo...>
-dnl @version $Date: 2007/06/23 01:51:22 $
-dnl @license AllPermissive
-dnl
-dnl $Id: ax_mysql.m4,v 1.3 2007/06/23 01:51:22 mloskot Exp $
-dnl
+dnl @version 1.2, 2007/02/20
+dnl @author Warren Young <my...@et...>
+
AC_DEFUN([AX_MYSQL],
[
- AC_ARG_WITH([mysql],
- AC_HELP_STRING([--with-mysql=@<:@ARG@:>@],
- [use MySQL client library @<:@default=yes@:>@, optionally specify path to mysql_config]
- ),
- [
- MYSQL_inc_check="$with_mysql/include"
- if test "$withval" = "no"; then
- want_mysql="no"
- elif test "$withval" = "yes"; then
- want_mysql="yes"
- else
- want_mysql="yes"
- MYSQL_CONFIG="$withval"
- fi
- ],
- [want_mysql="yes"
- MYSQL_inc_check="/usr/include /usr/local /opt"]
- )
+ #
+ # Set up configure script macros
+ #
+ AC_ARG_WITH(mysql,
+ [ --with-mysql=<path> root directory path of MySQL installation],
+ [MYSQL_lib_check="$with_mysql/lib64/mysql $with_mysql/lib/mysql $with_mysql/lib64 $with_mysql/lib"
+ MYSQL_inc_check="$with_mysql/include $with_mysql/include/mysql"
+ MYSQL_bin_check="$with_mysql/bin"],
+ [MYSQL_lib_check="/usr/lib64 /usr/lib /usr/lib64/mysql /usr/lib/mysql /usr/local/lib64 /usr/local/lib /usr/local/lib/mysql /usr/local/mysql/lib /usr/local/mysql/lib/mysql /opt/mysql/lib /opt/mysql/lib/mysql"
+ MYSQL_inc_check="/usr/include /usr/local/include /usr/local/mysql/include /opt/mysql/include"
+ MYSQL_bin_check="/usr/bin /usr/local/bin /usr/local/mysql/bin"])
- MYSQL_CFLAGS=""
- MYSQL_LDFLAGS=""
- MYSQL_VERSION=""
+ AC_ARG_WITH(mysql-lib,
+ [ --with-mysql-lib=<path> directory path of MySQL library installation],
+ [MYSQL_lib_check="$with_mysql_lib $with_mysql_lib/lib64 $with_mysql_lib/lib $with_mysql_lib/lib64/mysql $with_mysql_lib/lib/mysql"])
- dnl
- dnl Check MySQL libraries (libmysqlclient)
- dnl
-
- if test "$want_mysql" = "yes"; then
+ AC_ARG_WITH(mysql-include,
+ [ --with-mysql-include=<path> directory path of MySQL header installation],
+ [MYSQL_inc_check="$with_mysql_include $with_mysql_include/include $with_mysql_include/include/mysql"])
- if test -z "$MYSQL_CONFIG" -o test; then
- AC_PATH_PROG([MYSQL_CONFIG], [mysql_config], [])
- fi
- dnl MYSQL_CONFIG is provided but it does not point to executable file
- if test ! -x "$MYSQL_CONFIG"; then
- AC_MSG_ERROR([$MYSQL_CONFIG does not exist or it is not an exectuable file])
- MYSQL_CONFIG="no"
- found_mysql="no"
- fi
+ #
+ # Look for MySQL Configuration Script
+ #
+ AC_MSG_CHECKING([for MySQL configuration script])
+ MYSQL_CONFIG=
+ MYSQL_bindir=
+ for m in $MYSQL_bin_check
+ do
+ if test -d "$m" && test -f "$m/mysql_config"
+ then
+ MYSQL_CONFIG=$m/mysql_config
+ MYSQL_bindir=$m
+ break
+ fi
+ done
- if test "$MYSQL_CONFIG" != "no"; then
- #
- # Look for MySQL C API headers
- #
- AC_MSG_CHECKING([for MySQL include directory])
- MYSQL_incdir=
- for mysql_inc_dir in $MYSQL_inc_check
- do
- if test -d "$mysql_inc_dir" \
- && test -f "$mysql_inc_dir/mysql/mysql.h"
- then
- MYSQL_incdir=$mysql_inc_dir
- break
- fi
- done
+ if test -z "$MYSQL_bindir"
+ then
+ AC_MSG_ERROR([Didn't find $MYSQL_CONFIG binary in '$MYSQL_bin_check'])
+ fi
- if test -z "$MYSQL_incdir"
- then
- AC_MSG_ERROR([Didn't find the MySQL include dir in '$MYSQL_inc_check'])
- fi
+ case "$MYSQL_bindir" in
+ /* ) ;;
+ * ) AC_MSG_ERROR([The MySQL binary directory ($MYSQL_bindir) must be an absolute path.]) ;;
+ esac
- case "$MYSQL_incdir" in
- /* ) ;;
- * ) AC_MSG_ERROR([The MySQL include directory ($MYSQL_incdir) must be an absolute path.]) ;;
- esac
+ AC_MSG_RESULT([$MYSQL_bindir])
- AC_MSG_RESULT([$MYSQL_incdir])
+# AC_PATH_PROG(MYSQL_CONFIG, mysql_config, $MYSQL_bindir)
- if test "$MYSQL_incdir" = "/usr/include"; then
- MYSQL_CFLAGS=""
- else
- MYSQL_CFLAGS="-I${MYSQL_incdir}"
- fi
+ #
+ # Decide which C API library to use, based on thread support
+ #
+ if test "x$acx_pthread_ok" = xyes
+ then
+ MYSQL_C_LIB=mysqlclient_r
+ MYSQL_LIB_OPTION=libs_r
+ else
+ MYSQL_C_LIB=mysqlclient
+ MYSQL_LIB_OPTION=libs
+ fi
- MYSQL_LDFLAGS="`$MYSQL_CONFIG --libs`"
+ if test "x${MYSQL_CONFIG+set}" != xset
+ then
+ MYSQL_VERSION=`${MYSQL_CONFIG} --version`
+ MYSQL_CFLAGS=`${MYSQL_CONFIG} --cflags`
+ MYSQL_LIBS=`${MYSQL_CONFIG} --${MYSQL_LIB_OPTION}`
+ else
+ #
+ # Look for MySQL C API library
+ #
+ AC_MSG_CHECKING([for MySQL library directory])
+ MYSQL_libdir=
+ for m in $MYSQL_lib_check
+ do
+ if test -d "$m" && \
+ (test -f "$m/lib$MYSQL_C_LIB.so" \
+ || test -f "$m/lib$MYSQL_C_LIB.a")
+ then
+ MYSQL_libdir=$m
+ break
+ fi
+ done
- MYSQL_VERSION=`$MYSQL_CONFIG --version`
+ if test -z "$MYSQL_libdir"
+ then
+ AC_MSG_ERROR([Didn't find $MYSQL_C_LIB library in '$MYSQL_lib_check'])
+ fi
- AC_DEFINE([HAVE_MYSQL], [1], [Define to 1 if MySQL libraries are available])
-
- found_mysql="yes"
- else
- found_mysql="no"
- fi
- fi
+ case "$MYSQL_libdir" in
+ /* ) ;;
+ * ) AC_MSG_ERROR([The MySQL library directory ($MYSQL_libdir) must be an absolute path.]) ;;
+ esac
- dnl
- dnl Check if required version of MySQL is available
- dnl
-
+ AC_MSG_RESULT([$MYSQL_libdir])
- mysql_version_req=ifelse([$1], [], [], [$1])
+ case "$MYSQL_libdir" in
+ /usr/lib) ;;
+ *) LDFLAGS="$LDFLAGS -L${MYSQL_libdir}" ;;
+ esac
- if test "$found_mysql" = "yes" -a -n "$mysql_version_req"; then
+ #
+ # Look for MySQL C API headers
+ #
+ AC_MSG_CHECKING([for MySQL include directory])
+ MYSQL_incdir=
+ for m in $MYSQL_inc_check
+ do
+ if test -d "$m" && test -f "$m/mysql/mysql.h"
+ then
+ MYSQL_incdir=$m
+ break
+ fi
+ done
- AC_MSG_CHECKING([if MySQL version is >= $mysql_version_req])
+ if test -z "$MYSQL_incdir"
+ then
+ AC_MSG_ERROR([Didn't find the MySQL include dir in '$MYSQL_inc_check'])
+ fi
- dnl Decompose required version string of MySQL
- dnl and calculate its number representation
- mysql_version_req_major=`expr $mysql_version_req : '\([[0-9]]*\)'`
- mysql_version_req_minor=`expr $mysql_version_req : '[[0-9]]*\.\([[0-9]]*\)'`
- mysql_version_req_micro=`expr $mysql_version_req : '[[0-9]]*\.[[0-9]]*\.\([[0-9]]*\)'`
- if test "x$mysql_version_req_micro" = "x"; then
- mysql_version_req_micro="0"
- fi
+ case "$MYSQL_incdir" in
+ /* ) ;;
+ * ) AC_MSG_ERROR([The MySQL include directory ($MYSQL_incdir) must be an absolute path.]) ;;
+ esac
- mysql_version_req_number=`expr $mysql_version_req_major \* 1000000 \
- \+ $mysql_version_req_minor \* 1000 \
- \+ $mysql_version_req_micro`
+ AC_MSG_RESULT([$MYSQL_incdir])
- dnl Decompose version string of installed MySQL
- dnl and calculate its number representation
- mysql_version_major=`expr $MYSQL_VERSION : '\([[0-9]]*\)'`
- mysql_version_minor=`expr $MYSQL_VERSION : '[[0-9]]*\.\([[0-9]]*\)'`
- mysql_version_micro=`expr $MYSQL_VERSION : '[[0-9]]*\.[[0-9]]*\.\([[0-9]]*\)'`
- if test "x$mysql_version_micro" = "x"; then
- mysql_version_micro="0"
- fi
+ MYSQL_CFLAGS="-I${MYSQL_incdir}"
+ MYSQL_LIBS="-L${MYSQL_libdir} -l${MYSQL_C_LIB}"
+ fi
- mysql_version_number=`expr $mysql_version_major \* 1000000 \
- \+ $mysql_version_minor \* 1000 \
- \+ $mysql_version_micro`
+ AC_SUBST(MYSQL_VERSION)
+ AC_SUBST(MYSQL_CFLAGS)
+ AC_SUBST(MYSQL_LIBS)
- mysql_version_check=`expr $mysql_version_number \>\= $mysql_version_req_number`
- if test "$mysql_version_check" = "1"; then
- AC_MSG_RESULT([yes])
- else
- AC_MSG_RESULT([no])
- fi
- fi
-
- AC_SUBST([MYSQL_VERSION])
- AC_SUBST([MYSQL_CFLAGS])
- AC_SUBST([MYSQL_LDFLAGS])
-])
+ save_LIBS="$LIBS"
+ LIBS="$LIBS $MYSQL_LIBS"
+ AC_CHECK_LIB($MYSQL_C_LIB, mysql_store_result,
+ [],
+ [AC_MSG_ERROR([Could not find working MySQL client library!])]
+ )
+ AC_SUBST(MYSQL_C_LIB)
+ LIBS="$save_LIBS"
+]) dnl AX_MYSQL
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <den...@us...> - 2009-07-20 13:22:56
|
Revision: 146
http://opentrep.svn.sourceforge.net/opentrep/?rev=146&view=rev
Author: denis_arnaud
Date: 2009-07-20 13:22:52 +0000 (Mon, 20 Jul 2009)
Log Message:
-----------
[Test] Added support for Python wrapper (thanks to Boost Python).
Modified Paths:
--------------
trunk/opentrep/config/ax_boost.m4
trunk/opentrep/config/ax_mysql.m4
trunk/opentrep/configure.ac
trunk/opentrep/opentrep/batches/searcher.cpp
trunk/opentrep/test/python/Makefile.am
trunk/opentrep/test/python/boost_python.cpp
Added Paths:
-----------
trunk/opentrep/config/python.m4
trunk/opentrep/test/python/boost_python.py
trunk/opentrep/test/python/pyopentrep.cpp
trunk/opentrep/test/python/pyopentrep.py
Removed Paths:
-------------
trunk/opentrep/test/python/boost_hello.py
Modified: trunk/opentrep/config/ax_boost.m4
===================================================================
--- trunk/opentrep/config/ax_boost.m4 2009-07-20 09:59:43 UTC (rev 145)
+++ trunk/opentrep/config/ax_boost.m4 2009-07-20 13:22:52 UTC (rev 146)
@@ -18,6 +18,7 @@
dnl AC_SUBST(BOOST_SIGNALS_LIB)
dnl AC_SUBST(BOOST_DATE_TIME_LIB)
dnl AC_SUBST(BOOST_REGEX_LIB)
+dnl AC_SUBST(BOOST_PYTHON)
dnl AC_SUBST(BOOST_UNIT_TEST_FRAMEWORK_LIB)
dnl
dnl And sets:
@@ -31,6 +32,7 @@
dnl HAVE_BOOST_SIGNALS
dnl HAVE_BOOST_DATE_TIME
dnl HAVE_BOOST_REGEX
+dnl HAVE_BOOST_PYTHON
dnl HAVE_BOOST_UNIT_TEST_FRAMEWORK
dnl
dnl @category InstalledPackages
@@ -486,6 +488,27 @@
fi
fi
+ AC_CACHE_CHECK(whether the Boost::Python library is available,
+ ax_cv_boost_python,
+ [AC_LANG_SAVE
+ AC_LANG_CPLUSPLUS
+
+ saved_cflags="${CPPFLAGS}"
+ saved_ldflags="${LDFLAGS}"
+ CPPFLAGS="${CPPFLAGS} ${PYTHON_CFLAGS}"
+ LDFLAGS="${LDFLAGS} ${PYTHON_LIBS} ${PYTHON_ADD_LIBS}"
+
+ AC_COMPILE_IFELSE(AC_LANG_PROGRAM([[@%:@include <boost/python.hpp> char const* greet() { return "hello"; } BOOST_PYTHON_MODULE(hello_ext) { boost::python::def("greet", greet); }]]),
+ ax_cv_boost_python=yes, ax_cv_boost_python=no)
+ AC_LANG_RESTORE
+ ])
+
+ BOOST_PYTHON_LIB="-lboost_python"
+ AC_SUBST(BOOST_PYTHON_LIB)
+
+ CPPFLAGS="${saved_cflags}"
+ LDFLAGS="${saved_ldflags}"
+
AC_CACHE_CHECK(whether the Boost::UnitTestFramework library is available,
ax_cv_boost_unit_test_framework,
[AC_LANG_SAVE
Modified: trunk/opentrep/config/ax_mysql.m4
===================================================================
--- trunk/opentrep/config/ax_mysql.m4 2009-07-20 09:59:43 UTC (rev 145)
+++ trunk/opentrep/config/ax_mysql.m4 2009-07-20 13:22:52 UTC (rev 146)
@@ -33,7 +33,7 @@
dnl
dnl $Id: ax_mysql.m4,v 1.3 2007/06/23 01:51:22 mloskot Exp $
dnl
-AC_DEFUN([AX_LIB_MYSQL],
+AC_DEFUN([AX_MYSQL],
[
AC_ARG_WITH([mysql],
AC_HELP_STRING([--with-mysql=@<:@ARG@:>@],
Added: trunk/opentrep/config/python.m4
===================================================================
--- trunk/opentrep/config/python.m4 (rev 0)
+++ trunk/opentrep/config/python.m4 2009-07-20 13:22:52 UTC (rev 146)
@@ -0,0 +1,94 @@
+#
+# Autoconf macros for configuring the build of Python extension modules
+#
+
+# PGAC_PATH_PYTHON
+# ----------------
+# Look for Python and set the output variable 'PYTHON'
+# to 'python' if found, empty otherwise.
+AC_DEFUN([PGAC_PATH_PYTHON],
+[AC_PATH_PROG(PYTHON, python)
+if test x"$PYTHON" = x""; then
+ AC_MSG_ERROR([Python not found])
+fi
+])
+
+
+# _PGAC_CHECK_PYTHON_DIRS
+# -----------------------
+# Determine the name of various directories of a given Python installation.
+AC_DEFUN([_PGAC_CHECK_PYTHON_DIRS],
+[AC_REQUIRE([PGAC_PATH_PYTHON])
+AC_MSG_CHECKING([for Python distutils module])
+if "${PYTHON}" 2>&- -c 'import distutils'
+then
+ AC_MSG_RESULT(yes)
+else
+ AC_MSG_RESULT(no)
+ AC_MSG_ERROR([distutils module not found])
+fi
+AC_MSG_CHECKING([Python configuration directory])
+python_version=`${PYTHON} -c "import sys; print sys.version[[:3]]"`
+python_configdir=`${PYTHON} -c "from distutils.sysconfig import get_python_lib as f; import os; print os.path.join(f(plat_specific=1,standard_lib=1),'config')"`
+PYTHON_CFLAGS=`${PYTHON} -c "import distutils.sysconfig; print '-I'+distutils.sysconfig.get_python_inc()"`
+
+AC_SUBST(python_version)[]dnl
+AC_SUBST(python_configdir)[]dnl
+AC_SUBST(PYTHON_CFLAGS)[]dnl
+# This should be enough of a message.
+AC_MSG_RESULT([$python_configdir])
+])# _PGAC_CHECK_PYTHON_DIRS
+
+
+# PGAC_CHECK_PYTHON_EMBED_SETUP
+# -----------------------------
+#
+# Note: selecting libpython from python_configdir works in all Python
+# releases, but it generally finds a non-shared library, which means
+# that we are binding the python interpreter right into libplpython.so.
+# In Python 2.3 and up there should be a shared library available in
+# the main library location.
+AC_DEFUN([PGAC_CHECK_PYTHON_EMBED_SETUP],
+[AC_REQUIRE([_PGAC_CHECK_PYTHON_DIRS])
+AC_MSG_CHECKING([how to link an embedded Python application])
+
+python_libdir=`${PYTHON} -c "import distutils.sysconfig,string; print string.join(filter(None,distutils.sysconfig.get_config_vars('LIBDIR')))"`
+python_ldlibrary=`${PYTHON} -c "import distutils.sysconfig,string; print string.join(filter(None,distutils.sysconfig.get_config_vars('LDLIBRARY')))"`
+python_so=`${PYTHON} -c "import distutils.sysconfig,string; print string.join(filter(None,distutils.sysconfig.get_config_vars('SO')))"`
+ldlibrary=`echo "${python_ldlibrary}" | sed "s/${python_so}$//"`
+
+if test x"${python_libdir}" != x"" -a x"${python_ldlibrary}" != x"" -a x"${python_ldlibrary}" != x"${ldlibrary}"
+then
+ # New way: use the official shared library
+ ldlibrary=`echo "${ldlibrary}" | sed "s/^lib//"`
+ PYTHON_LIBS="-L${python_libdir} -l${ldlibrary}"
+else
+ # Old way: use libpython from python_configdir
+ python_libdir="${python_configdir}"
+ PYTHON_LIBS="-L${python_libdir} -lpython${python_version}"
+fi
+
+PYTHON_ADD_LIBS=`${PYTHON} -c "import distutils.sysconfig,string; print string.join(filter(None,distutils.sysconfig.get_config_vars('LIBS','LIBC','LIBM','LOCALMODLIBS','BASEMODLIBS')))"`
+
+AC_MSG_RESULT([${PYTHON_LIBS} ${PYTHON_ADD_LIBS}])
+
+AC_SUBST(python_incdir)[]dnl
+AC_SUBST(python_libdir)[]dnl
+AC_SUBST(PYTHON_LIBS)[]dnl
+AC_SUBST(PYTHON_ADD_LIBS)[]dnl
+
+# threaded python is not supported on bsd's
+AC_MSG_CHECKING(whether Python is compiled with thread support)
+pythreads=`${PYTHON} -c "import sys; print int('thread' in sys.builtin_module_names)"`
+if test "$pythreads" = "1"; then
+ AC_MSG_RESULT(yes)
+ case $host_os in
+ openbsd*|freebsd*)
+ AC_MSG_ERROR([threaded Python not supported on this platform])
+ ;;
+ esac
+else
+ AC_MSG_RESULT(no)
+fi
+
+])# PGAC_CHECK_PYTHON_EMBED_SETUP
Modified: trunk/opentrep/configure.ac
===================================================================
--- trunk/opentrep/configure.ac 2009-07-20 09:59:43 UTC (rev 145)
+++ trunk/opentrep/configure.ac 2009-07-20 13:22:52 UTC (rev 146)
@@ -103,6 +103,15 @@
fi
+# -----------------------------------------------------------
+# Python
+# -----------------------------------------------------------
+PGAC_CHECK_PYTHON_EMBED_SETUP
+AC_SUBST(PYTHON_VERSION)
+AC_SUBST(PYTHON_LIBS)
+AC_SUBST(PYTHON_CFLAGS)
+AC_SUBST(PYTHON_ADD_LIBS)
+
# --------------------------------------------------------
# Boost (STL Extensions: http://www.boost.org)
# --------------------------------------------------------
@@ -112,6 +121,7 @@
AC_SUBST(BOOST_LIBS)
AC_SUBST(BOOST_DATE_TIME_LIB)
AC_SUBST(BOOST_PROGRAM_OPTIONS_LIB)
+AC_SUBST(BOOST_PYTHON_LIB)
# --------------------------------------------------------------------
# Support for MySQL (C client API): http://www.mysql.org
@@ -287,12 +297,19 @@
- LIBS .............. : ${LIBS}
External libraries:
+ - Python ............. :
+ o PYTHON_version ... : ${PYTHON_VERSION}
+ o PYTHON_CFLAGS .... : ${PYTHON_CFLAGS}
+ o PYTHON_LIBS ...... : ${PYTHON_LIBS}
+ o PYTHON_ADD_LIBS .. : ${PYTHON_ADD_LIBS}
+
- Boost ............. :
o BOOST_VERSION ... : ${BOOST_VERSION}
o BOOST_CFLAGS .... : ${BOOST_CFLAGS}
o BOOST_LIBS ...... : ${BOOST_LIBS}
o BOOST_DT_LIB .... : ${BOOST_DATE_TIME_LIB}
o BOOST_PO_LIB .... : ${BOOST_PROGRAM_OPTIONS_LIB}
+ o BOOST_PYTH_LIB .. : ${BOOST_PYTHON_LIB}
- MySQL ............. :
o MYSQL_version ... : ${MYSQL_VERSION}
Modified: trunk/opentrep/opentrep/batches/searcher.cpp
===================================================================
--- trunk/opentrep/opentrep/batches/searcher.cpp 2009-07-20 09:59:43 UTC (rev 145)
+++ trunk/opentrep/opentrep/batches/searcher.cpp 2009-07-20 13:22:52 UTC (rev 146)
@@ -11,7 +11,7 @@
#include <boost/date_time/gregorian/gregorian.hpp>
#include <boost/tokenizer.hpp>
#include <boost/program_options.hpp>
-// OPENTREP
+// OpenTREP
#include <opentrep/OPENTREP_Service.hpp>
#include <opentrep/config/opentrep-paths.hpp>
@@ -235,7 +235,7 @@
lNonMatchedWordList);
std::cout << nbOfMatches << " (geographical) location(s) have been found "
- << "matching your query (`" << lTravelQuery << "´). "
+ << "matching your query (`" << lTravelQuery << "\xB4). "
<< lNonMatchedWordList.size() << " words were left unmatched."
<< std::endl;
Modified: trunk/opentrep/test/python/Makefile.am
===================================================================
--- trunk/opentrep/test/python/Makefile.am 2009-07-20 09:59:43 UTC (rev 145)
+++ trunk/opentrep/test/python/Makefile.am 2009-07-20 13:22:52 UTC (rev 146)
@@ -3,13 +3,19 @@
MAINTAINERCLEANFILES = Makefile.in
-# Library
-lib_LTLIBRARIES = libpy@PACKAGE@.la
+# Libraries
+lib_LTLIBRARIES = libpyboost.la libpy@PACKAGE@.la
-libpy@PACKAGE@_la_SOURCES = boost_python.cpp
-libpy@PACKAGE@_la_CXXFLAGS = -I/usr/include/python2.6
-#libpy@PACKAGE@_la_LIBADD =
-libpy@PACKAGE@_la_LDFLAGS = -L/usr/lib64 -lboost_python-mt -lpython2.6 \
- -version-info $(GENERIC_LIBRARY_VERSION)
+libpyboost_la_SOURCES = boost_python.cpp
+libpyboost_la_CXXFLAGS = ${PYTHON_CFLAGS}
+#libpyboost_la_LIBADD =
+libpyboost_la_LDFLAGS = ${PYTHON_LIBS} ${PYTHON_ADD_LIBS} \
+ ${BOOST_PYTHON_LIB} -version-info $(GENERIC_LIBRARY_VERSION)
+libpy@PACKAGE@_la_SOURCES = pyopentrep.cpp
+libpy@PACKAGE@_la_CXXFLAGS = ${PYTHON_CFLAGS} ${BOOST_CFLAGS}
+libpy@PACKAGE@_la_LIBADD = $(top_builddir)/@PACKAGE@/core/lib@PACKAGE@.la
+libpy@PACKAGE@_la_LDFLAGS = ${PYTHON_LIBS} ${PYTHON_ADD_LIBS} \
+ ${BOOST_PYTHON_LIB} -version-info $(GENERIC_LIBRARY_VERSION)
+
EXTRA_DIST =
Deleted: trunk/opentrep/test/python/boost_hello.py
===================================================================
--- trunk/opentrep/test/python/boost_hello.py 2009-07-20 09:59:43 UTC (rev 145)
+++ trunk/opentrep/test/python/boost_hello.py 2009-07-20 13:22:52 UTC (rev 146)
@@ -1,14 +0,0 @@
-#!/usr/bin/env python
-
-import sys
-sys.path.append('.libs')
-import libpyopentrep
-myWorld = libpyopentrep.World()
-myWorld.add('Bonjour', 5)
-myWorld.add('Hello', 7)
-myWorld.add('Gruss Gott', 5)
-#MsgList=myWorld.getMsgList()
-#IDList=myWorld.getIDList()
-#print ', '.join(['%s: %s' % (msg, id) for (msg, id) in zip(MsgList, IDList)])
-a = myWorld.toSimpleString().split(', ')
-print '^'.join(a[::-1])
Modified: trunk/opentrep/test/python/boost_python.cpp
===================================================================
--- trunk/opentrep/test/python/boost_python.cpp 2009-07-20 09:59:43 UTC (rev 145)
+++ trunk/opentrep/test/python/boost_python.cpp 2009-07-20 13:22:52 UTC (rev 146)
@@ -6,6 +6,12 @@
// Boost String
#include <boost/python.hpp>
+// //////////////////////////////////////////////
+char const* simpleGreet() {
+ return "hello, world";
+}
+
+// //////////////////////////////////////////////
namespace OPENTREP {
/** */
@@ -19,7 +25,7 @@
/** */
typedef std::pair<MsgList_T, IDList_T> OpenTrepList_T;
-
+
// //////////////////////////////////////////////
struct World {
public:
@@ -71,7 +77,7 @@
itMsg != _msgList.end() && itID != _idList.end();
++itMsg, ++itID, ++idx) {
if (idx != 0) {
- oStr << ", ";
+ oStr << ",";
}
oStr << *itMsg;
}
@@ -87,7 +93,14 @@
}
// //////////////////////////////////////////////
-BOOST_PYTHON_MODULE(libpyopentrep) {
+// using namespace boost::python;
+BOOST_PYTHON_MODULE(hello_ext) {
+ boost::python::def("greet", simpleGreet);
+}
+
+
+// //////////////////////////////////////////////
+BOOST_PYTHON_MODULE(libpyboost) {
boost::python::class_<OPENTREP::World> ("World")
.def ("getMsgList", &OPENTREP::World::getMsgList)
.def ("getIDList", &OPENTREP::World::getIDList)
Copied: trunk/opentrep/test/python/boost_python.py (from rev 145, trunk/opentrep/test/python/boost_hello.py)
===================================================================
--- trunk/opentrep/test/python/boost_python.py (rev 0)
+++ trunk/opentrep/test/python/boost_python.py 2009-07-20 13:22:52 UTC (rev 146)
@@ -0,0 +1,18 @@
+#!/usr/bin/env python
+
+import sys
+sys.path.append('.libs')
+import libpyboost
+
+myWorld = libpyboost.World()
+myWorld.add('Bonjour', 5)
+myWorld.add('Hello', 7)
+myWorld.add('Gruss Gott', 5)
+
+#MsgList=myWorld.getMsgList()
+#IDList=myWorld.getIDList()
+
+#print ', '.join(['%s: %s' % (msg, id) for (msg, id) in zip(MsgList, IDList)])
+
+a = myWorld.toSimpleString().split(',')
+print '^'.join(a[::-1])
Added: trunk/opentrep/test/python/pyopentrep.cpp
===================================================================
--- trunk/opentrep/test/python/pyopentrep.cpp (rev 0)
+++ trunk/opentrep/test/python/pyopentrep.cpp 2009-07-20 13:22:52 UTC (rev 146)
@@ -0,0 +1,69 @@
+// C
+#include <sstream>
+#include <string>
+#include <list>
+#include <vector>
+// Boost String
+#include <boost/python.hpp>
+// OpenTREP
+#include <opentrep/OPENTREP_Service.hpp>
+
+namespace OPENTREP {
+
+ struct OpenTrepSearcher {
+
+ /** Wrapper around the search use case. */
+ std::string search (const std::string& iTravelQuery,
+ const std::string& iXapianDatabaseFilepath) {
+ std::ostringstream oStr;
+
+ // Output stream (for logs)
+ std::ostringstream logOutputStream;
+
+ // Initialise the context
+ OPENTREP_Service opentrepService(logOutputStream, iXapianDatabaseFilepath);
+
+ // Query the Xapian database (index)
+ WordList_T lNonMatchedWordList;
+ LocationList_T lLocationList;
+ const NbOfMatches_T nbOfMatches =
+ opentrepService.interpretTravelRequest (iTravelQuery, lLocationList,
+ lNonMatchedWordList);
+
+ if (nbOfMatches != 0) {
+ NbOfMatches_T idx = 0;
+ for (LocationList_T::const_iterator itLocation = lLocationList.begin();
+ itLocation != lLocationList.end(); ++itLocation, ++idx) {
+ const Location& lLocation = *itLocation;
+ if (idx != 0) {
+ oStr << ",";
+ }
+ oStr << lLocation.getLocationCode();
+ }
+ }
+
+ if (lNonMatchedWordList.empty() == false) {
+ oStr << ";";
+ NbOfMatches_T idx = 0;
+ for (WordList_T::const_iterator itWord = lNonMatchedWordList.begin();
+ itWord != lNonMatchedWordList.end(); ++itWord, ++idx) {
+ const Word_T& lWord = *itWord;
+ if (idx != 0) {
+ oStr << ",";
+ }
+ oStr << lWord;
+ }
+ }
+
+ return oStr.str();
+ }
+
+ };
+
+}
+
+// /////////////////////////////////////////////////////////////
+BOOST_PYTHON_MODULE(libpyopentrep) {
+ boost::python::class_<OPENTREP::OpenTrepSearcher> ("OpenTrepSearcher")
+ .def ("search", &OPENTREP::OpenTrepSearcher::search);
+}
Added: trunk/opentrep/test/python/pyopentrep.py
===================================================================
--- trunk/opentrep/test/python/pyopentrep.py (rev 0)
+++ trunk/opentrep/test/python/pyopentrep.py 2009-07-20 13:22:52 UTC (rev 146)
@@ -0,0 +1,13 @@
+#!/usr/bin/env python
+
+import sys
+sys.path.append('.libs')
+import libpyopentrep
+
+mySearch = libpyopentrep.OpenTrepSearcher()
+a = mySearch.search('sna francicso rio de janero lso angles reykyavki','../traveldb')
+print a
+
+#print ', '.join(['%s: %s' % (msg, id) for (msg, id) in zip(MsgList, IDList)])
+#a = myWorld.toSimpleString().split(',')
+#print '^'.join(a[::-1])
Property changes on: trunk/opentrep/test/python/pyopentrep.py
___________________________________________________________________
Added: svn:executable
+ *
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <den...@us...> - 2009-07-20 09:59:46
|
Revision: 145
http://opentrep.svn.sourceforge.net/opentrep/?rev=145&view=rev
Author: denis_arnaud
Date: 2009-07-20 09:59:43 +0000 (Mon, 20 Jul 2009)
Log Message:
-----------
[Python] Added a test about Boost.Python
Modified Paths:
--------------
trunk/opentrep/configure.ac
Added Paths:
-----------
trunk/opentrep/opentrep/python/
trunk/opentrep/test/python/
trunk/opentrep/test/python/Makefile.am
trunk/opentrep/test/python/boost_hello.py
trunk/opentrep/test/python/boost_python.cpp
Modified: trunk/opentrep/configure.ac
===================================================================
--- trunk/opentrep/configure.ac 2009-07-19 22:25:34 UTC (rev 144)
+++ trunk/opentrep/configure.ac 2009-07-20 09:59:43 UTC (rev 145)
@@ -241,6 +241,7 @@
test/com/Makefile
test/parsers/Makefile
test/i18n/Makefile
+ test/python/Makefile
test/Makefile
win32/Makefile)
AC_OUTPUT
Property changes on: trunk/opentrep/test/python
___________________________________________________________________
Added: svn:ignore
+ .libs
.deps
Makefile
Makefile.in
Added: trunk/opentrep/test/python/Makefile.am
===================================================================
--- trunk/opentrep/test/python/Makefile.am (rev 0)
+++ trunk/opentrep/test/python/Makefile.am 2009-07-20 09:59:43 UTC (rev 145)
@@ -0,0 +1,15 @@
+## command sub-directory
+include $(top_srcdir)/Makefile.common
+
+MAINTAINERCLEANFILES = Makefile.in
+
+# Library
+lib_LTLIBRARIES = libpy@PACKAGE@.la
+
+libpy@PACKAGE@_la_SOURCES = boost_python.cpp
+libpy@PACKAGE@_la_CXXFLAGS = -I/usr/include/python2.6
+#libpy@PACKAGE@_la_LIBADD =
+libpy@PACKAGE@_la_LDFLAGS = -L/usr/lib64 -lboost_python-mt -lpython2.6 \
+ -version-info $(GENERIC_LIBRARY_VERSION)
+
+EXTRA_DIST =
Added: trunk/opentrep/test/python/boost_hello.py
===================================================================
--- trunk/opentrep/test/python/boost_hello.py (rev 0)
+++ trunk/opentrep/test/python/boost_hello.py 2009-07-20 09:59:43 UTC (rev 145)
@@ -0,0 +1,14 @@
+#!/usr/bin/env python
+
+import sys
+sys.path.append('.libs')
+import libpyopentrep
+myWorld = libpyopentrep.World()
+myWorld.add('Bonjour', 5)
+myWorld.add('Hello', 7)
+myWorld.add('Gruss Gott', 5)
+#MsgList=myWorld.getMsgList()
+#IDList=myWorld.getIDList()
+#print ', '.join(['%s: %s' % (msg, id) for (msg, id) in zip(MsgList, IDList)])
+a = myWorld.toSimpleString().split(', ')
+print '^'.join(a[::-1])
Property changes on: trunk/opentrep/test/python/boost_hello.py
___________________________________________________________________
Added: svn:executable
+ *
Added: trunk/opentrep/test/python/boost_python.cpp
===================================================================
--- trunk/opentrep/test/python/boost_python.cpp (rev 0)
+++ trunk/opentrep/test/python/boost_python.cpp 2009-07-20 09:59:43 UTC (rev 145)
@@ -0,0 +1,98 @@
+// C
+#include <sstream>
+#include <string>
+#include <list>
+#include <vector>
+// Boost String
+#include <boost/python.hpp>
+
+namespace OPENTREP {
+
+ /** */
+ typedef unsigned int OpenTrepID_T;
+
+ /** */
+ typedef std::vector<std::string> MsgList_T;
+
+ /** */
+ typedef std::vector<OpenTrepID_T> IDList_T;
+
+ /** */
+ typedef std::pair<MsgList_T, IDList_T> OpenTrepList_T;
+
+ // //////////////////////////////////////////////
+ struct World {
+ public:
+
+ /** */
+ MsgList_T getMsgList () const {
+ return _msgList;
+ }
+
+ /** */
+ IDList_T getIDList () const {
+ return _idList;
+ }
+
+ /** */
+ OpenTrepList_T getListPair () const {
+ return OpenTrepList_T (_msgList, _idList);
+ }
+
+ /** */
+ void add (const std::string& iMessage, const OpenTrepID_T& iID) {
+ _msgList.push_back (iMessage);
+ _idList.push_back (iID);
+ }
+
+
+ /** */
+ std::string toString() const {
+ std::ostringstream oStr;
+ IDList_T::const_iterator itID = _idList.begin();
+ OpenTrepID_T idx = 0;
+ for (MsgList_T::const_iterator itMsg = _msgList.begin();
+ itMsg != _msgList.end() && itID != _idList.end();
+ ++itMsg, ++itID, ++idx) {
+ if (idx != 0) {
+ oStr << ", ";
+ }
+ oStr << *itID << ": " << *itMsg;
+ }
+ return oStr.str();
+ }
+
+ /** */
+ std::string toSimpleString() const {
+ std::ostringstream oStr;
+ IDList_T::const_iterator itID = _idList.begin();
+ OpenTrepID_T idx = 0;
+ for (MsgList_T::const_iterator itMsg = _msgList.begin();
+ itMsg != _msgList.end() && itID != _idList.end();
+ ++itMsg, ++itID, ++idx) {
+ if (idx != 0) {
+ oStr << ", ";
+ }
+ oStr << *itMsg;
+ }
+ return oStr.str();
+ }
+
+ private:
+ /** */
+ MsgList_T _msgList;
+ IDList_T _idList;
+ };
+
+}
+
+// //////////////////////////////////////////////
+BOOST_PYTHON_MODULE(libpyopentrep) {
+ boost::python::class_<OPENTREP::World> ("World")
+ .def ("getMsgList", &OPENTREP::World::getMsgList)
+ .def ("getIDList", &OPENTREP::World::getIDList)
+ .def ("getListPair", &OPENTREP::World::getListPair)
+ .def ("add", &OPENTREP::World::add)
+ .def ("toSimpleString", &OPENTREP::World::toSimpleString)
+ .def ("toString", &OPENTREP::World::toString);
+}
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <den...@us...> - 2009-07-19 22:25:37
|
Revision: 144
http://opentrep.svn.sourceforge.net/opentrep/?rev=144&view=rev
Author: denis_arnaud
Date: 2009-07-19 22:25:34 +0000 (Sun, 19 Jul 2009)
Log Message:
-----------
[i18n] Added a few tests on i18n string handling.
Modified Paths:
--------------
trunk/opentrep/test/i18n/boost_string.cpp
trunk/opentrep/test/i18n/stdlocru.cpp
Modified: trunk/opentrep/test/i18n/boost_string.cpp
===================================================================
--- trunk/opentrep/test/i18n/boost_string.cpp 2009-07-19 21:48:27 UTC (rev 143)
+++ trunk/opentrep/test/i18n/boost_string.cpp 2009-07-19 22:25:34 UTC (rev 144)
@@ -11,6 +11,21 @@
boost::to_upper (str1);
std::cout << "After: " << str1 << std::endl;
+
+ // With STL locale feature
+ std::string mucDE ("München");
+ std::string mucRU ("Мюнхен");
+ std::string pacaFR ("Provence Alpes Côtes d'Azur");
+
+ // Test the Boost String conversions
+ boost::to_upper (mucDE);
+ boost::to_upper (mucRU);
+ boost::to_upper (pacaFR);
+ // Display the strings
+ std::cout << "de: " << mucDE << std::endl;
+ std::cout << "ru: " << mucRU << std::endl;
+ std::cout << "fr: " << pacaFR << std::endl;
+
return 0;
}
Modified: trunk/opentrep/test/i18n/stdlocru.cpp
===================================================================
--- trunk/opentrep/test/i18n/stdlocru.cpp 2009-07-19 21:48:27 UTC (rev 143)
+++ trunk/opentrep/test/i18n/stdlocru.cpp 2009-07-19 22:25:34 UTC (rev 144)
@@ -9,7 +9,8 @@
int main (int argc, char* argv[]) {
// Create the default locale from the user's environment
- std::locale langLocale ("ru_RU.UTF-8");
+ // std::locale langLocale ("ru_RU.UTF-8");
+ std::locale langLocale ("");
// and assign it to the standard output channel
//std::cout.imbue (langLocale);
@@ -18,15 +19,33 @@
//std::cin.imbue (langLocale);
// Display with no processing
+ std::cout << "Simple character literals without processing:" << std::endl;
std::cout << "de: München" << std::endl;
std::cout << "ru: Мюнхен" << std::endl;
- const std::string mucDE ("München");
- const std::string mucRU ("Мюнхен");
+ // With STL strings
+ std::string mucDE ("München");
+ std::string mucRU ("Мюнхен");
- // Display the strings
+ // Display the STL strings
+ std::cout << "STL strings without processing:" << std::endl;
std::cout << "de: " << mucDE << std::endl;
std::cout << "ru: " << mucRU << std::endl;
+
+ // With char*
+ char mucDEStr[] = "München";
+ char mucRUStr[] = "Мюнхен";
+
+ // STL ctypes on char*
+ std::use_facet<std::ctype<char> > (langLocale).toupper (mucDEStr,
+ mucDEStr+8);
+ std::use_facet<std::ctype<char> > (langLocale).toupper (mucRUStr,
+ mucRUStr+8);
+ // Display the altered STL strings
+ std::cout << "Character literals with STL locale processing:" << std::endl;
+ std::cout << "de: " << mucDEStr << std::endl;
+ std::cout << "ru: " << mucRUStr << std::endl;
+
return 0;
}
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <den...@us...> - 2009-07-19 21:48:31
|
Revision: 143
http://opentrep.svn.sourceforge.net/opentrep/?rev=143&view=rev
Author: denis_arnaud
Date: 2009-07-19 21:48:27 +0000 (Sun, 19 Jul 2009)
Log Message:
-----------
[i18n] Added a few simple tests on STL i18n (locale) features.
Modified Paths:
--------------
trunk/opentrep/configure.ac
trunk/opentrep/db/data/ref_place_names.csv
trunk/opentrep/opentrep/dbadaptor/DbaPlace.cpp
Added Paths:
-----------
trunk/opentrep/test/i18n/
trunk/opentrep/test/i18n/Makefile.am
trunk/opentrep/test/i18n/boost_string.cpp
trunk/opentrep/test/i18n/loc2.cpp
trunk/opentrep/test/i18n/stdlocru.cpp
Modified: trunk/opentrep/configure.ac
===================================================================
--- trunk/opentrep/configure.ac 2009-07-19 13:12:19 UTC (rev 142)
+++ trunk/opentrep/configure.ac 2009-07-19 21:48:27 UTC (rev 143)
@@ -240,6 +240,7 @@
db/data/Makefile
test/com/Makefile
test/parsers/Makefile
+ test/i18n/Makefile
test/Makefile
win32/Makefile)
AC_OUTPUT
Modified: trunk/opentrep/db/data/ref_place_names.csv
===================================================================
--- trunk/opentrep/db/data/ref_place_names.csv 2009-07-19 13:12:19 UTC (rev 142)
+++ trunk/opentrep/db/data/ref_place_names.csv 2009-07-19 21:48:27 UTC (rev 143)
@@ -1,7 +1,4 @@
concat('en,', code, ',', ticketing_name, ',', teleticketing_name, ',', extended_name)
-de,muc,münchen,muenchen,munich/de:franz j strauss
-fr,muc,munique,munique,munich/de:franz j strauss
-ru,muc,мюнхен,munich,munich/de:franz j strauss
en,cyz,cauayan,cauayan,cauayan/ph
en,cza,chichen itza,chichen itza,chichen itza/mx
en,czb,cruz alta,cruz alta,cruz alta/rs/br:carlos ruhl
@@ -3710,7 +3707,7 @@
en,bbo,berbera,berbera,berbera/so
en,bbp,bembridge,bembridge,bembridge/gb
en,bbq,barbuda,barbuda,barbuda/ag
-en,bbr,basse terre,basse terre,basse terre/gp:baillif
+en,bbr,basse terre,basse terre,basse terre/gp:baillif,guadeloupe
en,bbs,blackbush,blackbush,blackbush/gb
en,bbt,berberati,berberati,berberati/cf
en,bbu,bucharest baneasa,bucharest banea,bucharest/ro:baneasa
@@ -3838,7 +3835,7 @@
en,fct,yakima aaf,yakima aaf,yakima/wa/us:firing center aaf
en,fcy,forrest city,forrest city,forrest city/ar/us:municipal
en,fde,forde,forde,forde/no:bringeland
-en,fdf,fort de france,fort de france,fort de france/mq:lamentin
+en,fdf,fort de france,fort de france,fort de france/mq:lamentin,martinique
en,fdh,friedrichshafen,friedrichshafen,friedrichshafen/de:friedrichsh
en,fdk,frederick,frederick,frederick/md/us:municipal
en,fdr,frederick,frederick,frederick/ok/us:municipal
@@ -7731,7 +7728,7 @@
en,ptm,palmarito,palmarito,palmarito/ve
en,ptn,morgan city,morgan city,morgan city/la/us:mncpl hpt
en,pto,pato branco,pato branco,pato branco/pr/br:municipal
-en,ptp,pointe a pitre,pointe a pitre,pointe a pitr/gp:pole caraibes
+en,ptp,pointe a pitre,pointe a pitre,pointe a pitre/gp:pole caraibes,guadeloupe
en,ptq,porto de moz,porto de moz,porto de moz/pa/br
en,ptr,pleasant harbour,pleasant harbou,pleasant harbour/ak/us
en,pts,pittsburg,pittsburg,pittsburg/ks/us:municipal
@@ -8853,7 +8850,7 @@
en,pjz,puerto juarez,puerto juarez,puerto juarez/mx
en,pka,napaskiak,napaskiak,napaskiak/ak/us:spb
en,pkb,mid ohio valley,mid ohio valley,parkersburg/wv/us:mid ohio val
-en,pkc,petropavlovsk kam,petropavlovsk k,petropavlovsk kam/ru
+en,pkc,petropavlovsk kamchatka,petropavlovsk kam,petropavlovsk kam/ru,petropavlovsk,kamchatka
en,pkd,park rapids,park rapids,park rapids/mn/us
en,pke,parkes,parkes,parkes/ns/au
en,pkf,park falls,park falls,park falls/wi/us
@@ -9144,7 +9141,7 @@
en,rui,ruidoso,ruidoso,ruidoso/nm/us:municipal
en,ruk,rukumkot,rukumkot,rukumkot/np
en,rum,rumjatar,rumjatar,rumjatar/np
-en,run,st denis reunion,st denis reunio,st denis reunion/re:roland gar
+en,run,st denis reunion,st denis reunion,st denis reunion/re:roland gar,la reunion,reunion
en,rup,rupsi,rupsi,rupsi/in
en,rur,rurutu,rurutu,rurutu/pf
en,rus,marau sound,marau sound,marau sound/sb
Modified: trunk/opentrep/opentrep/dbadaptor/DbaPlace.cpp
===================================================================
--- trunk/opentrep/opentrep/dbadaptor/DbaPlace.cpp 2009-07-19 13:12:19 UTC (rev 142)
+++ trunk/opentrep/opentrep/dbadaptor/DbaPlace.cpp 2009-07-19 21:48:27 UTC (rev 143)
@@ -54,7 +54,7 @@
ioPlace.addName (lLanguageCode, lClassicalName);
const std::string& lClassicalName2 =
- iPlaceValues.get<std::string> ("classical_name", "");
+ iPlaceValues.get<std::string> ("classical_name2", "");
ioPlace.addName (lLanguageCode, lClassicalName2);
const std::string& lExtendedName =
Property changes on: trunk/opentrep/test/i18n
___________________________________________________________________
Added: svn:ignore
+ .libs
.deps
Makefile.in
Makefile
boost_string
loc2
stdlocru
Added: trunk/opentrep/test/i18n/Makefile.am
===================================================================
--- trunk/opentrep/test/i18n/Makefile.am (rev 0)
+++ trunk/opentrep/test/i18n/Makefile.am 2009-07-19 21:48:27 UTC (rev 143)
@@ -0,0 +1,20 @@
+## command sub-directory
+include $(top_srcdir)/Makefile.common
+
+MAINTAINERCLEANFILES = Makefile.in
+
+check_PROGRAMS = boost_string loc2 stdlocru
+
+boost_string_SOURCES = boost_string.cpp
+boost_string_CXXFLAGS = $(BOOST_CFLAGS)
+boost_string_LDADD = $(BOOST_LIB)
+
+loc2_SOURCES = loc2.cpp
+loc2_CXXFLAGS = $(BOOST_CFLAGS)
+loc2_LDADD = $(BOOST_LIB)
+
+stdlocru_SOURCES = stdlocru.cpp
+stdlocru_CXXFLAGS = $(BOOST_CFLAGS)
+stdlocru_LDADD = $(BOOST_LIB)
+
+EXTRA_DIST =
Added: trunk/opentrep/test/i18n/boost_string.cpp
===================================================================
--- trunk/opentrep/test/i18n/boost_string.cpp (rev 0)
+++ trunk/opentrep/test/i18n/boost_string.cpp 2009-07-19 21:48:27 UTC (rev 143)
@@ -0,0 +1,16 @@
+// C
+#include <iostream>
+// Boost String
+#include <boost/algorithm/string.hpp>
+
+// ////////////// M A I N //////////////
+int main (int argc, char* argv[]) {
+
+ std::string str1(" hello world! ");
+ std::cout << "Before: " << str1 << std::endl;
+
+ boost::to_upper (str1);
+ std::cout << "After: " << str1 << std::endl;
+
+ return 0;
+}
Added: trunk/opentrep/test/i18n/loc2.cpp
===================================================================
--- trunk/opentrep/test/i18n/loc2.cpp (rev 0)
+++ trunk/opentrep/test/i18n/loc2.cpp 2009-07-19 21:48:27 UTC (rev 143)
@@ -0,0 +1,60 @@
+// C
+#include <iostream>
+#include <locale>
+#include <string>
+#include <cstdlib>
+
+
+// ////////////// M A I N //////////////
+int main (int argc, char* argv[]) {
+
+ // Create the default locale from the user's environment
+ std::locale langLocale ("");
+
+ // and assign it to the standard output channel
+ std::cout.imbue (langLocale);
+
+ //
+ std::cout << "User's environment locale: " << langLocale.name() << std::endl;
+
+ bool isFrench = false;
+ if (langLocale.name() == "fr_FR"
+ || langLocale.name() == "fr"
+ || langLocale.name() == "french") {
+ isFrench = true;
+ }
+
+ // Read locale for the input
+ if (isFrench) {
+ std::cout << "Locale pour l'entrée: " << std::endl;
+
+ } else {
+ std::cout << "Locale for input: " << std::endl;
+ }
+
+ std::string tmpString;
+ std::cin >> tmpString;
+
+ if (!std::cin) {
+ if (isFrench) {
+ std::cerr << "Error while reading the locale" << std::endl;
+
+ } else {
+ std::cerr << "Error while reading the locale" << std::endl;
+ }
+ return EXIT_FAILURE;
+ }
+
+ std::locale cinLocale (tmpString.c_str());
+
+ // and assign it to the standard input channel
+ std::cin.imbue (cinLocale);
+
+ // Read and output floating-point values in a loop
+ double value;
+ while (std::cin >> value) {
+ std::cout << value << std::endl;
+ }
+
+ return 0;
+}
Added: trunk/opentrep/test/i18n/stdlocru.cpp
===================================================================
--- trunk/opentrep/test/i18n/stdlocru.cpp (rev 0)
+++ trunk/opentrep/test/i18n/stdlocru.cpp 2009-07-19 21:48:27 UTC (rev 143)
@@ -0,0 +1,32 @@
+// C
+#include <iostream>
+#include <locale>
+#include <string>
+#include <cstdlib>
+
+
+// ////////////// M A I N //////////////
+int main (int argc, char* argv[]) {
+
+ // Create the default locale from the user's environment
+ std::locale langLocale ("ru_RU.UTF-8");
+
+ // and assign it to the standard output channel
+ //std::cout.imbue (langLocale);
+
+ // and assign it to the standard input channel
+ //std::cin.imbue (langLocale);
+
+ // Display with no processing
+ std::cout << "de: München" << std::endl;
+ std::cout << "ru: Мюнхен" << std::endl;
+
+ const std::string mucDE ("München");
+ const std::string mucRU ("Мюнхен");
+
+ // Display the strings
+ std::cout << "de: " << mucDE << std::endl;
+ std::cout << "ru: " << mucRU << std::endl;
+
+ return 0;
+}
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <den...@us...> - 2009-07-19 13:12:22
|
Revision: 142
http://opentrep.svn.sourceforge.net/opentrep/?rev=142&view=rev
Author: denis_arnaud
Date: 2009-07-19 13:12:19 +0000 (Sun, 19 Jul 2009)
Log Message:
-----------
[DB] Added a few files for transliteration from Russian and Ukrainian to Latin characters.
Modified Paths:
--------------
trunk/opentrep/db/data/ref_place_names.csv
Added Paths:
-----------
trunk/opentrep/db/maintenance/i18n/
trunk/opentrep/db/maintenance/i18n/ref_ru_orig.txt
trunk/opentrep/db/maintenance/i18n/transliterate.sh
trunk/opentrep/db/maintenance/i18n/transliteration_ru_en.sed
trunk/opentrep/db/maintenance/i18n/transliteration_ru_en.txt
trunk/opentrep/db/maintenance/i18n/transliteration_ru_ua_en.txt
trunk/opentrep/db/maintenance/i18n/transliteration_ua_en.txt
Modified: trunk/opentrep/db/data/ref_place_names.csv
===================================================================
--- trunk/opentrep/db/data/ref_place_names.csv 2009-07-19 10:47:08 UTC (rev 141)
+++ trunk/opentrep/db/data/ref_place_names.csv 2009-07-19 13:12:19 UTC (rev 142)
@@ -1,4 +1,7 @@
concat('en,', code, ',', ticketing_name, ',', teleticketing_name, ',', extended_name)
+de,muc,münchen,muenchen,munich/de:franz j strauss
+fr,muc,munique,munique,munich/de:franz j strauss
+ru,muc,мюнхен,munich,munich/de:franz j strauss
en,cyz,cauayan,cauayan,cauayan/ph
en,cza,chichen itza,chichen itza,chichen itza/mx
en,czb,cruz alta,cruz alta,cruz alta/rs/br:carlos ruhl
Added: trunk/opentrep/db/maintenance/i18n/ref_ru_orig.txt
===================================================================
(Binary files differ)
Property changes on: trunk/opentrep/db/maintenance/i18n/ref_ru_orig.txt
___________________________________________________________________
Added: svn:mime-type
+ application/octet-stream
Added: trunk/opentrep/db/maintenance/i18n/transliterate.sh
===================================================================
--- trunk/opentrep/db/maintenance/i18n/transliterate.sh (rev 0)
+++ trunk/opentrep/db/maintenance/i18n/transliterate.sh 2009-07-19 13:12:19 UTC (rev 142)
@@ -0,0 +1,3 @@
+#!/bin/sh
+
+sed -f transliteration_ru_en.sed ref_ru_orig.txt
Property changes on: trunk/opentrep/db/maintenance/i18n/transliterate.sh
___________________________________________________________________
Added: svn:executable
+ *
Added: trunk/opentrep/db/maintenance/i18n/transliteration_ru_en.sed
===================================================================
--- trunk/opentrep/db/maintenance/i18n/transliteration_ru_en.sed (rev 0)
+++ trunk/opentrep/db/maintenance/i18n/transliteration_ru_en.sed 2009-07-19 13:12:19 UTC (rev 142)
@@ -0,0 +1,75 @@
+s/А/A/g
+s/а/a/g
+s/Б/B/g
+s/б/b/g
+s/В/V/g
+s/в/v/g
+s/Г/G/g
+s/г/g/g
+s/Ґ/G/g
+s/ґ/g/g
+s/Д/D/g
+s/д/d/g
+s/Е/E/g
+s/е/e/g
+s/Є/YE/g
+s/є/ye/g
+s/Ё/Ë/g
+s/ё/ë/g
+s/Ж/ZH/g
+s/ж/zh/g
+s/З/Z/g
+s/з/z/g
+s/И/I/g
+s/и/i/g
+s/І/I/g
+s/і/i/g
+s/Ї/YI/g
+s/ї/yi/g
+s/Й/Y/g
+s/й/y/g
+s/К/K/g
+s/к/k/g
+s/Л/L/g
+s/л/l/g
+s/М/M/g
+s/м/m/g
+s/Н/N/g
+s/н/n/g
+s/О/O/g
+s/о/o/g
+s/П/P/g
+s/п/p/g
+s/Р/R/g
+s/р/r/g
+s/С/S/g
+s/с/s/g
+s/Т/T/g
+s/т/t/g
+s/У/U/g
+s/у/u/g
+s/Ф/F/g
+s/ф/f/g
+s/Х/KH/g
+s/х/kh/g
+s/Ц/TS/g
+s/ц/ts/g
+s/Ч/CH/g
+s/ч/ch/g
+s/Ш/SH/g
+s/ш/sh/g
+s/Щ/SHCH/g
+s/щ/shch/g
+s/Ъ/"/g
+s/ъ/"/g
+s/Ы/Y/g
+s/ы/y/g
+s/Ь/'/g
+s/ь/'/g
+s/Э/E/g
+s/э/e/g
+s/Ю/YU/g
+s/ю/yu/g
+s/Я/YA/g
+s/я/ya/g
+s/’/"/g
Added: trunk/opentrep/db/maintenance/i18n/transliteration_ru_en.txt
===================================================================
--- trunk/opentrep/db/maintenance/i18n/transliteration_ru_en.txt (rev 0)
+++ trunk/opentrep/db/maintenance/i18n/transliteration_ru_en.txt 2009-07-19 13:12:19 UTC (rev 142)
@@ -0,0 +1,94 @@
+ISO 9 ISO/R 9 GOST ALA-LC BGN/PCGN
+1995(1.0) 1968 (2.0) 1983 (3.0) 1997(4.0) 1947(5.0)
+А а а a a a a
+Б б b b b b b
+В в v v v v v
+Г г g g g g g
+Д д d d d d d
+Е е e e e e e, ye (5.1)
+Ё ё ë ë ë ë ë, yë (5.1)
+Ж ж ž ž ž zh zh
+З з z z z z z
+И и i i i i i
+Й й j j j ĭ y
+К к k k k k k
+Л л l l l l l
+М м m m m m m
+Н н n n n n n
+О о o o o o o
+П п p p p p p
+Р р r r r r r
+С с s s s s s
+Т т t t t t t
+У у u u u u u
+Ф ф f f f f f
+Х х h ch h kh kh
+Ц ц c c c ts ts
+Ч ч č č č ch ch
+Ш ш š š š sh sh
+Щ щ s šč šč shch shch
+Ъ ъ " " " " "
+Ы ы y y y y y
+Ь ь ´ ´ ' ´ ’
+Э э è ė è ė e
+Ю ю û ju ju iu yu
+Я я â ja ja ia ya
+
+Notes
+1.0 International Standards Organisation. (http://www.iso.ch).
+
+2.0 International Standards Organisation. This standard was withdrawn
+ in 1986 and replaced by ISO 9:1986, which in turn was
+ replaced by ISO 9:1995. Included here as this standard is
+ still used and found in various publications.
+
+3.0 Also known as GOST 16876-71, ГОСТ (государственный
+ стандарт). Originally a Soviet, now Russian, state
+ standard. This standard corresponds to UN 1987.
+
+4.0 American Library Association/Library of Congress.
+
+5.0 United States Board on Geographic Names and the Permanent
+ Committee on Geographical Names for British Official Use.
+
+5.1 Transliterated ye/yë initially, after a vowel, й, ъ and ь.
+
+Sources
+• ALA-LC Romanization Tables: Transliteration Schemes for Non-Roman
+ Scripts. Randal K. Barry (ed.). Library of Congress,
+ 1997. (http://lcweb.loc.gov/catdir/cpso/roman.html).
+
+• Allworth, Edward: Nationalities of the Soviet East. Publications and
+ Writing Systems. New York, 1971.
+
+• Cubberley, Paul: "The Slavic Alphabets", in Peter T. Daniels &
+ William Bright, eds. The World’s Writing Systems. New
+ York/Oxford, 1996.
+
+• IDS (Informationsverbund Deutschschweiz):
+ Katalogisierungsregeln IDS (KIDS), Anhänge,
+ "IDS G: Transliterationstabellen".
+ Universität Zürich, 15.10.2001.
+ (http://www.zb3.unizh.ch/ids/KIDS/ KapG0.pdf).
+
+• ISO 9:1995. Information and documentation – Transliteration of
+ Cyrillic characters into Latin charac- ters – Slavic and
+ non-Slavic languages. International Organization for
+ Standardization, 1995.
+
+• ISO 9. Wikipedia, 2006-04-13. (http://en.wikipedia.org/wiki/ISO_9)
+
+• Latiniseeritud nimede hääldusjuhiseid/Guide to the Pronounciation of
+ Romanized Names. KNAB: Ko- hanimeandmebaas. Eesti Keeli
+ Instituut, 1998. Thomas T. Pedersen –
+ transliteration.eki.ee Rev. 2, 2006-04-25 Transliteration
+ of Russian 3/3
+
+• Russische Rechtschreibreform von 1918. Wikipedia,
+ 2006-03-27. (http://de.wikipedia.org/wiki/
+ Russische_Rechtschreibreform_von_1918)
+
+• United Nations Romanization Systems for Geographical Names. Report
+ on Their Current Status. Compiled by the UNGEGN Working
+ Group on Romanization Systems. Version 2.2. January
+ 2003. (http://www.eki.ee/wgrs).
Added: trunk/opentrep/db/maintenance/i18n/transliteration_ru_ua_en.txt
===================================================================
--- trunk/opentrep/db/maintenance/i18n/transliteration_ru_ua_en.txt (rev 0)
+++ trunk/opentrep/db/maintenance/i18n/transliteration_ru_ua_en.txt 2009-07-19 13:12:19 UTC (rev 142)
@@ -0,0 +1,38 @@
+А а a
+Б б b
+В в v
+Г г g
+Ґ ґ g
+Д д d
+Е е e
+Є є ye
+Ё ё ë
+Ж ж zh
+З з z
+И и i
+І і i
+Ї ї yi
+Й й y
+К к k
+Л л l
+М м m
+Н н n
+О о o
+П п p
+Р р r
+С с s
+Т т t
+У у u
+Ф ф f
+Х х kh
+Ц ц ts
+Ч ч ch
+Ш ш sh
+Щ щ shch
+Ъ ъ "
+Ы ы y
+Ь ь '
+Э э e
+Ю ю yu
+Я я ya
+ ’ "
Added: trunk/opentrep/db/maintenance/i18n/transliteration_ua_en.txt
===================================================================
--- trunk/opentrep/db/maintenance/i18n/transliteration_ua_en.txt (rev 0)
+++ trunk/opentrep/db/maintenance/i18n/transliteration_ua_en.txt 2009-07-19 13:12:19 UTC (rev 142)
@@ -0,0 +1,83 @@
+ ISO 9 National ALA/LC BGN/PCGN WWS
+ 1995 1996(2.0) 1997 1965 1996(5.0)
+А а a a a a a
+Б б b b b b b
+В в v v v v v
+Г г g h, gh(2.1) h h h
+Ґ ґ g̀ g g g g (5.1)
+Д д d d d d d
+Е е e e e e e
+Є є ê ie, ye(2.2) ie ye je (5.2)
+Ж ж ž zh zh zh ž
+З з z z z z z
+И и i y y y y
+І і ì i i i i
+Ї ї ï i, yi(2.2) ï yi ji
+Й й j i, y(2.2) ĭ y j (5.3)
+К к k k k k k
+Л л l l l l l
+М м m m m m m
+Н н n n n n n
+О о o o o o o
+П п p p p p p
+Р р r r r r r
+С с s s s s s
+Т т t t t t t
+У у u u u u u
+Ф ф f f f f f
+Х х h kh kh kh x
+Ц ц c ts ts ts c
+Ч ч č ch ch ch č
+Ш ш š sh sh sh š
+Щ щ s sch shch shch šč
+Ю ю û iu, yu(2.2) iu yu ju (5.2)
+Я я â ia, ya(2.2) ia ya ja (5.2)
+Ь ь ´ ’ ´ ’ ´ (5.3)
+ ’ ’ " - " - (5.3)
+
+Notes
+2.0. The current national system of romanization.
+2.1. gh is used in the romanization of зг zgh.
+2.2. In initial position.
+5.0. The following digraphs are not considered part of the
+ alphabetical ordering:
+ дж [dЗ]
+ дз [dz]
+
+5.1. During the Soviet period, the separate letter ґ g was not used,
+ and г h served for both [h] and [g], the latter occurring only in
+ onomatopoeia and loanwords. With national independence, ґ g has
+ been reintroduced.
+5.2. After Cj.
+5.3. Palatalization of consonants and [j] are indicated as follows:
+ [a] [e] [i] [o] [u]
+ –
+ Plain C Cа Cе Cи Cо Cу
+ Palatalized Cь Cя Cє Cі Cё Cю
+ [j] й я є ї йо ю
+ [j] between a vowel and a consonant is indicated by an apostrophe
+ followed by a yotated vowel, e.g. С’я [sja].
+
+
+Sources
+• ALA-LC Romanization Tables: Transliteration Schemes for Non-Roman
+ Scripts. Randal K. Berry (ed.). Library of Congress, 1997.
+ (http://lcweb.loc.gov/catdir/cpso/roman.html).
+
+• ISO 9:1995. Information and documentation – Transliteration of
+ Cyrillic characters into Latin characters – Slavic and non- Slavic
+ languages. International Organization for Standardization, 1995.
+
+• ISO 9. Wikipedia, 2006-04-13. (http://en.wikipedia.org/wiki/ISO_9)
+
+• Latiniseeritud nimede hääldusjuhiseid/Guide to the Pronounciation
+ of Romanized Names. KNAB: Kohanimeandmebaas. Eesti Keeli
+ Instituut, 1998.
+
+• United Nations Romanization Systems for Geographical Names. Report
+ on their Current Status. Compiled by the UNGEGN Working Group on
+ Romanization Systems. Version 2.1. June 2002.
+ (http://www.eki.ee/wgrs).
+
+• The World’s Writing Systems. Peter T. Daniels & William Bright
+ (eds.). New York, 1996.
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <den...@us...> - 2009-07-19 10:47:09
|
Revision: 141
http://opentrep.svn.sourceforge.net/opentrep/?rev=141&view=rev
Author: denis_arnaud
Date: 2009-07-19 10:47:08 +0000 (Sun, 19 Jul 2009)
Log Message:
-----------
[Dev] Improved the notification scheme when there are several matches with the same percentage.
Modified Paths:
--------------
trunk/opentrep/opentrep/bom/Document.cpp
trunk/opentrep/opentrep/bom/Document.hpp
trunk/opentrep/opentrep/bom/ResultHolder.cpp
trunk/opentrep/opentrep/bom/StringMatcher.cpp
Modified: trunk/opentrep/opentrep/bom/Document.cpp
===================================================================
--- trunk/opentrep/opentrep/bom/Document.cpp 2009-07-18 23:30:37 UTC (rev 140)
+++ trunk/opentrep/opentrep/bom/Document.cpp 2009-07-19 10:47:08 UTC (rev 141)
@@ -5,6 +5,7 @@
#include <cassert>
// OpenTREP
#include <opentrep/bom/Document.hpp>
+#include <opentrep/service/Logger.hpp>
namespace OPENTREP {
@@ -22,20 +23,20 @@
// //////////////////////////////////////////////////////////////////////
std::string Document::describeKey() const {
std::ostringstream oStr;
- oStr << describeShortKey();
+ oStr << "`" << describeShortKey() << "´";
if (_correctedQueryString.empty() == false) {
- oStr << " (corrected into " << _correctedQueryString << ")";
+ oStr << " (corrected into `" << _correctedQueryString << "´)";
}
return oStr.str();
}
// //////////////////////////////////////////////////////////////////////
- std::string Document::toString() const {
+ std::string Document::toShortString() const {
std::ostringstream oStr;
- oStr << describeKey() << std::endl;
+ oStr << describeShortKey();
const Xapian::docid& lDocID = _document.get_docid();
- oStr << "Document ID " << lDocID << "\t" << _percentage
+ oStr << " => Document ID " << lDocID << " matching at " << _percentage
<< "% [" << _document.get_data() << "]";
if (_documentList.empty() == false) {
@@ -62,6 +63,38 @@
}
// //////////////////////////////////////////////////////////////////////
+ std::string Document::toString() const {
+ std::ostringstream oStr;
+ oStr << describeKey();
+
+ const Xapian::docid& lDocID = _document.get_docid();
+ oStr << " => Document ID " << lDocID << " matching at " << _percentage
+ << "% [" << _document.get_data() << "]";
+
+ if (_documentList.empty() == false) {
+ oStr << " along with " << _documentList.size()
+ << " other matching document(s) { ";
+
+ unsigned short idx = 0;
+ for (XapianDocumentList_T::const_iterator itDoc = _documentList.begin();
+ itDoc != _documentList.end(); ++itDoc, ++idx) {
+ const Xapian::Document& lXapianDoc = *itDoc;
+ const Xapian::docid& lDocID = lXapianDoc.get_docid();
+ if (idx != 0) {
+ oStr << ", ";
+ }
+ oStr << "Doc ID " << lDocID << " [" << lXapianDoc.get_data() << "]";
+ }
+ oStr << " }." << std::endl;
+
+ } else {
+ oStr << std::endl;
+ }
+
+ return oStr.str();
+ }
+
+ // //////////////////////////////////////////////////////////////////////
void Document::toStream (std::ostream& ioOut) const {
ioOut << toString();
}
@@ -70,4 +103,22 @@
void Document::fromStream (std::istream& ioIn) {
}
+ // //////////////////////////////////////////////////////////////////////
+ NbOfMatches_T Document::notifyIfExtraMatch () const {
+ NbOfMatches_T oNbOfMatches = _documentList.size();
+
+ // DEBUG
+ if (oNbOfMatches != 0) {
+ OPENTREP_LOG_NOTIFICATION ("NOTE: the following document gets several "
+ << "matches with the same matching percentage."
+ << " You may want to alter the SQL database "
+ << "and re-index the Xapian database, so as "
+ << "to allow a more specific match:"
+ << std::endl << toString());
+ }
+
+ // Return the total number of matches (main plus extras)
+ return (1 + oNbOfMatches);
+ }
+
}
Modified: trunk/opentrep/opentrep/bom/Document.hpp
===================================================================
--- trunk/opentrep/opentrep/bom/Document.hpp 2009-07-18 23:30:37 UTC (rev 140)
+++ trunk/opentrep/opentrep/bom/Document.hpp 2009-07-19 10:47:08 UTC (rev 141)
@@ -80,20 +80,35 @@
_documentList.push_back (iMatchingDocument);
}
+
+ public:
+ // /////////// Business methods /////////
+ /** Retrieve the number of extra matches for the given query string,
+ and print a notification in the logs, so that the SQL and Xapian
+ databases can be altered in order to get more specific matches.
+ @param const TravelQuery_T& Query string having yielded that matching
+ document.
+ @return NbOfMatches_T The number of matches (i.e., one chosen match
+ added to the extra matches). */
+ NbOfMatches_T notifyIfExtraMatch() const;
+
public:
// /////////// Display support methods /////////
- /** Dump a Business Object into an output stream.
+ /** Dump the structure into an output stream.
@param ostream& the output stream. */
void toStream (std::ostream& ioOut) const;
- /** Read a Business Object from an input stream.
+ /** Read a structure from an input stream.
@param istream& the input stream. */
void fromStream (std::istream& ioIn);
- /** Get the serialised version of the Business Object. */
+ /** Get the serialised version of the structure. */
std::string toString() const;
+ /** Get a shorter serialised version of the structure. */
+ std::string toShortString() const;
+
/** Get a string describing the whole key (differentiating two objects
at any level). */
std::string describeKey() const;
Modified: trunk/opentrep/opentrep/bom/ResultHolder.cpp
===================================================================
--- trunk/opentrep/opentrep/bom/ResultHolder.cpp 2009-07-18 23:30:37 UTC (rev 140)
+++ trunk/opentrep/opentrep/bom/ResultHolder.cpp 2009-07-19 10:47:08 UTC (rev 141)
@@ -156,9 +156,8 @@
ioDocumentList.push_back (lMatchingDocument);
// DEBUG
- const XapianDocumentList_T& lXapianDocumentList =
- lMatchingDocument.getExtraDocumentList();
- const NbOfMatches_T lNbOfMatches = 1 + lXapianDocumentList.size();
+ const NbOfMatches_T lNbOfMatches =
+ lMatchingDocument.notifyIfExtraMatch();
OPENTREP_LOG_DEBUG ("==> " << lNbOfMatches
<< " matches for the query string: `"
<< lMatchedString << "´ (from `"
Modified: trunk/opentrep/opentrep/bom/StringMatcher.cpp
===================================================================
--- trunk/opentrep/opentrep/bom/StringMatcher.cpp 2009-07-18 23:30:37 UTC (rev 140)
+++ trunk/opentrep/opentrep/bom/StringMatcher.cpp 2009-07-19 10:47:08 UTC (rev 141)
@@ -295,21 +295,41 @@
*/
Xapian::MSetIterator itDoc = iMatchingSet.begin();
assert (itDoc != iMatchingSet.end());
-
+
+ // Store the percentage
const Xapian::percent& lBestPercentage = itDoc.get_percent();
ioMatchingDocument.setXapianPercentage (lBestPercentage);
- ioMatchingDocument.setXapianDocument (itDoc.get_document());
+ // Store the (Xapian) document itself
+ const Xapian::Document& lBestDocument = itDoc.get_document();
+ ioMatchingDocument.setXapianDocument (lBestDocument);
+
+ // Go on in the list of matches, if any
+ ++itDoc;
+
+ /*
+ if (itDoc != iMatchingSet.end()) {
+ // DEBUG
+ OPENTREP_LOG_DEBUG ("There are extra matches for Document ID "
+ << lBestDocument.get_docid()
+ << ", itself matching at " << lBestPercentage
+ << "%.");
+ }
+ */
+
/** Add all the Xapian documents having reached the same matching
percentage. */
NbOfMatches_T idx = 1;
- for (++itDoc ; itDoc != iMatchingSet.end(); ++itDoc, ++idx) {
+ for ( ; itDoc != iMatchingSet.end(); ++itDoc, ++idx) {
const Xapian::percent& lPercentage = itDoc.get_percent();
+ const Xapian::Document& lDocument = itDoc.get_document();
// DEBUG
- OPENTREP_LOG_DEBUG ("[" << idx << "] extra. Best " << lBestPercentage
- << "%, other " << lPercentage << "% ("
- << itDoc.get_document().get_docid() << ")");
+ /*
+ OPENTREP_LOG_DEBUG ("Extra #" << idx << ": Doc ID "
+ << lDocument.get_docid() << " matching at "
+ << lPercentage << "%.");
+ */
if (lPercentage == lBestPercentage) {
ioMatchingDocument.addExtraDocument (itDoc.get_document());
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <den...@us...> - 2009-07-18 23:30:44
|
Revision: 140
http://opentrep.svn.sourceforge.net/opentrep/?rev=140&view=rev
Author: denis_arnaud
Date: 2009-07-18 23:30:37 +0000 (Sat, 18 Jul 2009)
Log Message:
-----------
[Dev] The query parser service now returns the list of unmatched words (along with the list of Location structures).
Modified Paths:
--------------
trunk/opentrep/db/data/ref_place_names.csv
trunk/opentrep/opentrep/OPENTREP_Service.hpp
trunk/opentrep/opentrep/OPENTREP_Types.hpp
trunk/opentrep/opentrep/batches/searcher.cpp
trunk/opentrep/opentrep/bom/Document.cpp
trunk/opentrep/opentrep/bom/Document.hpp
trunk/opentrep/opentrep/bom/ResultHolder.cpp
trunk/opentrep/opentrep/bom/ResultHolder.hpp
trunk/opentrep/opentrep/bom/StringMatcher.cpp
trunk/opentrep/opentrep/bom/StringMatcher.hpp
trunk/opentrep/opentrep/bom/WordHolder.hpp
trunk/opentrep/opentrep/bom/sources.mk
trunk/opentrep/opentrep/command/RequestInterpreter.cpp
trunk/opentrep/opentrep/command/RequestInterpreter.hpp
trunk/opentrep/opentrep/service/OPENTREP_Service.cpp
Removed Paths:
-------------
trunk/opentrep/opentrep/bom/WordList.hpp
Modified: trunk/opentrep/db/data/ref_place_names.csv
===================================================================
--- trunk/opentrep/db/data/ref_place_names.csv 2009-07-18 20:30:08 UTC (rev 139)
+++ trunk/opentrep/db/data/ref_place_names.csv 2009-07-18 23:30:37 UTC (rev 140)
@@ -3550,7 +3550,7 @@
en,cdd,cauquira,cauquira,cauquira/hn:cauquira airport
en,cde,caledonia,caledonia,caledonia/pa
en,cdf,cortina d'ampezzo,cortina d'ampez,cortina d'ampezzo/it:fiames
-en,cdg,paris cdg,paris cdg,paris/fr:charles de gaulle
+en,cdg,paris cdg,paris cdg,paris/fr:charles de gaulle,cdg,cdg
en,cdh,camden,camden,camden/ar/us:harrell fld
en,cdi,cachoeiro,cachoeiro,cachoeiro de i/es/br:cachoeiro
en,cdj,conceicao do arag,conceicao do ar,conceicao do arag/pa/br
Modified: trunk/opentrep/opentrep/OPENTREP_Service.hpp
===================================================================
--- trunk/opentrep/opentrep/OPENTREP_Service.hpp 2009-07-18 20:30:08 UTC (rev 139)
+++ trunk/opentrep/opentrep/OPENTREP_Service.hpp 2009-07-18 23:30:37 UTC (rev 140)
@@ -30,9 +30,10 @@
"sna francicso rio de janero lso angles reykyavki nce iev mow").
@param LocationList_T& List of (geographical) locations, if any,
matching the given query string.
+ @param WordList_T& List of non-matched words of the query string.
@return NbOfMatches_T Number of matches. */
NbOfMatches_T interpretTravelRequest (const std::string& iTravelQuery,
- LocationList_T&);
+ LocationList_T&, WordList_T&);
// ////////// Constructors and destructors //////////
Modified: trunk/opentrep/opentrep/OPENTREP_Types.hpp
===================================================================
--- trunk/opentrep/opentrep/OPENTREP_Types.hpp 2009-07-18 20:30:08 UTC (rev 139)
+++ trunk/opentrep/opentrep/OPENTREP_Types.hpp 2009-07-18 23:30:37 UTC (rev 140)
@@ -7,6 +7,7 @@
// STL
#include <exception>
#include <string>
+#include <list>
namespace OPENTREP {
@@ -62,6 +63,12 @@
/** Number of matching documents. */
typedef unsigned short NbOfMatches_T;
+ /** Word, which is the atomic element of a query string. */
+ typedef std::string Word_T;
+
+ /** List of words. */
+ typedef std::list<Word_T> WordList_T;
+
}
#endif // __OPENTREP_OPENTREP_TYPES_HPP
Modified: trunk/opentrep/opentrep/batches/searcher.cpp
===================================================================
--- trunk/opentrep/opentrep/batches/searcher.cpp 2009-07-18 20:30:08 UTC (rev 139)
+++ trunk/opentrep/opentrep/batches/searcher.cpp 2009-07-18 23:30:37 UTC (rev 140)
@@ -228,13 +228,16 @@
lXapianDatabaseName);
// Query the Xapian database (index)
+ OPENTREP::WordList_T lNonMatchedWordList;
OPENTREP::LocationList_T lLocationList;
const OPENTREP::NbOfMatches_T nbOfMatches =
- opentrepService.interpretTravelRequest (lTravelQuery, lLocationList);
+ opentrepService.interpretTravelRequest (lTravelQuery, lLocationList,
+ lNonMatchedWordList);
- std::cout << nbOfMatches << " (geographical) location(s) have been found "
- << "matching your query (`" << lTravelQuery << "´)."
- << std::endl;
+ std::cout << nbOfMatches << " (geographical) location(s) have been found "
+ << "matching your query (`" << lTravelQuery << "´). "
+ << lNonMatchedWordList.size() << " words were left unmatched."
+ << std::endl;
if (nbOfMatches != 0) {
OPENTREP::NbOfMatches_T idx = 1;
@@ -245,6 +248,18 @@
std::cout << " [" << idx << "]: " << lLocation << std::endl;
}
}
+
+ if (lNonMatchedWordList.empty() == false) {
+ std::cout << "List of unmatched words:" << std::endl;
+
+ OPENTREP::NbOfMatches_T idx = 1;
+ for (OPENTREP::WordList_T::const_iterator itWord =
+ lNonMatchedWordList.begin();
+ itWord != lNonMatchedWordList.end(); ++itWord, ++idx) {
+ const OPENTREP::Word_T& lWord = *itWord;
+ std::cout << " [" << idx << "]: " << lWord << std::endl;
+ }
+ }
// Close the Log outputFile
logOutputFile.close();
Modified: trunk/opentrep/opentrep/bom/Document.cpp
===================================================================
--- trunk/opentrep/opentrep/bom/Document.cpp 2009-07-18 20:30:08 UTC (rev 139)
+++ trunk/opentrep/opentrep/bom/Document.cpp 2009-07-18 23:30:37 UTC (rev 140)
@@ -9,17 +9,6 @@
namespace OPENTREP {
// //////////////////////////////////////////////////////////////////////
- Document::Document () {
- }
-
- // //////////////////////////////////////////////////////////////////////
- Document::Document (const Document& iDocument)
- : _queryString (iDocument._queryString),
- _document (iDocument._document),
- _documentList (iDocument._documentList) {
- }
-
- // //////////////////////////////////////////////////////////////////////
Document::~Document () {
}
@@ -32,17 +21,42 @@
// //////////////////////////////////////////////////////////////////////
std::string Document::describeKey() const {
- return describeShortKey();
+ std::ostringstream oStr;
+ oStr << describeShortKey();
+ if (_correctedQueryString.empty() == false) {
+ oStr << " (corrected into " << _correctedQueryString << ")";
+ }
+ return oStr.str();
}
// //////////////////////////////////////////////////////////////////////
std::string Document::toString() const {
std::ostringstream oStr;
- oStr << describeShortKey() << std::endl;
+ oStr << describeKey() << std::endl;
const Xapian::docid& lDocID = _document.get_docid();
oStr << "Document ID " << lDocID << "\t" << _percentage
<< "% [" << _document.get_data() << "]";
+
+ if (_documentList.empty() == false) {
+ oStr << " along with " << _documentList.size()
+ << " other matching document(s) (";
+
+ unsigned short idx = 0;
+ for (XapianDocumentList_T::const_iterator itDoc = _documentList.begin();
+ itDoc != _documentList.end(); ++itDoc, ++idx) {
+ const Xapian::Document& lXapianDoc = *itDoc;
+ const Xapian::docid& lDocID = lXapianDoc.get_docid();
+ if (idx != 0) {
+ oStr << ", ";
+ }
+ oStr << lDocID;
+ }
+ oStr << ")." << std::endl;
+
+ } else {
+ oStr << std::endl;
+ }
return oStr.str();
}
Modified: trunk/opentrep/opentrep/bom/Document.hpp
===================================================================
--- trunk/opentrep/opentrep/bom/Document.hpp 2009-07-18 20:30:08 UTC (rev 139)
+++ trunk/opentrep/opentrep/bom/Document.hpp 2009-07-18 23:30:37 UTC (rev 140)
@@ -32,6 +32,12 @@
return _queryString;
}
+ /** Get the corrected query string.
+ <br>When empty, it means that no correction was necessary. */
+ const TravelQuery_T& getCorrectedTravelQuery() {
+ return _correctedQueryString;
+ }
+
/** Get the matching Xapian document. */
const Xapian::Document& getXapianDocument() const {
return _document;
@@ -49,10 +55,16 @@
// ////////////////// Setters ////////////////
+ /** Set the query string. */
void setQueryString (const TravelQuery_T& iQueryString) {
_queryString = iQueryString;
}
+ /** Set the corrected query string. */
+ void setCorrectedQueryString (const TravelQuery_T& iCorrectedQueryString) {
+ _correctedQueryString = iCorrectedQueryString;
+ }
+
/** Set the matching Xapian document. */
void setXapianDocument (const Xapian::Document& iMatchingDocument) {
_document = iMatchingDocument;
@@ -94,9 +106,9 @@
public:
// //////////////// Constructors and Destructors /////////////
/** Default constructor. */
- Document ();
+ // Document ();
/** Default copy constructor. */
- Document (const Document&);
+ // Document (const Document&);
/** Default destructor. */
~Document ();
@@ -106,6 +118,9 @@
/** Query string with which a Xapian full text search is done. */
TravelQuery_T _queryString;
+ /** Query string with which a Xapian full text search is done. */
+ TravelQuery_T _correctedQueryString;
+
/** Matching percentage, as returned by the Xapian full text search.
<br>Generally, that percentage is equal to, or close to, 100%. */
Xapian::percent _percentage;
Modified: trunk/opentrep/opentrep/bom/ResultHolder.cpp
===================================================================
--- trunk/opentrep/opentrep/bom/ResultHolder.cpp 2009-07-18 20:30:08 UTC (rev 139)
+++ trunk/opentrep/opentrep/bom/ResultHolder.cpp 2009-07-18 23:30:37 UTC (rev 140)
@@ -68,35 +68,41 @@
}
// //////////////////////////////////////////////////////////////////////
- bool ResultHolder::searchString (TravelQuery_T& ioPartialQueryString,
- Document& ioMatchingDocument) {
- bool oFoundDocument = false;
+ std::string ResultHolder::searchString (TravelQuery_T& ioPartialQueryString,
+ Document& ioMatchingDocument) {
+ std::string oMatchedString;
// Catch any Xapian::Error exceptions thrown
try {
bool shouldStop = false;
while (shouldStop == false) {
- // DEBUG
- OPENTREP_LOG_DEBUG ("Current query string: `"
- << ioPartialQueryString << "'");
-
// Retrieve the list of documents matching the query string
Xapian::MSet lMatchingSet;
- StringMatcher::searchString (lMatchingSet, ioPartialQueryString,
- _database);
+ oMatchedString = StringMatcher::searchString (lMatchingSet,
+ ioPartialQueryString,
+ _database);
- // Create the corresponding list of documents
- oFoundDocument = StringMatcher::
- extractBestMatchingDocumentFromMSet(lMatchingSet, ioMatchingDocument);
-
- // Stop if a result is found.
- if (oFoundDocument == true) {
+ // DEBUG
+ OPENTREP_LOG_DEBUG ("Current initial query string: `"
+ << ioPartialQueryString << "´ --- Kept query: `"
+ << oMatchedString << "´ for "
+ << lMatchingSet.size() << " matches.");
+
+ if (lMatchingSet.empty() == false) {
+ // Create the corresponding list of documents
+ StringMatcher::
+ extractBestMatchingDocumentFromMSet (lMatchingSet,
+ ioMatchingDocument);
+
+ // Since a result has been found, the search can be stopped
+ // for that part of the query.
shouldStop = true;
break;
}
- // Remove the furthest right word from the query string
+ // Since the query, as is, yield no match, the furthest right
+ // word must be removed from the query string.
StringMatcher::removeFurthestRightWord (ioPartialQueryString);
// Stop when the resulting string gets empty.
@@ -109,11 +115,12 @@
OPENTREP_LOG_ERROR ("Exception: " << error.get_msg());
}
- return oFoundDocument;
+ return oMatchedString;
}
// //////////////////////////////////////////////////////////////////////
- void ResultHolder::searchString (DocumentList_T& ioDocumentList) {
+ void ResultHolder::searchString (DocumentList_T& ioDocumentList,
+ WordList_T& ioWordList) {
// Catch any Xapian::Error exceptions thrown
try {
@@ -140,31 +147,39 @@
against the Xapian database).
*/
Document lMatchingDocument;
- const bool hasFoundDocument = searchString (lQueryString,
- lMatchingDocument);
+ const std::string lMatchedString = searchString (lQueryString,
+ lMatchingDocument);
- if (hasFoundDocument == true) {
+ if (lMatchedString.empty() == false) {
lMatchingDocument.setQueryString (lQueryString);
+ lMatchingDocument.setCorrectedQueryString (lMatchedString);
ioDocumentList.push_back (lMatchingDocument);
// DEBUG
- OPENTREP_LOG_DEBUG ("==> Matching of the query string: `"
- << lQueryString << "'");
+ const XapianDocumentList_T& lXapianDocumentList =
+ lMatchingDocument.getExtraDocumentList();
+ const NbOfMatches_T lNbOfMatches = 1 + lXapianDocumentList.size();
+ OPENTREP_LOG_DEBUG ("==> " << lNbOfMatches
+ << " matches for the query string: `"
+ << lMatchedString << "´ (from `"
+ << lQueryString << "´)");
/**
Remove, from the lRemainingQueryString string, the part
- which has been already successfully parsed. <br>For
- instance, when 'sna francisco rio de janeiro' is the
- initial full clean query string, the searchString()
+ which has been already successfully parsed.
+ <br>For instance, when 'sna francisco rio de janeiro' is
+ the initial full clean query string, the searchString()
method first reduce the query string to 'sna francisco',
which successfully matches against SFO (San Francisco
- airport). <br>Then, the remaining part of the query
- string to be parsed is 'rio de janeiro'. So, the already
- parsed part, namely 'sna francisco', must be subtracted
- from the initial query string.
+ airport).
+ <br>Then, the remaining part of the query string to be
+ parsed is 'rio de janeiro'. So, the already parsed part,
+ namely 'sna francisco', must be subtracted from the
+ initial query string.
*/
StringMatcher::subtractParsedToRemaining (lQueryString,
lRemainingQueryString);
+
} else {
// DEBUG
OPENTREP_LOG_DEBUG ("==> No matching of the query string: `"
@@ -180,8 +195,12 @@
therefore be empty, and the loop will therefore be exited
in the next step below.
*/
- // Remove the furthest right word from the query string
- StringMatcher::removeFurthestLeftWord (lRemainingQueryString);
+ // Remove the furthest right word from the query string...
+ const Word_T& lNonMatchedWord =
+ StringMatcher::removeFurthestLeftWord (lRemainingQueryString);
+
+ // ... and add it to the list of unmatched words.
+ ioWordList.push_back (lNonMatchedWord);
}
// If there is nothing left to be parsed, we have then finished
Modified: trunk/opentrep/opentrep/bom/ResultHolder.hpp
===================================================================
--- trunk/opentrep/opentrep/bom/ResultHolder.hpp 2009-07-18 20:30:08 UTC (rev 139)
+++ trunk/opentrep/opentrep/bom/ResultHolder.hpp 2009-07-18 23:30:37 UTC (rev 140)
@@ -38,8 +38,10 @@
public:
// /////////// Business methods /////////
- /** Retrieve the list of documents matching the query string. */
- void searchString (DocumentList_T&);
+ /** Retrieve the list of documents matching the query string.
+ @param DocumentList_T& List of matched documents by the query string.
+ @param WordList_T& List of non-matched words of the query string. */
+ void searchString (DocumentList_T&, WordList_T&);
private:
@@ -47,7 +49,7 @@
@param TravelQuery_T& The partial query string.
@param MatchingDocument_T& The best matching Xapian document (if found).
@return bool Whether such a best matching document has been found. */
- bool searchString (TravelQuery_T& ioPartialQueryString, Document&);
+ std::string searchString (TravelQuery_T& ioPartialQueryString, Document&);
public:
Modified: trunk/opentrep/opentrep/bom/StringMatcher.cpp
===================================================================
--- trunk/opentrep/opentrep/bom/StringMatcher.cpp 2009-07-18 20:30:08 UTC (rev 139)
+++ trunk/opentrep/opentrep/bom/StringMatcher.cpp 2009-07-18 23:30:37 UTC (rev 140)
@@ -79,9 +79,10 @@
}
// ///////////////////////////////////////////////////////////////////
- void StringMatcher::searchString (Xapian::MSet& ioMatchingSet,
- const std::string& iSearchString,
- const Xapian::Database& ioDatabase) {
+ std::string StringMatcher::searchString (Xapian::MSet& ioMatchingSet,
+ const std::string& iSearchString,
+ const Xapian::Database& ioDatabase) {
+ std::string oMatchedString;
// Catch any Xapian::Error exceptions thrown
try {
@@ -108,10 +109,11 @@
WordHolder::tokeniseStringIntoWordList (iSearchString, lOriginalWordList);
/**
- We rebuild a clean query string from the word list. Indeed, the original
- string may have contained a few separators (e.g., '/', ';', etc.), which
- have been removed by the tokeniseStringIntoWordList() method. All those
- separators are thus replaced by spaces.
+ We rebuild a clean query string from the word list. Indeed,
+ the original string may have contained a few separators
+ (e.g., '/', ';', etc.), which have been removed by the
+ tokeniseStringIntoWordList() method. All those separators are
+ thus replaced by spaces.
For instance, the 'san francisco, ca, us' initial string would be
replaced by 'san francisco ca us'.
*/
@@ -133,14 +135,15 @@
const EditDistance_T lEditDistance =
calculateEditDistance (lOriginalQueryString);
const std::string lFullWordCorrectedString =
- ioDatabase.get_spelling_suggestion (lOriginalQueryString, lEditDistance);
+ ioDatabase.get_spelling_suggestion(lOriginalQueryString, lEditDistance);
// DEBUG
/*
- OPENTREP_LOG_DEBUG ("Query string `" << lOriginalQueryString
- << "' ==> corrected query string: `" << lCorrectedQueryString
- << "' and correction for the full query string: `"
- << lFullWordCorrectedString << "'");
+ OPENTREP_LOG_DEBUG ("Query string `" << lOriginalQueryString
+ << "' ==> corrected query string: `"
+ << lCorrectedQueryString
+ << "' and correction for the full query string: `"
+ << lFullWordCorrectedString << "'");
*/
// Build the query object
@@ -220,9 +223,7 @@
int nbMatches = ioMatchingSet.size();
// DEBUG
- /*
- OPENTREP_LOG_DEBUG (nbMatches << " results found");
- */
+ // OPENTREP_LOG_DEBUG (nbMatches << " results found");
/**
When no match is found, we search on the corrected phrase/string
@@ -236,9 +237,10 @@
nbMatches = ioMatchingSet.size();
// DEBUG
- /*
- OPENTREP_LOG_DEBUG(nbMatches << " results found on corrected string");
- */
+ //OPENTREP_LOG_DEBUG(nbMatches << " results found on corrected string");
+
+ } else {
+ oMatchedString = lOriginalQueryString;
}
/**
@@ -253,12 +255,16 @@
nbMatches = ioMatchingSet.size();
// DEBUG
- /*
- OPENTREP_LOG_DEBUG (nbMatches
- << " results found on corrected full string");
- */
+ // OPENTREP_LOG_DEBUG (nbMatches
+ // << " results found on corrected full string");
+
+ } else {
+ oMatchedString = lCorrectedQueryString;
}
+ if (nbMatches != 0 && oMatchedString.empty() == true) {
+ oMatchedString = lFullWordCorrectedString;
+ }
// DEBUG
/*
@@ -270,19 +276,16 @@
} catch (const Xapian::Error& error) {
OPENTREP_LOG_ERROR ("Exception: " << error.get_msg());
}
+
+ return oMatchedString;
}
// //////////////////////////////////////////////////////////////////////
- bool StringMatcher::
+ void StringMatcher::
extractBestMatchingDocumentFromMSet (const Xapian::MSet& iMatchingSet,
Document& ioMatchingDocument) {
- bool oFoundDocument = false;
+ assert (iMatchingSet.empty() == false);
- if (iMatchingSet.empty() == true) {
- return oFoundDocument;
- }
- oFoundDocument = true;
-
/**
Retrieve the best matching document. If there are several such
best matching documents (for instance, several at, say, 100%),
@@ -291,14 +294,22 @@
same: it appears random).
*/
Xapian::MSetIterator itDoc = iMatchingSet.begin();
+ assert (itDoc != iMatchingSet.end());
+
const Xapian::percent& lBestPercentage = itDoc.get_percent();
ioMatchingDocument.setXapianPercentage (lBestPercentage);
ioMatchingDocument.setXapianDocument (itDoc.get_document());
/** Add all the Xapian documents having reached the same matching
percentage. */
- for ( ; itDoc != iMatchingSet.end(); ++itDoc) {
+ NbOfMatches_T idx = 1;
+ for (++itDoc ; itDoc != iMatchingSet.end(); ++itDoc, ++idx) {
const Xapian::percent& lPercentage = itDoc.get_percent();
+
+ // DEBUG
+ OPENTREP_LOG_DEBUG ("[" << idx << "] extra. Best " << lBestPercentage
+ << "%, other " << lPercentage << "% ("
+ << itDoc.get_document().get_docid() << ")");
if (lPercentage == lBestPercentage) {
ioMatchingDocument.addExtraDocument (itDoc.get_document());
@@ -307,8 +318,6 @@
break;
}
}
-
- return oFoundDocument;
}
// //////////////////////////////////////////////////////////////////////
@@ -328,19 +337,24 @@
}
// //////////////////////////////////////////////////////////////////////
- void StringMatcher::removeFurthestLeftWord (std::string& ioQueryString) {
+ Word_T StringMatcher::removeFurthestLeftWord (std::string& ioQueryString) {
assert (ioQueryString.empty() == false);
WordList_T lWordList;
WordHolder::tokeniseStringIntoWordList (ioQueryString, lWordList);
assert (lWordList.empty() == false);
+ // Store (a copy of) the furthest left word
+ const Word_T lFurthestLeftWord = lWordList.front();
+
// Remove the furthest left word
lWordList.pop_front();
const std::string& lReducedString =
WordHolder::createStringFromWordList (lWordList);
ioQueryString = lReducedString;
+
+ return lFurthestLeftWord;
}
// //////////////////////////////////////////////////////////////////////
Modified: trunk/opentrep/opentrep/bom/StringMatcher.hpp
===================================================================
--- trunk/opentrep/opentrep/bom/StringMatcher.hpp 2009-07-18 20:30:08 UTC (rev 139)
+++ trunk/opentrep/opentrep/bom/StringMatcher.hpp 2009-07-18 23:30:37 UTC (rev 140)
@@ -7,8 +7,8 @@
// STL
#include <string>
// OpenTREP
+#include <opentrep/OPENTREP_Types.hpp>
#include <opentrep/bom/BomAbstract.hpp>
-#include <opentrep/bom/WordList.hpp>
#include <opentrep/bom/Document.hpp>
// Forward declarations
@@ -25,36 +25,36 @@
class StringMatcher : public BomAbstract {
public:
/** Search, within the Xapian database, for occurrences of the
- words of the search string. */
- static void searchString (Xapian::MSet&, const std::string& iSearchString,
- const Xapian::Database&);
+ words of the search string.
+ @param Xapian::MSet& The Xapian matching set. It can be empty.
+ @param const std::string& The query string.
+ @param const Xapian::Database& The Xapian index/database.
+ @return std::string The query string, potentially corrected,
+ which has yielded matches. */
+ static std::string searchString (Xapian::MSet&,
+ const std::string& iSearchString,
+ const Xapian::Database&);
- /**
- Extract the best matching Xapian document.
- <br>If there are several such best matching documents (for
- instance, several at, say, 100%), one is taken randomly. Well,
- as we take the first one of the STL multimap, it is not exactly
- randomly, but the result is the same: it appears to be random.
- @return bool Whether or not there was a matching document.
- */
- static bool
+ /** Extract the best matching Xapian document.
+ <br>If there are several such best matching documents (for
+ instance, several at, say, 100%), one is taken randomly. Well,
+ as we take the first one of the STL multimap, it is not exactly
+ randomly, but the result is the same: it appears to be random.
+ @param Xapian::MSet& The Xapian matching set. It can be empty. */
+ static void
extractBestMatchingDocumentFromMSet (const Xapian::MSet&, Document&);
/** Remove the word furthest at right. */
static void removeFurthestRightWord (std::string& ioQueryString);
/** Remove the word furthest at left. */
- static void removeFurthestLeftWord (std::string& ioQueryString);
+ static Word_T removeFurthestLeftWord (std::string& ioQueryString);
/** Remove, from a string, the part corresponding to the one given
as parameter. */
static void
subtractParsedToRemaining (const std::string& iAlreadyParsedQueryString,
std::string& ioRemainingQueryString);
-
-
- private:
-
};
}
Modified: trunk/opentrep/opentrep/bom/WordHolder.hpp
===================================================================
--- trunk/opentrep/opentrep/bom/WordHolder.hpp 2009-07-18 20:30:08 UTC (rev 139)
+++ trunk/opentrep/opentrep/bom/WordHolder.hpp 2009-07-18 23:30:37 UTC (rev 140)
@@ -5,8 +5,8 @@
// Import section
// //////////////////////////////////////////////////////////////////////
// OpenTREP
+#include <opentrep/OPENTREP_Types.hpp>
#include <opentrep/bom/BomAbstract.hpp>
-#include <opentrep/bom/WordList.hpp>
namespace OPENTREP {
Deleted: trunk/opentrep/opentrep/bom/WordList.hpp
===================================================================
--- trunk/opentrep/opentrep/bom/WordList.hpp 2009-07-18 20:30:08 UTC (rev 139)
+++ trunk/opentrep/opentrep/bom/WordList.hpp 2009-07-18 23:30:37 UTC (rev 140)
@@ -1,18 +0,0 @@
-#ifndef __OPENTREP_BOM_WORDLIST_HPP
-#define __OPENTREP_BOM_WORDLIST_HPP
-
-// //////////////////////////////////////////////////////////////////////
-// Import section
-// //////////////////////////////////////////////////////////////////////
-// STL
-#include <string>
-#include <list>
-
-namespace OPENTREP {
-
- /** List of simple words (STL strings). */
- typedef std::list<std::string> WordList_T;
-
-}
-#endif // __OPENTREP_BOM_WORDLIST_HPP
-
Modified: trunk/opentrep/opentrep/bom/sources.mk
===================================================================
--- trunk/opentrep/opentrep/bom/sources.mk 2009-07-18 20:30:08 UTC (rev 139)
+++ trunk/opentrep/opentrep/bom/sources.mk 2009-07-18 23:30:37 UTC (rev 140)
@@ -3,7 +3,6 @@
$(top_srcdir)/opentrep/bom/Language.hpp \
$(top_srcdir)/opentrep/bom/GenericBom.hpp \
$(top_srcdir)/opentrep/bom/World.hpp \
- $(top_srcdir)/opentrep/bom/WordList.hpp \
$(top_srcdir)/opentrep/bom/WordHolder.hpp \
$(top_srcdir)/opentrep/bom/Names.hpp \
$(top_srcdir)/opentrep/bom/Place.hpp \
Modified: trunk/opentrep/opentrep/command/RequestInterpreter.cpp
===================================================================
--- trunk/opentrep/opentrep/command/RequestInterpreter.cpp 2009-07-18 20:30:08 UTC (rev 139)
+++ trunk/opentrep/opentrep/command/RequestInterpreter.cpp 2009-07-18 23:30:37 UTC (rev 140)
@@ -111,7 +111,8 @@
interpretTravelRequest (soci::session& ioSociSession,
const TravelDatabaseName_T& iTravelDatabaseName,
const TravelQuery_T& iTravelQuery,
- LocationList_T& ioLocationList) {
+ LocationList_T& ioLocationList,
+ WordList_T& ioWordList) {
NbOfMatches_T oNbOfMatches = 0;
// Create a PlaceHolder object, to collect the matching Place objects
@@ -132,7 +133,7 @@
// Main algorithm
DocumentList_T lDocumentList;
- lResultHolder.searchString (lDocumentList);
+ lResultHolder.searchString (lDocumentList, ioWordList);
/** Create the list of Result objects corresponding to the list
of documents. */
Modified: trunk/opentrep/opentrep/command/RequestInterpreter.hpp
===================================================================
--- trunk/opentrep/opentrep/command/RequestInterpreter.hpp 2009-07-18 20:30:08 UTC (rev 139)
+++ trunk/opentrep/opentrep/command/RequestInterpreter.hpp 2009-07-18 23:30:37 UTC (rev 140)
@@ -32,11 +32,12 @@
"sna francicso rio de janero lso angles reykyavki nce iev mow").
@param LocationList_T& List of (geographical) locations, if any,
matching the given query string.
+ @param WordList_T& List of non-matched words of the query string.
@return NbOfMatches_T Number of matches. */
static NbOfMatches_T interpretTravelRequest (soci::session&,
const TravelDatabaseName_T&,
const TravelQuery_T&,
- LocationList_T&);
+ LocationList_T&, WordList_T&);
private:
/** Constructors. */
Modified: trunk/opentrep/opentrep/service/OPENTREP_Service.cpp
===================================================================
--- trunk/opentrep/opentrep/service/OPENTREP_Service.cpp 2009-07-18 20:30:08 UTC (rev 139)
+++ trunk/opentrep/opentrep/service/OPENTREP_Service.cpp 2009-07-18 23:30:37 UTC (rev 140)
@@ -112,7 +112,9 @@
// //////////////////////////////////////////////////////////////////////
NbOfMatches_T OPENTREP_Service::
interpretTravelRequest (const std::string& iTravelQuery,
- LocationList_T& ioLocationList) {
+ LocationList_T& ioLocationList,
+ WordList_T& ioWordList) {
+
if (_opentrepServiceContext == NULL) {
throw NonInitialisedServiceException();
}
@@ -133,7 +135,8 @@
const NbOfMatches_T nbOfMatches =
RequestInterpreter::interpretTravelRequest (lSociSession,
lTravelDatabaseName,
- iTravelQuery, ioLocationList);
+ iTravelQuery, ioLocationList,
+ ioWordList);
const double lRequestInterpreterMeasure =
lRequestInterpreterChronometer.elapsed();
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
