[Opentrep-svn] SF.net SVN: opentrep:[171] trunk/opentrep
Status: Beta
Brought to you by:
denis_arnaud
Menu
▾
▴
[Opentrep-svn] SF.net SVN: opentrep:[171] trunk/opentrep
|
From: <den...@us...> - 2009-08-08 14:55:48
|
Revision: 171
http://opentrep.svn.sourceforge.net/opentrep/?rev=171&view=rev
Author: denis_arnaud
Date: 2009-08-08 14:55:34 +0000 (Sat, 08 Aug 2009)
Log Message:
-----------
1. [Dev] Finished the work on bringing extra and additional Location objects into the API.
2. [DB] In the search batch, looking in the database is now based on the airport/city code, rather than on the Xapian document ID. That way, no database update is necessary when re-indexing, and any search from any Xapian index will find the corresponding result details within the database. The easiest way is to extract the first three letters of the Xapian document data.
3. [Dev] Wrote a (Python-based) PSP page in order to render in HTML the output of the search batch. There is still some work to do in order to adapt it to the new API (with extra and alternate locations).
Modified Paths:
--------------
trunk/opentrep/Makefile.am
trunk/opentrep/config/soci.m4
trunk/opentrep/configure.ac
trunk/opentrep/db/data/ref_place_names.csv
trunk/opentrep/opentrep/Location.hpp
trunk/opentrep/opentrep/OPENTREP_Service.hpp
trunk/opentrep/opentrep/OPENTREP_Types.hpp
trunk/opentrep/opentrep/batches/opentrep_indexer.cfg
trunk/opentrep/opentrep/batches/opentrep_searcher.cfg
trunk/opentrep/opentrep/batches/searcher.cpp
trunk/opentrep/opentrep/bom/Place.cpp
trunk/opentrep/opentrep/bom/Place.hpp
trunk/opentrep/opentrep/bom/ResultHolder.cpp
trunk/opentrep/opentrep/bom/StringMatcher.cpp
trunk/opentrep/opentrep/bom/StringMatcher.hpp
trunk/opentrep/opentrep/command/DBManager.cpp
trunk/opentrep/opentrep/command/DBManager.hpp
trunk/opentrep/opentrep/command/IndexBuilder.cpp
trunk/opentrep/opentrep/command/RequestInterpreter.cpp
trunk/opentrep/opentrep/factory/FacPlace.cpp
trunk/opentrep/opentrep/python/pyopentrep.cpp
trunk/opentrep/opentrep/python/pyopentrep.py
trunk/opentrep/test/i18n/Makefile.am
Added Paths:
-----------
trunk/opentrep/TODO
trunk/opentrep/config/ax_icu.m4
trunk/opentrep/gui/
trunk/opentrep/gui/Makefile.am
trunk/opentrep/gui/icons/
trunk/opentrep/gui/icons/Makefile.am
trunk/opentrep/gui/icons/opentrep.png
trunk/opentrep/gui/icons/opentrep.xcf
trunk/opentrep/gui/icons/sources.mk
trunk/opentrep/gui/psp/
trunk/opentrep/gui/psp/Makefile.am
trunk/opentrep/gui/psp/index.html
trunk/opentrep/gui/psp/libpyopentrep_proxy.py
trunk/opentrep/gui/psp/localize.py
trunk/opentrep/gui/psp/log_service.py
trunk/opentrep/gui/psp/opentrep.psp
trunk/opentrep/gui/psp/result_parser.py
trunk/opentrep/gui/psp/sources.mk
trunk/opentrep/opentrep/LocationList.hpp
trunk/opentrep/test/i18n/icufmt.cpp
trunk/opentrep/test/i18n/ref/
trunk/opentrep/test/i18n/ref/ref_text_en.txt
trunk/opentrep/test/i18n/ref/ref_text_ru.txt
trunk/opentrep/test/i18n/ref/ref_text_ru_koi8r.txt
trunk/opentrep/test/i18n/ref/ref_text_ru_koi8ru.txt
trunk/opentrep/test/i18n/ref/ref_text_ru_windows_1251.txt
trunk/opentrep/test/i18n/ref/ref_text_ua.txt
trunk/opentrep/test/i18n/ref/ref_text_ua_koi8r.txt
trunk/opentrep/test/i18n/ref/ref_text_ua_koi8u.txt
trunk/opentrep/test/i18n/ref/ref_text_ua_windows_1251.txt
trunk/opentrep/test/i18n/simple_io.cpp
Property Changed:
----------------
trunk/opentrep/
trunk/opentrep/test/i18n/
Property changes on: trunk/opentrep
___________________________________________________________________
Modified: svn:ignore
- configure
config.log
config.status
autom4te.cache
aclocal.m4
ABOUT-NLS
INSTALL
COPYING
libtool
Makefile.in
Makefile
opentrep.spec
opentrep-config
opentrep.m4
opentrep.pc
opentrep-*.*.*.tar.*
opentrep-html-doc-*.*.*.tar.*
+ configure
config.log
config.status
autom4te.cache
aclocal.m4
ABOUT-NLS
INSTALL
COPYING
libtool
Makefile.in
Makefile
opentrep.spec
opentrep-config
opentrep.m4
opentrep.pc
opentrep-*.*.*.tar.*
opentrep-html-doc-*.*.*.tar.*
psp.tar.*
Modified: trunk/opentrep/Makefile.am
===================================================================
--- trunk/opentrep/Makefile.am 2009-07-27 05:56:43 UTC (rev 170)
+++ trunk/opentrep/Makefile.am 2009-08-08 14:55:34 UTC (rev 171)
@@ -24,7 +24,8 @@
EXTRA_DIST = @PACKAGE@.spec @PACKAGE@.m4 @PACKAGE@.pc Makefile.common
# Build in these directories:
-SUBDIRS = opentrep win32 po man $(INFO_DOC_DIR) $(HTML_DOC_DIR) db $(TEST_DIR)
+SUBDIRS = @PACKAGE@ win32 po man $(INFO_DOC_DIR) $(HTML_DOC_DIR) db \
+ gui $(TEST_DIR)
# Configuration helpers
@@ -43,8 +44,10 @@
dist-html:
$(MAKE) -C doc dist-html
+dist-gui:
+ $(MAKE) -C gui dist-gui
-snapshot: snapshot-src snapshot-html
+snapshot: snapshot-src snapshot-html snapshot-gui
snapshot-src:
@@ -53,8 +56,11 @@
snapshot-html:
$(MAKE) -C doc dist-html html_tarname=@PACKAGE_TARNAME@-html-doc-`date +"%Y%m%d"`
-upload: upload-src upload-html
+snapshot-gui:
+ $(MAKE) -C gui dist-gui
+upload: upload-src upload-html upload-gui
+
upload-src: dist
@UPLOAD_COMMAND@ @PACKAGE_TARNAME@-@VERSION@.tar.gz \
@PACKAGE_TARNAME@-@VERSION@.tar.bz2
@@ -63,3 +69,6 @@
@UPLOAD_COMMAND@ @PACKAGE_TARNAME@-html-doc-@VERSION@.tar.gz \
@PACKAGE_TARNAME@-html-doc-@VERSION@.tar.bz2
+upload-gui: dist-gui
+ @UPLOAD_COMMAND@ @PACKAGE_TARNAME@-gui-@VERSION@.tar.gz \
+ @PACKAGE_TARNAME@-gui-@VERSION@.tar.bz2
Added: trunk/opentrep/TODO
===================================================================
--- trunk/opentrep/TODO (rev 0)
+++ trunk/opentrep/TODO 2009-08-08 14:55:34 UTC (rev 171)
@@ -0,0 +1,20 @@
+Todo list for the OpenTrep project
+----------------------------------
+
+* [01/08/2009] Finish the work on bringing extra and additional
+ Location objects into the API.
+OK
+
+* [01/08/2009] In the search batch, when looking in the database, do
+ it based on the airport/city code, rather than on the Xapian
+ document ID. That way, no database update will be necessary when
+ re-indexing, and any search from any Xapian index will find the
+ corresponding result details within the database. The easiest way is
+ to extract the first three letters of the Xapian document data.
+OK
+
+* [01/08/2009] Write a (Python-based) PSP page, in order to test the
+ different locales of the browsers.
+The Python (PSP) page has been created, but there is still some work
+to do in order to adapt it to the new API (with extra and alternate
+locations).
\ No newline at end of file
Added: trunk/opentrep/config/ax_icu.m4
===================================================================
--- trunk/opentrep/config/ax_icu.m4 (rev 0)
+++ trunk/opentrep/config/ax_icu.m4 2009-08-08 14:55:34 UTC (rev 171)
@@ -0,0 +1,160 @@
+dnl @synopsis AX_ICU
+dnl
+dnl This macro tries to find Icu C API header and library locations.
+dnl
+dnl We define the following configure script flags:
+dnl
+dnl --with-icu: Give prefix for both library and headers, and try
+dnl to guess subdirectory names for each. (e.g. Tack /lib and
+dnl /include onto given dir name, and other common schemes.)
+dnl --with-icu-lib: Similar to --with-icu, but for library only.
+dnl --with-icu-include: Similar to --with-icu, but for headers
+dnl only.
+dnl
+dnl @version 1.2, 2007/02/20
+dnl @author Warren Young <ic...@et...>
+
+AC_DEFUN([AX_ICU],
+[
+ #
+ # Set up configure script macros
+ #
+ AC_ARG_WITH(icu,
+ [ --with-icu=<path> root directory path of Icu installation],
+ [ICU_lib_check="$with_icu/lib64/icu $with_icu/lib/icu $with_icu/lib64 $with_icu/lib"
+ ICU_inc_check="$with_icu/include $with_icu/include/icu"
+ ICU_bin_check="$with_icu/bin"],
+ [ICU_lib_check="/usr/lib64 /usr/lib /usr/lib64/icu /usr/lib/icu /usr/local/lib64 /usr/local/lib /usr/local/lib/icu /usr/local/icu/lib /usr/local/icu/lib/icu /opt/icu/lib /opt/icu/lib/icu"
+ ICU_inc_check="/usr/include /usr/local/include /usr/local/icu/include /opt/icu/include"
+ ICU_bin_check="/usr/bin /usr/local/bin /usr/local/icu/bin"])
+
+ AC_ARG_WITH(icu-lib,
+ [ --with-icu-lib=<path> directory path of Icu library installation],
+ [ICU_lib_check="$with_icu_lib $with_icu_lib/lib64 $with_icu_lib/lib $with_icu_lib/lib64/icu $with_icu_lib/lib/icu"])
+
+ AC_ARG_WITH(icu-include,
+ [ --with-icu-include=<path> directory path of Icu header installation],
+ [ICU_inc_check="$with_icu_include $with_icu_include/include $with_icu_include/include/icu"])
+
+
+ #
+ # Look for Icu Configuration Script
+ #
+ AC_MSG_CHECKING([for Icu configuration script])
+ ICU_CONFIG=
+ ICU_bindir=
+ for m in $ICU_bin_check
+ do
+ if test -d "$m" && test -f "$m/icu-config"
+ then
+ ICU_CONFIG=$m/icu-config
+ ICU_bindir=$m
+ break
+ fi
+ done
+
+ if test -z "$ICU_bindir"
+ then
+ AC_MSG_ERROR([Didn't find $ICU_CONFIG binary in '$ICU_bin_check'])
+ fi
+
+ case "$ICU_bindir" in
+ /* ) ;;
+ * ) AC_MSG_ERROR([The Icu binary directory ($ICU_bindir) must be an absolute path.]) ;;
+ esac
+
+ AC_MSG_RESULT([$ICU_bindir])
+
+ AC_PATH_PROG(ICU_CONFIG, icu-config, $ICU_bindir)
+
+ if test "x${ICU_CONFIG+set}" != xset
+ then
+ ICU_VERSION=`${ICU_CONFIG} --version`
+ ICU_CFLAGS=`${ICU_CONFIG} --cppflags`
+ ICU_LIBS=`${ICU_CONFIG} --ldflags`
+ else
+ #
+ # Look for Icu C API library
+ #
+ AC_MSG_CHECKING([for Icu library directory])
+ ICU_libdir=
+ ICU_IO_LIB=icuio
+ for m in $ICU_lib_check
+ do
+ if test -d "$m" && \
+ (test -f "$m/lib$ICU_IO_LIB.so" \
+ || test -f "$m/lib$ICU_IO_LIB.a")
+ then
+ ICU_libdir=$m
+ break
+ fi
+ done
+
+ if test -z "$ICU_libdir"
+ then
+ AC_MSG_ERROR([Didn't find $ICU_IO_LIB library in '$ICU_lib_check'])
+ fi
+
+ case "$ICU_libdir" in
+ /* ) ;;
+ * ) AC_MSG_ERROR([The Icu library directory ($ICU_libdir) must be an absolute path.]) ;;
+ esac
+
+ AC_MSG_RESULT([$ICU_libdir])
+
+ case "$ICU_libdir" in
+ /usr/lib64) ;;
+ /usr/lib) ;;
+ *) LDFLAGS="$LDFLAGS -L${ICU_libdir}" ;;
+ esac
+
+ #
+ # Look for Icu C API headers
+ #
+ AC_MSG_CHECKING([for Icu include directory])
+ ICU_incdir=
+ for m in $ICU_inc_check
+ do
+ if test -d "$m" && test -f "$m/unicode/utf8.h"
+ then
+ ICU_incdir=$m
+ break
+ fi
+ done
+
+ if test -z "$ICU_incdir"
+ then
+ AC_MSG_ERROR([Didn't find the Icu include dir in '$ICU_inc_check'])
+ fi
+
+ case "$ICU_incdir" in
+ /* ) ;;
+ * ) AC_MSG_ERROR([The Icu include directory ($ICU_incdir) must be an absolute path.]) ;;
+ esac
+
+ AC_MSG_RESULT([$ICU_incdir])
+
+ ICU_CFLAGS="-D_REENTRANT -I${ICU_incdir}"
+ ICU_LIBS="-licui18n -licuuc -licudata -lpthread -lm"
+
+ case "$ICU_libdir" in
+ /usr/lib64) ;;
+ /usr/lib) ;;
+ *) ICU_LIBS="-L${ICU_libdir} $ICU_LIBS" ;;
+ esac
+ fi
+
+ AC_SUBST(ICU_VERSION)
+ AC_SUBST(ICU_CFLAGS)
+ AC_SUBST(ICU_LIBS)
+
+ save_LIBS="$LIBS"
+ LIBS="$LIBS $ICU_LIBS"
+# AC_CHECK_LIB($ICU_IO_LIB, utext_isWritable,
+# [],
+# [AC_MSG_ERROR([Could not find working Icu client library!])]
+# )
+ ICU_IO_LIB="-l${ICU_IO_LIB}"
+ AC_SUBST(ICU_IO_LIB)
+ LIBS="$save_LIBS"
+]) dnl AX_ICU
Modified: trunk/opentrep/config/soci.m4
===================================================================
--- trunk/opentrep/config/soci.m4 2009-07-27 05:56:43 UTC (rev 170)
+++ trunk/opentrep/config/soci.m4 2009-08-08 14:55:34 UTC (rev 171)
@@ -60,9 +60,9 @@
SOCI_CORE_LIB=${SOCI_CORE_LIB}-${SOCI_LIB_SUFFIX}
SOCI_MYSQL_LIB=${SOCI_MYSQL_LIB}-${SOCI_LIB_SUFFIX}
SOCI_libdir=$m
+ break
fi
done
- break
fi
done
Modified: trunk/opentrep/configure.ac
===================================================================
--- trunk/opentrep/configure.ac 2009-07-27 05:56:43 UTC (rev 170)
+++ trunk/opentrep/configure.ac 2009-08-08 14:55:34 UTC (rev 171)
@@ -147,7 +147,16 @@
AC_SUBST(XAPIAN_CFLAGS)
AC_SUBST(XAPIAN_LIBS)
+# --------------------------------------------------------------------
+# Support for ICU (i18n C API): http://www.icu-project.org
+# --------------------------------------------------------------------
+AX_ICU
+AC_SUBST(ICU_VERSION)
+AC_SUBST(ICU_CFLAGS)
+AC_SUBST(ICU_LIBS)
+AC_SUBST(ICU_IO_LIB)
+
# -------------------------------------------------------------------
# Support for documentation
# -------------------------------------------------------------------
@@ -249,6 +258,9 @@
db/maintenance/Makefile
db/maintenance/tables/Makefile
db/data/Makefile
+ gui/Makefile
+ gui/icons/Makefile
+ gui/psp/Makefile
test/com/Makefile
test/parsers/Makefile
test/i18n/Makefile
@@ -327,6 +339,12 @@
o XAPIAN_CFLAGS ... : ${XAPIAN_CFLAGS}
o XAPIAN_LIBS ..... : ${XAPIAN_LIBS}
+ - ICU ............... :
+ o ICU_version ..... : ${ICU_VERSION}
+ o ICU_CFLAGS ...... : ${ICU_CFLAGS}
+ o ICU_LIBS ........ : ${ICU_LIBS}
+ o ICU_IO_LIB ...... : ${ICU_IO_LIB}
+
- CPPUNIT ........... :
o CPPUNIT_VERSION . : ${CPPUNIT_VERSION}
o CPPUNIT_CFLAGS .. : ${CPPUNIT_CFLAGS}
Modified: trunk/opentrep/db/data/ref_place_names.csv
===================================================================
--- trunk/opentrep/db/data/ref_place_names.csv 2009-07-27 05:56:43 UTC (rev 170)
+++ trunk/opentrep/db/data/ref_place_names.csv 2009-08-08 14:55:34 UTC (rev 171)
@@ -6253,7 +6253,7 @@
en,yvo,val d'or,val d'or/qc/ca
en,yvp,kuujjuaq,kuujjuaq/qc/ca
en,yvq,norman wells,norman wells/nt/ca
-en,yvr,vancouver int,vancouver/bc/ca:intl
+en,yvr,vancouver int,vancouver/bc/ca:intl,vancouver
en,yvs,ski rail station,ski/no:ski rail station
en,yvt,buffalo narrows,buffalo narrows/sk/ca
en,yvv,wiarton,wiarton/on/ca
@@ -9446,7 +9446,7 @@
en,xdx,sarnia,sarnia/on/ca:railway station
en,xdy,sudbury,sudbury/on/ca:junction rail st
en,xdz,the pas,the pas/mb/ca:railway station
-en,xea,vancouver,vancouver/bc/ca:railway statio
+en,xea,vancouver railway,vancouver/bc/ca:railway statio
en,xeb,evian les bains,evian les bains/fr:off-
en,xec,windsor,windsor/on/ca:railway station
en,xed,disneyland paris,paris/fr:disneyland paris
Property changes on: trunk/opentrep/gui
___________________________________________________________________
Added: svn:ignore
+ .libs
.deps
Makefile
Makefile.in
Added: trunk/opentrep/gui/Makefile.am
===================================================================
--- trunk/opentrep/gui/Makefile.am (rev 0)
+++ trunk/opentrep/gui/Makefile.am 2009-08-08 14:55:34 UTC (rev 171)
@@ -0,0 +1,27 @@
+# Python Server Pages (PSP)
+
+SUBDIRS = icons psp
+
+MAINTAINERCLEANFILES = Makefile.in Makefile
+
+datadir = @datadir@
+pkgdatadir = $(datadir)/@PACKAGE@
+guidir = $(pkgdatadir)/gui
+
+psp_sources = psp
+psp_dests = $(foreach ext,.tar.gz .tar.bz2,$(addsuffix $(ext),$(psp_sources)))
+
+# Targets
+$(top_builddir)/%.tar.gz $(builddir)/%.tar.gz: %/*.html %/*.py %/*.psp %/../icons/*.png
+ tar chof - $^ | gzip --best -c > $@
+
+$(top_builddir)/%.tar.bz2 $(builddir)/%.tar.bz2: %/*.html %/*.py %/*.psp %/../icons/*.png
+ tar chof - $^ | bzip2 -9 -c > $@
+
+dist-gui: $(addprefix $(top_builddir)/,$(psp_dests))
+
+clean-local:
+ rm -f $(addprefix $(top_builddir)/,$(psp_dests))
+
+snapshot-gui:
+ $(MAKE) dist-gui gui_tarname=@PACKAGE_TARNAME@-gui-`date +"%Y%m%d"`
Property changes on: trunk/opentrep/gui/icons
___________________________________________________________________
Added: svn:ignore
+ .libs
.deps
Makefile
Makefile.in
Added: trunk/opentrep/gui/icons/Makefile.am
===================================================================
--- trunk/opentrep/gui/icons/Makefile.am (rev 0)
+++ trunk/opentrep/gui/icons/Makefile.am 2009-08-08 14:55:34 UTC (rev 171)
@@ -0,0 +1,25 @@
+# gui/icons sub-directory: Images (.png, .gif, etc)
+include $(srcdir)/sources.mk
+
+datadir = @datadir@
+pkgdatadir = $(datadir)/@PACKAGE@
+imgdir = $(pkgdatadir)/gui/icons
+
+MAINTAINERCLEANFILES = Makefile.in Makefile
+
+noinst_DATA = $(img_sources)
+
+EXTRA_DIST = $(noinst_DATA)
+
+# Targets
+install-data-local:
+ $(mkinstalldirs) $(DESTDIR)$(imgdir); \
+ for f in $(noinst_DATA); do \
+ $(INSTALL_DATA) $$f $(DESTDIR)$(imgdir); \
+ done
+
+uninstall-local:
+ rm -rf $(DESTDIR)$(imgdir)
+
+clean-local:
+ rm -rf *.log *.tag
Added: trunk/opentrep/gui/icons/opentrep.png
===================================================================
(Binary files differ)
Property changes on: trunk/opentrep/gui/icons/opentrep.png
___________________________________________________________________
Added: svn:mime-type
+ application/octet-stream
Added: trunk/opentrep/gui/icons/opentrep.xcf
===================================================================
(Binary files differ)
Property changes on: trunk/opentrep/gui/icons/opentrep.xcf
___________________________________________________________________
Added: svn:mime-type
+ application/octet-stream
Added: trunk/opentrep/gui/icons/sources.mk
===================================================================
--- trunk/opentrep/gui/icons/sources.mk (rev 0)
+++ trunk/opentrep/gui/icons/sources.mk 2009-08-08 14:55:34 UTC (rev 171)
@@ -0,0 +1 @@
+img_sources = $(top_srcdir)/gui/icons/opentrep.png
Property changes on: trunk/opentrep/gui/psp
___________________________________________________________________
Added: svn:ignore
+ .libs
.deps
Makefile
Makefile.in
Added: trunk/opentrep/gui/psp/Makefile.am
===================================================================
--- trunk/opentrep/gui/psp/Makefile.am (rev 0)
+++ trunk/opentrep/gui/psp/Makefile.am 2009-08-08 14:55:34 UTC (rev 171)
@@ -0,0 +1,25 @@
+# Python Server Pages (PSP)
+include $(srcdir)/sources.mk
+
+datadir = @datadir@
+pkgdatadir = $(datadir)/@PACKAGE@
+pspdir = $(pkgdatadir)/gui/psp
+
+MAINTAINERCLEANFILES = Makefile.in Makefile
+
+noinst_DATA = $(html_sources) $(py_sources) $(psp_sources)
+
+EXTRA_DIST = $(noinst_DATA)
+
+# Targets
+install-data-local:
+ $(mkinstalldirs) $(DESTDIR)$(pspdir); \
+ for f in $(noinst_DATA); do \
+ $(INSTALL_DATA) $$f $(DESTDIR)$(pspdir); \
+ done
+
+uninstall-local:
+ rm -rf $(DESTDIR)$(pspdir)
+
+clean-local:
+ rm -rf *.log *.tag
Added: trunk/opentrep/gui/psp/index.html
===================================================================
--- trunk/opentrep/gui/psp/index.html (rev 0)
+++ trunk/opentrep/gui/psp/index.html 2009-08-08 14:55:34 UTC (rev 171)
@@ -0,0 +1,14 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
+<html xmlns="http://www.w3.org/1999/xhtml">
+<head>
+<meta http-equiv="content-type" content="text/html; charset=UTF-8" />
+<meta http-equiv="refresh"
+ content="0; url=http://localhost/opentrep/opentrep.psp" />
+<title>Redirection</title>
+<meta name="robots" content="noindex,follow" />
+</head>
+
+<body>
+</body>
+</html>
Added: trunk/opentrep/gui/psp/libpyopentrep_proxy.py
===================================================================
--- trunk/opentrep/gui/psp/libpyopentrep_proxy.py (rev 0)
+++ trunk/opentrep/gui/psp/libpyopentrep_proxy.py 2009-08-08 14:55:34 UTC (rev 171)
@@ -0,0 +1,8 @@
+#!/usr/bin/python
+
+import sys
+
+def import_libpyopentrep(libpyopentrep_path):
+ sys.path.append(libpyopentrep_path)
+ import libpyopentrep
+ return libpyopentrep
Added: trunk/opentrep/gui/psp/localize.py
===================================================================
--- trunk/opentrep/gui/psp/localize.py (rev 0)
+++ trunk/opentrep/gui/psp/localize.py 2009-08-08 14:55:34 UTC (rev 171)
@@ -0,0 +1,14 @@
+#!/usr/bin/python
+
+import socket
+
+www_log_filename = '/var/log/opentrep/www.log'
+trep_log_filename = '/var/log/opentrep/opentrep.log'
+tmp_trep_log_filename = '/var/log/opentrep/tmp_opentrep.log'
+
+hostname = socket.gethostname()
+main_name = hostname.split('.')[0]
+
+traveldb_path = '/var/www/opentrep/traveldb'
+libpyopentrep_path = '/tmp/opentrep/lib'
+opentrep_dbparams = {'user': 'opentrep', 'password': 'opentrep', 'host': 'localhost', 'port': '3306', 'db': 'trep_opentrep'}
Added: trunk/opentrep/gui/psp/log_service.py
===================================================================
--- trunk/opentrep/gui/psp/log_service.py (rev 0)
+++ trunk/opentrep/gui/psp/log_service.py 2009-08-08 14:55:34 UTC (rev 171)
@@ -0,0 +1,25 @@
+#!/usr/bin/python
+
+import socket, os, datetime
+
+def log(filename, req, query, codes, unrecognized):
+ req.add_common_vars()
+ # determine ip
+ remote_client_ip = req.connection.remote_ip
+ # determine hostname
+ hostname = req.connection.remote_host
+ if hostname == None: hostname = 'localhost'
+ # determine time
+ str_time = datetime.datetime.now().strftime('%y%m%d%H%M%S')
+ # determine user agent
+ agent = ''
+ if req.subprocess_env.has_key("HTTP_USER_AGENT"): agent = req.subprocess_env["HTTP_USER_AGENT"]
+ # determine user allowed languages
+ languages = ''
+ if req.subprocess_env.has_key("HTTP_ACCEPT_LANGUAGE"): languages = req.subprocess_env["HTTP_ACCEPT_LANGUAGE"]
+ # determine user allowed character sets
+ charsets = ''
+ if req.subprocess_env.has_key("HTTP_ACCEPT_CHARSET"): charsets = req.subprocess_env["HTTP_ACCEPT_CHARSET"]
+ # write to file
+ str_out = '^'.join([str_time,remote_client_ip, hostname, query, ','.join(codes), unrecognized, agent, languages, charsets])
+ os.system('echo "%s" >> %s' % (str_out, filename))
Added: trunk/opentrep/gui/psp/opentrep.psp
===================================================================
--- trunk/opentrep/gui/psp/opentrep.psp (rev 0)
+++ trunk/opentrep/gui/psp/opentrep.psp 2009-08-08 14:55:34 UTC (rev 171)
@@ -0,0 +1,80 @@
+<%
+import os
+local_path = '/var/www/opentrep'
+from mod_python import apache
+localize = apache.import_module('localize', path=[local_path])
+log_service = apache.import_module('log_service', path=[local_path])
+
+# defaults
+msg, head, form_value, unrecognized = '', '', '', ''
+#body_declaration = '<body>'
+quiet = True
+
+# parsing: recognize sequence of three-letter codes
+codes = []
+alter_locations = []
+queryStringForm = form
+if queryStringForm.has_key('data'):
+ form_value = queryStringForm['data']
+ quiet = False
+ if form_value.rstrip(' ') == '':
+ pass
+ else:
+ # Use opentrep
+ libpyopentrep_proxy = apache.import_module('libpyopentrep_proxy', path=[local_path])
+ libpyopentrep = libpyopentrep_proxy.import_libpyopentrep(localize.libpyopentrep_path)
+ mySearch = libpyopentrep.OpenTrepSearcher()
+ mySearch.init(localize.traveldb_path, localize.tmp_trep_log_filename, localize.opentrep_dbparams['user'], localize.opentrep_dbparams['password'], localize.opentrep_dbparams['host'], localize.opentrep_dbparams['port'], localize.opentrep_dbparams['db'])
+ str_matches = mySearch.search(form_value)
+ if ';' in str_matches:
+ str_matches, unrecognized = str_matches.split(';')
+ msg = 'unrecognized: %s. ' % unrecognized
+ str_value = unrecognized
+ if str_matches != '':
+ alter_locations = [x for x in str_matches.split(',')]
+ for alter_location_list in alter_locations:
+ alter_location_list = [x for x in alter_location_list.split('-')]
+ for extra_location_list in alter_location_list:
+ extra_location_list = [x for x in extra_location_list.split(':')]
+
+ codes = [x[0].upper() for x in alter_locations]
+ if len(codes)>0: form_value = ' '.join(codes)
+ if str_value != '': form_value += ' ' + str_value
+
+ # Logging
+ log_service.log(localize.www_log_filename, req, queryStringForm['data'], codes, unrecognized)
+ os.system('cat %s >> %s' % (localize.tmp_trep_log_filename, localize.trep_log_filename))
+
+%>
+
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
+<html xmlns="http://www.w3.org/1999/xhtml">
+<head>
+<title>OpenTREP</title>
+<%= head %>
+</head>
+
+<body>
+<div align="center">
+<a href="opentrep.psp"><img src="/icons/opentrep.png" height="80px" border=0></a>
+</div>
+<br>
+
+<div align="center">
+<table border="0">
+ <tr>
+ <td>
+ <form value="queryStringForm" action="opentrep.psp" method="post">
+ <input type="text" size=80% name="data" value="<%= form_value%>">
+ <input type="submit" value="Send">
+ </form>
+ </td>
+ </tr>
+</table>
+</div>
+
+<p style="font-size:small;"><%= msg %></p>
+
+</body>
+</html>
Added: trunk/opentrep/gui/psp/result_parser.py
===================================================================
--- trunk/opentrep/gui/psp/result_parser.py (rev 0)
+++ trunk/opentrep/gui/psp/result_parser.py 2009-08-08 14:55:34 UTC (rev 171)
@@ -0,0 +1,64 @@
+#!/usr/bin/env python
+
+import sys
+
+# Default result string
+defaultResultString = 'yvr:xea/98-xtw/87,sfo/100,led:dft:htl/96;niznayou'
+
+# If no result string was supplied as arguments of the command-line,
+# ask the user for some
+resultString = ','.join(sys.argv[1:])
+if resultString == '' : resultString = defaultResultString
+
+# Function to parse the result string
+def parseResultString(iResultString):
+ form_value, unrecognized = '', ''
+ msg = '(parsing successful)'
+ str_matches = iResultString
+ alter_locations = []
+
+ if ';' in str_matches:
+ str_matches, unrecognized = str_matches.split(';')
+ msg = '(unrecognized: %s)' % unrecognized
+ str_value = unrecognized
+
+ if str_matches != '':
+ alter_locations = str_matches.split(',')
+
+ print 'alter_locations: ', alter_locations
+
+ idx1 = 0
+ while idx1 != len(alter_locations):
+
+# print 'Before - alter_locations['+str(idx1)+']: ', alter_locations[idx1]
+ alter_locations[idx1] = alter_locations[idx1].split('-')
+# print 'After - alter_locations['+str(idx1)+']: ', alter_locations[idx1], alter_locations
+
+ idx2 = 0
+ while idx2 != len(alter_locations[idx1]):
+
+ alter_locations[idx1][idx2] = alter_locations[idx1][idx2].split(':')
+
+ idx3 = 0
+ while idx3 != len(alter_locations[idx1][idx2]):
+
+ alter_locations[idx1][idx2][idx3] = alter_locations[idx1][idx2][idx3].split('/')
+ idx3 += 1
+
+ idx2 += 1
+
+ idx1 += 1
+
+# codes = [x.upper() for x in alter_locations]
+# if len(codes) > 0: form_value = ' '.join(codes)
+ if str_value != '': form_value += ' ' + str_value
+
+ print 'After - alter_locations: ', alter_locations
+
+ print 'Result ' + msg + ':'
+ return form_value
+
+# Main
+print 'Before: ' + resultString
+resultString = parseResultString(resultString)
+print 'After: ' + resultString
Property changes on: trunk/opentrep/gui/psp/result_parser.py
___________________________________________________________________
Added: svn:executable
+ *
Added: trunk/opentrep/gui/psp/sources.mk
===================================================================
--- trunk/opentrep/gui/psp/sources.mk (rev 0)
+++ trunk/opentrep/gui/psp/sources.mk 2009-08-08 14:55:34 UTC (rev 171)
@@ -0,0 +1,6 @@
+html_sources = $(top_srcdir)/gui/psp/index.html
+psp_sources = $(top_srcdir)/gui/psp/opentrep.psp
+py_sources = \
+ $(top_srcdir)/gui/psp/localize.py \
+ $(top_srcdir)/gui/psp/log_service.py \
+ $(top_srcdir)/gui/psp/libpyopentrep_proxy.py
Modified: trunk/opentrep/opentrep/Location.hpp
===================================================================
--- trunk/opentrep/opentrep/Location.hpp 2009-07-27 05:56:43 UTC (rev 170)
+++ trunk/opentrep/opentrep/Location.hpp 2009-08-08 14:55:34 UTC (rev 171)
@@ -11,6 +11,7 @@
#include <list>
// OpenTrep
#include <opentrep/OPENTREP_Types.hpp>
+#include <opentrep/LocationList.hpp>
#include <opentrep/OPENTREP_Abstract.hpp>
namespace OPENTREP {
@@ -73,6 +74,26 @@
return _nameList;
}
+ /** Get the matching percentage. */
+ const MatchingPercentage_T& getPercentage() const {
+ return _percentage;
+ }
+
+ /** Get the allowed edit distance/error. */
+ const NbOfErrors_T& getEditDistance() const {
+ return _editDistance;
+ }
+
+ /** Get the list of extra matching (similar) locations. */
+ const LocationList_T& getExtraLocationList() const {
+ return _extraLocationList;
+ }
+
+ /** Get the list of alternate matching (less similar) locations. */
+ const LocationList_T& getAlternateLocationList() const {
+ return _alternateLocationList;
+ }
+
// ///////// Setters //////////
/** Set the Location code. */
@@ -125,7 +146,27 @@
_nameList = iNameList;
}
+ /** Set the Xapian matching percentage. */
+ void setPercentage (const MatchingPercentage_T& iPercentage) {
+ _percentage = iPercentage;
+ }
+
+ /** Set the allowed edit distance/error. */
+ void setEditDistance (const NbOfErrors_T& iEditDistance) {
+ _editDistance = iEditDistance;
+ }
+
+ /** Add an extra matching location. */
+ void addExtraLocation (const Location& iExtraLocation) {
+ _extraLocationList.push_back (iExtraLocation);
+ }
+ /** Add an alternate matching location. */
+ void addAlternateLocation (const Location& iAlternateLocation) {
+ _alternateLocationList.push_back (iAlternateLocation);
+ }
+
+
public:
// ///////// Display methods ////////
/** Dump a structure into an output stream.
@@ -145,7 +186,17 @@
oStr << _locationCode << ", " << _cityCode << ", " << _stateCode
<< ", " << _countryCode << ", " << _regionCode
<< ", " << _continentCode << ", " << _timeZoneGroup
- << ", " << _longitude << ", " << _latitude;
+ << ", " << _longitude << ", " << _latitude
+ << ", " << _percentage << ", " << _editDistance;
+
+ if (_extraLocationList.empty() == false) {
+ oStr << " " << _extraLocationList.size() << " extra match(es)";
+ }
+
+ if (_alternateLocationList.empty() == false) {
+ oStr << " " << _alternateLocationList.size() << " alternate match(es)";
+ }
+
return oStr.str();
}
@@ -157,6 +208,36 @@
itName != _nameList.end(); ++itName) {
oStr << ", " << *itName;
}
+
+ if (_extraLocationList.empty() == false) {
+ oStr << "; Extra matches: {";
+ unsigned short idx = 0;
+ for (LocationList_T::const_iterator itLoc = _extraLocationList.begin();
+ itLoc != _extraLocationList.end(); ++itLoc, ++idx) {
+ if (idx != 0) {
+ oStr << ", ";
+ }
+ const Location& lExtraLocation = *itLoc;
+ oStr << lExtraLocation.toShortString();
+ }
+ oStr << "}";
+ }
+
+ if (_alternateLocationList.empty() == false) {
+ oStr << "; Alternate matches: {";
+ unsigned short idx = 0;
+ for (LocationList_T::const_iterator itLoc =
+ _alternateLocationList.begin();
+ itLoc != _alternateLocationList.end(); ++itLoc, ++idx) {
+ if (idx != 0) {
+ oStr << ", ";
+ }
+ const Location& lAlternateLocation = *itLoc;
+ oStr << lAlternateLocation.toShortString();
+ }
+ oStr << "}";
+ }
+
return oStr.str();
}
@@ -168,12 +249,15 @@
const std::string& iRegionCode, const std::string& iContinentCode,
const std::string& iTimeZoneGroup,
const double iLongitude, const double iLatitude,
- const LocationNameList_T& iNameList)
+ const LocationNameList_T& iNameList,
+ const MatchingPercentage_T& iPercentage,
+ const NbOfErrors_T& iEditDistance)
: _locationCode (iPlaceCode), _cityCode (iCityCode),
_stateCode (iStateCode), _countryCode (iCountryCode),
_regionCode (iRegionCode), _continentCode (iContinentCode),
_timeZoneGroup (iTimeZoneGroup), _longitude (iLongitude),
- _latitude (iLatitude), _nameList (iNameList) {
+ _latitude (iLatitude), _nameList (iNameList),
+ _percentage (iPercentage), _editDistance (iEditDistance) {
}
/** Default Constructor. */
@@ -207,11 +291,19 @@
double _latitude;
/** List of (American) English names. */
LocationNameList_T _nameList;
- };
+ /** Matching percentage. */
+ MatchingPercentage_T _percentage;
- /** List of (geographical) location structures. */
- typedef std::list<Location> LocationList_T;
+ /** Allowed edit error/distance. */
+ NbOfErrors_T _editDistance;
+ /** List of extra matching (similar) locations. */
+ LocationList_T _extraLocationList;
+
+ /** List of alternate matching (less similar) locations. */
+ LocationList_T _alternateLocationList;
+ };
+
}
#endif // __OPENTREP_LOCATION_HPP
Added: trunk/opentrep/opentrep/LocationList.hpp
===================================================================
--- trunk/opentrep/opentrep/LocationList.hpp (rev 0)
+++ trunk/opentrep/opentrep/LocationList.hpp 2009-08-08 14:55:34 UTC (rev 171)
@@ -0,0 +1,20 @@
+#ifndef __OPENTREP_LOCATIONLIST_HPP
+#define __OPENTREP_LOCATIONLIST_HPP
+
+// //////////////////////////////////////////////////////////////////////
+// Import section
+// //////////////////////////////////////////////////////////////////////
+// STL
+#include <list>
+
+namespace OPENTREP {
+
+ // Forward declaration
+ struct Location;
+
+ /** List of (geographical) location structures. */
+ typedef std::list<Location> LocationList_T;
+
+}
+#endif // __OPENTREP_LOCATIONLIST_HPP
+
Modified: trunk/opentrep/opentrep/OPENTREP_Service.hpp
===================================================================
--- trunk/opentrep/opentrep/OPENTREP_Service.hpp 2009-07-27 05:56:43 UTC (rev 170)
+++ trunk/opentrep/opentrep/OPENTREP_Service.hpp 2009-08-08 14:55:34 UTC (rev 171)
@@ -10,7 +10,7 @@
// OpenTREP
#include <opentrep/OPENTREP_Types.hpp>
#include <opentrep/DBParams.hpp>
-#include <opentrep/Location.hpp>
+#include <opentrep/LocationList.hpp>
#include <opentrep/DistanceErrorRule.hpp>
namespace OPENTREP {
Modified: trunk/opentrep/opentrep/OPENTREP_Types.hpp
===================================================================
--- trunk/opentrep/opentrep/OPENTREP_Types.hpp 2009-07-27 05:56:43 UTC (rev 170)
+++ trunk/opentrep/opentrep/OPENTREP_Types.hpp 2009-08-08 14:55:34 UTC (rev 171)
@@ -45,6 +45,9 @@
class XapianTravelDatabaseEmptyException : public XapianException {
};
+ class XapianTravelDatabaseNotInSyncWithSQLDatabaseException : public XapianException {
+ };
+
class SQLDatabaseException : public RootException {
};
@@ -87,6 +90,9 @@
/** Xapian document ID. */
typedef int XapianDocID_T;
+ /** Xapian percentage. */
+ typedef unsigned int MatchingPercentage_T;
+
/** Travel search query. */
typedef std::string TravelQuery_T;
Modified: trunk/opentrep/opentrep/batches/opentrep_indexer.cfg
===================================================================
--- trunk/opentrep/opentrep/batches/opentrep_indexer.cfg 2009-07-27 05:56:43 UTC (rev 170)
+++ trunk/opentrep/opentrep/batches/opentrep_indexer.cfg 2009-08-08 14:55:34 UTC (rev 171)
@@ -1,4 +1,4 @@
-database=../../test/traveldb
+database=/tmp/opentrep/share/opentrep/traveldb
log=opentrep_indexer.log
user=opentrep
passwd=opentrep
Modified: trunk/opentrep/opentrep/batches/opentrep_searcher.cfg
===================================================================
--- trunk/opentrep/opentrep/batches/opentrep_searcher.cfg 2009-07-27 05:56:43 UTC (rev 170)
+++ trunk/opentrep/opentrep/batches/opentrep_searcher.cfg 2009-08-08 14:55:34 UTC (rev 171)
@@ -1,4 +1,4 @@
-database=../../test/traveldb
+database=/tmp/opentrep/share/opentrep/traveldb
log=opentrep_searcher.log
user=opentrep
passwd=opentrep
Modified: trunk/opentrep/opentrep/batches/searcher.cpp
===================================================================
--- trunk/opentrep/opentrep/batches/searcher.cpp 2009-07-27 05:56:43 UTC (rev 170)
+++ trunk/opentrep/opentrep/batches/searcher.cpp 2009-08-08 14:55:34 UTC (rev 171)
@@ -13,6 +13,7 @@
#include <boost/program_options.hpp>
// OpenTREP
#include <opentrep/OPENTREP_Service.hpp>
+#include <opentrep/Location.hpp>
#include <opentrep/DBParams.hpp>
#include <opentrep/config/opentrep-paths.hpp>
Modified: trunk/opentrep/opentrep/bom/Place.cpp
===================================================================
--- trunk/opentrep/opentrep/bom/Place.cpp 2009-07-27 05:56:43 UTC (rev 170)
+++ trunk/opentrep/opentrep/bom/Place.cpp 2009-08-08 14:55:34 UTC (rev 171)
@@ -22,7 +22,8 @@
_regionCode (iPlace._regionCode), _continentCode (iPlace._continentCode),
_timeZoneGroup (iPlace._timeZoneGroup), _longitude (iPlace._longitude),
_latitude (iPlace._latitude), _nameMatrix (iPlace._nameMatrix),
- _docID (iPlace._docID) {
+ _docID (iPlace._docID), _percentage (iPlace._percentage),
+ _editDistance (iPlace._editDistance) {
}
// //////////////////////////////////////////////////////////////////////
@@ -77,7 +78,9 @@
oStr << ", " << lCityCode << ", " << _stateCode
<< ", " << _countryCode << ", " << _regionCode
<< ", " << _continentCode << ", " << _timeZoneGroup
- << ", " << _longitude << ", " << _latitude << ", " << _docID << ". ";
+ << ", " << _longitude << ", " << _latitude
+ << ", " << _docID << ", " << _percentage
+ << ", " << _editDistance << ". ";
for (NameMatrix_T::const_iterator itNameList = _nameMatrix.begin();
itNameList != _nameMatrix.end(); ++itNameList) {
@@ -85,6 +88,37 @@
oStr << lNameList.toString();
}
+ if (_extraPlaceList.empty() == false) {
+ oStr << "; Extra matches: {";
+ unsigned short idx = 0;
+ for (PlaceOrderedList_T::const_iterator itLoc = _extraPlaceList.begin();
+ itLoc != _extraPlaceList.end(); ++itLoc, ++idx) {
+ if (idx != 0) {
+ oStr << "; ";
+ }
+ const Place* lExtraPlace_ptr = *itLoc;
+ assert (lExtraPlace_ptr != NULL);
+ oStr << lExtraPlace_ptr->toShortString();
+ }
+ oStr << "}";
+ }
+
+ if (_alternatePlaceList.empty() == false) {
+ oStr << "; Alternate matches: {";
+ unsigned short idx = 0;
+ for (PlaceOrderedList_T::const_iterator itLoc =
+ _alternatePlaceList.begin();
+ itLoc != _alternatePlaceList.end(); ++itLoc, ++idx) {
+ if (idx != 0) {
+ oStr << "; ";
+ }
+ const Place* lAlternatePlace_ptr = *itLoc;
+ assert (lAlternatePlace_ptr != NULL);
+ oStr << lAlternatePlace_ptr->toShortString();
+ }
+ oStr << "}";
+ }
+
return oStr.str();
}
@@ -100,7 +134,9 @@
oStr << ", " << lCityCode << ", " << _stateCode
<< ", " << _countryCode << ", " << _regionCode
<< ", " << _continentCode << ", " << _timeZoneGroup
- << ", " << _longitude << ", " << _latitude << ", " << _docID;
+ << ", " << _longitude << ", " << _latitude
+ << ", " << _docID << ", " << _percentage
+ << ", " << _editDistance;
NameMatrix_T::const_iterator itNameHolder = _nameMatrix.begin();
if (itNameHolder != _nameMatrix.end()) {
@@ -113,6 +149,14 @@
}
}
+ if (_extraPlaceList.empty() == false) {
+ oStr << " " << _extraPlaceList.size() << " extra match(es)";
+ }
+
+ if (_alternatePlaceList.empty() == false) {
+ oStr << " " << _alternatePlaceList.size() << " alternate match(es)";
+ }
+
return oStr.str();
}
@@ -143,6 +187,8 @@
<< ", longitude = " << _longitude
<< ", latitude = " << _latitude
<< ", docID = " << _docID
+ << ", percentage = " << _percentage << "%"
+ << ", edit distance = " << _editDistance
<< std::endl;
return oStr.str();
}
@@ -215,7 +261,37 @@
// Copy the parameters from the Place object to the Location structure
Location oLocation (_placeCode, lCityCode, _stateCode, _countryCode,
_regionCode, _continentCode, _timeZoneGroup,
- _longitude, _latitude, lNameList);
+ _longitude, _latitude, lNameList,
+ _percentage, _editDistance);
+
+ // Add extra matching locations, whenever they exist
+ if (_extraPlaceList.empty() == false) {
+ for (PlaceOrderedList_T::const_iterator itLoc = _extraPlaceList.begin();
+ itLoc != _extraPlaceList.end(); ++itLoc) {
+ const Place* lExtraPlace_ptr = *itLoc;
+ assert (lExtraPlace_ptr != NULL);
+
+ // Add the extra matching location
+ const Location& lExtraLocation = lExtraPlace_ptr->createLocation();
+ oLocation.addExtraLocation (lExtraLocation);
+ }
+ }
+
+ // Add alternate matching locations, whenever they exist
+ if (_alternatePlaceList.empty() == false) {
+ for (PlaceOrderedList_T::const_iterator itLoc =
+ _alternatePlaceList.begin();
+ itLoc != _alternatePlaceList.end(); ++itLoc) {
+ const Place* lAlternatePlace_ptr = *itLoc;
+ assert (lAlternatePlace_ptr != NULL);
+
+ // Add the alternate matching location
+ const Location& lAlternateLocation =
+ lAlternatePlace_ptr->createLocation();
+ oLocation.addAlternateLocation (lAlternateLocation);
+ }
+ }
+
return oLocation;
}
}
Modified: trunk/opentrep/opentrep/bom/Place.hpp
===================================================================
--- trunk/opentrep/opentrep/bom/Place.hpp 2009-07-27 05:56:43 UTC (rev 170)
+++ trunk/opentrep/opentrep/bom/Place.hpp 2009-08-08 14:55:34 UTC (rev 171)
@@ -81,6 +81,16 @@
return _docID;
}
+ /** Get the matching percentage. */
+ const MatchingPercentage_T& getPercentage() const {
+ return _percentage;
+ }
+
+ /** Get the allowed edit distance/error. */
+ const NbOfErrors_T& getEditDistance() const {
+ return _editDistance;
+ }
+
/** Get the map of name lists. */
const NameMatrix_T& getNameMatrix () const {
return _nameMatrix;
@@ -156,6 +166,16 @@
_docID = iDocID;
}
+ /** Set the Xapian matching percentage. */
+ void setPercentage (const MatchingPercentage_T& iPercentage) {
+ _percentage = iPercentage;
+ }
+
+ /** Set the allowed edit distance/error. */
+ void setEditDistance (const NbOfErrors_T& iEditDistance) {
+ _editDistance = iEditDistance;
+ }
+
public:
// ////////// Setters in underlying names ////////
@@ -247,9 +267,16 @@
double _latitude;
/** List of names, for each given language. */
NameMatrix_T _nameMatrix;
+
/** Xapian document ID. */
XapianDocID_T _docID;
+ /** Matching percentage. */
+ MatchingPercentage_T _percentage;
+
+ /** Allowed edit error/distance. */
+ NbOfErrors_T _editDistance;
+
/** List of extra matching (similar) places. */
PlaceOrderedList_T _extraPlaceList;
Modified: trunk/opentrep/opentrep/bom/ResultHolder.cpp
===================================================================
--- trunk/opentrep/opentrep/bom/ResultHolder.cpp 2009-07-27 05:56:43 UTC (rev 170)
+++ trunk/opentrep/opentrep/bom/ResultHolder.cpp 2009-08-08 14:55:34 UTC (rev 171)
@@ -280,7 +280,7 @@
const NbOfMatches_T lNbOfMatches =
lMatchingDocument.notifyIfExtraMatch();
OPENTREP_LOG_DEBUG ("==> " << lNbOfMatches
- << " matches for the query string: `"
+ << " main matches for the query string: `"
<< lMatchedString << "' (from `"
<< lQueryString << "')");
Modified: trunk/opentrep/opentrep/bom/StringMatcher.cpp
===================================================================
--- trunk/opentrep/opentrep/bom/StringMatcher.cpp 2009-07-27 05:56:43 UTC (rev 170)
+++ trunk/opentrep/opentrep/bom/StringMatcher.cpp 2009-08-08 14:55:34 UTC (rev 171)
@@ -621,4 +621,21 @@
WordHolder::createStringFromWordList (lRemainingWordList);
}
+ // //////////////////////////////////////////////////////////////////////
+ std::string StringMatcher::getPlaceCode (const Xapian::Document& iDocument) {
+ // Retrieve the Xapian document data
+ const std::string& lDocumentData = iDocument.get_data();
+
+ // Tokenise the string into words
+ WordList_T lWordList;
+ WordHolder::tokeniseStringIntoWordList (lDocumentData, lWordList);
+ assert (lWordList.empty() == false);
+
+ // By convention (within OpenTrep), the first word of the Xapian
+ // document data string is the place code
+ const std::string& lPlaceCode = lWordList.front();
+
+ return lPlaceCode;
+ }
+
}
Modified: trunk/opentrep/opentrep/bom/StringMatcher.hpp
===================================================================
--- trunk/opentrep/opentrep/bom/StringMatcher.hpp 2009-07-27 05:56:43 UTC (rev 170)
+++ trunk/opentrep/opentrep/bom/StringMatcher.hpp 2009-08-08 14:55:34 UTC (rev 171)
@@ -15,6 +15,7 @@
namespace Xapian {
class MSet;
class Database;
+ class Document;
}
namespace OPENTREP {
@@ -24,6 +25,7 @@
for more information. */
class StringMatcher : public BomAbstract {
public:
+ // ///////////////////////////////////////////////
/** Search, within the Xapian database, for occurrences of the
words of the search string.
@param Xapian::MSet& The Xapian matching set. It can be empty.
@@ -63,6 +65,15 @@
static void
subtractParsedToRemaining (const std::string& iAlreadyParsedQueryString,
std::string& ioRemainingQueryString);
+
+
+ public:
+ // ///////////////////////////////////////////////
+ /** Extract the place code from the document data.
+ <br>The place code is the first 3-letter string of the Xapian
+ document data/content. */
+ static std::string getPlaceCode (const Xapian::Document&);
+
};
}
Modified: trunk/opentrep/opentrep/command/DBManager.cpp
===================================================================
--- trunk/opentrep/opentrep/command/DBManager.cpp 2009-07-27 05:56:43 UTC (rev 170)
+++ trunk/opentrep/opentrep/command/DBManager.cpp 2009-08-08 14:55:34 UTC (rev 171)
@@ -60,6 +60,52 @@
// //////////////////////////////////////////////////////////////////////
void DBManager::
+ prepareSelectOnPlaceCodeStatement (soci::session& ioSociSession,
+ soci::statement& ioSelectStatement,
+ const std::string& iPlaceCode,
+ Place& ioPlace) {
+
+ try {
+
+ // Instanciate a SQL statement (no request is performed at that stage)
+ /**
+ select rpd.code AS code, city_code, xapian_docid, is_airport, is_city,
+ is_main, is_commercial, state_code, country_code, region_code,
+ continent_code, time_zone_grp, longitude, latitude, language_code,
+ classical_name, extended_name, alternate_name1, alternate_name2,
+ alternate_name3, alternate_name4, alternate_name5, alternate_name6,
+ alternate_name7, alternate_name8, alternate_name9, alternate_name10
+ from ref_place_details rpd, ref_place_names rpn
+ where rpd.code = iPlaceCode
+ and rpn.code = rpd.code;
+ */
+
+ ioSelectStatement =
+ (ioSociSession.prepare
+ << "select rpd.code AS code, city_code, xapian_docid, is_airport, "
+ << "is_city, is_main, is_commercial, state_code, country_code, "
+ << "region_code, continent_code, time_zone_grp, longitude, latitude, "
+ << "language_code, classical_name, extended_name, "
+ << "alternate_name1, alternate_name2, alternate_name3, "
+ << "alternate_name4, alternate_name5, alternate_name6, "
+ << "alternate_name7, alternate_name8, alternate_name9, "
+ << "alternate_name10 "
+ << "from ref_place_details rpd, ref_place_names rpn "
+ << "where rpd.code = :place_code "
+ << "and rpn.code = rpd.code",
+ soci::into (ioPlace), soci::use (iPlaceCode));
+
+ // Execute the SQL query
+ ioSelectStatement.execute();
+
+ } catch (std::exception const& lException) {
+ OPENTREP_LOG_ERROR ("Error: " << lException.what());
+ throw SQLDatabaseException();
+ }
+ }
+
+ // //////////////////////////////////////////////////////////////////////
+ void DBManager::
prepareSelectOnDocIDStatement (soci::session& ioSociSession,
soci::statement& ioSelectStatement,
const XapianDocID_T& iDocID,
@@ -164,7 +210,7 @@
// //////////////////////////////////////////////////////////////////////
bool DBManager::retrievePlace (soci::session& ioSociSession,
- const XapianDocID_T& iDocID,
+ const std::string& iPlaceCode,
Place& ioPlace) {
bool oHasRetrievedPlace = false;
@@ -172,8 +218,9 @@
// Prepare the SQL request corresponding to the select statement
soci::statement lSelectStatement (ioSociSession);
- DBManager::prepareSelectOnDocIDStatement (ioSociSession, lSelectStatement,
- iDocID, ioPlace);
+ DBManager::prepareSelectOnPlaceCodeStatement (ioSociSession,
+ lSelectStatement,
+ iPlaceCode, ioPlace);
const bool shouldDoReset = true;
bool hasStillData = iterateOnStatement (lSelectStatement, ioPlace,
shouldDoReset);
Modified: trunk/opentrep/opentrep/command/DBManager.hpp
===================================================================
--- trunk/opentrep/opentrep/command/DBManager.hpp 2009-07-27 05:56:43 UTC (rev 170)
+++ trunk/opentrep/opentrep/command/DBManager.hpp 2009-08-08 14:55:34 UTC (rev 171)
@@ -22,28 +22,41 @@
from the database. */
class DBManager {
public:
+ /** Update the Xapian document ID field of the database row
+ corresponding to the given Place object. */
+ static void updatePlaceInDB (soci::session&, const Place&);
+
+ /** Retrieve, from the (MySQL) database, the row corresponding to
+ the given place code (e.g., 'sfo' for San Francisco Intl
+ airport), and fill the given Place object with that retrieved
+ data. */
+ static bool retrievePlace (soci::session&, const std::string& iPlaceCode,
+ Place&);
+
+
+ public:
/** Prepare (parse and put in cache) the SQL statement. */
static void prepareSelectStatement (soci::session&, soci::statement&,
Place&);
- /** Prepare (parse and put in cache) the SQL statement. */
- static void prepareSelectOnDocIDStatement (soci::session&, soci::statement&,
- const XapianDocID_T&, Place&);
-
/** Iterate on the SQL statement.
<br>The SQL has to be already prepared.
@parameter const bool Tells whether the Place object should be reset. */
static bool iterateOnStatement (soci::statement&, Place&,
const bool iShouldDoReset);
- /** Update the Xapian document ID field of the database row
- corresponding to the given Place object. */
- static void updatePlaceInDB (soci::session&, const Place&);
+
+ private:
+ /** Prepare (parse and put in cache) the SQL statement. */
+ static void prepareSelectOnPlaceCodeStatement(soci::session&,
+ soci::statement&,
+ const std::string& iPlaceCode,
+ Place&);
+
+ /** Prepare (parse and put in cache) the SQL statement. */
+ static void prepareSelectOnDocIDStatement (soci::session&, soci::statement&,
+ const XapianDocID_T&, Place&);
- /** Retrieve, from the (MySQL) database, the row corresponding to the
- given Xapian Document ID, and fill the given Place object with
- that retrieved data. */
- static bool retrievePlace (soci::session&, const XapianDocID_T&, Place&);
private:
/** Constructors. */
Modified: trunk/opentrep/opentrep/command/IndexBuilder.cpp
===================================================================
--- trunk/opentrep/opentrep/command/IndexBuilder.cpp 2009-07-27 05:56:43 UTC (rev 170)
+++ trunk/opentrep/opentrep/command/IndexBuilder.cpp 2009-08-08 14:55:34 UTC (rev 171)
@@ -170,16 +170,10 @@
lPlace,
shouldDoReset);
while (hasStillData == true) {
- // Add the document corresponding to the Place object to the
+ // Add the document, corresponding to the Place object, to the
// Xapian index
IndexBuilder::addDocumentToIndex (lDatabase, lPlace);
- // Update the row in (MySQL) database for the given Place object:
- // The Xapian document ID is generated by Xapian when inserting
- // the document into the index; that document ID has to be updated
- // in the (MySQL) database.
- DBManager::updatePlaceInDB (ioSociSession, lPlace);
-
// DEBUG
OPENTREP_LOG_DEBUG ("[" << idx << "] " << lPlace);
Modified: trunk/opentrep/opentrep/command/RequestInterpreter.cpp
===================================================================
--- trunk/opentrep/opentrep/command/RequestInterpreter.cpp 2009-07-27 05:56:43 UTC (rev 170)
+++ trunk/opentrep/opentrep/command/RequestInterpreter.cpp 2009-08-08 14:55:34 UTC (rev 171)
@@ -14,6 +14,7 @@
#include <opentrep/bom/ResultHolder.hpp>
#include <opentrep/bom/Result.hpp>
#include <opentrep/bom/PlaceHolder.hpp>
+#include <opentrep/bom/StringMatcher.hpp>
#include <opentrep/factory/FacPlaceHolder.hpp>
#include <opentrep/factory/FacPlace.hpp>
#include <opentrep/factory/FacResultHolder.hpp>
@@ -59,58 +60,171 @@
<< "========================================="
<< std::endl << std::endl);
}
+
+ /** Helper function. */
+ // //////////////////////////////////////////////////////////////////////
+ bool retrieveAndFillPlace (const Xapian::Document& iDocument,
+ const Xapian::percent& iDocPercentage,
+ soci::session& ioSociSession, Place& ioPlace) {
+ bool hasRetrievedPlace = false;
+
+ // Set the matching percentage
+ ioPlace.setPercentage (iDocPercentage);
+
+ // Retrieve the parameters of the best matching document
+ const std::string& lPlaceCode = StringMatcher::getPlaceCode (iDocument);
+
+ // DEBUG
+ const Xapian::docid& lDocID = iDocument.get_docid();
+ const std::string& lDocData = iDocument.get_data();
+ OPENTREP_LOG_DEBUG ("Place code: " << lPlaceCode << " - Document ID "
+ << lDocID << ", " << iDocPercentage
+ << "% [" << lDocData << "]");
+
+ // Fill the Place object with the row retrieved from the
+ // (MySQL) database and corresponding to the given place code
+ // (e.g., 'sfo' for the San Francisco Intl airport).
+ hasRetrievedPlace = DBManager::retrievePlace (ioSociSession, lPlaceCode,
+ ioPlace);
+
+ if (hasRetrievedPlace == false) {
+ /**
+ The Xapian database/index should contain only places
+ available within the SQL database, as the first is built from
+ the latter. If that happens, it means that the user gave a
+ wrong Xapian database.
+ */
+ OPENTREP_LOG_ERROR ("There is no document corresponding to "
+ << lPlaceCode << " (Xapian document ID" << lDocID
+ << " [" << lDocData << "]) in the SQL database. "
+ << "It usually means that the Xapian index/database "
+ << "is not synchronised with the SQL database. "
+ << "[Hint] Rebuild the Xapian index/database "
+ << "from the SQL database.");
+ throw XapianTravelDatabaseNotInSyncWithSQLDatabaseException();
+ }
+
+ return hasRetrievedPlace;
+ }
+ /** Helper function. */
// //////////////////////////////////////////////////////////////////////
+ bool retrieveAndFillPlace (const Document& iDocument,
+ soci::session& ioSociSession, Place& ioPlace) {
+ // Delegate
+ const Xapian::Document& lXapianDocument = iDocument.getXapianDocument();
+ const Xapian::percent& lDocPercentage = iDocument.getXapianPercentage();
+ return retrieveAndFillPlace (lXapianDocument, lDocPercentage,
+ ioSociSession, ioPlace);
+ }
+
+ // //////////////////////////////////////////////////////////////////////
void createPlaces (const ResultHolder& iResultHolder,
soci::session& ioSociSession, PlaceHolder& ioPlaceHolder) {
- // Browse the list of result objects
- const ResultList_T& lResultList = iResultHolder.getResultList();
- for (ResultList_T::const_iterator itResult = lResultList.begin();
- itResult != lResultList.end(); ++itResult) {
- // Retrieve the result object
- const Result* lResult_ptr = *itResult;
- assert (lResult_ptr != NULL);
+ // Browse the list of result objects
+ const ResultList_T& lResultList = iResultHolder.getResultList();
+ for (ResultList_T::const_iterator itResult = lResultList.begin();
+ itResult != lResultList.end(); ++itResult) {
+ // Retrieve the result object
+ const Result* lResult_ptr = *itResult;
+ assert (lResult_ptr != NULL);
- /**
- TODO: Add a loop for retrieving both extra and alternate Documents
- Use FacPlace::initLinkWithExtraPlace() and
- FacPlace::initLinkWithAlternatePlace()
- */
+ // Retrieve the matching document
+ const Document& lDocument = lResult_ptr->getMatchingDocument();
+
+ // Instanciate an empty place object, which will be filled from the
+ // rows retrieved from the database.
+ Place& lPlace = FacPlace::instance().create();
+
+ // Retrieve, in the MySQL database, the place corresponding to
+ // the place code located as the first word of the Xapian
+ // document data.
+ bool hasRetrievedPlace = retrieveAndFillPlace (lDocument, ioSociSession,
+ lPlace);
+ // If there was no place corresponding to the place code with
+ // the SQL database, an exception is thrown. Hence, here, by
+ // construction, the place has been retrieved from the SQL
+ // database.
+ assert (hasRetrievedPlace == true);
+
+ // Insert the Place object within the PlaceHolder object
+ FacPlaceHolder::initLinkWithPlace (ioPlaceHolder, lPlace);
+
+ // DEBUG
+ OPENTREP_LOG_DEBUG ("Retrieved Document: " << lPlace.toString());
+
+ // Retrieve the list of extra matching documents (documents
+ // matching with the same weight/percentage)
+ const Xapian::percent& lExtraDocPercentage =
+ lDocument.getXapianPercentage();
+ const XapianDocumentList_T& lExtraDocumentList =
+ lDocument.getExtraDocumentList();
+ for (XapianDocumentList_T::const_iterator itExtraDoc =
+ lExtraDocumentList.begin();
+ itExtraDoc != lExtraDocumentList.end(); ++itExtraDoc) {
+ // Retrieve the extra matching Xapian document
+ const Xapian::Document& lExtraDocument = *itExtraDoc;
- // Retrieve the parameters of the best matching document
- const Xapian::Document& lDocument = lResult_ptr->getXapianDocument();
- const Xapian::percent& lDocPercentage =
- lResult_ptr->getXapianPercentage();
- const Xapian::docid& lDocID = lDocument.get_docid();
- const std::string& lDocData = lDocument.get_data();
-
+ // Instanciate an empty place object, which will be filled from the
+ // ...
[truncated message content] |
