Enhance CQP with custom SpheroscopeDebug option (experimental)
Make sure cwb-encode aborts with meaningful error message if there are unparseable arguments
Encourage users to upgrade to v3.5.1
Fix bug in cl_string_canoncial(), which would fail on diacritic-folding of long strings (where NFC exceeds CL_MAX_LINE_LENGTH)
A kwic concordance (where left and right context might not even contain complete tokens!) is very different from a list of sentences with pre-determined annotation as in the FreeLing output. I don't think we can learn much from it to help us address the challenges of kwic XML output. SGML print mode is really badly broken if you display s-attributes in the concordance. It also includes them (and any p-attributes) as plain text in the tokens rather than in a way that allows them to be processed e.g....
You seem to have forgotten to activate the corpus: info PRUEBA; but PRUEBA; show cd;
Note to those not familiar with CQP print modes: Their implementation is a horrible mess, so we are reluctant to add extensions and very limited in what can be achieved. Moreover, the print modes only affect some CQP output (kwic concordances, frequency tables from group) but by far not all.
XML output mode for CQP
Because that's how the orginal developer decided to do things in 1994. It's a quirk that we live with for the sake of backwards compatibility. Note that the filename of the registry file also has to be in lowercase, while corpus IDs are to be specified in all caps everywhere else. You can get the list of attributes with show cd or using cwb-describe-corpus -s on the command line. Canonical attribute names (both positional and structural) should be all lowercase and only use ASCII characters. While...
SGML invalid structure
PS: If you want to pursue this, please add a feature request “XML output mode for CQP” for CWB v3.6.
Not a bug: SGML allows omission of closing tags – you just have to assume a suitable DTD for the output produced by CQP. Note that your second suggestion isn't valid SGML and would have to be written <attribute// instead. If your SGML output also included the kwic line with some s-attributes shown, you'd get many more validation errors (because nothing guarantees that open/close tags match up within a kwic line, and they can also overlap between context and match). If we ever find the nerves to implement...
cwb-scan-corpus now also reports type count before applying frequency filter
Ziggurat design: even more B-tree algorithms
Ziggurat design: added binsearch benchmark observations with Rust on MacOS
Ziggurat design: try yet another binary search algo (unsuccessfully)
Ziggurat design: fix uint typedef conflict (now zuint)
Added binary search benchmarks for random walks and exponential search algorithm
Convert README to Markdown + HTML for easier reading
Ziggurat design: estimate disk size of compressed sparse inverted index
Add HTML version of Markdown README for convenience
Ziggurat design: benchmark results for binary search in sort index, with thorough discussion
Ziggurat design: benchmark binary lookup in large tables vs. b-tree
Ziggurat design: estimate size of sparse inverted index
Fix extremely embarrassing as well as catastrophic bug in cwb-scan-corpus introduced by r1851
minor display fix in cwb-scan-corpus
cwb-scan-corpus now obtains total token/document counts if no regular keys are specified
Re. 3: If you installed this particular corpus in CQPweb, you must have ignored all the error messages that it shot at you. It should have outright refused to install the corpus, but perhaps it went far enough to get its database into an inconsistent state that causes the lock-up.
(a) Does this happen for a specific query, or for every query that returns some matches? In the latter case there' s probably sth in the corpus that confuses CQPweb. (b) Are there also problems if you run the query directly in CQP? (c) You can't possibly have installed this corpus in CQPweb because it's lacking the mandatory <text id="..."> elements!
(a) Does this happen for a specific query, or for every query that returns some matches? In the latter case there' s probably sth in the corpus that confuses CQPweb. (b) Are there also problems if you run the query directly in CQP? (c) You can't possibly have installed this corpus in CQPweb because it's lacking the mandatory <text id="..."> ... </text> elements!
(a) Does this happen for a specific query, or for every query that returns some matches? In the latter case there' s probably sth in the corpus that confuses CQPweb. (b) Are there also problems if you run the query directly in CQP? (c) You can't possibly have installed this corpus in CQPweb because it's lacking the mandatory <text id="..."> elements!</text>
(a) Does this happen for a specific query, or for every query that returns some matches? In the latter case there' s probably sth in the corpus that confuses CQPweb. (b) Are there also problems if you run the query directly in CQP?
cwb-ccc should give you most (if not all) of what you can get from the CQPweb API, and it will be faster (as everything is directly in Python and doesn't need to be serialised and de-serialised) and more directly under your control. I think CQPweb is useful for your use case only if you need its GUI.
Best approach for interfacing CQP with other software
We don't currently have plans to continue the development of CQi, though cqpserver and the Perl client are still fully functional. CQi doesn't wrap all relevant CQP functionality yet and has a few suboptimal design choices, and currently there doesn't seem to be much demand for a client-server API of this type. Direct APIs such as the ones available for Python, Perl and R are more efficient and convenient to use. As a point of interest, the R API (RCppCWB) is modeled after CQi, but accesses the internal...
Since you're working in Python, you should use the native API provided by cwb-ccc package. It is well-maintained and up to date, but doesn't support Windows platforms yet (help wanted!).
Merge bugfix r1839 into branches/3.5
Fix undefined behaviour / segmentation fault of cwb-scan-corpus if more than 32 keys are specified.
Document handling of empty element tags in Corpus Encoding Manual
Fix bug in r1836
Make cwb-encode silently discard empty element tags for s-attributes with nesting specifiers
Merge bugfix r1826 into branches/3.5
Create CWB release/3.5.0 as stable archive w/o any bug fixes added to branches/3.5 after release.
Downgrade CWB trunk to v3.5.1 rather than v3.6
Varint benchmark for batch processing (inlined loop) + results for M1Max
Update varint encoding benchmarks:
First bug fix after release of v3.5.0 :-}
Repair last changes to CWB::CL configuration script
Improve CWB::CL makefile
A few final bugfixes
Correct typo in CWB/Perl test
Platform configuration for MacOS binary release based on HomeBrew
Typo in config/platform/darwin-release
Update INSTALL and INSTALL-MACOS with instructiosn for building self-contained binary releases.
More work on self-contained binary release package for MacOS
New cwb-config Perl script for self-contained binary packages; remove install-cwb.sh
First attempt to build self-contained package for MacOS (work in progress)
Same for hombrew-formula configuration
Change linker flags to -Xlinker notation, which seems most widely supported, and remove the special case on MacOS
Use correct synatx for -Wl,-rpath flag (stupid thinko)
Minor fix to uninstall-cwb.perl
Uninstall script for CWB (e.g. to switch from locally installed version to Debian package)
Proper bugfix for CL regexp optimizer, so grains aren't incorrectly anchored at end of string
Explain in README how to test CQP with set Optimize on; or other options
Some more regression tests for regexp optimizer
Ignore some local files in CWB source tree (svn propedit svn:ignore .)
Enable dynamic linking on MacOS by default
Some further cleanup of config files
Amend previous checkin
Improve dynamic linking in homebrew formula (experimental)
Fix rpath problem in homebrew on MacOS (but may persist on Linux?)
Enable shared linking in homebrew formula (experimental)
Get dynamic linking to work properly on MacOS
Improve Makefile for automated Perl module testing
Make CWB-Web package available on CPAN (needs dummy CWB::Web module)
Clean up beta-install config; enable optional "local_config.mk" for developers
Fix .dylib building on MacOS
Forgot to save a file before last check-in
Re-enable regular expression optimizer with cl_optimize=1;, which had been disabled due to a stupid bug.
Fix segfault in cwb-s-encode (using corpus handle to get charset when not adding to existing corpus)
Remove a few left-over references to editline/ from config
Finally switch config.mk to default to standard installation rather than separate beta tree
"set RightContext n character" is off by 1
CQPweb: Collocation analysis with POS filter is inconsistent
CQPweb: Collocation analysis with POS filter is inconsistent
Fix bug #76: token separator next to match included in RightContext character count
Reason: the token separator between match and co-text is included in the character count for the right context size, but excluded from the count for the left context size. Seems wrong to me.
CQP can save a named query results file larger than 2 GiB but cannot read it
CQP can save a named query results file larger than 2 GiB but cannot read it
Thanks for the report and thorough diagnosis. Should be fixed in r1708.
Bug fix: CQP fails to load saved query results if file size is > 2 GiB
Inconsistency in CQP regexp matching
Fixed in r1705. The macro CL_REGEX_METACHARACTERS is defined in cl/cl.h (and documentation explains that it's not a list of all "unsafe" characters despite the name).
Make sure that CQP always matches search strings that contain ^ or $ as regular expressions
Inconsistency in CQP regexp matching
Perl/CWB-CQi: v3.0.0 for CPAN release
(amend previous check-in: forgot one file)
Perl/CWB-Web: v3.4.0 for CPAN release
Perl/CWB-CL: v3.4.33 for CPAN release; update author name
(amend previous check-in: forgot one file)