eyedb-general — General discussion list for EyeDB users

[eyedb-general] Sadly, EyeDB no longer build under openSuSE 13.2 ...

[Michael G. H. <mic...@gm...>]

I guess this is an indication that EyeDB is getting long in the tooth and
need a little attention.

I can no longer build it under openSuSE 13.2 --I can still build the system
under openSusSE 13.1 and prior versions.

I guess it it time to start taking care of my much beloved database and
move into this decade technology.

The first efforts will be to make EyeDB compile using "cmake" then work on
bring it in to the world of C++11 through C++17

When I get the EyeDB compiling using cmake perhaps some of you will want
join me in updating the EyeDB system with C++17.... perhaps adding some
nice GUI front end and addressing some big data usage for EyeDB.

This effort will probably be a multi-year effort, hopefully ending in 2017.

Later
Michael
-- 
*"...Hope is what makes us strong. It is why we are here. It is what we
fight with when all is lost..."*

[eyedb-general] Big Data System; lessons in reality.

[Michael G. H. <mic...@gm...>]

To those of you who think that EyeDB is not usefully BigData capture.

Big Data System; lessons in reality.
Summary: (see below what is included in all markets)
  *Average 7800 tics/second, with burst to more than 18000 tics/second
insertion rate of everything from all the markets NOTE: The results of the
testing  done on an AMD Phenom II X6 1075t with 8 GB RAM. Substantial
performance is to be expected on a more modern system.

   *Referential integrity between tics/fields and symbol; maintained on the
fly.

   *Referential integrity between tics/fields and symbol and time of
occurrence; maintained on the fly.

    *Referential integrity between symbols and all unformulated text data
--higher resolution is possible. eg the text information could be
categorized. Perhaps the duties of data mining algorithms.

   *Referential integrity between symbols and the time of occurrence;
maintained on the fly

   *All tic information (such as bid and ask) is being stored not just the
trade information.

   *Storing all tic information at about 50% of rate of all information
coming from all the markets (see below for the list of all markets). So
that should mean we can store all equity tics in real time should we filter
on NYSE, NASDAQ, etc. NOTE: The results of the testing  was done on an AMD
Phenom II X6 1075t with 8 GB RAM. Substantial performance is to be expected
on a more modern system.

    *The disk space use by OODBMS, to every 512MB of raw market tic data,
from the server is 7.3GB. NOTE: 512MD of raw market-data becomes about 1GB
of human readable text.

The project:
    With 8 to 9 gigabyte of market-data pouring in everyday I needed a
place to store the data so that it can be easily analyze, with the
additional consideration, to  directly feed the market-data into the
database in real-time. Those are initial parameters for the Big Data system
I envisioned.

   For a number of reasons I chose the EyeDB (www.eyedb.org), an Object
Oriented Database Management System (OODBMS) based on the ODMG 3
specification. IMHO the advantages of the EyeDB system cannot be
understated in comparison to Relational database management system (RDBMS)
such as SYBASE, ORACLE etc; OQL alone is an absolute pleasure when compared
to plain SQL and PL --I cannot speak to other DBMS such as Mongo DB and it
like since I have limited exposure to them.

   With the database chosen I designed a wonderful set of objects/classes
describing market-data that could come from various markets and sources. I
even made allowances for free form text information right down to the tic
of a particular symbol/company/option etc. --there was always the
possibility firing off triggers on patterns of of text.

   As, a general rule, I have developed over the many years of practicing
object oriented development, if you design a good set of objects with
linear relationships, performance almost always a side effect; especially
when using a language like C++. So I had very high expectation for my
market-data delivery system. And those expectation were being met for the
first several hundred thousand tic/market data force feeding the EyeDB
OODBMS.

Many developer suffer from the symptom of not reading the manual of
potential complex systems such as EyeDB OODBMS --luckily for me EyeDB isn't
as complex as SYBASE or an  ORACLE systems to manage or use. But, there was
one particular bit of information I should have paid particular attention
to that was found in the EyeDB administrator manual under the topic
database turnings "..As EyeDB file access relies heavily on virtual memory
and file mapping, it is mandatory to either unset these (memory) limits or
set them to a high value..." -- in my case unlimited using the "ulimit"
command.

EyeDB is extremely fast and efficient database and will request more and
more memory resources as needed. So, as I stuffed several hundred thousand
tic/market EyeDB was maintaining referential integrity between: all symbol
and there fields, all symbols and the time of occurrence of their fields
within a market. Additional, EyeDB for this design maintaining  at a
minimum symbol index, tic time index since it would likely we would be
querying the system for symbols and their fields that occurred between a
range of times through a period of time or simply wanted to study field
delivery patterns.

As I stuffed the data into the EyeDB OODBMS my application it eventually
hit the memory/RAM and virtual limits of the system I was running EyeDB
application and then started an extremely slow grind as it was now working
totally as best as it could from disk. I was actually surprised that EyeDB
had pretty good performance as it entered virtual memory and maintained
relatively good performance until the LINUX swap space maxed out.

The hardware running EyeDB OODBMS is a AMD Phenom II X6, with 8 gigabyte
system with 2 terabyte hard disk. It seemed unacceptable to me not being
able load on this system with potentially tens/hundreds/thousands of
billions of object into an EyeDB OODBMS.

After much thought I decided on an approach that would not compromise any
of my initial design requirements for the system with respect to how and
what the user can access from the OODBMS and eventually kept my application
from using any more than 17% of memory.

Sorry guys I was told that this technique is an extremely valuable (and I
could really use the money right now) in pushing market-data into the
database in real-time --this is valuable since there are companies making
millions selling real-time market-data capture systems. So I won't be
providing any further details about the technique. But I'm guessing you
smart guys already have an idea of how I got such a dramatic improvements
in performances without resorting to buying more hardware like big
organization typically do.

I do believe this technique will work on many other forms of big data. But
as I said at the start of this article this was lesson and a reality check.

Now, besides the normal data mining technique I need to discover what type
of Analytics traders and other users market-data  would want to see.

Do feel free to contact me for a real-time demo if you happen to be in the
 Charlotte NC

Later from this side of the peanut gallery ;)




I, the unwilling, was lead by the unqualified, to do the unbelievable for
so long with so little, that I attempted the impossible with nothing

Sent from my iPad



-- 
*"...Hope is what makes us strong. It is why we are here. It is what we
fight with when all is lost..."*

Re: [eyedb-general] NOTE: edbodl --gencode=C++, server side methods; incorrect instructions given

[Michael G. H. <mic...@gm...>]

I am am using and have been using openSuSE for many years with eyeDB. I am
currently using openSuSE 12.2 and notice that things are not building as
expected. In a recent build on my development machine the build would stop
in the middle of making the documentation I had to manually answer yes/no
and directory to put the documentation during the make... On my clean
distribution of openSuSE 12.2 there were no issues on the build and
install... so it was something nonstandard on my development box.

That been said, I think automake and tools are getting a bit long in the
tooth  and since there have been a number or updates to the tool chain
that,  "I have heard/read causing grief for people whose automake script
have not been update", I think someone, should either revisit and update
the existing script.

I personally was thinking of moving the build process over to "cmake"; it
is something I will consider tackelling about two months from now when I
free up sometime.

Michael





On Fri, Jan 31, 2014 at 12:50 AM, luca paganotti
<luc...@gm...>wrote:

> Hi Michael, not so great issues with server side methods, I'm fighting
> with eyedb build, as I recently switch from fedora 18 to 19, something is
> broken in the build process and eyedbodl is not able anymore to process
> .odl files without the --nocpp flag. At least, for the time being, this
> seems the most probable cause ...
>
>
>
>
>
> On Fri, Jan 31, 2014 at 1:20 AM, Michael George Hart <
> mic...@gm...> wrote:
>
>> Hi Luca,
>>   I saw that you were having so issues with server side methods perhaps
>> this could help.
>>
>> BTW, where do I look for more substantial examples of server side method
>> calls?
>>
>> Thanks
>>   Michael.
>>
>> I, the unwilling, was lead by the unqualified, to do the unbelievable for
>> so long with so little, that I attempted the impossible with nothing
>>
>> Sent from my iPad
>>
>> Begin forwarded message:
>>
>> *From:* Michael George Hart <mic...@gm...>
>> *Date:* January 30, 2014 at 6:33:41 PM EST
>> *To:* eye...@li..., luca paganotti <
>> luc...@gm...>, "eye...@li..." <
>> eye...@li...>
>> *Subject:* *NOTE: edbodl --gencode=C++, server side methods; incorrect
>> instructions given*
>>
>> After running
>>     edbodl --gencode=C++ --package=BcBinary binary.odl
>>
>> The instruction given in the server side skeleton code says
>> "....
>> #4. Copy the shared library to the eyedb loadable library directory
>> cp BcBinarymthbe-2.8.8.so <eyedbinstalldir>/lib/eyedb
>>
>> #5. Change the file access mode
>> chmod a+r <eyedbinstalldir>/lib/eyedb/BcBinarymthbe-2.8.8.so"
>>
>> That instruction is incorrect
>> EyeDB expect the shared file to be "binarymthbe-2.8.8.so"
>>
>> Notice the share library has no "lib" prefix and it is in all lower case.
>> It also seem important that version of EyeDB is posfix as see here to be
>> "-2.8.8"
>>
>>  Additionally the chmod should be chmod a+r and chmod a+x
>>
>>
>>
>>
>> --
>> *"...Hope is what makes us strong. It is why we are here. It is what we
>> fight with when all is lost..."*
>>
>>
>


-- 
*"...Hope is what makes us strong. It is why we are here. It is what we
fight with when all is lost..."*

[eyedb-general] NOTE: edbodl --gencode=C++, server side methods; incorrect instructions given

[Michael G. H. <mic...@gm...>]

After running
    edbodl --gencode=C++ --package=BcBinary binary.odl

The instruction given in the server side skeleton code says
"....
#4. Copy the shared library to the eyedb loadable library directory
cp BcBinarymthbe-2.8.8.so <eyedbinstalldir>/lib/eyedb

#5. Change the file access mode
chmod a+r <eyedbinstalldir>/lib/eyedb/BcBinarymthbe-2.8.8.so"

That instruction is incorrect
EyeDB expect the shared file to be "binarymthbe-2.8.8.so"

Notice the share library has no "lib" prefix and it is in all lower case.
It also seem important that version of EyeDB is posfix as see here to be
"-2.8.8"

Additionally the chmod should be chmod a+r and chmod a+x




-- 
*"...Hope is what makes us strong. It is why we are here. It is what we
fight with when all is lost..."*

[eyedb-general] using cmake for you EyeDB build: A basic CMakeLists.txt

[Michael G. H. <mic...@gm...>]

I use kdevelop 4.x for all my C++ application development and I have come
to like very much the simplicity of cmake's CMakeLists.txt file. So I share
with you a basic CMakeLists.txt for EyeDB builds

CMakeLists.txt

cmake_minimum_required(VERSION 2.8)

project(<YOUR PROJECT NAME>)


set(CMAKE_CXX_FLAGS "-fPIC -std=c++11 -D_REENTRANT -D_LARGEFILE_SOURCE
-D_FILE_OFFSET_BITS=64")

#set(CMAKE_VERBOSE_MAKEFILE ON)

find_package(KDeclarative QUIET)

include_directories(BEFORE
                            <INSTALLED PATH TO EYEDB>/include
                            <ADDITIONAL SPACE INCLUDES USED BY YOUR
APPLICATION>)

link_directories(<INSTALLED PATH TO EYEDB>/lib64
                       <ADDITIONAL SPACE LIBRARY PATHS USED BY YOUR
APPLICATION>)

add_executable(<YOUR PROJECT NAME>
                        <SPACE SEPARATED C++ PROJECT FILES>
                        <THE TWO FILES; .h AND .cc GENERATED BY eyedbodl
--gencode=C++ --package=DTN dtn.odl>
)


target_link_libraries(<YOUR PROJECT NAME>
                             <SPACE SEPARATED LIBRARIES USED BY YOUR
APPLICATION>
                             eyedb eyedbrpcfe eyedbsm eyedbutils)


install(TARGETS <YOUR PROJECT NAME> RUNTIME DESTINATION bin)


-- 
*"...Hope is what makes us strong. It is why we are here. It is what we
fight with when all is lost..."*

[eyedb-general] Is EyeDB as a place for big data: EyeDB crashes when inserting data from multiple sources...

[Michael G. H. <mic...@gm...>]

I have several EyeDB databases since I have several sources of market data

I start several different process; where each deliver data exclusively to
their own assigned database.

All but one process crashes. The one that does not crash continues on;
doing what it supposes to do without incident.

Hopefully someone can explain the crashing of the other processes. The
crashing processes basically give the same dump to my screen

29*** glibc detected *** eyedb.copy.from.passThru: free(): invalid pointer:
0x00000000248c5710 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x78b66)[0x7f5452532b66]
/usr/local/eyedb/lib64/libeyedb-2.8.8.so
(_ZN5eyedb4DateC2EPKNS_6StructENS_4BoolE+0xa4)[0x7f5453a21f44]
/usr/local/eyedb/lib64/libeyedb-2.8.8.so
(_ZN5eyedb14Date_constructEPKNS_6ObjectENS_4BoolE+0x34)[0x7f5453a21fa4]
/usr/local/eyedb/lib64/libeyedb-2.8.8.so
(_ZN5eyedb15utilsMakeObjectEPNS_6ObjectENS_4BoolE+0x64)[0x7f5453a24994]
/usr/local/eyedb/lib64/libeyedb-2.8.8.so
(_ZN5eyedb9utilsMakeEPPNS_6ObjectE+0x11)[0x7f54539284d1]
/usr/local/eyedb/lib64/libeyedb-2.8.8.so
(_ZNK5eyedb11StructClass6newObjEPhNS_4BoolE+0xe6)[0x7f54539db8c6]
/usr/local/eyedb/lib64/libeyedb-2.8.8.so
(_ZNK5eyedb10AttrDirect13newObjRealizeEPNS_6ObjectE+0xb1)[0x7f5453876761]
/usr/local/eyedb/lib64/libeyedb-2.8.8.so
(_ZNK5eyedb12AgregatClass13newObjRealizeEPNS_6ObjectE+0x37)[0x7f54538675d7]
eyedb.copy.from.passThru(_ZN10date_field10initializeEPN5eyedb8DatabaseE+0x92)[0x451d22]
eyedb.copy.from.passThru(_ZN10date_fieldC1EPN5eyedb8DatabaseEPKNS0_9DataspaceE+0x32)[0x451d82]
eyedb.copy.from.passThru(_ZN8passthru13eyeDB_visitorclERKNS_11market_dateE+0x39)[0x43c4a9]
eyedb.copy.from.passThru(main+0x1179)[0x434059]
/lib64/libc.so.6(__libc_start_main+0xf5)[0x7f54524db455]
eyedb.copy.from.passThru[0x434b41]
======= Memory map: ========
00400000-00478000 r-xp 00000000 08:02 922247
/usr/local/bin/eyedb.copy.from.passThru
00677000-00683000 r--p 00077000 08:02 922247
/usr/local/bin/eyedb.copy.from.passThru
00683000-00684000 rw-p 00083000 08:02 922247
/usr/local/bin/eyedb.copy.from.passThru
0119f000-248d1000 rw-p 00000000 00:00 0
 [heap]
7f54517c8000-7f54517cd000 rw-p 00000000 00:00 0
7f54517cd000-7f54517f6000 r-xp 00000000 08:02 526370
/lib64/libtinfo.so.5.9
7f54517f6000-7f54519f5000 ---p 00029000 08:02 526370
/lib64/libtinfo.so.5.9
7f54519f5000-7f54519f9000 r--p 00028000 08:02 526370
/lib64/libtinfo.so.5.9
7f54519f9000-7f54519fe000 rw-p 0002c000 08:02 526370
/lib64/libtinfo.so.5.9
7f54519fe000-7f54519ff000 rw-p 00000000 00:00 0
7f54519ff000-7f5451a3d000 r-xp 00000000 08:02 525573
/lib64/libreadline.so.6.2
7f5451a3d000-7f5451c3d000 ---p 0003e000 08:02 525573
/lib64/libreadline.so.6.2
7f5451c3d000-7f5451c3f000 r--p 0003e000 08:02 525573
/lib64/libreadline.so.6.2
7f5451c3f000-7f5451c45000 rw-p 00040000 08:02 525573
/lib64/libreadline.so.6.2
7f5451c45000-7f5451c47000 rw-p 00000000 00:00 0
7f5451c47000-7f5451c5c000 r-xp 00000000 08:02 525600
/lib64/libnsl-2.15.so
7f5451c5c000-7f5451e5b000 ---p 00015000 08:02 525600
/lib64/libnsl-2.15.so
7f5451e5b000-7f5451e5c000 r--p 00014000 08:02 525600
/lib64/libnsl-2.15.so
7f5451e5c000-7f5451e5d000 rw-p 00015000 08:02 525600
/lib64/libnsl-2.15.so
7f5451e5d000-7f5451e5f000 rw-p 00000000 00:00 0
7f5451e5f000-7f5451e62000 r-xp 00000000 08:02 525577
/lib64/libdl-2.15.so
7f5451e62000-7f5452061000 ---p 00003000 08:02 525577
/lib64/libdl-2.15.so
7f5452061000-7f5452062000 r--p 00002000 08:02 525577
/lib64/libdl-2.15.so
7f5452062000-7f5452063000 rw-p 00003000 08:02 525577
/lib64/libdl-2.15.so
7f5452063000-7f545206f000 r-xp 00000000 08:02 525567
/lib64/libcrypt-2.15.so
7f545206f000-7f545226e000 ---p 0000c000 08:02 525567
/lib64/libcrypt-2.15.so
7f545226e000-7f545226f000 r--p 0000b000 08:02 525567
/lib64/libcrypt-2.15.so
7f545226f000-7f5452270000 rw-p 0000c000 08:02 525567
/lib64/libcrypt-2.15.so
7f5452270000-7f545229e000 rw-p 00000000 00:00 0
7f545229e000-7f54522b5000 r-xp 00000000 08:02 526358
/lib64/libpthread-2.15.so
7f54522b5000-7f54524b4000 ---p 00017000 08:02 526358
/lib64/libpthread-2.15.so
7f54524b4000-7f54524b5000 r--p 00016000 08:02 526358
/lib64/libpthread-2.15.so
7f54524b5000-7f54524b6000 rw-p 00017000 08:02 526358
/lib64/libpthread-2.15.so
7f54524b6000-7f54524ba000 rw-p 00000000 00:00 0
7f54524ba000-7f5452655000 r-xp 00000000 08:02 525356
/lib64/libc-2.15.so
7f5452655000-7f5452855000 ---p 0019b000 08:02 525356
/lib64/libc-2.15.so
7f5452855000-7f5452859000 r--p 0019b000 08:02 525356
/lib64/libc-2.15.so
7f5452859000-7f545285b000 rw-p 0019f000 08:02 525356
/lib64/libc-2.15.so
7f545285b000-7f545285f000 rw-p 00000000 00:00 0
7f545285f000-7f5452874000 r-xp 00000000 08:02 525604
/lib64/libgcc_s.so.1
7f5452874000-7f5452a73000 ---p 00015000 08:02 525604
/lib64/libgcc_s.so.1
7f5452a73000-7f5452a74000 r--p 00014000 08:02 525604
/lib64/libgcc_s.so.1
7f5452a74000-7f5452a75000 rw-p 00015000 08:02 525604
/lib64/libgcc_s.so.1
7f5452a75000-7f5452b6a000 r-xp 00000000 08:02 525581
/lib64/libm-2.15.so
7f5452b6a000-7f5452d6a000 ---p 000f5000 08:02 525581
/lib64/libm-2.15.so
7f5452d6a000-7f5452d6b000 r--p 000f5000 08:02 525581
/lib64/libm-2.15.so
7f5452d6b000-7f5452d6c000 rw-p 000f6000 08:02 525581
/lib64/libm-2.15.so
7f5452d6c000-7f5452e54000 r-xp 00000000 08:02 924653
/usr/lib64/libstdc++.so.6.0.17
7f5452e54000-7f5453054000 ---p 000e8000 08:02 924653
/usr/lib64/libstdc++.so.6.0.17
7f5453054000-7f545305c000 r--p 000e8000 08:02 924653
/usr/lib64/libstdc++.so.6.0.17
7f545305c000-7f545305e000 rw-p 000f0000 08:02 924653
/usr/lib64/libstdc++.so.6.0.17
7f545305e000-7f5453073000 rw-p 00000000 00:00 0
7f5453073000-7f5453081000 r-xp 00000000 08:02 1057788
 /usr/local/eyedb/lib64/libeyedbutils-2.8.8.so
7f5453081000-7f5453280000 ---p 0000e000 08:02 1057788
 /usr/local/eyedb/lib64/libeyedbutils-2.8.8.so
7f5453280000-7f5453281000 r--p 0000d000 08:02 1057788
 /usr/local/eyedb/lib64/libeyedbutils-2.8.8.so
7f5453281000-7f5453282000 rw-p 0000e000 08:02 1057788
 /usr/local/eyedb/lib64/libeyedbutils-2.8.8.so
7f5453282000-7f54532e2000 r-xp 00000000 08:02 1058996
 /usr/local/eyedb/lib64/libeyedbsm-2.8.8.so
7f54532e2000-7f54534e1000 ---p 00060000 08:02 1058996
 /usr/local/eyedb/lib64/libeyedbsm-2.8.8.so
7f54534e1000-7f54534e2000 r--p 0005f000 08:02 1058996
 /usr/local/eyedb/lib64/libeyedbsm-2.8.8.so
7f54534e2000-7f54534e4000 rw-p 00060000 08:02 1058996
 /usr/local/eyedb/lib64/libeyedbsm-2.8.8.so
7f54534e4000-7f54534e6000 rw-p 00000000 00:00 0
7f54534e6000-7f54534ec000 r-xp 00000000 08:02 1058986
 /usr/local/eyedb/lib64/libeyedbrpcfe-2.8.8.so
7f54534ec000-7f54536eb000 ---p 00006000 08:02 1058986
 /usr/local/eyedb/lib64/libeyedbrpcfe-2.8.8.so
7f54536eb000-7f54536ec000 r--p 00005000 08:02 1058986
 /usr/local/eyedb/lib64/libeyedbrpcfe-2.8.8.so
7f54536ec000-7f54536ed000 rw-p 00006000 08:02 1058986
 /usr/local/eyedb/lib64/libeyedbrpcfe-2.8.8.so
7f54536ed000-7f5453adb000 r-xp 00000000 08:02 1059046
 /usr/local/eyedb/lib64/libeyedb-2.8.8.so
7f5453adb000-7f5453cda000 ---p 003ee000 08:02 1059046
 /usr/local/eyedb/lib64/libeyedb-2.8.8.so
7f5453cda000-7f5453d00000 r--p 003ed000 08:02 1059046
 /usr/local/eyedb/lib64/libeyedb-2.8.8.so
7f5453d00000-7f5453d08000 rw-p 00413000 08:02 1059046
 /usr/local/eyedb/lib64/libeyedb-2.8.8.so
7f5453d08000-7f5453d11000 rw-p 00000000 00:00 0
7f5453d11000-7f5453d6f000 r-xp 00000000 08:02 928091
/usr/lib64/libboost_program_options.so.1.49.0
7f5453d6f000-7f5453f6f000 ---p 0005e000 08:02 928091
/usr/lib64/libboost_program_options.so.1.49.0
7f5453f6f000-7f5453f72000 r--p 0005e000 08:02 928091
/usr/lib64/libboost_program_options.so.1.49.0
7f5453f72000-7f5453f73000 rw-p 00061000 08:02 928091
/usr/lib64/libboost_program_options.so.1.49.0
7f5453f73000-7f5453f94000 r-xp 00000000 08:02 537144
/lib64/ld-2.15.so
7f545415f000-7f5454164000 rw-p 00000000 00:00 0
7f5454191000-7f5454194000 rw-p 00000000 00:00 0
7f5454194000-7f5454195000 r--p 00021000 08:02 537144
/lib64/ld-2.15.so
7f5454195000-7f5454196000 rw-p 00022000 08:02 537144
/lib64/ld-2.15.so
7f5454196000-7f5454197000 rw-p 00000000 00:00 0
7fffeeb51000-7fffeeb8d000 rw-p 00000000 00:00 0
 [stack]
7fffeebf0000-7fffeebf1000 r-xp 00000000 00:00 0
 [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0
 [vsyscall]
Aborted00: 121784614
X

-- 
*"...Hope is what makes us strong. It is why we are here. It is what we
fight with when all is lost..."*

[eyedb-general] Is EyeDB as a place for big data: eyedbadmin

[Michael G. H. <mic...@gm...>]

Doing:
     eyedbadmin database setmaxobjcount market-data 10000000000

eyedbadmin database data list reports:
-- Max Object Count
  1410065408

So assuming that 1410065408 is the maximum number of possible objects then
it is clear in my case I will have to break up the data across multiple
database since about 564694 packets represent about 29052460 objects and
currently without any further analysis I need to process about (8000 x
564694) = 4.517552 x 10^9 packets per day

It is a good thing that terabyte drives are so cheap these days...

*"...Hope is what makes us strong. It is why we are here. It is what we
fight with when all is lost..."*

Re: [eyedb-general] EyeDB compatibility with Apple iOS

[Michael G. H. <mic...@gm...>]

Gcc and posix so if it isn't there shouldn't be much of and effort to compile it there.

I, the unwilling, was lead by the unqualified, to do the unbelievable for so long with so little, that I attempted the impossible with nothing

Sent from my iPad

On Mar 21, 2013, at 4:13 AM, Jussi Niskanen <jus...@sp...> wrote:

> Hei
> 
> Do you know is EyeDB compatible with Apple iOS ?
> 
> Thanks for your info,
> Jussi Niskanen
> ------------------------------------------------------------------------------
> Everyone hates slow websites. So do we.
> Make your web apps faster with AppDynamics
> Download AppDynamics Lite for free today:
> http://p.sf.net/sfu/appdyn_d2d_mar
> _______________________________________________
> eyedb-general mailing list
> eye...@li...
> https://lists.eyedb.org/lists/listinfo/eyedb-general

[eyedb-general] EyeDB compatibility with Apple iOS

[Jussi N. <jus...@sp...>]

Hei

Do you know is EyeDB compatible with Apple iOS ?

Thanks for your info,
Jussi Niskanen

Re: [eyedb-general] Eyedb manager

[Michael H. <mic...@gm...>]

Thank Luca!!!
  I do believe this will be very useful to all that are using or planning
to use EyeDB for one of their projects.

Michael

On Sat, Dec 29, 2012 at 10:49 AM, luca paganotti
<luc...@gm...>wrote:

> Hi all,
>
> I've recently made a trivial qt project to be be able to interface with an
> eyedb database.
>
> if anyone interested, he/she can get the sources issuing:
>
> svn co https://enml.svn.sourceforge.net/svnroot/enml/eyedb/eyedbmgreyedbmgr
>
> The executable depends on qt, eyedb, libconfig++ and log4cxx libraries and
> is built for x86_64 arch on a fedora box using eclipse and qt integration
> plugin. It's very far from being a complete management application for an
> eyedb database server but I hope that someone can get interested in and
> make it better expanding its functionalities.
>
> As attachment you find a trivial screenshot.
>
> lp
>
>
>
>
>
> ------------------------------------------------------------------------------
> Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS,
> MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current
> with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft
> MVPs and experts. SALE $99.99 this month only -- learn more at:
> http://p.sf.net/sfu/learnmore_122412
> _______________________________________________
> eyedb-general mailing list
> eye...@li...
> https://lists.eyedb.org/lists/listinfo/eyedb-general
>
>

Re: [eyedb-general] Code generation

[Michael H. <mic...@gm...>]

These days, it would see you are the EyeDB expert.

That would be a cool feature to know about especially if a person was
interested in writing something like an expert system.

Please do let me know how goes your efforts.

Good luck
Michael

On Wed, Jan 2, 2013 at 5:33 AM, luca paganotti <luc...@gm...>wrote:

> Hi,
>
> I would like to generate C++ and java code programatically, without
> calling eyedbodl with corrected parameters.
>
> At the moment I'm able to generate an odl file from a database schema in a
> custom folder calling:
>
> schema->genODL(_db, NULL, NULL, schema->getName(), NULL, NULL,
> filepath.toAscii().data(), NULL, NULL);
>
> where schema is a valid Schema*.
>
> I would like to do the same for C++ and java.
>
> I tried to use schema->genC_API(...) but, after some retries I get up an
> now I try to ask because I always get crashes ...
>
> I noticed that one genC_API parameter is of type GenCodeHints* but if I
> try to declare a GenCodeHints variable in my code I get linking errors
> because GenCodeHints seems to be only a forward declaration, I found that
> GenCodeHints class is defined in src/odlgen_utils.h and src/odlgen_utils.cc
> so I tried to import these two files in my project but they have so many
> dependencies on other files ...
>
> Which are the correct ways to call genC_API / genJava_API against a loaded
> Schema*?
>
> Thank you for any answer.
>
> Best regards
>
> lp
>
>
>
>
>
> ------------------------------------------------------------------------------
> Master Java SE, Java EE, Eclipse, Spring, Hibernate, JavaScript, jQuery
> and much more. Keep your Java skills current with LearnJavaNow -
> 200+ hours of step-by-step video tutorials by Java experts.
> SALE $49.99 this month only -- learn more at:
> http://p.sf.net/sfu/learnmore_122612
> _______________________________________________
> eyedb-general mailing list
> eye...@li...
> https://lists.eyedb.org/lists/listinfo/eyedb-general
>
>

[eyedb-general] Code generation

[luca p. <luc...@gm...>]

Hi,

I would like to generate C++ and java code programatically, without calling
eyedbodl with corrected parameters.

At the moment I'm able to generate an odl file from a database schema in a
custom folder calling:

schema->genODL(_db, NULL, NULL, schema->getName(), NULL, NULL,
filepath.toAscii().data(), NULL, NULL);

where schema is a valid Schema*.

I would like to do the same for C++ and java.

I tried to use schema->genC_API(...) but, after some retries I get up an
now I try to ask because I always get crashes ...

I noticed that one genC_API parameter is of type GenCodeHints* but if I try
to declare a GenCodeHints variable in my code I get linking errors because
GenCodeHints seems to be only a forward declaration, I found that
GenCodeHints class is defined in src/odlgen_utils.h and src/odlgen_utils.cc
so I tried to import these two files in my project but they have so many
dependencies on other files ...

Which are the correct ways to call genC_API / genJava_API against a loaded
Schema*?

Thank you for any answer.

Best regards

lp

Re: [eyedb-general] uer 'lolveley' : opening database 'EYEDBBM' in sread mode : invalid password

[olivier SAINT-E. <lol...@la...>]

well I found the answer : create a user lolveley of type strict_unix_user


-------- Message original --------
Sujet: 	[eyedb-general] uer 'lolveley' : opening database 'EYEDBBM' in 
sread mode : invalid password
Date : 	Mon, 18 Jun 2012 19:55:59 +0200
De : 	olivier SAINT-EVE <lol...@la...>
Répondre à : 	eye...@li...
Pour : 	eye...@li...



hello,

in the title there is the error I have after the statement : eyeadmin
user list.

can you tell me whart to do to remove this error?

thank you,

olivier

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
eyedb-general mailing list
eye...@li...
https://lists.eyedb.org/lists/listinfo/eyedb-general

[eyedb-general] uer 'lolveley' : opening database 'EYEDBBM' in sread mode : invalid password

[olivier SAINT-E. <lol...@la...>]

hello,

in the title there is the error I have after the statement : eyeadmin 
user list.

can you tell me whart to do to remove this error?

thank you,

olivier

[eyedb-general] Lots of stuff missing in the generated or downloaded documentation...

[Michael H. <mic...@gm...>]

Fantastic job of the OODBMS system but it seems lots of documentation is
not being generated

Lots of stuff missing from the documentation generated or directly down
loaded from eyedb.org

You are better off use the following:
 www.eyedb.org/wp-content/uploads/documentation/api/C++/hierarchy.html

[eyedb-general] Missing documentation from eyedb

[Michael G. H. <mic...@gm...>]

It's is interesting to note that the doxygen documentation for eyedb is not being generated correctly.  The URL www.eyedb.org/wp-content/uploads/documentation/api/C++/hierarchy.html L shows many more classes available for the current build.

I, the unwilling, was lead by the unqualified, to do the unbelievable for so long with so little, that I attempted the impossible with nothing

Sent from my iPad

On Dec 16, 2011, at 8:27 AM, luca paganotti <luc...@gm...> wrote:

> sorry I did a mistake ...
> 
> you should look at eyedbdbm.odl that should be in the distribution you should find the definition of date class and there i found it.
> 
> As to indexing probably my error was coming because I did something like:
> 
> class C1
> {
> ....
> attribute date* dt;
> ....
> };
> 
> class C2
> {
> ....
> attribute C1*;
> ....
> 
> index on C1.dt;
> 
> ....
> };
> 
> and eyedbodl --checkfile schema.odl says thet C1 is not of a literal type.
> 
> Sorry for my mistakes.
> 
> 
> 
> On Fri, Dec 16, 2011 at 1:22 PM, Michael Hart <mic...@gm...> wrote:
> How did you discover the date and time time. It was purely by accident I did. Is there some documentation describing them. 
> 
> Is so happens that I am building a major Web base application for a start up that will be well funded if I can get at demo system going by march. So before I start integrating eyedb into a well funded company what need to know what is actually in the ODL as I design my model.
> 
> 
> 
> On Fri, Dec 16, 2011 at 2:08 AM, luca paganotti <luc...@gm...> wrote:
> Hi Michael,
> 
> I'm not sure but one drawback in the usage of Date type is that you can't index on a field of that type. I think that this holds also for the other data types.
> 
> Best regards.
> 
> ------------------------------------------------------------------------------
> Learn Windows Azure Live!  Tuesday, Dec 13, 2011
> Microsoft is holding a special Learn Windows Azure training event for
> developers. It will provide a great way to learn Windows Azure and what it
> provides. You can attend the event by watching it streamed LIVE online.
> Learn more at http://p.sf.net/sfu/ms-windowsazure
> 
> _______________________________________________
> eyedb-general mailing list
> eye...@li...
> https://lists.eyedb.org/lists/listinfo/eyedb-general
> 
> 
> 
> ------------------------------------------------------------------------------
> Learn Windows Azure Live!  Tuesday, Dec 13, 2011
> Microsoft is holding a special Learn Windows Azure training event for
> developers. It will provide a great way to learn Windows Azure and what it
> provides. You can attend the event by watching it streamed LIVE online.
> Learn more at http://p.sf.net/sfu/ms-windowsazure
> _______________________________________________
> eyedb-general mailing list
> eye...@li...
> https://lists.eyedb.org/lists/listinfo/eyedb-general
> 
> 
> ------------------------------------------------------------------------------
> Learn Windows Azure Live!  Tuesday, Dec 13, 2011
> Microsoft is holding a special Learn Windows Azure training event for 
> developers. It will provide a great way to learn Windows Azure and what it 
> provides. You can attend the event by watching it streamed LIVE online.  
> Learn more at http://p.sf.net/sfu/ms-windowsazure
> _______________________________________________
> eyedb-general mailing list
> eye...@li...
> https://lists.eyedb.org/lists/listinfo/eyedb-general

Re: [eyedb-general] ODL date, time data types

[luca p. <luc...@gm...>]

sorry I did a mistake ...

you should look at eyedbdbm.odl that should be in the distribution you
should find the definition of date class and there i found it.

As to indexing probably my error was coming because I did something like:

class C1
{
....
attribute date* dt;
....
};

class C2
{
....
attribute C1*;
....

index on C1.dt;

....
};

and eyedbodl --checkfile schema.odl says thet C1 is not of a literal type.

Sorry for my mistakes.



On Fri, Dec 16, 2011 at 1:22 PM, Michael Hart <mic...@gm...
> wrote:

> How did you discover the date and time time. It was purely by accident I
> did. Is there some documentation describing them.
>
> Is so happens that I am building a major Web base application for a start
> up that will be well funded if I can get at demo system going by march. So
> before I start integrating eyedb into a well funded company what need to
> know what is actually in the ODL as I design my model.
>
>
>
> On Fri, Dec 16, 2011 at 2:08 AM, luca paganotti <luc...@gm...>wrote:
>
>> Hi Michael,
>>
>> I'm not sure but one drawback in the usage of Date type is that you can't
>> index on a field of that type. I think that this holds also for the other
>> data types.
>>
>> Best regards.
>>
>>
>> ------------------------------------------------------------------------------
>> Learn Windows Azure Live!  Tuesday, Dec 13, 2011
>> Microsoft is holding a special Learn Windows Azure training event for
>> developers. It will provide a great way to learn Windows Azure and what it
>> provides. You can attend the event by watching it streamed LIVE online.
>> Learn more at http://p.sf.net/sfu/ms-windowsazure
>>
>> _______________________________________________
>> eyedb-general mailing list
>> eye...@li...
>> https://lists.eyedb.org/lists/listinfo/eyedb-general
>>
>>
>
>
> ------------------------------------------------------------------------------
> Learn Windows Azure Live!  Tuesday, Dec 13, 2011
> Microsoft is holding a special Learn Windows Azure training event for
> developers. It will provide a great way to learn Windows Azure and what it
> provides. You can attend the event by watching it streamed LIVE online.
> Learn more at http://p.sf.net/sfu/ms-windowsazure
> _______________________________________________
> eyedb-general mailing list
> eye...@li...
> https://lists.eyedb.org/lists/listinfo/eyedb-general
>
>

Re: [eyedb-general] ODL date, time data types

[luca p. <luc...@gm...>]

Don't remember well, but ...

I think that in some example there is the usage of date type or simply I
tried to put it in a odl file. As to indexing I simply tried but I don't
remember well, I can be wrong .... to be sure of it I should find my old
files ... and try again





On Fri, Dec 16, 2011 at 1:22 PM, Michael Hart <mic...@gm...
> wrote:

> How did you discover the date and time time. It was purely by accident I
> did. Is there some documentation describing them.
>
> Is so happens that I am building a major Web base application for a start
> up that will be well funded if I can get at demo system going by march. So
> before I start integrating eyedb into a well funded company what need to
> know what is actually in the ODL as I design my model.
>
>
>
> On Fri, Dec 16, 2011 at 2:08 AM, luca paganotti <luc...@gm...>wrote:
>
>> Hi Michael,
>>
>> I'm not sure but one drawback in the usage of Date type is that you can't
>> index on a field of that type. I think that this holds also for the other
>> data types.
>>
>> Best regards.
>>
>>
>> ------------------------------------------------------------------------------
>> Learn Windows Azure Live!  Tuesday, Dec 13, 2011
>> Microsoft is holding a special Learn Windows Azure training event for
>> developers. It will provide a great way to learn Windows Azure and what it
>> provides. You can attend the event by watching it streamed LIVE online.
>> Learn more at http://p.sf.net/sfu/ms-windowsazure
>>
>> _______________________________________________
>> eyedb-general mailing list
>> eye...@li...
>> https://lists.eyedb.org/lists/listinfo/eyedb-general
>>
>>
>
>
> ------------------------------------------------------------------------------
> Learn Windows Azure Live!  Tuesday, Dec 13, 2011
> Microsoft is holding a special Learn Windows Azure training event for
> developers. It will provide a great way to learn Windows Azure and what it
> provides. You can attend the event by watching it streamed LIVE online.
> Learn more at http://p.sf.net/sfu/ms-windowsazure
> _______________________________________________
> eyedb-general mailing list
> eye...@li...
> https://lists.eyedb.org/lists/listinfo/eyedb-general
>
>

Re: [eyedb-general] ODL date, time data types

[Michael H. <mic...@gm...>]

How did you discover the date and time time. It was purely by accident I
did. Is there some documentation describing them.

Is so happens that I am building a major Web base application for a start
up that will be well funded if I can get at demo system going by march. So
before I start integrating eyedb into a well funded company what need to
know what is actually in the ODL as I design my model.



On Fri, Dec 16, 2011 at 2:08 AM, luca paganotti <luc...@gm...>wrote:

> Hi Michael,
>
> I'm not sure but one drawback in the usage of Date type is that you can't
> index on a field of that type. I think that this holds also for the other
> data types.
>
> Best regards.
>
>
> ------------------------------------------------------------------------------
> Learn Windows Azure Live!  Tuesday, Dec 13, 2011
> Microsoft is holding a special Learn Windows Azure training event for
> developers. It will provide a great way to learn Windows Azure and what it
> provides. You can attend the event by watching it streamed LIVE online.
> Learn more at http://p.sf.net/sfu/ms-windowsazure
> _______________________________________________
> eyedb-general mailing list
> eye...@li...
> https://lists.eyedb.org/lists/listinfo/eyedb-general
>
>

Re: [eyedb-general] ODL date, time data types

[luca p. <luc...@gm...>]

Hi Michael,

I'm not sure but one drawback in the usage of Date type is that you can't
index on a field of that type. I think that this holds also for the other
data types.

Best regards.

[eyedb-general] ODL date, time data types

[Michael G. H. <mic...@gm...>]

I have been meaning to ask this question for a long time;

The eyedb ODL manual show the various type in section 1.2, 1.3, 1.4, 1.5, and 1.6

I did not see anything about date and time types. Just by accident I was looking through the class hierarchy and discovered

eyedb::Date
eyedb::Time
eyedb::TimeInterval
eyedb::TimeStamp
eyedb::Image

I was able to use the Date and Time in a ODL script as follows
class foo
{
   date myDate;
   time myTime;
}

Am I missing something here? I don't see any of matching ODL to C++ types in the eyedb ODL manual and it was just by experimenting I discovered that I could use date and time in the ODL file without issue. Is there some other parallel documentation that tells what are the full set of eyedb ODL data type. In particular I really would like to be able to define the full set of time, date and image object in the ODL.

Thanks for the help you maybe able to offer.

Michael



Sent from my iPad

Re: [eyedb-general] trying to get info about EYEDBDBM system classes

[luca p. <luc...@gm...>]

Hi Eric,
see below.


On Fri, Nov 11, 2011 at 9:29 PM, Eric Viara <vi...@sy...> wrote:

>
> Hi Luca,
>
> Hi Eric,
>
> I've found some of the answers to my questions making some 'try and retry
> cycles' about code generation via odl compiler, comments below. I've some
> problem with preprocessor directives that I don't find in the generated
> code.
>
>
> As I said in my previous mail, eyedbodl calls cpp on the given odl file,
> then parses the result and generates the C++ .cc and .h files => the
> preprocessor directives have been absorbed by cpp then you will not see
> them anymore in the generated C++ file. Is that a problem for what you want
> to do ?
>
> No, at least now, just wondering where my #define went ...
>
>  I would like to learn more about eyedb architecture / design.
>
> If I use the --gen-class-stubs swicth with eyedbodl I get a bunch of .h
> files named <class_name>_stub.h containing only some comments and an
> #include  directive pointing to <eyedb/eyedb.h> i.e the main eyedb include
> file. What should be in these files?
>
>
> The idea of these files is to avoid large C++ block code in the ODL file:
> - write your odl file and generate the C++ files using eyedbodl
> --gencode=C++ --gen-class-stubs
> - when you want to add C++ code within a given class, write your C++ code
> in <class_name>_stub.h, for instance:
> private:
>   int attr1, attr2;
> public:
>   MyClass(int attr1, int attr2) : attr1(attr1), attr2(2) { }
>   void perform(......)
>   etc. // could be very large
>

so this is the right place where to write methods implementations, I'm
talking about the methods define in odl %C++{ ... %} blocks.


> - Note that, when using --gen-class-stubs,  #include <class_name>_stub.h
> have been added within each class in the generated <schema_name>.h
> - when compiling <schema_name>.cc using the generated
> Makefile_<schema_name>, the generated classes will also include the code
> from <class_name>_stub.h
>
> Note that the files <class_name>_stub.h should not contain #include
> <eyedb/eyedb.h> (it is a mistake), you have to get rid of this line before
> writing you own code.
>
>
>
> Some other files are generated by eyedbodl --> skeleton files named
> <package_name>mthbe-skel.cc and <package_name>mthfe-skel.cc, what are they
> for?
> I get also <package_name>stubsbe.cc and <package_name>stubsfe.cc, what are
> they for?
>
> Which is the difference between ...be and ...fe files? Does mth means
> multi threaded?
>
>
> mth means method, fe means frontend (client side) and be means backend
> (server side)
> These files are useful when methods are declared in the ODL class
> definitions, for instance:
>
> class C {
>   attribute string name;
>   string foo(in string str, out int count);
> };
>
> will generate the function:
> Status
> __method__OUT_string_foo_C__IN_string__OUT_int32(eyedb::Database *_db,
> eyedb::BEMethod_C *_m, eyedb::Object *_o, const char * str, eyedblib::int32
> &count, char * &retarg)
> {
>   // User implementation
> }
>
> in <schema_name>mthbe-skel.cc
>
> You have to write the method implementation using the function arguments.
> Then, you have to compile this code using the generated
> Makefile.<schema_name>. The Makefile will also compile
> <schema_name>stubsbe.cc which contains glue code. Then follow the
> instructions contained at the beginning of <schema_name>mthbe-skel.cc.
> This method can be called from C++. This is not yet implemented for Java.
>
> The method will be executed on the server side (eyedbd).
> If you want this method to be executed on the client side (your C++
> program), you have to declare your method as follows:
> instance_method<client> string foo(in string str, out int count);
> Then, you have to use <schema_name>mthfe-skel.cc.
>
> Note that writing:
> string foo(in string str, out int count);
> is equivalent to:
> instance_method<server> string foo(in string str, out int count);
>
>
>
>
> Skeleton files give some short hints on what to do, for example:
>
> // To implement and use user methods, perform the following operations
> /*
> Does it mean that here I have to implement the user methods defined in
> <package_name>.h ?
>
> #1. Copy the skeleton file
> cp testmthbe-skel.cc testmthbe.cc
>
> #2. Implement the user methods in testmthbe.cc using a text editor
>
> #3. Compile the shared library
> *?????? Which shared library?* Do you mean that this way I'm able to
> create a shared library containing all I need to use what I've defined in
> my .odl schema? Is there a Makefile for this library?
>
>
> Yes As I mentionned previously, Makefile.<schema_name> is generated by
> eyedbodl --gencode=C++
>
>
>
> #4. Copy the shared library to the eyedb loadable library directory
> cp testmthbe-2.8.8.so <eyedbinstalldir>/lib/eyedb
>
> #5. Change the file access mode
> chmod a+r <eyedbinstalldir>/lib/eyedb/testmthbe-2.8.8.so
>
> */
>
> I'm thinking to give eyedb a serious try and to spend on it some of my
> time, but try and retry is a very time consuming task, can you please give
> me some suggestion on the way to follow? Thanks.
>
>
> Difficult to answer in a pertinent way... The material you have is only a
> non exhaustive documentation, a few examples, but... you haveall  the C++
> code. Not very easy to navigate into it, but it can be very helpful in some
> cases. For instance, if you need the accurate grammar of ODL and OQL, the
> flex and bison files are a good documentation.
> Have you a precise idea of what you want to implement using eyedb ?
>
>
I'm requested to build a huge database containing meteo observations.

I need to store a grid of about 30000 points and for each point I will need
to store about 20 numeric values. I have one of such grids once an hour for
all the days of a year.

There are already some data about the past years and there will be data
regarding future years.

Once these data are stored in some structured way I need to be able to do
some kind of analisys, mainly statistical analisys but may be also some
kind of spatial query --> this the reason why I talked to you about gist
indexes.

The analisys time interval could be a month, a year or many years in case
of climatological analisys we can have also for example 20 years.

So I have to choose my database system.

Last, some data that need to be stored are raster images or grib2 files ..

The guys using these data are using a bunch of databases, excel files,
etc... and they would like to store their info in more structured way to be
able to work well with their data.






>
> What do you think about the possibility of gist index integration/usage in
> eyedb? Do you think it feasible?
>
>
> We have to write documentation, reference manuals, programming manuals,
> but... this will take a long time. I think the best way is to begin by
> resurrect eyedb wiki and to add FAQ, hints etc. I will think about that.
>

I do not mean to ask you to integrate gist index in eyedb, I see that you
are all very busy. I was asking if do you think that it can be done. I'm
offering some help
and maybe I could learn something more about eyedb and understand how to
add some kind of spatial index to eyedb.

I don't like rdbms even if I use them every day because all customers trust
them. But I work in C++, C# java etc... and many times we have to map
tables to software objects. Why not to use an oodbms? But oodbms should
give at least the same range of possibilities that rdbms give ...

Bye

Luca


>
> Best regards,
>
> Eric
>
>
>
> ------------------------------------------------------------------------------
> RSA(R) Conference 2012
> Save $700 by Nov 18
> Register now
> http://p.sf.net/sfu/rsa-sfdev2dev1
> _______________________________________________
> eyedb-general mailing list
> eye...@li...
> https://lists.eyedb.org/lists/listinfo/eyedb-general
>
>

Re: [eyedb-general] trying to get info about EYEDBDBM system classes

[luca p. <luc...@gm...>]

Hi Eric,

consider these two code fragments taken from eyedb odl files

>
>    1. *C++ blocks* with variable and methods declarations
>     1. For example: %C++{
>           Datafile *xdatfile;
>           virtual void userInitialize();
>           virtual void userCopy(const Object &);
>         %}
>
> *Complex C++ blocks* including nested struct (not found nested classes,
> is it possible?)
>
>
>     1. %C++{
>           void userInitialize();
>           void userCopy(const Object &);
>           void userGarbage();
>
>           Status find(const char *, Index *&);
>           Status find(const char *, NotNullConstraint *&);
>           Status find(const char *, UniqueConstraint *&);
>           Status find(const char *, CollAttrImpl *&);
>           Status find(const char *, CardinalityConstraint_Test *&);
>           Status getAttrComponents(const Class *, LinkedList &);
>           Status hasIndex(bool &has_index, std::string &idx_str);
>
>         private:
>           struct Cache {
>             unsigned int comp_count;
>             unsigned int comp_alloc;
>             struct Comp {
>           char *attrpath;
>           AttributeComponent *comp;
>           Comp();
>           ~Comp();
>             } *comps;
>             Cache();
>             void add(AttributeComponent *);
>             void getComponents(const char *, int, LinkedList &);
>             AttributeComponent *find(const char *attrpath);
>             ~Cache();
>           };
>
>           Status makeCache();
>           void invalidateCache();
>
>           Cache *index_cache, *notnull_cache, *unique_cache, *card_cache,
>       *collimpl_cache;
>           friend class Attribute;
>           %}
>
> Is this a way to make your own classes skeletons and then complete them
> with methods implementation?
>
> How do want to define your class skeletons ? In C++ ? In ODL ? Can you
> give me a small example ?
>

I would like to work in C++.
Take for example the first code fragment, it declares a variable Datafile*
xdatafile an then two methods. Where is the code that uses xdatafile? Where
are the two functions implementations? I imagine that I have to write them
but in which file I have to put them?



>  Is this a way to make eyedb extensible?
>
>
> What do you mean exactly by extensible ?
>
> I mean that by adding declarations to database classes that i'll find
after generating the .h files via odl and then adding the corresponding
implementations I can extend classes behaviour with functionalities not
directly tied to database.

>
>
> Should I take a look to flex and/or bison files for a full syntax?
>
>
> It is the only full documentation.
>
> Eric
>
> _______________________________________________
> eyedb-general mailing lis...@li...https://lists.eyedb.org/lists/listinfo/eyedb-general
>
>
>
> ------------------------------------------------------------------------------
> RSA(R) Conference 2012
> Save $700 by Nov 18
> Register now
> http://p.sf.net/sfu/rsa-sfdev2dev1
> _______________________________________________
> eyedb-general mailing list
> eye...@li...
> https://lists.eyedb.org/lists/listinfo/eyedb-general
>
>

Re: [eyedb-general] trying to get info about EYEDBDBM system classes

[Eric V. <vi...@sy...>]

Hi Luca,

> Hi Eric,
>
> I've found some of the answers to my questions making some 'try and 
> retry cycles' about code generation via odl compiler, comments below. 
> I've some problem with preprocessor directives that I don't find in 
> the generated code.

As I said in my previous mail, eyedbodl calls cpp on the given odl file, 
then parses the result and generates the C++ .cc and .h files => the 
preprocessor directives have been absorbed by cpp then you will not see 
them anymore in the generated C++ file. Is that a problem for what you 
want to do ?


> I would like to learn more about eyedb architecture / design.
>
> If I use the --gen-class-stubs swicth with eyedbodl I get a bunch of 
> .h files named <class_name>_stub.h containing only some comments and 
> an #include  directive pointing to <eyedb/eyedb.h> i.e the main eyedb 
> include file. What should be in these files?

The idea of these files is to avoid large C++ block code in the ODL file:
- write your odl file and generate the C++ files using eyedbodl 
--gencode=C++ --gen-class-stubs
- when you want to add C++ code within a given class, write your C++ 
code in <class_name>_stub.h, for instance:
private:
   int attr1, attr2;
public:
   MyClass(int attr1, int attr2) : attr1(attr1), attr2(2) { }
   void perform(......)
   etc. // could be very large
- Note that, when using --gen-class-stubs,  #include <class_name>_stub.h 
have been added within each class in the generated <schema_name>.h
- when compiling <schema_name>.cc using the generated 
Makefile_<schema_name>, the generated classes will also include the code 
from <class_name>_stub.h

Note that the files <class_name>_stub.h should not contain #include 
<eyedb/eyedb.h> (it is a mistake), you have to get rid of this line 
before writing you own code.

>
> Some other files are generated by eyedbodl --> skeleton files named 
> <package_name>mthbe-skel.cc and <package_name>mthfe-skel.cc, what are 
> they for?
> I get also <package_name>stubsbe.cc and <package_name>stubsfe.cc, what 
> are they for?
>
> Which is the difference between ...be and ...fe files? Does mth means 
> multi threaded?

mth means method, fe means frontend (client side) and be means backend 
(server side)
These files are useful when methods are declared in the ODL class 
definitions, for instance:

class C {
   attribute string name;
   string foo(in string str, out int count);
};

will generate the function:
Status
__method__OUT_string_foo_C__IN_string__OUT_int32(eyedb::Database *_db, 
eyedb::BEMethod_C *_m, eyedb::Object *_o, const char * str, 
eyedblib::int32 &count, char * &retarg)
{
   // User implementation
}

in <schema_name>mthbe-skel.cc

You have to write the method implementation using the function arguments.
Then, you have to compile this code using the generated 
Makefile.<schema_name>. The Makefile will also compile 
<schema_name>stubsbe.cc which contains glue code. Then follow the 
instructions contained at the beginning of <schema_name>mthbe-skel.cc.
This method can be called from C++. This is not yet implemented for Java.

The method will be executed on the server side (eyedbd).
If you want this method to be executed on the client side (your C++ 
program), you have to declare your method as follows:
instance_method<client> string foo(in string str, out int count);
Then, you have to use <schema_name>mthfe-skel.cc.

Note that writing:
string foo(in string str, out int count);
is equivalent to:
instance_method<server> string foo(in string str, out int count);


>
> Skeleton files give some short hints on what to do, for example:
>
> // To implement and use user methods, perform the following operations
> /*
> Does it mean that here I have to implement the user methods defined in 
> <package_name>.h ?
>
> #1. Copy the skeleton file
> cp testmthbe-skel.cc testmthbe.cc
>
> #2. Implement the user methods in testmthbe.cc using a text editor
>
> #3. Compile the shared library
> *?????? Which shared library?* Do you mean that this way I'm able to 
> create a shared library containing all I need to use what I've defined 
> in my .odl schema? Is there a Makefile for this library?

Yes As I mentionned previously, Makefile.<schema_name> is generated by 
eyedbodl --gencode=C++

>
> #4. Copy the shared library to the eyedb loadable library directory
> cp testmthbe-2.8.8.so <http://testmthbe-2.8.8.so> 
> <eyedbinstalldir>/lib/eyedb
>
> #5. Change the file access mode
> chmod a+r <eyedbinstalldir>/lib/eyedb/testmthbe-2.8.8.so 
> <http://testmthbe-2.8.8.so>
>
> */
>
> I'm thinking to give eyedb a serious try and to spend on it some of my 
> time, but try and retry is a very time consuming task, can you please 
> give me some suggestion on the way to follow? Thanks.

Difficult to answer in a pertinent way... The material you have is only 
a non exhaustive documentation, a few examples, but... you haveall  the 
C++ code. Not very easy to navigate into it, but it can be very helpful 
in some cases. For instance, if you need the accurate grammar of ODL and 
OQL, the flex and bison files are a good documentation.
Have you a precise idea of what you want to implement using eyedb ?

>
> What do you think about the possibility of gist index 
> integration/usage in eyedb? Do you think it feasible?

We have to write documentation, reference manuals, programming manuals, 
but... this will take a long time. I think the best way is to begin by 
resurrect eyedb wiki and to add FAQ, hints etc. I will think about that.

Best regards,

Eric

Re: [eyedb-general] trying to get info about EYEDBDBM system classes

[Eric V. <vi...@sy...>]

Hi Luca,

> Hi Eric,
>
> I began to take a look to eyedb odl files .. interesting ...
>
> I notice some features of odl syntax without detailed documentation. 
> For example:
>
> DBM.odl file
>
>  1. class user_entry *[UserEntry]*
>      1. What's the Meaning of *'[UserEntry]'*? Does it mean that
>         user-entry database class will be mapped on UserEntry C++ class?
>

Yes.

>      1. The same construct is used for example also in enum user_type
>         values: EYEDB_USER [EyeDBUser] = 1, ...
>  1. user_type*#type*;
>      1. what's the meaning of *'#'*? Does it mean that I'm using a
>         language keyword as variable name or something similar?
>

type is an ODL language keyword. If you want to use 'type' as a class or 
attribute name you need to escape 'type' using a starting # character.

> oqlctb.odl file
>
>  1. keyword *native*
>      1. Does it mean that object is 'native' to eyedb? I.e. internally
>         defined? For example native object.
>

Yes, object is natively defined.

>      1. native #class : what is the meaning of '#'
>

class is a keyword (see above).

>  1. *C++ blocks* with variable and methods declarations
>      1. For example: %C++{
>             Datafile *xdatfile;
>             virtual void userInitialize();
>             virtual void userCopy(const Object &);
>           %}
>

The contents of this C++ block code will be added "as is" to the 
definition of the generated C++ class.
You can also include Java block codes using %Java{ .. %} to add code in 
Java class definition (when using --gencode=Java).

>  1. Again the meaning of '#'
>      1. instance_method<client, called_from=OQL> oid reimplement(in
>         int idxtype, in string #hints);
>

hints is also an ODL keyword.

> syscls.odl
>
>  1. *Complex C++ blocks* including nested struct (not found nested
>     classes, is it possible?)
>      1. %C++{
>             void userInitialize();
>             void userCopy(const Object &);
>             void userGarbage();
>
>             Status find(const char *, Index *&);
>             Status find(const char *, NotNullConstraint *&);
>             Status find(const char *, UniqueConstraint *&);
>             Status find(const char *, CollAttrImpl *&);
>             Status find(const char *, CardinalityConstraint_Test *&);
>             Status getAttrComponents(const Class *, LinkedList &);
>             Status hasIndex(bool &has_index, std::string &idx_str);
>
>           private:
>             struct Cache {
>               unsigned int comp_count;
>               unsigned int comp_alloc;
>               struct Comp {
>             char *attrpath;
>             AttributeComponent *comp;
>             Comp();
>             ~Comp();
>               } *comps;
>               Cache();
>               void add(AttributeComponent *);
>               void getComponents(const char *, int, LinkedList &);
>               AttributeComponent *find(const char *attrpath);
>               ~Cache();
>             };
>
>             Status makeCache();
>             void invalidateCache();
>
>             Cache *index_cache, *notnull_cache, *unique_cache,
>         *card_cache, *collimpl_cache;
>             friend class Attribute;
>             %}
>

Same as previously. All this code will be included in the generated C++ 
class.

>  1. Usage of *preprocessor directives*
>      1. For example: #define TriggerBEFORE 0x01
>         #define TriggerAFTER  0x02
>
>         #define TriggerCREATE 0x10
>         #define TriggerUPDATE 0x20
>         #define TriggerLOAD   0x40
>         #define TriggerREMOVE 0x80
>
>         #define MK(X,W) ((X)|(W))
>
>         enum trigger_type [TriggerType] {
>           TRIGGER_CREATE_BEFORE [TriggerCREATE_BEFORE]
>             = MK(TriggerCREATE, TriggerBEFORE),
>
> utils.odl

eyedboql uses internally the C preprocessor (cpp) : cpp is called before 
parsing. So you can add preprocessor directives.

>  1. Usage of *static keyword*.
>      1. Does it mean the same as class_method?
>

Yes.

> It seems like %C++ blocks are part of a c++ header file that will be 
> used by odl compiler, is this right?

Yes.

> Is this a way to make your own classes skeletons and then complete 
> them with methods implementation?

How do want to define your class skeletons ? In C++ ? In ODL ? Can you 
give me a small example ?

> Is this a way to make eyedb extensible?

What do you mean exactly by extensible ?

>
> Should I take a look to flex and/or bison files for a full syntax?

It is the only full documentation.

Eric


>
> Sorry for the long request list, hope doesn't bother you.


>
> Bye
>
> Luca
>
>
> On Mon, Nov 7, 2011 at 10:12 PM, Eric Viara <vi...@sy... 
> <mailto:vi...@sy...>> wrote:
>
>
>     Hi Luca.
>
>     The best way to look at EYEDBDBM schema is to do as follows:
>     eyedboql -d EYEDBDBM -c "select class.type = \"user\""  -p
>
>     Or in an interactive way:
>     eyedboql -d EYEDBDBM
>     ? select class.type = "user";
>     = bag(.....:oid, ....:oid);
>     ? \p
>
>     There is no documentation about that schema. A few hints:
>     - EYEDBDBM database is the database gathering all the information
>     about database users and databases (including itself, EYEDBDBM)
>     - the classes used to store these information are the non-system
>     classes =>
>         select class.type != "system" <=>
>         select class.type = "user";
>        (select class.type != "system")->name;
>        = bag("user_type", "system_access_mode",
>     "database_access_mode", "database_property", "user_entry",
>     "database_entry", "system_user_access", "database_user_access",
>     "database_property_value")
>
>     - user_entry class contains the database users
>     - database_entry class contains the databases
>     - database_user_access contains the user access for each pair
>     (database, user)
>       select database_user_access
>       \p
>       or, to display also the referenced objects
>       \p full
>
>     - the system classes (that you can find in any database) contains
>     the schema of the database including constraints, methods,
>     triggers, indices etc. System classes include also some super
>     abstract classes such as object (root superclass), class, agregat
>     etc..  you can browse them using eyedboql, for instance:
>       - select hashindex; (then \p)
>       - select btreeindex;
>       - select object; (mind, you will display all the database)
>      - etc.
>
>     Main of these system classes are generated using a few .odl files
>     that you can find in src/eyedb/*.odl. Other ones, such as object,
>     class etc. are generated natively in the code.
>     You may have a look to these files : syscls.odl, oqlctb.oql and
>     utils.odl (and also DBM.odl for EYEDBDBM user classes). The
>     contents of these files are very close to what you get by browsing
>     the schema using eyedboql.
>
>     Hope that may help.
>
>     Eric
>
>
>     Le 07/11/2011 16:43, luca paganotti a écrit :
>>     Hi all,
>>
>>     I'm trying to get info about EYEDBDBM internals issuing this command:
>>
>>     eyedbodl --gencode=ODL -d EYEDBDBM --system-class -o eyedbdbm.odl
>>
>>     I get the following error:
>>
>>     eyedbodl: executable error: invalid internal format '{    
>>     this.s[offset] := c;   }'
>>
>>     output file eyedbdbm.odl attached.
>>
>>     I would like to learn more about eyedb internals, is this the
>>     correct way, can I found more doc about it?
>>
>>     Bye
>>
>>     Luca
>>
>>
>>
>>
>>     ------------------------------------------------------------------------------
>>     RSA(R) Conference 2012
>>     Save $700 by Nov 18
>>     Register now
>>     http://p.sf.net/sfu/rsa-sfdev2dev1
>>
>>
>>     _______________________________________________
>>     eyedb-general mailing list
>>     eye...@li...  <mailto:eye...@li...>
>>     https://lists.eyedb.org/lists/listinfo/eyedb-general
>
>     ------------------------------------------------------------------------------
>     RSA(R) Conference 2012
>     Save $700 by Nov 18
>     Register now
>     http://p.sf.net/sfu/rsa-sfdev2dev1
>     _______________________________________________
>     eyedb-general mailing list
>     eye...@li...
>     <mailto:eye...@li...>
>     https://lists.eyedb.org/lists/listinfo/eyedb-general
>
>
>
>
> ------------------------------------------------------------------------------
> RSA(R) Conference 2012
> Save $700 by Nov 18
> Register now
> http://p.sf.net/sfu/rsa-sfdev2dev1
>
>
> _______________________________________________
> eyedb-general mailing list
> eye...@li...
> https://lists.eyedb.org/lists/listinfo/eyedb-general