I think we're going in the same direction here. I'm in the process of storing quotes in postgres also. Though I'm using python at the moment. Yahoo EOD is working fine, OpenTick is next, but their Python API is really just a wrapper around the C api; so I may need to port it to python/twisted before I have anything to show.
I really have no desire to write 'yet another quote engine', seems like there are 20-30 different approaches around, all doing the same thing. I'd prefer to concentrate on UI and analysis.
Have you decided on a schema/table design for the real time data? Will you store each intrument in it's own table or mix them together (ala beancounter)?
Thanks,
Mark
markpmc at users dot sourceforge dot net
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I've done some work with Opentick. I made a couple of posts on the opentick forum about this, mainly because I was stuck at one stage.
I have some code (using it daily) that uses the c++ opentick api and cmake. Thus it compiles using the QT4 library on Linux. The sample code supplied by opentick was no good for this. It took me a while to get it to work, so you may find it interesting.
Ben Glazier
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Sorry I am late - have not moitored this Forum (until yet).
> I really have no desire to write 'yet another quote engine', seems like there are 20-30 different approaches around, all doing the
> same thing. I'd prefer to concentrate on UI and analysis.
I was not able to find a solution fitting my needs. I work as a Business Analyst in Banks for years and I can see every day what solutions are available - and expensive:-). So I decided to start a project for open source community which provides todays state of the art - and a little more :-)
> Have you decided on a schema/table design for the real time data? Will you store each intrument in it's own table or mix them
> together (ala beancounter)?
I have one schema for fimi and diffrent tables for eod, tick and layer2 data. But the point is how to manage the financial instruments and not how to manage the quotes. For now there is only a concept available in german language. It is difficult enough to write understandable in german so translating is really hard and needs some more time. But you are welcome to take a look into CVS Files. There will also be a concept in english soon.
Thanks
Chris
PS Ben cool that you have oT up and running. I can not connect to oT from my production Server. Its holding 2 network cards, which I think is the problem - I don't know why cause every other application is working pretty good but on my notebook oT connection is OK. But for now the priority to fix this is low for me, first I wanna get fimi running.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Thanks for the reply. This is what I have been working on:
1. End of day perl script for opentick.
Works like the yahoo script currently available with FiMi.
This is working but needs a few tweaks before it is sent in to cvs (mainly involving date ranges).
2. Real-time tick retrieval for opentick.
A number of additional tables for storing tick data and timeframe-volume data added to Fimi (currently as a new schema).
A c++ application using the qtsql module and the opentick c++ api.
It is a standalone application that writes directly to the postgres database.
At the moment it runs from the command line and has no interface.
It requires adding records to Fimi/postgres to enable tick-trade retrieval.
This is currently working and storing data successfully.
However, it has some stuff hard coded which needs to be looked at.
Also, I am looking to use plpgsql in place of some of the c++ code.
3. Client Application that uses the timeframe-volume data.
The above 1 & 2. just about does enough for me to be able to run and develop my client application which is the reason the above it not elegant and complete.
I am not yet making this code available because I don't know what to do with it.
I am considering merging it with fimi/pgstalker (as Lothar has proposed) if this gets underway or with qtstalker.
Otherwise, it will get released as a new open source project as an add-on or standalone application.
I understand that at present Chris & Lot are working on the EODay Fimi stuff.
And Lothar has been working on getting qtstalker working off directly off the postgres database rather than Berkeley. He has proposed a re-write of qtstalker.
1. & 2. needs to be merged at some point with what the other guys are doing (Chris & Lothar), especially when they move on to doing tick data. I hope what I have achieved so far will be beneficial.
I will e-mail you what I have done so far, although, it is not particularly pretty, it does work.
Code for 1 & 2. and the corresponding table create code. I am using the latest FiMi from cvs.
Regards,
Ben.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Just to kepp you up to date. Now you can define a marketsymbol analogue to the fi symbols. so the agentf is able to send your script a defined symbol for the market.
what you need:
at least one fi (of type future i think) - example corn_nearest, corn_2nd
a market something like cbot
a stype (caption something like opentick with providerfla = true)
at least one symbol holding the symbol which oT uses - example CH08
a market symbol for your stype
So long
Chris
PS I am on the way to make a future rolling procedure. you must use the symbol issuedate and maturitydate to use this function in future. the benefit will be that the cart is rolled automatically for nearest, 2nd and so on.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
For the future rolling, I did some research on this. One thing I found which I thought was a good idea: if you add another field for the continuous price, it enables you to also keep the original price data. This allows perhaps allows for easier data management, e.g. filling in gaps and for changing to a different rolling procedure.
I have been playing with managerf, and it is coming along nicely so far.
I will be e-mailing you some documentation soon on what I am working on, as described above. I am doing some interesting things at the moment.
Regards,
Ben
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I think we're going in the same direction here. I'm in the process of storing quotes in postgres also. Though I'm using python at the moment. Yahoo EOD is working fine, OpenTick is next, but their Python API is really just a wrapper around the C api; so I may need to port it to python/twisted before I have anything to show.
I really have no desire to write 'yet another quote engine', seems like there are 20-30 different approaches around, all doing the same thing. I'd prefer to concentrate on UI and analysis.
Have you decided on a schema/table design for the real time data? Will you store each intrument in it's own table or mix them together (ala beancounter)?
Thanks,
Mark
markpmc at users dot sourceforge dot net
Guys,
I've done some work with Opentick. I made a couple of posts on the opentick forum about this, mainly because I was stuck at one stage.
I have some code (using it daily) that uses the c++ opentick api and cmake. Thus it compiles using the QT4 library on Linux. The sample code supplied by opentick was no good for this. It took me a while to get it to work, so you may find it interesting.
Ben Glazier
Hi Mark and Ben!
Sorry I am late - have not moitored this Forum (until yet).
> I really have no desire to write 'yet another quote engine', seems like there are 20-30 different approaches around, all doing the
> same thing. I'd prefer to concentrate on UI and analysis.
I was not able to find a solution fitting my needs. I work as a Business Analyst in Banks for years and I can see every day what solutions are available - and expensive:-). So I decided to start a project for open source community which provides todays state of the art - and a little more :-)
> Have you decided on a schema/table design for the real time data? Will you store each intrument in it's own table or mix them
> together (ala beancounter)?
I have one schema for fimi and diffrent tables for eod, tick and layer2 data. But the point is how to manage the financial instruments and not how to manage the quotes. For now there is only a concept available in german language. It is difficult enough to write understandable in german so translating is really hard and needs some more time. But you are welcome to take a look into CVS Files. There will also be a concept in english soon.
Thanks
Chris
PS Ben cool that you have oT up and running. I can not connect to oT from my production Server. Its holding 2 network cards, which I think is the problem - I don't know why cause every other application is working pretty good but on my notebook oT connection is OK. But for now the priority to fix this is low for me, first I wanna get fimi running.
Ben,
Sorry. I checked out for a while due to work....
I'd like to take a look at your OpenTick code if possible.
Mark
Mark,
Thanks for the reply. This is what I have been working on:
1. End of day perl script for opentick.
Works like the yahoo script currently available with FiMi.
This is working but needs a few tweaks before it is sent in to cvs (mainly involving date ranges).
2. Real-time tick retrieval for opentick.
A number of additional tables for storing tick data and timeframe-volume data added to Fimi (currently as a new schema).
A c++ application using the qtsql module and the opentick c++ api.
It is a standalone application that writes directly to the postgres database.
At the moment it runs from the command line and has no interface.
It requires adding records to Fimi/postgres to enable tick-trade retrieval.
This is currently working and storing data successfully.
However, it has some stuff hard coded which needs to be looked at.
Also, I am looking to use plpgsql in place of some of the c++ code.
3. Client Application that uses the timeframe-volume data.
The above 1 & 2. just about does enough for me to be able to run and develop my client application which is the reason the above it not elegant and complete.
I am not yet making this code available because I don't know what to do with it.
I am considering merging it with fimi/pgstalker (as Lothar has proposed) if this gets underway or with qtstalker.
Otherwise, it will get released as a new open source project as an add-on or standalone application.
I understand that at present Chris & Lot are working on the EODay Fimi stuff.
And Lothar has been working on getting qtstalker working off directly off the postgres database rather than Berkeley. He has proposed a re-write of qtstalker.
1. & 2. needs to be merged at some point with what the other guys are doing (Chris & Lothar), especially when they move on to doing tick data. I hope what I have achieved so far will be beneficial.
I will e-mail you what I have done so far, although, it is not particularly pretty, it does work.
Code for 1 & 2. and the corresponding table create code. I am using the latest FiMi from cvs.
Regards,
Ben.
Mark,
Sourceforge doesn't seem to like the e-mails. Can you send me a regular e-mail address. bglazier@users.sourceforge.net
Thanks
Hi Ben
Just to kepp you up to date. Now you can define a marketsymbol analogue to the fi symbols. so the agentf is able to send your script a defined symbol for the market.
what you need:
at least one fi (of type future i think) - example corn_nearest, corn_2nd
a market something like cbot
a stype (caption something like opentick with providerfla = true)
at least one symbol holding the symbol which oT uses - example CH08
a market symbol for your stype
So long
Chris
PS I am on the way to make a future rolling procedure. you must use the symbol issuedate and maturitydate to use this function in future. the benefit will be that the cart is rolled automatically for nearest, 2nd and so on.
Chris,
Thanks for the update.
For the future rolling, I did some research on this. One thing I found which I thought was a good idea: if you add another field for the continuous price, it enables you to also keep the original price data. This allows perhaps allows for easier data management, e.g. filling in gaps and for changing to a different rolling procedure.
I have been playing with managerf, and it is coming along nicely so far.
I will be e-mailing you some documentation soon on what I am working on, as described above. I am doing some interesting things at the moment.
Regards,
Ben