geniustrader-devel — participate in geniustrader development, ask for help, suggest improvements, etc

Re: [GT] the geniustrader-devel mail list might actually be operational ...

[Robert A. S. <ra...@ac...>]

gt'ers

in reference to my message http://sourceforge.net/mailarchive/message.php?msg_id=26756392
which the mailing list never managed to reflect a copy of to me ...


i've 'relocated' my gt svn repo clone to the sourceforge server, it seems
to have been done without any badness what-so-ever, but only time will tell.

below is the summary of csh commands i used to do the deed, you bashers will
need to adjust the for loop and everyone will likely have to tweak the paths.

ras


   %   cd  /usr/local/src/genius_trader

# ok in this gt top level dir i've got my svn repo directory ./svn_repo

   %   gtar -cjf ./svn_repo_backup.`date '+%d%b%y' \
   | tr '[A-Z]' '[a-z]'`.tar.bz ./svn_repo/

# just insurance ...

   %  foreach d ( GT Scripts website )
   ? svn switch --relocate \
   https://devel.geniustrader.org:8082/svn/trunk/${d} \
   https://geniustrader.svn.sourceforge.net/svnroot/geniustrader/trunk/${d} \
   ./svn_repo/${d}
   ? end

ok -- no errors, status value of 0

   %   svn info ./svn_repo/GT 
   Path: svn_repo/GT
   URL: https://geniustrader.svn.sourceforge.net/svnroot/geniustrader/trunk/GT
   Repository Root: https://geniustrader.svn.sourceforge.net/svnroot/geniustrader
   Repository UUID: bb1899a8-85f5-0310-a327-fe9f4e15340a
   Revision: 664
   Node Kind: directory
   Schedule: normal
   Last Changed Author: ras
   Last Changed Rev: 664
   Last Changed Date: 2009-04-20 12:03:02 -0700 (Mon, 20 Apr 2009)

tkcvs seemed to connect with the sf server so things look good to go ...

Re: [GT] GeniusTrader: any windows working version?

[Robert A. S. <ra...@ac...>]

Gianni76 wrote:
> 
i am backing away from the statement below:
>> after doing a lot of checking and thinking and more checking
>> the bottom line, i'm afraid, is that gt is currently unsuited
>> for your expressed need to trigger buys or sells on an intraday
>> basis: as far as i can tell the intraday stuff was added as an
>> add-on to the basic design and it was never accounted for in
>> the context of a trading system. what this means is that the
>> trading system signals are processed at the next days open.
see the reason why at the very bottom ...

this reply is chronological based on analysis i've done serially
to reach the results reported in this summary. see the details of
how i arrived here below.

but in summary:
 * gt trading-systems will operate on intraday timeframes
 * not all gt script apps work correctly with intraday timeframes
 * time-zone consideration might be needed to correctly handle
   multi-timezone prices data streams
 * gianni -- you might be in an ideal position to respond to this question:
   how to international financial businesses handle time-stamps
   when across time-zones?
 * display_signal.pl might the be best available script for you
   to work with, but the output format probably isn't ideal for
   your purposes.


i've rolled all the miscellaneous files into a tarball
(humm, maybe a zipfile is better for the windozers ;-) out there)
gianni_analysis.zip to simplify attaching and compressing ...
it doesn't have the aug 1m database but the sed command i used
to create it is in this email or in one of the files in the zf.

lastly and more trivially i think your primary problem was with
the system you defined -- the one below is the correction -- more
details are in the file /tmp/gianni.scan -- grep for 'ahh! i think i see the problem'

--add=VotingLine( SY:G \
    { S:G:And \
       {S:G:Increase      {I:Prices CLOSE}       } \
       {S:G:CrossOverDown {I:SMA 5} {I:SMA 20}   } \
       {S:G:Decrease      {I:ADX}                } \
       {S:G:Below         {I:Prices} {I:EMA  30} } \
       {S:G:Below         {I:Prices} {I:EMA 150} } \
    } \
    { S:G:And \
       {S:G:Decrease      {I:Prices CLOSE}       } \
       {S:G:CrossOverUp   {I:SMA 5} {I:SMA 20}   } \
       {S:G:Increase      {I:ADX}                } \
       {S:G:Above         {I:Prices} {I:EMA  30} } \
       {S:G:Above         {I:Prices} {I:EMA 150} } \
    } \
  , 10)

and i ran into trouble getting all the days 1min bars onto a single plot.
i tried many variations of date and time string without get the image
(/tmp/FIAT_13aug10_fix.png) to change any
%    graphic.pl --file /tmp/frank_scan1_fixed.gconf \
    -timeframe 1min --start '13aug2010 09:00:00' --end '13aug2010 16:00:00' \
    --out /tmp/FIAT_13aug10_fix.png FIAT1M_AUG2010

then looking harder i noticed the price data and the bar plots don't
appear to be matching up. what is happening is still unknown at this point.

end summary -- i really want to get this posted so others can review and
comment or expand on it.



aloha

merry christmas

ras


> 
> 
> I don't know GT well enough, but for what concerns all other trading 
> programs I know of, they use the concept of 'period' or 'candle' very 
> generically and not in absolute terms.
> This means that if you feed them 30 lines of 1min or 30 lines of 1 day 
> nothing will change in the logic behind the calculations...
> They will process things link EMA (30) on the last 30 data available 
> (and if it is 30 samples taken at 1min intervals or 30 samples taken at 
> 1day intervals or 30 samples taken at 1 week intervals nothing changes 
> at all!!)....

i guess a way to empirically determine if the next trading period can
be in the sub-day realm would be to set up a simple system and run it
on backtest.pl. the alternate would be to see if buysellarrows or even
a votingline plot would indicate intraday buy-sell signals, but keep in
mind the way a gt trading-system functions -- the initial 'systems' signals
are only opportunities to open a position, the managers, the filters and
the orderfactory determine if (and probably when) the position will actually
be taken, finally once a new position is taken the closestragy(ies) define
the signals that ultimately close or otherwise adjust the position.

now armed with lots of real intraday data i will start to ponder how one
might be able to evaluate if intraday trading can work with gt.

a simple, but complete trading system might look something like this:

note this is off the top of my head, so it may be incomplete or simply wrong

SY:G \
  { S:G:CrossOverUp   {I:SMA 30} {I:SMA 100} } \
  { S:G:CrossOverDown {I:SMA 30} {I:SMA 100} } \
| OF:MarketPrice \
| TF:OneTrade \
| TF:LongOnly \
| MM:Basic \
| CS:OppositeSignal \


> So if you trigger a buy (just to make a simple example) on the cross 
> EMA(5) >EMA(30) they will check the EMA value of the last 5 samples, the 
> ema value of the last 30 samples and make the numeric comparison, 
> without ever worry about the absolute time (which is just a label on a 
> graph or maybe a temporal filter for a trade/no trade precondition, but 
> not much more...).
> This is what I thought GT was doing as well.
> 
> And if the logic is like this, there is no problem in getting my 
> BUY/SELL signals intraday.
> It means that each signal will come at the OPEN of the next candle 
> (which, if a feed 1min data, will be at the open of the next min), as 
> happens with all programs triggering buy/sell signals.... (I am not sure 
> why you talk about opening of the next day... GT should simply process 
> last xx data depending on the requested functions and regardless of 
> absolute time!!!).
> 
> Who added the intraday patches?
> he must have an answer to this question about the logic in GT...

well, that would be 'a lot of people, over a long period of time',
they just sort of evolved and became part of the code body.

> 
> 
>>
>> note that the signals themselves might be indicated at the appropriate
>> intraday time, but the trading system controller (don't know if it is
>> OrderFactory, SystemManager, PortfolioManager, or something else)
>> would need to pass the signal immediately and not wait for the
>> next days open to see if the designated trade can be made or not.
> 
> 
> See above, my thinking is about a logic divided in 'periods' or 'candles'.

right -- i think of them as bars (like much of the technical analysis
literature calls them, and i think most of the easylanguage codes too).
for the most part in gt they are the current timeframe.

> The logic controller has to process the last x data (say the last 30  
> 1min candles if we want the EMA30) and pass out the result at the open 
> of the following candle.
> This means that in absolute terms the processing is done in batches 
> corresponding to the timeframe period.
> So if I have a 1min timeframe the answer should come at the open of the 
> next period/candle (which means the max delay is 1min).
> I think that any logic trigger needs to be based on the selected 
> timeframe and not on the basis of 'absolute time' (i.e. next day) as you 
> mention.
> (this means that if you set a timeframe 1min, the answer should be 
> delayed 1min max; but also that if you select a timeframe of a week, the 
> processing should be done at the open of the next candle, which means 
> the following week and not the following day....).
> 
> Unless I totally misunderstood GT, the only rational conclusion is that 
> it has to work on relative time-frames intervals, rather than absolute 
> 1-day periods.
> But I would ask confirmation from those who wrote and studied the logic 
> behind it....
> 

gt trading systems involve a lot more logic than the simple presentation
you outline -- so if we put trading systems aside that leaves indicators and
signals and those can do what you want. however, the only applications that
use them are scan.pl and the display_signal.pl script. you might find these
inadequate (or at least poorly suited) for your purpose, but you can likely
design one or more scripts that do exactly what you want to do.


display_signal.pl can be configured to report only when a signal changes:
  --change ( or -c )
      show on output only those dates that signal changed


>> in summary all the problems pointed out with respect to the intraday
>> timeframe data are confirmed in various different forms. the gt apps
>> all need to consider date and time fields when operating on sub-eod
>> timeframes. there also needs to be a way to set or specify how to
>> treat the time fields in the database:
>>   i) do we assume local timezone applies? if not are times assumed
>>      to be gmt(utc)?
>>  ii) regardless can an offset be applied, and if so how is it
>>      controlled and varied
>> iii) there has to be a 3rd ...
> 
> 
> Can anyone explain here the concept of a timezone in GT?

the incorporation of Date::Manip likely introduced, or at least
highlighted the problems associated with price data streams especially
the time-stamp part. end-of-day and larger bars (timeframes) are
presumed to have time-stamp component of 00:00:00 if memory serves
(a better choice might have been 23:59:59). but note there isn't
an explicit time zone and presumably Date::Manip has some heuristic
that sets it appropriately.

recognizing more aspects of the problem led me to try this command
on both my 'close to v_690 toolkit' and my very different toolkit:

  %  display_indicator.pl -timeframe 1min --start '2010-09-03' --end '2010-09-04'  I:Prices fiat1m_3sep10 OPEN

in each instance the data starts and ends with the correct bars.

but there is still an issue when command is run without specific time-stamp values

 %   display_indicator.pl -timeframe 1min   I:Prices fiat1m_3sep10 OPEN |head -4
 Calculating indicator Prices[OPEN] ...
 Prices[OPEN]        [2010-09-03 15:21:00] = 10.0400
 Prices[OPEN]        [2010-09-03 15:22:00] = 10.0400
 Prices[OPEN]        [2010-09-03 15:23:00] = 10.0400

and my hack version
 %   display_indicator.pl -timeframe 1min   I:Prices fiat1m_3sep10 OPEN |head -4
 Indicator I:Prices has 1 value ... all values selected
         I:Prices/1  <=> Prices[OPEN]

         timeframe 1min, time periods 372 .. 572
 Calculating indicator ...
 Prices[OPEN]        [2010-09-03 15:20:00] = 10.0400
 Prices[OPEN]        [2010-09-03 15:21:00] = 10.0400
 Prices[OPEN]        [2010-09-03 15:22:00] = 10.0400

for gt to be usable world-wide with intra-day time-stamped data
the time-stamp must either include the time-zone or the database
needs to be converted to gmt (universal coordinate time) when
stored or there needs to be a way to infor gt what the correct
time-zone value is for the data being used.

in either case there will be the usual issues of day rollover
that need to be handled correctly for the data to retain it's
integrity.


so gianni -- based on your projects description i presume you are
in italy, that's likely gmt or maybe an hour earlier, so is your
fiat data your local time?

i guess the large question is -- how to international financial
businesses handle time-stamps? however it is done in the financial
industry should be good enough for gt.


> 
> Just to get back to simple steps to what I want to achieve as an 
> absolute minimum (i.e. triggering of signals based on the knowledge of 
> the last xx data...), at the end is "in simple terms" just a comparison 
> of two (or more) mathematical functions.
> 
> if (mathfunc 1> mathfunc 2) then triggerlong;
> if (mathfunc3> mathfunc4) then triggershort;

you can do these operations with signals and indicators.

> 
> where mathfunc1,2,3,4 can be derived with simple use of the GT 
> Indicators (without even using the scripts at all...).
> So any time issue is solved at root as the indicators are functions 
> based on input samples, i.e. based on your own sampling time, which is 
> the timeframe (and not depending on absolute times of end of day)....
> 
> And in this case it would be quite easy to write a simple logic wrapper 
> around the GT libraries. This wrapper takes xx data, calculates the long 
> and short logic and passes out the related signals...


exactly! use the tool kit as the basis of your new application(s),
and where necessary add, correct and enhance the tool kit itself.

please when making a change to an existing method or function keep the
api the same so the change doesn't ripple through the rest of the code.
if that isn't possible add a new method/function instead. also, try to
limit adding new prerequisites, when unavoidable try to wrap them in
a conditional so unless the new feature or capability is enabled the
user doesn't need to upgrade unless she is using the new feature.


> 
> Of course the backtesting is a different matter as in that case the 
> logic needs to execute the function check many times over sliding 
> time-window where a time sample must correspond to a data sample (so if 
> sampling is 1min then the processing has to be made minute after minute 
> for all the period duration).
> 
> I will try this simpler approach before getting down again in the nicer 
> and more complex GT scripts...
> But would welcome any suggestions fro other users (or developers) of 
> intraday data....
> 
> Thanks
> Gianni
> 


messin' around with scan.pl and your 'systems' description:
 #  made a monthly fiat database of 1min data (sed -n '/2010-08-[0-3][0-9]/p' /tmp/fiat1mr.txt > /tmp/fiat1m_aug2010.txt)
 #  made a multiple signal .scan file (/tmp/gianni.scan)
 #  made a market file (/tmp/gianni_market) with one line 'FIAT1M_AUG2010'

command line
  scan.pl --timeframe 1min /tmp/gianni_market 3aug2010 /tmp/gianni.scan



noted problems -- a month of 1min data is still a huge amount, so i tried
to view the price action using graphic.pl with a 1 hour time frame (hoping
gt would upshift the 1min data into 1 hour chunks -- it didn't. since it
works that way for an end-of-day database and week, month, year timeframes
it really ought to do that for other smaller timeframes as well ...

with these files and this command line i'm getting no errors, but no signals
and unfortunately no signals, since scan.pl only works on a given 'date' i'm
not sure it will actually work on that 'date' for all every timeframe and
without knowing when a signal should be true it is hard to determine what
dates to try.

so it's time to switch to display_signal.pl ... bad (well more of a big
limitation) thing is the display_*.pl apps don't read a file containing a
the sys-sig-indic description, so you have to somehow get it onto the command
line with all the correct quoting and/or special char escapes.

i generally resort to entering the command in an editor window and cut+paste
it to a terminal just to keep my sanity, you may find a different method

but here's the file i developed (/tmp/gianni_ds_cmds) for use with the csh,
but it would work cut+paste wise just fine with bash as well


based on this effort i've found a few intraday 'go long' signals:
[2010-08-02 09:02:00] = yes
[2010-08-02 09:04:00] = yes
[2010-08-02 09:07:00] = yes
[2010-08-02 09:11:00] = yes
[2010-08-02 09:15:00] = yes
[2010-08-02 09:19:00] = yes
[2010-08-02 10:03:00] = yes
[2010-08-09 14:23:00] = yes
[2010-08-11 14:51:00] = yes
[2010-08-11 15:58:00] = yes
[2010-08-11 16:04:00] = yes
[2010-08-12 14:12:00] = yes
[2010-08-13 11:26:00] = yes
[2010-08-13 12:37:00] = yes
[2010-08-13 14:14:00] = yes
[2010-08-16 12:12:00] = yes
[2010-08-16 14:44:00] = yes
[2010-08-19 10:10:00] = yes
[2010-08-19 15:00:00] = yes
[2010-08-19 15:32:00] = yes
[2010-08-19 15:51:00] = yes
[2010-08-19 16:54:00] = yes
[2010-08-20 10:08:00] = yes
[2010-08-20 15:50:00] = yes
[2010-08-23 18:03:00] = yes
[2010-08-24 15:18:00] = yes
[2010-08-25 15:45:00] = yes


i'm going to discount all for 2 aug and start evaluating with scan.pl
for each of these dates: 2010-08-09, 2010-08-11, 2010-08-12, 2010-08-13,
2010-08-16, 2010-08-19, 2010-08-20, 2010-08-23, 2010-08-24 and 2010-08-25



something like: for d in 09 11 12 13 16 19 20 23 24 25; do
  echo "working 2010-08-${d}"
  scan.pl --timeframe 1min --start '2aug2010 09:00:00' --end '27aug2010 17:24:00' /tmp/gianni_market 2010-08-${d} /tmp/gianni.scan
  echo "done"
done

unfortunately something seems amiss -- using scan.pl no signals are reported
-- it is likely scan.pl doesn't have the intraday smarts needed

==   during my review before sending it occured to me that scan.pl probably
==   wants/expects the <date> field to be the same as the --end value
==   --- something to look at anyway.



so far display_signal.pl seems to be the best existing tool for the task
you want done ...





the next big hurdle is to see how backtest.pl operates with this dataset and
these signals ...


  ./backtest.pl <options> { "<full_system_name>" | individual tradesys components } <code>
<options>
  --start '2aug2010 09:00:00'
  --end '27aug2010 17:24:00'
  --graph="/tmp/bt_fiat_sys.png"
  --dt
  --store="/tmp/gianni_pf.xml"
  --timeframe=1min
{ "<full_system_name>" | individual tradesys components }
for now lets keep it simple and define individual tradesys components
 --system ...
 --money-management ...
 --trade-filter ...
 --order-factory ...
 --close-strategy ...
<code> FIAT1M_AUG2010


so again with a command cut+paste file (see file /tmp/gianni_bt_cmds)



well well -- with the 'similar to v_690 toolkit' i'm getting
trades with this trading system ...

## Analysis of SY:Generic {S:G:And {S:G:Increase {I:Prices CLOSE}} {S:G:CrossOverDown {I:SMA 5 {I:Prices CLOSE}} {I:SMA 20 {I:Prices CLOSE}}} {S:G:Decrease {I:ADX 14 {I:Prices HIGH} {I:Prices LOW} {I:Prices CLOSE}}} {S:G:Below {I:Prices } {I:EMA 30 {I:Prices CLOSE}}} {S:G:Below {I:Prices } {I:EMA 130 {I:Prices CLOSE}}}} {S:G:And {S:G:Decrease {I:Prices CLOSE}} {S:G:CrossOverUp {I:SMA 5 {I:Prices CLOSE}} {I:SMA 20 {I:Prices CLOSE}}} {S:G:Increase {I:ADX 14 {I:Prices HIGH} {I:Prices LOW} {I:Prices CLOSE}}} {S:G:Above {I:Prices } {I:EMA 30 {I:Prices CLOSE}}} {S:G:Above {I:Prices } {I:EMA 150 {I:Prices CLOSE}}}}|OF:MarketPrice |TF:OneTrade |TF:LongOnly |CS:OppositeSignal |MM:Basic
History of the portfolio :
--------------------------
Long position (0) on FIAT1M_AUG2010
  2010-08-02  Buy     1023 at 9.7800
  2010-08-02  Sell    1023 at 9.7750
Long position (1) on FIAT1M_AUG2010
  2010-08-02  Buy     1014 at 9.7750
  2010-08-02  Sell    1014 at 9.8150
Long position (2) on FIAT1M_AUG2010
  2010-08-02  Buy     1006 at 9.8150
  2010-08-02  Sell    1006 at 9.8000
Long position (3) on FIAT1M_AUG2010
  2010-08-02  Buy     1000 at 9.7850
  2010-08-02  Sell    1000 at 9.7700
Long position (4) on FIAT1M_AUG2010
  2010-08-02  Buy      992 at 9.7650
  2010-08-05  Sell     992 at 10.3100
Long position (5) on FIAT1M_AUG2010
  2010-08-11  Buy     1041 at 9.7450
  2010-08-17  Sell    1041 at 9.6350
Long position (6) on FIAT1M_AUG2010
  2010-08-19  Buy     1016 at 9.7900
  2010-08-26  Sell    1016 at 9.2200
## Global analysis (full portfolio always invested)
Analysis of the portfolio (2010-08-02 09:00:00 / 2010-08-26 20:27:00) :
-----------------------------------------------------
Performance :  -7.0% (-58.6%)     Buy & Hold :  -5.5% (-50.0%) () => by year
MaxDrawDown :        8.4%  B&H MaxDrawDown :    14.8%
Best performance :   1.5%  Worst performance :  -7.0%
Net gain :        -698.47  Gross gain :       -147.63

Trades statistics :
Number of trades :      7  Trades/Year :       104.45
Number of gains :       1  Number of losses :       6  Win. ratio :   14.3%
Max consec. win :       1  Max consec. loss :       4  Expectancy :   -0.01
Average gain :      4.76%  Average loss :      -1.96%  Avg. perf  :  -1.03%
Biggest gain :      4.76%  Biggest loss :      -6.58%  Profit fac :    2.42
Sum of gains :     461.15  Sum of losses :   -1159.62  Risk of ruin : 100.0

the graphic trades file (/tmp/bt_fiat_sys_svn.png)

(well the v_690 backtest graphics are for crap -- the one from my version
marks the long and short (green and red) and the holding periods inside
|> ... <| marks, plus it plots price bars so you can get a visual indication
of what the trading system and closestrategies are doing ...





my toolkit and backtest version isn't using the Date::Manip date
processing so i need to tweak the date strings to comply with the
defacto 'YYYY-MM-DD\ hh:mm:ss' standard (maybe the timestamp is valid)
yes ! it appears so my report differs slightly:
##
## Analysis of code FIAT1M_AUG2010 using System Spec

SY:Generic {S:G:And {S:G:Increase {I:Prices CLOSE }}
 {S:G:CrossOverDown {I:SMA 5 {I:Prices CLOSE }} {I:SMA 20 {I:Prices CLOSE }}}
 {S:G:Decrease {I:ADX 14 {I:Prices HIGH } {I:Prices LOW } {I:Prices CLOSE }}}
 {S:G:Below {I:Prices } {I:EMA 30 {I:Prices CLOSE }}} {S:G:Below {I:Prices }
 {I:EMA 130 {I:Prices CLOSE }}}} {S:G:And {S:G:Decrease {I:Prices CLOSE }}
 {S:G:CrossOverUp {I:SMA 5 {I:Prices CLOSE }} {I:SMA 20 {I:Prices CLOSE }}}
 {S:G:Increase {I:ADX 14 {I:Prices HIGH } {I:Prices LOW } {I:Prices CLOSE }}}
 {S:G:Above {I:Prices } {I:EMA 30 {I:Prices CLOSE }}} {S:G:Above {I:Prices }
 {I:EMA 150 {I:Prices CLOSE }}}} | OF:MarketPrice | TF:OneTrade |
 TF:LongOnly | CS:OppositeSignal | MM:Basic

History of the portfolio :
--------------------------
Long position (0) on FIAT1M_AUG2010
  2010-08-02  Buy     1023 at 9.7800
  2010-08-02  Sell    1023 at 9.7750
Long position (1) on FIAT1M_AUG2010
  2010-08-02  Buy     1014 at 9.7750
  2010-08-02  Sell    1014 at 9.8150
Long position (2) on FIAT1M_AUG2010
  2010-08-02  Buy     1006 at 9.8150
  2010-08-02  Sell    1006 at 9.8000
Long position (3) on FIAT1M_AUG2010
  2010-08-02  Buy     1000 at 9.7850
  2010-08-02  Sell    1000 at 9.7700
Long position (4) on FIAT1M_AUG2010
  2010-08-02  Buy      992 at 9.7650
  2010-08-03  Sell     992 at 10.1100
Long position (5) on FIAT1M_AUG2010
  2010-08-11  Buy     1021 at 9.7450
  2010-08-16  Sell    1021 at 9.5350
Long position (6) on FIAT1M_AUG2010
  2010-08-19  Buy      987 at 9.7800
  2010-08-19  Sell     987 at 9.8200
Long position (7) on FIAT1M_AUG2010
  2010-08-19  Buy      982 at 9.7900
  2010-08-20  Sell     982 at 9.3950
Long position (8) on FIAT1M_AUG2010
  2010-08-24  Buy      999 at 9.1800
  2010-08-25  Sell     999 at 9.0950
Long position (9) on FIAT1M_AUG2010
  2010-08-27  Buy      986 at 9.1500
  2010-08-27  Sell     986 at 9.3400

## Global analysis (full portfolio always invested)
Analysis of the portfolio (2010-08-02 09:00:00 / 2010-08-27 17:24:00)
-----------------------------------------------------
Performance :  -8.7% (-66.7%)     Buy & Hold :  95.7% (338124) () => by year
MaxDrawDown :        9.9%  B&H MaxDrawDown :    14.8%
Best performance :   0.0%  Worst performance :  -9.9%
Net gain :        -868.63  Gross gain :       -112.80

Trades statistics
Number of trades :     10  Trades/Year :       144.08
Number of gains :       2  Number of losses :       8  Win. ratio :   20.0%
Max consec. win :       1  Max consec. loss :       4  Expectancy :   -0.01
Average gain :      2.02%  Average loss :      -1.62%  Avg. perf  :  -0.90%
Biggest gain :      2.73%  Biggest loss :      -4.79%  Profit fac :    1.24
Sum of gains :     383.02  Sum of losses :   -1251.65  Risk of ruin : 100.0

portfolio performance graph at "/tmp/bt_fiat_sys_rh.png"




so in summary:
 a) i'm incorrect in thinking that gt will only make trades
    at the next days open, i've been able to show otherwise.
 b) there are problems with some gt apps when using
    intraday data streams, scan.pl being one of them
    and graphic.pl maybe also ...
 c) dunno' but there's always a third.

Re: [GT] GeniusTrader: any windows working version?

[Thomas W. <we...@ms...>]

Gianni,

please see comments below...

Th.

On 12/25/2010 12:12 AM, Gianni76 wrote:
>
> I don't know GT well enough, but for what concerns all other trading 
> programs I know of, they use the concept of 'period' or 'candle' very 
> generically and not in absolute terms.
> This means that if you feed them 30 lines of 1min or 30 lines of 1 day 
> nothing will change in the logic behind the calculations...
> They will process things link EMA (30) on the last 30 data available 
> (and if it is 30 samples taken at 1min intervals or 30 samples taken at 
> 1day intervals or 30 samples taken at 1 week intervals nothing changes 
> at all!!)....
> So if you trigger a buy (just to make a simple example) on the cross 
> EMA(5) >EMA(30) they will check the EMA value of the last 5 samples, the 
> ema value of the last 30 samples and make the numeric comparison, 
> without ever worry about the absolute time (which is just a label on a 
> graph or maybe a temporal filter for a trade/no trade precondition, but 
> not much more...).
> This is what I thought GT was doing as well.
>
> And if the logic is like this, there is no problem in getting my 
> BUY/SELL signals intraday.
> It means that each signal will come at the OPEN of the next candle 
> (which, if a feed 1min data, will be at the open of the next min), as 
> happens with all programs triggering buy/sell signals.... (I am not sure 
> why you talk about opening of the next day... GT should simply process 
> last xx data depending on the requested functions and regardless of 
> absolute time!!!).
>
what you describe is how GT also works.

I think that what RAS was concerned about was how other tools, such as
back testing, are responding to the results of evaluating a trading system.

However, if I understand you right, that is not your concern. Rather, I
think you are planning to feed the information that a signal triggered
into some other program which will execute your trade.

Assuming that I understand you right, you would have to develop some
machinery that would
(i) continuously update your data source
(ii) continuously evaluate your system to generate triggers
(iii) react to these triggers

Such machinery does not exist, but the raw materials are there for you
to put this together.

However, I am not sure how well the performance of this will be (here I
mean the execution speed of the program you will have to put together).
This will depend a lot on the complexity of the system and the amount of
data you need to consider.

Just for reference, my application is rather different. I compute very
complex systems on a weekly basis on long histories. These evaluations
take many hours to compute. Trading on small timesteps (minutes, say)
will also have to deal with enormous amounts of history, so you will
have to consider performance carefully, as you do not have hours to wait
then. Remember, GT is written in perl so there is naturally a
performance penalty.
> Can anyone explain here the concept of a timezone in GT?
What RAS was saying is that there is no concept of a timezone in GT. So
when you are talking about intraday trading, you need to worry about
what 1:00 PM means. You will have to explore what the base time is that
GT operates in. The parsing extensions that have been added recently
allow you to feed in time in arbitrary time zones. You will need to
study what the system internally converts that into.
> Just to get back to simple steps to what I want to achieve as an 
> absolute minimum (i.e. triggering of signals based on the knowledge of 
> the last xx data...), at the end is "in simple terms" just a comparison 
> of two (or more) mathematical functions.
>
> if (mathfunc 1> mathfunc 2) then triggerlong;
> if (mathfunc3> mathfunc4) then triggershort;
>
> where mathfunc1,2,3,4 can be derived with simple use of the GT 
> Indicators (without even using the scripts at all...).
> So any time issue is solved at root as the indicators are functions 
> based on input samples, i.e. based on your own sampling time, which is 
> the timeframe (and not depending on absolute times of end of day)....
>
> And in this case it would be quite easy to write a simple logic wrapper 
> around the GT libraries. This wrapper takes xx data, calculates the long 
> and short logic and passes out the related signals...
>
If you study the existing scripts you will see how they are using the GT
libraries to accomplish what you describe. I believe, as I said above,
the challenge for you will be dealing with continuously changing data as
the day progresses, and processing this data fast enough to respond. I
don't know what type of intraday trading you are planning to do, but I
do suggest not to attempt to compete (at least not using a perl-based
system) with the computer based trading systems of professional trading
houses in arbitrage trading unless you are aware of and are able to
exploit some inefficiencies of the markets (such as the recent examples
of east european bond trading).

Re: [GT] GeniusTrader: any windows working version?

[Gianni76 <gia...@em...>]

> after doing a lot of checking and thinking and more checking
> the bottom line, i'm afraid, is that gt is currently unsuited
> for your expressed need to trigger buys or sells on an intraday
> basis: as far as i can tell the intraday stuff was added as an
> add-on to the basic design and it was never accounted for in
> the context of a trading system. what this means is that the
> trading system signals are processed at the next days open.

I don't know GT well enough, but for what concerns all other trading 
programs I know of, they use the concept of 'period' or 'candle' very 
generically and not in absolute terms.
This means that if you feed them 30 lines of 1min or 30 lines of 1 day 
nothing will change in the logic behind the calculations...
They will process things link EMA (30) on the last 30 data available 
(and if it is 30 samples taken at 1min intervals or 30 samples taken at 
1day intervals or 30 samples taken at 1 week intervals nothing changes 
at all!!)....
So if you trigger a buy (just to make a simple example) on the cross 
EMA(5) >EMA(30) they will check the EMA value of the last 5 samples, the 
ema value of the last 30 samples and make the numeric comparison, 
without ever worry about the absolute time (which is just a label on a 
graph or maybe a temporal filter for a trade/no trade precondition, but 
not much more...).
This is what I thought GT was doing as well.

And if the logic is like this, there is no problem in getting my 
BUY/SELL signals intraday.
It means that each signal will come at the OPEN of the next candle 
(which, if a feed 1min data, will be at the open of the next min), as 
happens with all programs triggering buy/sell signals.... (I am not sure 
why you talk about opening of the next day... GT should simply process 
last xx data depending on the requested functions and regardless of 
absolute time!!!).

Who added the intraday patches?
he must have an answer to this question about the logic in GT...


>
> note that the signals themselves might be indicated at the appropriate
> intraday time, but the trading system controller (don't know if it is
> OrderFactory, SystemManager, PortfolioManager, or something else)
> would need to pass the signal immediately and not wait for the
> next days open to see if the designated trade can be made or not.

See above, my thinking is about a logic divided in 'periods' or 'candles'.
The logic controller has to process the last x data (say the last 30  
1min candles if we want the EMA30) and pass out the result at the open 
of the following candle.
This means that in absolute terms the processing is done in batches 
corresponding to the timeframe period.
So if I have a 1min timeframe the answer should come at the open of the 
next period/candle (which means the max delay is 1min).
I think that any logic trigger needs to be based on the selected 
timeframe and not on the basis of 'absolute time' (i.e. next day) as you 
mention.
(this means that if you set a timeframe 1min, the answer should be 
delayed 1min max; but also that if you select a timeframe of a week, the 
processing should be done at the open of the next candle, which means 
the following week and not the following day....).

Unless I totally misunderstood GT, the only rational conclusion is that 
it has to work on relative time-frames intervals, rather than absolute 
1-day periods.
But I would ask confirmation from those who wrote and studied the logic 
behind it....

> in summary all the problems pointed out with respect to the intraday
> timeframe data are confirmed in various different forms. the gt apps
> all need to consider date and time fields when operating on sub-eod
> timeframes. there also needs to be a way to set or specify how to
> treat the time fields in the database:
>   i) do we assume local timezone applies? if not are times assumed
>      to be gmt(utc)?
>  ii) regardless can an offset be applied, and if so how is it
>      controlled and varied
> iii) there has to be a 3rd ...

Can anyone explain here the concept of a timezone in GT?

Just to get back to simple steps to what I want to achieve as an 
absolute minimum (i.e. triggering of signals based on the knowledge of 
the last xx data...), at the end is "in simple terms" just a comparison 
of two (or more) mathematical functions.

if (mathfunc 1> mathfunc 2) then triggerlong;
if (mathfunc3> mathfunc4) then triggershort;

where mathfunc1,2,3,4 can be derived with simple use of the GT 
Indicators (without even using the scripts at all...).
So any time issue is solved at root as the indicators are functions 
based on input samples, i.e. based on your own sampling time, which is 
the timeframe (and not depending on absolute times of end of day)....

And in this case it would be quite easy to write a simple logic wrapper 
around the GT libraries. This wrapper takes xx data, calculates the long 
and short logic and passes out the related signals...

Of course the backtesting is a different matter as in that case the 
logic needs to execute the function check many times over sliding 
time-window where a time sample must correspond to a data sample (so if 
sampling is 1min then the processing has to be made minute after minute 
for all the period duration).

I will try this simpler approach before getting down again in the nicer 
and more complex GT scripts...
But would welcome any suggestions fro other users (or developers) of 
intraday data....

Thanks
Gianni

Re: [GT] GeniusTrader: any windows working version?

[Robert A. S. <ra...@ac...>]

> Hello ras and all,
> thank you for your answers and patience :-)
> I will go in order to make it clearer and probably I will follow up with 
> more data on the new mailing list.


aloha gianni


after doing a lot of checking and thinking and more checking
the bottom line, i'm afraid, is that gt is currently unsuited
for your expressed need to trigger buys or sells on an intraday
basis: as far as i can tell the intraday stuff was added as an
add-on to the basic design and it was never accounted for in
the context of a trading system. what this means is that the
trading system signals are processed at the next days open.

note that the signals themselves might be indicated at the appropriate
intraday time, but the trading system controller (don't know if it is
OrderFactory, SystemManager, PortfolioManager, or something else)
would need to pass the signal immediately and not wait for the
next days open to see if the designated trade can be made or not.


how to change this behavior is unknown -- it may be trivial or
next to impossible without a major rewrite. if anyone has ideas
this is the place to discuss them ...



in summary all the problems pointed out with respect to the intraday
timeframe data are confirmed in various different forms. the gt apps
all need to consider date and time fields when operating on sub-eod
timeframes. there also needs to be a way to set or specify how to
treat the time fields in the database:
   i) do we assume local timezone applies? if not are times assumed
      to be gmt(utc)?
  ii) regardless can an offset be applied, and if so how is it
      controlled and varied
 iii) there has to be a 3rd ...
 
lastly the votingline graphic directive -- it is a complex system
(not a trading-system) but a system signal pair. to diagnose the
issue i'd break it into smaller pieces using display_signal.pl and
maybe some graphic.pl plots using Marks ...

aloha

merry christmas

ras



> 
> 1) Unfortunately I have no choice about OS. Either I make GeniusTrader 
> work properly on Windows or I have to abandon it totally ( I have an 
> alternative commercial program using Easylanguage as a basis, even if I 
> cannot customize it at all... but that one works out of the box!).
> This is because I need it (almost) exclusively to give me trading 
> signals (buy/sell) to pass them a broker platform which ONLY and 
> exclusively runs on Windows.
> This broker platform is from one of the most active trading banks in 
> Italy and they only release their platform for Windows.
> 
> I have some experience programming Perl, so I am ready to spend some 
> time to tweak GT to the stage where I need it...
> BTW: I am using Windows XP for my programming needs, Xampp (standard 
> apache) as server and ActiveState ActivePerl 5.8 as perl interpreter.
> (and Date:manip is installed in the default Activeperl installation, so 
> there is no need for adding it to the windows installer!)

all this is good info, even if it generally meaningless to me.

however, just to make sure Date::Manip is working as expected you
should get the following results from the following commands:

(website)-ras [ 2949 ] %    perl -MDate::Manip -e'while(my$d=shift(@ARGV)){print "$d is ", UnixDate( ParseDate( "$d" ), "%Y-%m-%d" ), "\n"}'  yesterday now tomorrow 'last friday'
yesterday is 2010-12-23
now is 2010-12-24
tomorrow is 2010-12-25
last friday is 2010-12-17

well the actual results should reflect the date you actually exec the command.

> 
> 2) My (desired) workflow is relatively simple:
> I have real time data (coming from the broker platform, written to text 
> files (or to MySQL if necessary) refreshed at a given interval (1min, 
> 5min etc), which contains the last 'timeframe' of data for a particular 
> ticker symbol(s) (usually only one/two). What I would like from 
> GeniusTrader is that is read the data, analyse it against a given 
> system/signal which i have written (and is relatively simple) and pass 
> me back either a buy or sell indicator. I would pack this signal out to 
> the trading platform for a corresponding order buy or sell.
> (so I thought to a customised version of display_signal.pl or graphic.pl 
> to be called in a cron process every 1min to get the signals buy/sell out)

this sounds totally reasonable. so make sure the prices files format is
correctly described in your $HOME/.gt/options file in accordance with the
info in GT/DB/Text.pm pod.

likely it already is because you are able to run graphic.pl, but
as a simple test what about something like

display_indicator.pl -timeframe 1min I:Prices fiat1mr OPEN | head -20



note -- using the text files base database requires that gt load
the entire file into memory, the sql based systems provides ways
to avoid this significant issue ...

my first attempt took a while without returning anything so
as an experiment i exploded fiat1m into day based files

(Scripts)-ras [ 2999 ] %    display_indicator.pl -timeframe 1min --start '2010-09-03 09:00:00' --end '2010-09-03 16:00:00'  I:Prices fiat1m_3sep10 OPEN | head -20
Error: --start date 2010-09-03 09:00:00 must be prior to --end date 2010-09-03 16:00:00
oops this isn't good!

(Scripts)-ras [ 3000 ] %    display_indicator.pl -timeframe 1min  I:Prices fiat1m_3sep10 OPEN | head -20
Calculating indicator Prices[OPEN] ...
Prices[OPEN]        [2010-09-03 15:21:00] = 10.0400
Prices[OPEN]        [2010-09-03 15:22:00] = 10.0400
Prices[OPEN]        [2010-09-03 15:23:00] = 10.0400
Prices[OPEN]        [2010-09-03 15:24:00] = 10.0400
Prices[OPEN]        [2010-09-03 15:25:00] = 10.0400
Prices[OPEN]        [2010-09-03 15:26:00] = 10.0300
Prices[OPEN]        [2010-09-03 15:27:00] = 10.0400
Prices[OPEN]        [2010-09-03 15:28:00] = 10.0500
Prices[OPEN]        [2010-09-03 15:29:00] = 10.0400
Prices[OPEN]        [2010-09-03 15:30:00] = 10.0400
Prices[OPEN]        [2010-09-03 15:31:00] = 10.0500
Prices[OPEN]        [2010-09-03 15:32:00] = 10.0400
Prices[OPEN]        [2010-09-03 15:33:00] = 10.0400
Prices[OPEN]        [2010-09-03 15:34:00] = 10.0600
Prices[OPEN]        [2010-09-03 15:35:00] = 10.0600
Prices[OPEN]        [2010-09-03 15:36:00] = 10.0700
Prices[OPEN]        [2010-09-03 15:37:00] = 10.0700
Prices[OPEN]        [2010-09-03 15:38:00] = 10.0800
Prices[OPEN]        [2010-09-03 15:39:00] = 10.1100

but this suggests things sort-of work so far, but for some reason
the starting time is being shifted ... there must be time-zone conversion
issues that need additional logic to sort out. since i've got no
access to sub-day price feeds it isn't high on my need to do list.

a fast double-check against my toolkit and app:
(see notes below regarding the various versions i'm using)

(Scripts)-ras [ 3011 ] %     display_indicator.pl -timeframe 1min --start '2010-09-03 09:00:00' --end '2010-09-03 16:00:00'  I:Prices fiat1m_3sep10 OPEN | head -20
pre check dates: $start=2010-09-03  $end=2010-09-03
gt:prices: Can't open /tmp/FIAT1M_3SEP10.txt: No such file or directory
Indicator I:Prices has 1 value ... all values selected
        I:Prices/1  <=> Prices[OPEN]
where the heck is that filename upcase occurring? this ain't good at all!
humm, well that stems from the legacy of a symbol, which are upcased as
a convenience, guess that isn't always the case ... created symlink

(Scripts)-ras [ 3014 ] %      display_indicator.pl -timeframe 1min --start '2010-09-03 09:00:00' --end '2010-09-03 16:00:00'  I:Prices fiat1m_3sep10 OPEN | head -20
pre check dates: $start=2010-09-03  $end=2010-09-03
pre check dates: $start=2010-09-03 $first=372
                 $end=2010-09-03 $last=572
display_indicator.pl: error: 
--start "2010-09-03 00:00:00" must be prior to
--end   "2010-09-03 00:00:00"
Indicator I:Prices has 1 value ... all values selected
        I:Prices/1  <=> Prices[OPEN]

(Scripts)-ras [ 3015 ] %    display_indicator.pl -timeframe 1min --full   I:Prices fiat1m_3sep10 OPEN | head -20
pre check dates: $start=  $end=
pre check dates: $start= $first=372
                 $end= $last=572
post check dates: $start= $first=0
                  $end= $last=572
doin' the 99: $ival=99, $nb_val=1
but now: $ival=0, $nb_val=1
display_indicator.pl: interval: 0 .. 572
Indicator I:Prices has 1 value ... all values selected
        I:Prices/1  <=> Prices[OPEN]

        timeframe 1min, time periods 0 .. 572
Calculating indicator ...
Prices[OPEN]        [2010-09-03 09:00:00] = 9.9550
Prices[OPEN]        [2010-09-03 09:01:00] = 9.9650
Prices[OPEN]        [2010-09-03 09:02:00] = 9.9400
Prices[OPEN]        [2010-09-03 09:03:00] = 9.9500
Prices[OPEN]        [2010-09-03 09:04:00] = 9.9600
Prices[OPEN]        [2010-09-03 09:05:00] = 9.9650
Prices[OPEN]        [2010-09-03 09:06:00] = 9.9650
Prices[OPEN]        [2010-09-03 09:07:00] = 9.9850
Prices[OPEN]        [2010-09-03 09:08:00] = 9.9850
Prices[OPEN]        [2010-09-03 09:09:00] = 9.9700

that's more like it ...



but looking at the data a bit more i see that the intervals are not all
that consistent ... i'm not sure this fact has been taken into consideration
when the sub-day provisions were incorporated ...


but continuing ... (see notes below regarding the various versions i'm using)
(Scripts)-ras [ 3004 ] %    graphic.pl -timeframe 1min fiat1m_3sep10 -out /tmp/fiat_3sep10.png

and the image seems reasonable /tmp/fiat_3sep10_svn.png (attached), but i'm not sure about
the hours markings. somehow there is a time-zone shift occurring which isn't expected
and isn't understood ...

using my versions of toolkit and graphic.pl yields about the same thing, but
with all the axis stuff ...
graphic.pl -timeframe 1min fiat1m_3sep10 -out /tmp/fiat_3sep10.png
/tmp/fiat_3sep10.png (attached)


> 
> GeniusTrader should also allow me to do (offline) backtesting of the 
> algorithm to check its potential (and tweak it/improve it) against a 
> given set of old data. (this is what I think backtest.pl should be doing)


yes that is the intent of the backtest*.pl apps. they work differently
from graphic.pl and scan.pl in that they take a trading-system (like
that you have as the votingline, and make trades based on it and the
other trading system attributes like trade-filters, closestrategies, etc.
review the 'user guide' wike pages and the pod in GT/Docs where gt
trading systems are described in some detail ...


> Finally  I would like to be able to check other data/symbols to verify 
> when the given conditions of my Long/short are met also there.. (which 
> is what scan.pl is supposed to do).

yep. exactly.

> 
> So, when I (recently) read about GeniusTrader I was extremely happy, as 
> it seems to do exactly what I need!
> 
> 3) The three exact versions of GeniusTrader I tested are:
>    - v 690 (I used the nice Windows installer), which is the official 
> offered on site.
> 
>     - v 709 - development tree , taken out from 
> http://www.geniustrader.org/development.html, i.e. the two links:
>              
> http://geniustrader.org/websvn/dl.php?repname=group.Geniustrader&path=%2Ftrunk%2FGT%2F&rev=709&peg=709&isdir=1  
> (modules)
> and    
> http://geniustrader.org/websvn/dl.php?repname=group.Geniustrader&path=%2Ftrunk%2FScripts%2F&rev=709&peg=709&isdir=1  
> (scripts)
>    This version was not complete (it started trowing out missing GT:xx, 
> so I copied all the new files over the complete 690 version).
> 
>    - the clkao version 'daytrade' (which I hoped was an intraday 
> version, but wasn't :) ):
>      https://github.com/clkao/finance-geniustrader/tree/topic/daytrade
> 

well, the only source versions of gt i can speak to are those
available via
   http://geniustrader.org/development.html
and/or
   http://geniustrader.org/download
and/or
   http://sourceforge.net/projects/geniustrader/

and of these only the trunk and exp branches are known to me
to be functional.


> 
> 3)  I have tested simple instructions, either copied from the help or 
> from the develop mailing list, just to understand if GeniusTrader is 
> working properly, before doing any change/hack on it or wrapper 
> around...  And here I got into trouble.
> 
> Let's go in order:
> 
> * Testing graphic.pl on V690
> perl graphic.pl 13000 > test1.png  (ok)
> perl graphic.pl --start=2002-05-01 --end=2002-06-01 13000 > test2.png  (ok)
> 
> perl graphic.pl --file=frank_scan1.gconf 13000 >test3.png,
> where  franks_scan1.gconf is attached (basically puts a start and an end 
> to the graph and should add a voting line based a given multiline 
> system, i.e. a system divided in multiple lines connected with a "\" at 
> the end of each of them).
>   --> here no errors are given, but test3.png results exactly the same 
> as test1.png, which means no processing was done on the file at all.
>       So this option definitely does not work on r690!


humm -- locally i put the gconfs in /tmp and have set my database to text
using my version of graphic.pl and my version of gt toolkit:
(Scripts)-ras [ 2955 ] %    graphic.pl --file /tmp/frank_scan1.gconf  13000 -out /tmp/fs1_13000.png
warn: Bad number of arguments given to S:Generic:Increase!
got 6 expected 1 at ../GT/Signals/Generic/Increase.pm line 42.
Bad number of arguments given to S:Generic:Decrease
got 6 arguments, expected 1

and the chart image is:
/tmp/fs1_13000.png (attached) which looks about right to me

using some version of the trunk head, but probably not exactly v_690
(Scripts)-ras [ 2961 ] %     graphic.pl --file /tmp/frank_scan1.gconf  13000 -out /tmp/fs1_13000_svn.png
VotingLine( Systems::Generic \ is not a valid object description at ./graphic.pl line 861.

and the chart image is:
/tmp/fs1_13000_svn.png (attached) which looks about right to me, given
that this combined version fails to correctly handle the VotingLine
directive ...

so there's something really amiss with VotingLine directive -- i'd start
with the error message from my version of the gt toolkit ...

you might try diagnosing the directive with display_system.pl and display_signal.pl
and or use the alternate BuySellArrows to put markers on the chart when
a buy or sell action is triggered ...


> 
> perl graphic.pl --file=frank_scan2.gconf 13000 >test4.png,
> where  franks_scan2.gconf is attached (basically puts a start and an end 
> to the graph and should add a voting line based a given single line system).
> --> here no errors are given, but test4.png results exactly the same as 
> test1.png, which means no processing was done on the file at all.
>       So this option definitely does not work on r690!
> 
> perl graphic.pl --file=e:\cgi-bin\geniustrader\Scripts\frank_scan1.gconf 
> 13000 >test5.png
> (full patch to conf file used used!)
> --> config file read, start/end limits work, but no voting line is added 
> (multiline problem?)
> Error given:
> VotingLine( Systems::Generic \ is not a valid object description at 
> graphic.pl line 744.
> 
> perl graphic.pl --file=e:\cgi-bin\geniustrader\Scripts\frank_scan2.gconf 
> 13000 >test6.png
> (full patch to conf file used used!)
> --> config file read, start/end limits work, but no voting line is added
> VotingLine( Systems::Generic \ is not a valid object description at 
> graphic.pl line 744.
> 


arghhh! what the #%$@! are '\' doing in a pathname?


> 
> * Testing graphic.pl on V709 (exactly as provided on the development 
> page, placing modules under "GT" and scripts under "Scripts", and 
> copying the same "data" folder as used in r 690).
> 
> perl graphic.pl 13000 > test1.png
> --> processing takes at least 10x longer than with v690, a file 
> test1.png is generated, but cannot be opened ("invalid png file" tells 
> me Paint Shop Pro or MS Paint)
> 
> 
> * Testing graphic.pl on V. daytrade (clkao)
> perl graphic.pl 13000 > test1.png  (ok)
> perl graphic.pl --start=2002-05-01 --end=2002-06-01 13000 > test2.png  (ok)
> 
> perl graphic.pl --file=frank_scan1.gconf 13000 >test3.png,
> where  franks_scan1.gconf is attached as before.
>   --> errors due to multiline systems not supported
> 
> perl graphic.pl --file=frank_scan2.gconf 13000 >test4.png,
> where  franks_scan2.gconf is attached
> --> works ok
> 
> changing now data to fiat1mr (my own data; 1 minute scan for 5 months; 
> 3MB file; zipped with this email)
> perl graphic.pl --file=frank_scan2b.gconf fiat1mr >test8.png,
> (frank_scan2b.gconf simply changes timeframe to timeframe=1min)
> "Intraday support not implemented in DB::Text"
> 
> perl graphic.pl --file=frank_scan3.gconf fiat1mr >test9.png,
> (my data is 1m candles, the timeframe in the config file is 'day')
> the png is generated (attached file test9.png), but there are 'only' 120 
> candles (out of 62165 rows of data!) and I cannot understand what has 
> been plotted.
> In case when you have timeframe 1m data, but ask for 'day' processing 
> what is Geniustrader doing?
> (does it process automatically all the 1min data to form the day candle?)


you need to really study the graphic.pl manpage (pod) using something
like perldoc graphic.pl or pod2man graphic.pl

but to set the timeframe to 1min you need to set --timeframe 1min
either on the command line or in the specified gconf file.

if i recall correctly command line parameters will over-ride those
read from a gconfig file, but don't take this as true (it should work
that way, but it may not (yet)), so to be sure change it in the gconf
file.

in addition, you need to verify that the gt configuration key
DB::timeframes_available includes 1min and any other sub-day timeframes
you support.


> ....
> 
> Similar tests have been done for the other functions, but as you can see 
> from the above results report at the moment I am stuck.
> V 690 and v 709 do not seem to work properly  (graphic.pl does not seem 
> to process at all the system I passed them in a configuration file and I 
> need this to work to be able to pass my system to see if the buy/sell 
> points are correct).
> V. daytrade (clkao) works to a certain extent (without multiline 
> systems) but only with day support (no intraday allowed) and I cannot 
> even understand the result graphic for my 1min data...
> 
> Even more problems are coming from the other scripts.... and debugging 
> is now taking far more time than I hoped for.

well, gt is a very complex set of tools and it operates on an even
larger set of data, so it is going to take some bit of time and effort
to get your particular use model configured in a way that works for you.

in addition, you need to realize the sub-day features are an add-on to
the initial design. and because sub-day price data isn't freely available
(at least i'm not aware of any free feeds) these features are probably
not used all that much. i don't even consider them unless a user points
out problems and provides data that i can test against.

(both of which you've managed to do, so i suppose i need to start debugging)

> 
> This is why I ask you great users/developers:
>   - do you have a decent version which works for you under Windows that 
> i can try to start with, something like the clkao version, but with 
> intraday (absolutely a must) and multiline support (nice to have to 

ok -- i take it that 'multiline support' refers to what i call
'configuration file continuation lines' -- make sure there isn't
any trailing whitespace between '\' and end-of-line marker.

> understand the systems I test and easily vary their parameters...).
> - did anyone attempt to write a different scan.pl without process 
> priorities (so it can be compatible with Windows) or have a different 
> solution for scanning signals under Windows?

not that i'm aware of -- as an experienced perler you should be able to
hack that out without a lot of trouble. just get rid of the loop that
controls the forked children, suppress the forking and eliminate the
references to ipc and it should be good to go.

> 
> Thanks you once again for all your help.
> I would like very much to help the community and do some live testing 
> and improvements to the program, but starting from something that works 
> not to add my mistakes to others already (maybe) solved!

well, i'd say you are well on you way already. but you need to select
one of the myriad versions and work from that baseline. i'd recommend
either thomas' exp branch or the trunk branch. you can then add clkao
daytrade patches, but the big problem i see given your expressed needs
is that the gt trading system actions that trigger a buy or sell are
likely to only work on a next-trading day basis.

> 
> Thanks
> Gianni
> 
> 
> 
> On 23/12/2010 20.14, Robert A. Schmied wrote:
> > Gianni76 wrote:
> >
> > aloha gianni
> >
> > this is ras, i haven't heard from nick in some time ...
> >
> >> Dear Nick, Ras
> >> I am trying to set GeniusTrader up and running on Windows. I have 
> >> seen your names on the GeniusTrader developers list (isn't the list 
> >> working anymore? the last message stops in August... 
> >
> > we have a new home at sourceforge, new mailing lists (ml) there as well:
> >
> > http://sourceforge.net/projects/geniustrader/
> > http://sourceforge.net/mailarchive/forum.php?forum_name=geniustrader-system-traders 
> >
> > http://sourceforge.net/mailarchive/forum.php?forum_name=geniustrader-devel 
> >
> >
> > i've had a load of trouble getting ml -devel up and working at sf. it 
> > seems
> > to be working now, but it is hard to tell unless members post messages
> > ... so this is perfect opportunity to post there.
> >
> > you have to subscribe to post messages, and don't need a sourceforge id.
> >
> > the extensively revised home page is at http://geniustrader.org/
> >
> >> http://www.geniustrader.org/lists/devel/index.html) and I hope you 
> >> can help.
> >> I need it on Windows as I wish to interface it to a trading (order) 
> >> platform which only runs on Windows.
> >
> > that is a terrible dilemma to be on ;-)
> >
> > solution is to install and run a lamp
> >
> >> My setup is a default Xampp server + ActiveState Perl 5.8
> >>
> >> But I am having trouble in making Geniustrader work properly
> >> I tested v 690, v709 and then went back to some older versions.
> >> But they all have issues.
> >
> > i don't have a clue about m$ crap, nor understand why anyone would
> > want to use it when other compatible options exist, so i'm not going
> > to be too helpful, specifically, but i will make some general comments
> > that might help you, provided you can hack on the gt perl code ...
> >
> > João Costa { joaocosta at zonalivre dot org } is/was the principle
> > involved with the windoze installer thingy, maybe he can/will help
> > with an updated version ... as it is multimegabytes of (useless)
> > binary data i keep my versions of it on /dev/null.
> >
> >>
> >> Basically graphic.pl works perfectly (with all options) on older 
> >> versions
> >
> > older versions of what?
> >  windblows?
> > or
> >  graphic.pl and gt tool-kit?
> >
> >
> > probably the best way to access the entire svn repo is via the web page:
> > http://geniustrader.org/websvn/listing.php?repname=group.Geniustrader
> >
> > the head on the trunk is 709 -- 690 for tool-kit and 691 for scripts
> > the head on the exp branch is 650 for tool-kit and 642 for scripts
> >
> > note: Weigert, Thomas { weigert at mst dot edu } is the principle on
> > the exp branch and if memory serves is also a 'dozer.
> >
> > looks like 519 for the windows installer on any of the branches
> >
> >
> > (but I can only get it to work with a day timeframe and not
> >> with intraday
> >> support from a newer Text.pm)
> >
> > have you checked the new advanced configuration features available in 
> > Text.pm?
> >
> > without specifics on your intraday format and your specific scan 
> > sys-sig-indic
> > description spec i cannot evaluate, but last i checked i believe 
> > intraday was
> > working at least with graphic.pl.
> >
> > if you can provide specifics i will evaluate against my version, but 
> > it is *nix based.
> >
> > or on the web/cgi interface (limited
> >> though as it does not read configuration files).
> >
> > yea! that is a major oversight -- can you suggest a way to safely 
> > (website wise) fix this?
> >
> >> The newer 690 and 709 versions do not work properly (the -- start and 
> >> -- end for example are not recognised as many other options...).
> >
> > for this you must have perl module Date::Manip. it might not be part 
> > of the windozer installer?
> >
> >>
> >> scan.pl which seems the most useful script never worked for me in any 
> >> version.
> >> The problem is with the IPC SysV library, which does not exist for 
> >> Windows
> >> ....
> >> Can't locate IPC/SysV.pm in @INC (@INC contains: .. 
> >> E:\servers\xampp\cgi-bin\gtd
> >> ay E:/servers/perl/site/lib E:/servers/perl/lib .) at scan.pl line 19.
> >> BEGIN failed--compilation aborted at scan.pl line 19.
> >> .....
> >
> > possible options:
> > 1) windblows must surely have some sort of ipc support by now
> >    (or maybe not) but assuming it does and there is a perl module
> >    that works with it you could port scan.pl to use that ipc 
> > interface/api.
> > 2) strip out all multi-processing features from scan.pl entirely.
> > 3) wrap the ipc stuff inside eval blocks and only use when
> >    a) platform supports it
> >    b) user calls for it
> >
> > of these two options i think 2) is the easiest solution ... others
> > have complained about the ipc stuff in the past and i had to hack
> > it some just to get it working locally (solaris), so my take is
> > that the implementor was just learning/experimenting with the ipc
> > api and maybe a dual processor system.
> > option 3) is the best technical approach, provided it includes
> > hooks that would allow/support option 1) should that be available.
> >
> >>
> >> I haven't seen this mentioned anywhere on the main list (and I read 
> >> dozens of messages to get a good understanding of the program...).
> >> Is there any way to make it run (even with no priorities)?
> >> Has anyone written another version of it compatible with Windows 
> >> distributions?
> >>
> >> Or in general do you have a (recent) version of GeniusTrader which 
> >> has been tested and runs on Windows?
> >> I miss the scan.pl, the intraday (1min, 5min) support and the 
> >> multiline command support...
> >
> > multiline command support? -- i don't understand this.
> >
> >>
> >> Thank you
> >> Gianni
> >>
> >>
> >>
> >
> > in summary:
> >
> > you *must* have perl module Date::Manip with any of the newer gt 
> > versions.
> > it may or may not be part of the current windoze installer.
> >
> > windoze isn't something i can/will develop or test on, so volunteers are
> > needed to test and advance that port.
> >
> > removing the ipc stuff from scan.pl (or wrapping it inside a 
> > conditional eval block)
> > is probably a worth-while endeavor.
> >
> > aloha
> >
> > merry christmas and a happy and profitable new year
> >
> > ras
> >
> >
> >

Re: [GT] GeniusTrader: any windows working version?

[Thomas W. <we...@ms...>]

Gianni,

in the past (before I switched to Linux as my main computer) I have ran
GT on cygwin on Windows. cygwin is a set of libraries that provides you
a pretty good unix emulation on windows. Once I managed to compile the
GD libraries everything worked as it should under Windows. The only
issue is that the concurrency mechanism used in one of the scripts is
not supported by GT but you probably don't need that anyway...

In that setup, I did not use ActiveState perl, but the standard perl
that comes with the cygwin distribution...

Best regards, Th.

On 12/24/2010 06:02 AM, Gianni76 wrote:
> Hello ras and all,
> thank you for your answers and patience :-)
> I will go in order to make it clearer and probably I will follow up
> with more data on the new mailing list.
>
> 1) Unfortunately I have no choice about OS. Either I make GeniusTrader
> work properly on Windows or I have to abandon it totally ( I have an
> alternative commercial program using Easylanguage as a basis, even if
> I cannot customize it at all... but that one works out of the box!).
> This is because I need it (almost) exclusively to give me trading
> signals (buy/sell) to pass them a broker platform which ONLY and
> exclusively runs on Windows.
> This broker platform is from one of the most active trading banks in
> Italy and they only release their platform for Windows.
>
> I have some experience programming Perl, so I am ready to spend some
> time to tweak GT to the stage where I need it...
> BTW: I am using Windows XP for my programming needs, Xampp (standard
> apache) as server and ActiveState ActivePerl 5.8 as perl interpreter.
> (and Date:manip is installed in the default Activeperl installation,
> so there is no need for adding it to the windows installer!)
>
>

Re: [GT] GeniusTrader: any windows working version?

[Gianni76 <gia...@em...>]

Hello ras and all,
thank you for your answers and patience :-)
I will go in order to make it clearer and probably I will follow up with 
more data on the new mailing list.

1) Unfortunately I have no choice about OS. Either I make GeniusTrader 
work properly on Windows or I have to abandon it totally ( I have an 
alternative commercial program using Easylanguage as a basis, even if I 
cannot customize it at all... but that one works out of the box!).
This is because I need it (almost) exclusively to give me trading 
signals (buy/sell) to pass them a broker platform which ONLY and 
exclusively runs on Windows.
This broker platform is from one of the most active trading banks in 
Italy and they only release their platform for Windows.

I have some experience programming Perl, so I am ready to spend some 
time to tweak GT to the stage where I need it...
BTW: I am using Windows XP for my programming needs, Xampp (standard 
apache) as server and ActiveState ActivePerl 5.8 as perl interpreter.
(and Date:manip is installed in the default Activeperl installation, so 
there is no need for adding it to the windows installer!)

2) My (desired) workflow is relatively simple:
I have real time data (coming from the broker platform, written to text 
files (or to MySQL if necessary) refreshed at a given interval (1min, 
5min etc), which contains the last 'timeframe' of data for a particular 
ticker symbol(s) (usually only one/two). What I would like from 
GeniusTrader is that is read the data, analyse it against a given 
system/signal which i have written (and is relatively simple) and pass 
me back either a buy or sell indicator. I would pack this signal out to 
the trading platform for a corresponding order buy or sell.
(so I thought to a customised version of display_signal.pl or graphic.pl 
to be called in a cron process every 1min to get the signals buy/sell out)

GeniusTrader should also allow me to do (offline) backtesting of the 
algorithm to check its potential (and tweak it/improve it) against a 
given set of old data. (this is what I think backtest.pl should be doing)
Finally  I would like to be able to check other data/symbols to verify 
when the given conditions of my Long/short are met also there.. (which 
is what scan.pl is supposed to do).

So, when I (recently) read about GeniusTrader I was extremely happy, as 
it seems to do exactly what I need!

3) The three exact versions of GeniusTrader I tested are:
   - v 690 (I used the nice Windows installer), which is the official 
offered on site.

    - v 709 - development tree , taken out from 
http://www.geniustrader.org/development.html, i.e. the two links:
             
http://geniustrader.org/websvn/dl.php?repname=group.Geniustrader&path=%2Ftrunk%2FGT%2F&rev=709&peg=709&isdir=1  
(modules)
and    
http://geniustrader.org/websvn/dl.php?repname=group.Geniustrader&path=%2Ftrunk%2FScripts%2F&rev=709&peg=709&isdir=1  
(scripts)
   This version was not complete (it started trowing out missing GT:xx, 
so I copied all the new files over the complete 690 version).

   - the clkao version 'daytrade' (which I hoped was an intraday 
version, but wasn't :) ):
     https://github.com/clkao/finance-geniustrader/tree/topic/daytrade


3)  I have tested simple instructions, either copied from the help or 
from the develop mailing list, just to understand if GeniusTrader is 
working properly, before doing any change/hack on it or wrapper 
around...  And here I got into trouble.

Let's go in order:

* Testing graphic.pl on V690
perl graphic.pl 13000 > test1.png  (ok)
perl graphic.pl --start=2002-05-01 --end=2002-06-01 13000 > test2.png  (ok)

perl graphic.pl --file=frank_scan1.gconf 13000 >test3.png,
where  franks_scan1.gconf is attached (basically puts a start and an end 
to the graph and should add a voting line based a given multiline 
system, i.e. a system divided in multiple lines connected with a "\" at 
the end of each of them).
  --> here no errors are given, but test3.png results exactly the same 
as test1.png, which means no processing was done on the file at all.
      So this option definitely does not work on r690!

perl graphic.pl --file=frank_scan2.gconf 13000 >test4.png,
where  franks_scan2.gconf is attached (basically puts a start and an end 
to the graph and should add a voting line based a given single line system).
--> here no errors are given, but test4.png results exactly the same as 
test1.png, which means no processing was done on the file at all.
      So this option definitely does not work on r690!

perl graphic.pl --file=e:\cgi-bin\geniustrader\Scripts\frank_scan1.gconf 
13000 >test5.png
(full patch to conf file used used!)
--> config file read, start/end limits work, but no voting line is added 
(multiline problem?)
Error given:
VotingLine( Systems::Generic \ is not a valid object description at 
graphic.pl line 744.

perl graphic.pl --file=e:\cgi-bin\geniustrader\Scripts\frank_scan2.gconf 
13000 >test6.png
(full patch to conf file used used!)
--> config file read, start/end limits work, but no voting line is added
VotingLine( Systems::Generic \ is not a valid object description at 
graphic.pl line 744.


* Testing graphic.pl on V709 (exactly as provided on the development 
page, placing modules under "GT" and scripts under "Scripts", and 
copying the same "data" folder as used in r 690).

perl graphic.pl 13000 > test1.png
--> processing takes at least 10x longer than with v690, a file 
test1.png is generated, but cannot be opened ("invalid png file" tells 
me Paint Shop Pro or MS Paint)


* Testing graphic.pl on V. daytrade (clkao)
perl graphic.pl 13000 > test1.png  (ok)
perl graphic.pl --start=2002-05-01 --end=2002-06-01 13000 > test2.png  (ok)

perl graphic.pl --file=frank_scan1.gconf 13000 >test3.png,
where  franks_scan1.gconf is attached as before.
  --> errors due to multiline systems not supported

perl graphic.pl --file=frank_scan2.gconf 13000 >test4.png,
where  franks_scan2.gconf is attached
--> works ok

changing now data to fiat1mr (my own data; 1 minute scan for 5 months; 
3MB file; zipped with this email)
perl graphic.pl --file=frank_scan2b.gconf fiat1mr >test8.png,
(frank_scan2b.gconf simply changes timeframe to timeframe=1min)
"Intraday support not implemented in DB::Text"

perl graphic.pl --file=frank_scan3.gconf fiat1mr >test9.png,
(my data is 1m candles, the timeframe in the config file is 'day')
the png is generated (attached file test9.png), but there are 'only' 120 
candles (out of 62165 rows of data!) and I cannot understand what has 
been plotted.
In case when you have timeframe 1m data, but ask for 'day' processing 
what is Geniustrader doing?
(does it process automatically all the 1min data to form the day candle?)

....

Similar tests have been done for the other functions, but as you can see 
from the above results report at the moment I am stuck.
V 690 and v 709 do not seem to work properly  (graphic.pl does not seem 
to process at all the system I passed them in a configuration file and I 
need this to work to be able to pass my system to see if the buy/sell 
points are correct).
V. daytrade (clkao) works to a certain extent (without multiline 
systems) but only with day support (no intraday allowed) and I cannot 
even understand the result graphic for my 1min data...

Even more problems are coming from the other scripts.... and debugging 
is now taking far more time than I hoped for.

This is why I ask you great users/developers:
  - do you have a decent version which works for you under Windows that 
i can try to start with, something like the clkao version, but with 
intraday (absolutely a must) and multiline support (nice to have to 
understand the systems I test and easily vary their parameters...).
- did anyone attempt to write a different scan.pl without process 
priorities (so it can be compatible with Windows) or have a different 
solution for scanning signals under Windows?

Thanks you once again for all your help.
I would like very much to help the community and do some live testing 
and improvements to the program, but starting from something that works 
not to add my mistakes to others already (maybe) solved!

Thanks
Gianni



On 23/12/2010 20.14, Robert A. Schmied wrote:
> Gianni76 wrote:
>
> aloha gianni
>
> this is ras, i haven't heard from nick in some time ...
>
>> Dear Nick, Ras
>> I am trying to set GeniusTrader up and running on Windows. I have 
>> seen your names on the GeniusTrader developers list (isn't the list 
>> working anymore? the last message stops in August... 
>
> we have a new home at sourceforge, new mailing lists (ml) there as well:
>
> http://sourceforge.net/projects/geniustrader/
> http://sourceforge.net/mailarchive/forum.php?forum_name=geniustrader-system-traders 
>
> http://sourceforge.net/mailarchive/forum.php?forum_name=geniustrader-devel 
>
>
> i've had a load of trouble getting ml -devel up and working at sf. it 
> seems
> to be working now, but it is hard to tell unless members post messages
> ... so this is perfect opportunity to post there.
>
> you have to subscribe to post messages, and don't need a sourceforge id.
>
> the extensively revised home page is at http://geniustrader.org/
>
>> http://www.geniustrader.org/lists/devel/index.html) and I hope you 
>> can help.
>> I need it on Windows as I wish to interface it to a trading (order) 
>> platform which only runs on Windows.
>
> that is a terrible dilemma to be on ;-)
>
> solution is to install and run a lamp
>
>> My setup is a default Xampp server + ActiveState Perl 5.8
>>
>> But I am having trouble in making Geniustrader work properly
>> I tested v 690, v709 and then went back to some older versions.
>> But they all have issues.
>
> i don't have a clue about m$ crap, nor understand why anyone would
> want to use it when other compatible options exist, so i'm not going
> to be too helpful, specifically, but i will make some general comments
> that might help you, provided you can hack on the gt perl code ...
>
> João Costa { joaocosta at zonalivre dot org } is/was the principle
> involved with the windoze installer thingy, maybe he can/will help
> with an updated version ... as it is multimegabytes of (useless)
> binary data i keep my versions of it on /dev/null.
>
>>
>> Basically graphic.pl works perfectly (with all options) on older 
>> versions
>
> older versions of what?
>  windblows?
> or
>  graphic.pl and gt tool-kit?
>
>
> probably the best way to access the entire svn repo is via the web page:
> http://geniustrader.org/websvn/listing.php?repname=group.Geniustrader
>
> the head on the trunk is 709 -- 690 for tool-kit and 691 for scripts
> the head on the exp branch is 650 for tool-kit and 642 for scripts
>
> note: Weigert, Thomas { weigert at mst dot edu } is the principle on
> the exp branch and if memory serves is also a 'dozer.
>
> looks like 519 for the windows installer on any of the branches
>
>
> (but I can only get it to work with a day timeframe and not
>> with intraday
>> support from a newer Text.pm)
>
> have you checked the new advanced configuration features available in 
> Text.pm?
>
> without specifics on your intraday format and your specific scan 
> sys-sig-indic
> description spec i cannot evaluate, but last i checked i believe 
> intraday was
> working at least with graphic.pl.
>
> if you can provide specifics i will evaluate against my version, but 
> it is *nix based.
>
> or on the web/cgi interface (limited
>> though as it does not read configuration files).
>
> yea! that is a major oversight -- can you suggest a way to safely 
> (website wise) fix this?
>
>> The newer 690 and 709 versions do not work properly (the -- start and 
>> -- end for example are not recognised as many other options...).
>
> for this you must have perl module Date::Manip. it might not be part 
> of the windozer installer?
>
>>
>> scan.pl which seems the most useful script never worked for me in any 
>> version.
>> The problem is with the IPC SysV library, which does not exist for 
>> Windows
>> ....
>> Can't locate IPC/SysV.pm in @INC (@INC contains: .. 
>> E:\servers\xampp\cgi-bin\gtd
>> ay E:/servers/perl/site/lib E:/servers/perl/lib .) at scan.pl line 19.
>> BEGIN failed--compilation aborted at scan.pl line 19.
>> .....
>
> possible options:
> 1) windblows must surely have some sort of ipc support by now
>    (or maybe not) but assuming it does and there is a perl module
>    that works with it you could port scan.pl to use that ipc 
> interface/api.
> 2) strip out all multi-processing features from scan.pl entirely.
> 3) wrap the ipc stuff inside eval blocks and only use when
>    a) platform supports it
>    b) user calls for it
>
> of these two options i think 2) is the easiest solution ... others
> have complained about the ipc stuff in the past and i had to hack
> it some just to get it working locally (solaris), so my take is
> that the implementor was just learning/experimenting with the ipc
> api and maybe a dual processor system.
> option 3) is the best technical approach, provided it includes
> hooks that would allow/support option 1) should that be available.
>
>>
>> I haven't seen this mentioned anywhere on the main list (and I read 
>> dozens of messages to get a good understanding of the program...).
>> Is there any way to make it run (even with no priorities)?
>> Has anyone written another version of it compatible with Windows 
>> distributions?
>>
>> Or in general do you have a (recent) version of GeniusTrader which 
>> has been tested and runs on Windows?
>> I miss the scan.pl, the intraday (1min, 5min) support and the 
>> multiline command support...
>
> multiline command support? -- i don't understand this.
>
>>
>> Thank you
>> Gianni
>>
>>
>>
>
> in summary:
>
> you *must* have perl module Date::Manip with any of the newer gt 
> versions.
> it may or may not be part of the current windoze installer.
>
> windoze isn't something i can/will develop or test on, so volunteers are
> needed to test and advance that port.
>
> removing the ipc stuff from scan.pl (or wrapping it inside a 
> conditional eval block)
> is probably a worth-while endeavor.
>
> aloha
>
> merry christmas and a happy and profitable new year
>
> ras
>
>
>

Re: [GT] GeniusTrader: any windows working version?

[Robert A. S. <ra...@ac...>]

Gianni76 wrote:

aloha gianni

this is ras, i haven't heard from nick in some time ...

> Dear Nick, Ras
> I am trying to set GeniusTrader up and running on Windows. I have seen 
> your names on the GeniusTrader developers list (isn't the list working 
> anymore? the last message stops in August... 

we have a new home at sourceforge, new mailing lists (ml) there as well:

http://sourceforge.net/projects/geniustrader/
http://sourceforge.net/mailarchive/forum.php?forum_name=geniustrader-system-traders
http://sourceforge.net/mailarchive/forum.php?forum_name=geniustrader-devel

i've had a load of trouble getting ml -devel up and working at sf. it seems
to be working now, but it is hard to tell unless members post messages
... so this is perfect opportunity to post there.

you have to subscribe to post messages, and don't need a sourceforge id.

the extensively revised home page is at http://geniustrader.org/

> http://www.geniustrader.org/lists/devel/index.html) and I hope you can 
> help.
> I need it on Windows as I wish to interface it to a trading (order) 
> platform which only runs on Windows.

that is a terrible dilemma to be on ;-)

solution is to install and run a lamp

> My setup is a default Xampp server + ActiveState Perl 5.8
> 
> But I am having trouble in making Geniustrader work properly
> I tested v 690, v709 and then went back to some older versions.
> But they all have issues.

i don't have a clue about m$ crap, nor understand why anyone would
want to use it when other compatible options exist, so i'm not going
to be too helpful, specifically, but i will make some general comments
that might help you, provided you can hack on the gt perl code ...

João Costa { joaocosta at zonalivre dot org } is/was the principle
involved with the windoze installer thingy, maybe he can/will help
with an updated version ... as it is multimegabytes of (useless)
binary data i keep my versions of it on /dev/null.

> 
> Basically graphic.pl works perfectly (with all options) on older 
> versions

older versions of what?
  windblows?
or
  graphic.pl and gt tool-kit?


probably the best way to access the entire svn repo is via the web page:
http://geniustrader.org/websvn/listing.php?repname=group.Geniustrader

the head on the trunk is 709 -- 690 for tool-kit and 691 for scripts
the head on the exp branch is 650 for tool-kit and 642 for scripts

note: Weigert, Thomas { weigert at mst dot edu } is the principle on
the exp branch and if memory serves is also a 'dozer.

looks like 519 for the windows installer on any of the branches


 (but I can only get it to work with a day timeframe and not 
> with intraday
> support from a newer Text.pm)

have you checked the new advanced configuration features available in Text.pm?

without specifics on your intraday format and your specific scan sys-sig-indic
description spec i cannot evaluate, but last i checked i believe intraday was
working at least with graphic.pl.

if you can provide specifics i will evaluate against my version, but it is *nix based.

 or on the web/cgi interface (limited 
> though as it does not read configuration files).

yea! that is a major oversight -- can you suggest a way to safely (website wise) fix this?

> The newer 690 and 709 versions do not work properly (the -- start and -- 
> end for example are not recognised as many other options...).

for this you must have perl module Date::Manip. it might not be part of the windozer installer?

> 
> scan.pl which seems the most useful script never worked for me in any 
> version.
> The problem is with the IPC SysV library, which does not exist for Windows
> ....
> Can't locate IPC/SysV.pm in @INC (@INC contains: .. 
> E:\servers\xampp\cgi-bin\gtd
> ay E:/servers/perl/site/lib E:/servers/perl/lib .) at scan.pl line 19.
> BEGIN failed--compilation aborted at scan.pl line 19.
> .....

possible options:
 1) windblows must surely have some sort of ipc support by now
    (or maybe not) but assuming it does and there is a perl module
    that works with it you could port scan.pl to use that ipc interface/api.
 2) strip out all multi-processing features from scan.pl entirely.
 3) wrap the ipc stuff inside eval blocks and only use when
    a) platform supports it
    b) user calls for it

of these two options i think 2) is the easiest solution ... others
have complained about the ipc stuff in the past and i had to hack
it some just to get it working locally (solaris), so my take is
that the implementor was just learning/experimenting with the ipc
api and maybe a dual processor system.
option 3) is the best technical approach, provided it includes
hooks that would allow/support option 1) should that be available.

> 
> I haven't seen this mentioned anywhere on the main list (and I read 
> dozens of messages to get a good understanding of the program...).
> Is there any way to make it run (even with no priorities)?
> Has anyone written another version of it compatible with Windows 
> distributions?
> 
> Or in general do you have a (recent) version of GeniusTrader which has 
> been tested and runs on Windows?
> I miss the scan.pl, the intraday (1min, 5min) support and the multiline 
> command support...

multiline command support? -- i don't understand this.

> 
> Thank you
> Gianni
> 
> 
> 

in summary:

you *must* have perl module Date::Manip with any of the newer gt versions.
it may or may not be part of the current windoze installer.

windoze isn't something i can/will develop or test on, so volunteers are
needed to test and advance that port.

removing the ipc stuff from scan.pl (or wrapping it inside a conditional eval block)
is probably a worth-while endeavor.

aloha

merry christmas and a happy and profitable new year

ras

[GT] the geniustrader-devel mail list might actually be operational ...

[Robert A. S. <ra...@ac...>]

gt developers

maybe this -devel list is finally working, which includes forwarding
messages to subscribers, archiving them, and handling replys from
subscribers ... on time and use will tell if this is true.

this pose will recap all my prior postings about the relocation of gt
to sourceforge and describe the new website primarily to ensure it gets
into the archive, but also to hopefully get it the the subscribers
in the chance they failed to get them (i know i only got 2 of the 7
or so posts).

the new gt repository home -- http://sourceforge.net/projects/geniustrader/



check out the new gt web home -- http://geniustrader.org

it has lots of changes including:

  a screenshots how to page
  an extensively expanded faq page
  expanded wiki (user guide)
  bugtrackers one for the site and the geniustrader code
  use the development page to get the head of trunk tarball for GT and Scripts

unfortunately it seems to me anyway that this sourceforge hosted -devel mailing
list is somehow wedged, i don't see any new messages since late aug 2010 being
added to the archive, and get very few of the messages i've tried to post since
it went active at sourceforge. maybe this one will be different. this issue is
being worked on, stay tuned ...

however the -system-trades mailing list does seem operational.

feel free to contact me directly if you have any questions or comments about the
relocated gt.



notes on the new repository and a 'how to' on how to re-aim your local svn clone
(checkout, repo, copy, whatever you call it) to the new svn server at sourceforge.
still as of now (13dec2010) i have yet to attempt this ...

aloha

ras



how to relocate your svn checkout clone

none of the following is necessary if you haven't made any changes in your svn
clone. you can either ignore (for now) the server change or simply start with a clean
checkout per the instructions (http://sourceforge.net/projects/geniustrader/develop
or http://geniustrader.org/development.html)

but if you have anything valuable in your svn checkout area i'd recommend you make
a backup of it before doing the following. i've made a bzipped tarball with

   %   gtar -cjf svn_repo_backup.tar.bz ./svn_repo/

that appears to be intact.

you could also (probably) just copy recursively the entire dir hierarchy to
a new location, after all a svn checkout is just a filesystem dir hierarchy
(correct?).

finally, before you try any of these commands read thru this entire note, and
then maybe wait a few days to see if anyone suggests corrections or a better way.
also carefully evaluate your situation compared to the one i describe here. while
the general solution might be similar i doubt it will be exactly the same.

it appears relatively straight forward and simple to change the URL that's embedded
throughout a svn repo clone. just use the svn command

    ''svn switch --relocate FROM TO path''

(http://svnbook.red-bean.com/nightly/en/svn.ref.svn.c.switch.html)

so for a svn repo clone like my partial gt (only 3 dirs on the trunk branch)
at ./svn_repo/
GT/
Scripts/
website/

the svn client command (might be) [note: i've yet to attempt this -- hopefully
smarter people will acknowledge this is the correct course of action]:

%   svn switch --relocate \
    https://devel.geniustrader.org:8082/svn/trunk/GT \
    https://geniustrader.svn.sourceforge.net/svnroot/geniustrader/trunk/GT \
    ./svn_repo/GT

notice both URL references identify the branch (trunk) and the directory (GT)
but the path (last argument) reflects my actual filesystem pathname.

so for each of my 3 gt dirs ( GT Scripts and website) i might run a csh loop like

 %  foreach d ( GT Scripts website )
 ?    svn switch --relocate \
      https://devel.geniustrader.org:8082/svn/trunk/${d} \
      https://geniustrader.svn.sourceforge.net/svnroot/geniustrader/trunk/${d} \
      ./svn_repo/${d}
 end


so how does one 'find' this information you are probably thinking? use the svn command

   ''svn info target''

(http://svnbook.red-bean.com/nightly/en/svn.ref.svn.c.info.html)

and do it for both your local checkout clone as well as for the new server, just to
ensure things do indeed match up.


%     svn info ./svn_repo/GT
Path: svn_repo/GT
URL: https://devel.geniustrader.org:8082/svn/trunk/GT
Repository Root: https://devel.geniustrader.org:8082/svn
Repository UUID: bb1899a8-85f5-0310-a327-fe9f4e15340a
Revision: 664
Node Kind: directory
Schedule: normal
Last Changed Author: ras
Last Changed Rev: 664
Last Changed Date: 2009-04-20 12:03:02 -0700 (Mon, 20 Apr 2009)

%    svn info https://geniustrader.svn.sourceforge.net/svnroot/geniustrader/trunk/GT
Path: GT
URL: https://geniustrader.svn.sourceforge.net/svnroot/geniustrader/trunk/GT
Repository Root: https://geniustrader.svn.sourceforge.net/svnroot/geniustrader
Repository UUID: bb1899a8-85f5-0310-a327-fe9f4e15340a
Revision: 709
Node Kind: directory
Last Changed Author: ras
Last Changed Rev: 690
Last Changed Date: 2009-07-29 22:18:51 -0700 (Wed, 29 Jul 2009)


note that my svn clone is way out of date with respect to the current trunk.
none of that matters it seems, but in order for me to be able to continue to
use my svn clone i must 'fix' it to use the new server url. i'm hopeful this
'how to' describes the steps that will do just that.

[GT] Re: git versus svn

[Thomas W. <we...@ms...>]

I am using git and svn, and I still believe that svn is preferable for
our application and development mode.  Th.

On 08/10/2010 10:04 AM, Robert A. Schmied wrote:
> to git or svn -- that is the question
>
> and this is the discussion thread for it at least as far as gt is
> concerned.
>
> ras
>
>
> immediate benefits for git
>                    -- it provides distributed scm, where svn does not
>                    -- it (reportedly) simplifies patch-set or
> change-set management
>
> immediate negatives for git
>                    -- the current geniustrader repo, process(es) and
> methodology(ies)
>                       are grounded in svn and cvs before that.
>                    -- the switch to git will entail much work most of
> it not actually
>                       in switching the repository, but with the svn to
> git changeover
>
> concerns about git?
>                    -- all or nothing access rights?
>                    -- windows port?
>                    -- git does not allow partial (repo) checkouts?
>
>
>
> looks to me, a noted change resistant unix gray-beard, that git is the
> preferred choice
> *provided* we can put in place sufficient process to make the
> transition. this includes
> converting all the legacy stuff in existing geniustrader repository
> and purge whatever
> is unneeded (or put it in (take your pick) the attic, basement,
> cellar, dustbin, junkyard.
>
> unfortunately, without guidance and generalized instruction i'm not
> confident i'm the
> one for that job ...
>
> assuming we decide *not* to go that route, or just *not* right now git
> fans can still
> use git as their local scm -- see the url
> http://flavio.castelli.name/howto_use_git_with_svn
>
>
> some interesting urls -- i'm sure there are many many others
> http://www.viget.com/extend/effectively-using-git-with-subversion/
> http://flavio.castelli.name/howto_use_git_with_svn
>
>
>
>
>
>
> from http://techblog.floorplanner.com/2008/12/09/git-vs-svn-for-bosses/
>
> it seems that in svn branches are problematic, but greatly improved in
> svn 1.5 ...
>
> [i don't know what versions are involved at either the
> geniustrader.org or the
> geniustrader sf site (http://sourceforge.net/projects/geniustrader/),
> but locally i'm running 'svn, version 1.4.3 (r23084)' which i could
> upgrade (probably),
> but note that i only used it for raphaels gt site commits ... as
> locally i still
> operate caveman style with rcs]
>
> but back to branches and svn -- i think if there are/is the processes
> and methods
> in place that dictate how branches are to be employed (over the
> complete lifecycle)
> then maybe the svn implementation is adequate ...
>
>   create branch (svn)
>   svn copy URL_TO_TRUNK URL_TO_BRANCH
>
>   merge branch back to trunk (svn)
>   svn merge --reintegrate URL_TO_TRUNK
>
> this works best (only works at all) if the change(s) are isolated or
> the whole branch
> is to be merged -- again, it comes down to actually having and then
> actually following a process ...
>
> if some-one has written such a process, or has seen one that makes
> sense and can provide
> a reference to it, or wants to contribute a draft strawman process
> please do so.
>
>

[GT] git versus svn

[Robert A. S. <ra...@ac...>]

to git or svn -- that is the question

and this is the discussion thread for it at least as far as gt is concerned.

ras


immediate benefits for git
                    -- it provides distributed scm, where svn does not
                    -- it (reportedly) simplifies patch-set or change-set management

immediate negatives for git
                    -- the current geniustrader repo, process(es) and methodology(ies)
                       are grounded in svn and cvs before that.
                    -- the switch to git will entail much work most of it not actually
                       in switching the repository, but with the svn to git changeover

concerns about git?
                    -- all or nothing access rights?
                    -- windows port?
                    -- git does not allow partial (repo) checkouts?



looks to me, a noted change resistant unix gray-beard, that git is the preferred choice
*provided* we can put in place sufficient process to make the transition. this includes
converting all the legacy stuff in existing geniustrader repository and purge whatever
is unneeded (or put it in (take your pick) the attic, basement, cellar, dustbin, junkyard.

unfortunately, without guidance and generalized instruction i'm not confident i'm the
one for that job ...

assuming we decide *not* to go that route, or just *not* right now git fans can still
use git as their local scm -- see the url http://flavio.castelli.name/howto_use_git_with_svn


some interesting urls -- i'm sure there are many many others
http://www.viget.com/extend/effectively-using-git-with-subversion/
http://flavio.castelli.name/howto_use_git_with_svn






from http://techblog.floorplanner.com/2008/12/09/git-vs-svn-for-bosses/

it seems that in svn branches are problematic, but greatly improved in svn 1.5 ...

[i don't know what versions are involved at either the geniustrader.org or the
geniustrader sf site (http://sourceforge.net/projects/geniustrader/),
but locally i'm running 'svn, version 1.4.3 (r23084)' which i could upgrade (probably),
but note that i only used it for raphaels gt site commits ... as locally i still
operate caveman style with rcs]

but back to branches and svn -- i think if there are/is the processes and methods
in place that dictate how branches are to be employed (over the complete lifecycle)
then maybe the svn implementation is adequate ...

   create branch (svn)
   svn copy URL_TO_TRUNK URL_TO_BRANCH

   merge branch back to trunk (svn)
   svn merge --reintegrate URL_TO_TRUNK

this works best (only works at all) if the change(s) are isolated or the whole branch
is to be merged -- again, it comes down to actually having and then actually following a process ...

if some-one has written such a process, or has seen one that makes sense and can provide
a reference to it, or wants to contribute a draft strawman process please do so.

[GT] Re: RE: Moving to Sourceforge

[Raphael H. <ra...@ge...>]

On Thu, 05 Aug 2010, Bja...@no... wrote:
> Hi,
> Ok, sounds like the right decision at this stage. Let's just stick with the current mode of operation for now, and have a separate discussion on how with might be able to improve the workflow in the future.
> 
> > 1/ I have picked the basic set of features to match the existing setup. No we
> > need to find out how to load all the data that we have in sourceforge.
> > Bjorne, do you feel like driving this process?
> 
> Yes, I can drive this part.
> 
> My username at SF is "bjchrist"

I added you and ras to the project. You are both project admin so that you
can more people and fix anything that need fixing.

Cheers,
-- 
Raphaël Hertzog ◈ Writer/Consultant ◈ Debian Developer ◈ [Flattr=20693]

Master Your Debian/Ubuntu System ▶ http://RaphaelHertzog.com (English)
                                 ▶ http://RaphaelHertzog.fr (Français)

[GT] Re: Moving to Sourceforge

[Robert A. S. <ra...@ac...>]

Thomas Weigert wrote:
> Guys,
> 
> I would, at least for the time being, like to continue the current mode
> where checkins are vetted by RAS before they are added into the official
> repository. We have exploratory branches where everybody can check in,
> but the main trunk should be carefully vetted. RAS did a great job at
> that and I would hope he is willing to consider doing so.

thanks thomas for the ata boy, but i wouldn't use 'great' as the adjective.
however, i am willing to continue in that role but still believe it is a
shared responsibility...

> 
> In any case, we should agree on the mode of operation as we transition
> to a new environment...

i too would like to see some sort of 'order of business' guidelines
going forward.

specifically two (at least) items need to be addressed ...

first the need to use a tracker in lieu of the devel mailing list to manage
user comments. i don't know enough about how sourceforge (sf) works yet
vis-a-vis the myriad add-on applications, but do know that many bugs and
improvements went missing because we (that's the collective gt we) failed
to use any sort of tracker to manage them in a controlled/visible way.

secondly, i'd like to see a defined strategy on naming and tagging releases
and maybe a defined strategy on branches. in the past i found that a
''branch on purpose'' method worked well when code regularly evolved,
but still had a requirement to re-establish any prior 'release' for either
re-creation or re-creation and improvement.

the result was a main line trunk with branches that usually dead-ended at
a named (tagged) release. merges back to the truck happened when there
was an agreed need/want/willingness to do. this might be over-kill for gt,
but the model fits, since merges are a lot of work and time is limited.
but it's only one of many naming/tagging/branching schemes, so long as
one is defined that is simple enough to follow and then followed i'd be
happy(ier).

currently we use the head is current release and there is no ''official''
naming/tagging scheme.

aloha

ras (on sf ras2010)

> 
> Cheers, Th.
> 
> On 08/05/2010 01:29 AM, Bja...@no... wrote:
> 
>>Hi,
>>Ok, sounds like the right decision at this stage. Let's just stick with the current mode of operation for now, and have a separate discussion on how with might be able to improve the workflow in the future.
>>
>>  
>>
>>>1/ I have picked the basic set of features to match the existing setup. No we
>>>need to find out how to load all the data that we have in sourceforge.
>>>Bjorne, do you feel like driving this process?
>>>    
>>
>>Yes, I can drive this part.
>>
>>My username at SF is "bjchrist"
>>
>>Cheers,
>>Bjarne  
>>
>>
>>-----Original Message-----
>>From: ext Raphael Hertzog [mailto:ra...@ou...] 
>>Sent: 2. august 2010 23:38
>>To: de...@ge...
>>Subject: [GT] Moving to Sourceforge
>>
>>Hello,
>>
>>so I took the decision and sourceforge it's going to be:
>>https://sourceforge.net/projects/geniustrader
>>
>>Please give the login of your sourceforge account so that I can add you to
>>the project.
>>
>>Here's what I just did:
>>
>>1/ I have picked the basic set of features to match the existing setup. No we
>>need to find out how to load all the data that we have in sourceforge.
>>Bjorne, do you feel like driving this process?
>>
>>2/ I have dumped the content of the current website in the web space but it
>>will need some editing I guess:
>>http://geniustrader.sourceforge.net
>>
>>I copied the "/doc" and "/lists" as of today but those are supposed to be
>>auto-updated and that won't be the case on sourceforge, we should
>>probably keep /lists for archives. Don't know about /doc.
>>
>>3/ I have made an archive of trac related files but I have no idea how to
>>load this into the SF trac instance. I'll provide the tarball if needed.
>>
>>4/ For the mailing list I had to open a ticket because it used to exist in
>>the past:
>>https://sourceforge.net/apps/trac/sourceforge/ticket/12952
>>
>>5/ For SVN, I made the current repository read-only. I already moved it
>>to sourceforge:
>>https://sourceforge.net/scm/?type=svn&group_id=339432
>>
>>Cheers,
>>  
> 
> 

[GT] Re: RE: Moving to Sourceforge

[Thomas W. <we...@ms...>]

Guys,

I would, at least for the time being, like to continue the current mode
where checkins are vetted by RAS before they are added into the official
repository. We have exploratory branches where everybody can check in,
but the main trunk should be carefully vetted. RAS did a great job at
that and I would hope he is willing to consider doing so.

In any case, we should agree on the mode of operation as we transition
to a new environment...

Cheers, Th.

On 08/05/2010 01:29 AM, Bja...@no... wrote:
> Hi,
> Ok, sounds like the right decision at this stage. Let's just stick with the current mode of operation for now, and have a separate discussion on how with might be able to improve the workflow in the future.
>
>   
>> 1/ I have picked the basic set of features to match the existing setup. No we
>> need to find out how to load all the data that we have in sourceforge.
>> Bjorne, do you feel like driving this process?
>>     
> Yes, I can drive this part.
>
> My username at SF is "bjchrist"
>
> Cheers,
> Bjarne  
>
>
> -----Original Message-----
> From: ext Raphael Hertzog [mailto:ra...@ou...] 
> Sent: 2. august 2010 23:38
> To: de...@ge...
> Subject: [GT] Moving to Sourceforge
>
> Hello,
>
> so I took the decision and sourceforge it's going to be:
> https://sourceforge.net/projects/geniustrader
>
> Please give the login of your sourceforge account so that I can add you to
> the project.
>
> Here's what I just did:
>
> 1/ I have picked the basic set of features to match the existing setup. No we
> need to find out how to load all the data that we have in sourceforge.
> Bjorne, do you feel like driving this process?
>
> 2/ I have dumped the content of the current website in the web space but it
> will need some editing I guess:
> http://geniustrader.sourceforge.net
>
> I copied the "/doc" and "/lists" as of today but those are supposed to be
> auto-updated and that won't be the case on sourceforge, we should
> probably keep /lists for archives. Don't know about /doc.
>
> 3/ I have made an archive of trac related files but I have no idea how to
> load this into the SF trac instance. I'll provide the tarball if needed.
>
> 4/ For the mailing list I had to open a ticket because it used to exist in
> the past:
> https://sourceforge.net/apps/trac/sourceforge/ticket/12952
>
> 5/ For SVN, I made the current repository read-only. I already moved it
> to sourceforge:
> https://sourceforge.net/scm/?type=svn&group_id=339432
>
> Cheers,
>   

[GT] RE: Moving to Sourceforge

[<Bja...@no...>]

Hi,
Ok, sounds like the right decision at this stage. Let's just stick with the current mode of operation for now, and have a separate discussion on how with might be able to improve the workflow in the future.

> 1/ I have picked the basic set of features to match the existing setup. No we
> need to find out how to load all the data that we have in sourceforge.
> Bjorne, do you feel like driving this process?

Yes, I can drive this part.

My username at SF is "bjchrist"

Cheers,
Bjarne  


-----Original Message-----
From: ext Raphael Hertzog [mailto:ra...@ou...] 
Sent: 2. august 2010 23:38
To: de...@ge...
Subject: [GT] Moving to Sourceforge

Hello,

so I took the decision and sourceforge it's going to be:
https://sourceforge.net/projects/geniustrader

Please give the login of your sourceforge account so that I can add you to
the project.

Here's what I just did:

1/ I have picked the basic set of features to match the existing setup. No we
need to find out how to load all the data that we have in sourceforge.
Bjorne, do you feel like driving this process?

2/ I have dumped the content of the current website in the web space but it
will need some editing I guess:
http://geniustrader.sourceforge.net

I copied the "/doc" and "/lists" as of today but those are supposed to be
auto-updated and that won't be the case on sourceforge, we should
probably keep /lists for archives. Don't know about /doc.

3/ I have made an archive of trac related files but I have no idea how to
load this into the SF trac instance. I'll provide the tarball if needed.

4/ For the mailing list I had to open a ticket because it used to exist in
the past:
https://sourceforge.net/apps/trac/sourceforge/ticket/12952

5/ For SVN, I made the current repository read-only. I already moved it
to sourceforge:
https://sourceforge.net/scm/?type=svn&group_id=339432

Cheers,
-- 
Raphaël Hertzog ◈ Writer/Consultant ◈ Debian Developer ◈ [Flattr=20693]

Master Your Debian/Ubuntu System ▶ http://RaphaelHertzog.com (English)
                                 ▶ http://RaphaelHertzog.fr (Français)

[GT] Re: Re: Are your have plans to switch from plain-text data to a DBMS?

[Alex D. <ald...@di...>]

> For whatever it is worth, I use only text files. I run almost continuous 
> analysis on data and found that the text files suite me just fine, and that 
> there is no need for monkeying with a data base. I update the text files from 
> yahoo.com programmatically.... Th.
>   
Thanks Thomas,

in fact, it is possible to use modules DBI::CVS, DBI::AnyData for the
data from text files you have. In this sense, DBI:: by itself the level
of abstraction of "data sources", but standardized. Using some code,
which itself serves as DBI::CVS, in my view makes sense when you need to
increase speed. Thus, the existing version is optimized for text. If we
assume that mainstream use database to store both data and an array of
financial (trade) functions and settings and other information, then it
makes sense to switch completely to the DBI::, which allows for backward
compatibility, as well as for those who not happy with a DBMS, to
continue to use the text at little loss of speed.

btw, monkeying is not so monkeying now, e.g.:
http://demo.phpmyadmin.net/trunk-config/
http://phppgadmin.sourceforge.net/?page=screenshots
so it is so hard as install and create some DB users with some right.

Cheers,
Alex.
> On 08/02/2010 10:05 AM, Alex Dubenetsky wrote:
> >
> > So I have a question that developers think about the refusal of text files and 
> > transfer all and everything in the DBMS. I wonder if they do consider it 
> > useful? Most likely it one way or another will have a number of issues on 
> > further development of GT and will require big changes.
> >
>   

[GT] Re: Are your have plans to switch from plain-text data to a DBMS?

[Thomas W. <we...@ms...>]

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
  <meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#ffffff">
For whatever it is worth, I use only text files. I run almost
continuous analysis on data and found that the text files suite me just
fine, and that there is no need for monkeying with a data base. I
update the text files from yahoo.com programmatically.... Th.<br>
<br>
On 08/02/2010 10:05 AM, Alex Dubenetsky wrote:
<blockquote cite="mid:4C5...@di..." type="cite">
  <meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
  <title></title>
  <span id="result_box" class="long_text"><span
 style="background-color: rgb(255, 255, 255);"
 title="Если всё устраивает - это одно, если систему можно (или нужно) развивать - это другое."
 onmouseover="this.style.backgroundColor='#ebeff9'"
 onmouseout="this.style.backgroundColor='#fff'"><br>
  </span><span style="background-color: rgb(255, 255, 255);"
 title="Поэтому у меня возник вопрос, что думают разработчики об отказе от текстовых файлов и переносу всего и вся в DBMS."
 onmouseover="this.style.backgroundColor='#ebeff9'"
 onmouseout="this.style.backgroundColor='#fff'">So I have a question
that developers think about the refusal of text files and transfer all
and everything in the DBMS. </span><span
 style="background-color: rgb(255, 255, 255);"
 title="Считают ли они это полезным?"
 onmouseover="this.style.backgroundColor='#ebeff9'"
 onmouseout="this.style.backgroundColor='#fff'"></span></span><span
 id="result_box" class="long_text"><span title="Интересно ли им это?"
 onmouseover="this.style.backgroundColor='#ebeff9'"
 onmouseout="this.style.backgroundColor='#fff'">I wonder</span></span><span
 id="result_box" class="long_text"><span
 style="background-color: rgb(255, 255, 255);"
 title="Считают ли они это полезным?"
 onmouseover="this.style.backgroundColor='#ebeff9'"
 onmouseout="this.style.backgroundColor='#fff'"> if they do consider it
useful? </span><span style="background-color: rgb(255, 255, 255);"
 title="Скорее всего это так или иначе вызовет ещё ряд вопросов о дальнейшей разработке GT и потребует больших изменений."
 onmouseover="this.style.backgroundColor='#ebeff9'"
 onmouseout="this.style.backgroundColor='#fff'">Most likely it one way
or another will have a number of issues on further development of GT
and will require big changes.</span></span><br>
  <br>
</blockquote>
</body>
</html>

[GT] Moving to Sourceforge

[Raphael H. <ra...@ou...>]

Hello,

so I took the decision and sourceforge it's going to be:
https://sourceforge.net/projects/geniustrader

Please give the login of your sourceforge account so that I can add you to
the project.

Here's what I just did:

1/ I have picked the basic set of features to match the existing setup. No we
need to find out how to load all the data that we have in sourceforge.
Bjorne, do you feel like driving this process?

2/ I have dumped the content of the current website in the web space but it
will need some editing I guess:
http://geniustrader.sourceforge.net

I copied the "/doc" and "/lists" as of today but those are supposed to be
auto-updated and that won't be the case on sourceforge, we should
probably keep /lists for archives. Don't know about /doc.

3/ I have made an archive of trac related files but I have no idea how to
load this into the SF trac instance. I'll provide the tarball if needed.

4/ For the mailing list I had to open a ticket because it used to exist in
the past:
https://sourceforge.net/apps/trac/sourceforge/ticket/12952

5/ For SVN, I made the current repository read-only. I already moved it
to sourceforge:
https://sourceforge.net/scm/?type=svn&group_id=339432

Cheers,
-- 
Raphaël Hertzog ◈ Writer/Consultant ◈ Debian Developer ◈ [Flattr=20693]

Master Your Debian/Ubuntu System ▶ http://RaphaelHertzog.com (English)
                                 ▶ http://RaphaelHertzog.fr (Français)

[GT] Re: [possible-spam:3.93] RE: Are your have plans to switch from plain-text data to a DBMS?

[Alex D. <ald...@di...>]

> Hi Alex,
> Your plans for developing a trading system sounds interesting. How far did you get and what are your plans for future development?
>   
Hi Bjarne,
    I have not gone too far: it loads the data from the web but a
tunable way, see below.
My plans for future development are build scalable trading system (I
mean software) that is allow relatively fast automated trading (from
tick or 1min. TF and above)  and use it...
My nearest to-do is to start to implement financial logic - indicators,
signals, strategies and so on, then analyzing, reporting, backtracking,
gateway to the trade system, GUI.

On the one hand there is GT, which is already part of the functionality
is implemented, on the other hand, as someone who is closer to the
technical side than the financial, the code and the decisions I have
ever seen in the GT, far from optimal. But they are, they work, and it
is great.
However, there is a sense that the system of GT is aimed at trading on
the day TF few tools. I am also interested in intraday trade. Also I am
interested in the use of a large number of indicators and strategies, in
search of certain dynamic trading strategies. This implies a greater
volume of data and tend to drive the use of DBMS. And not only for
quotations, as is done now, but for the entire program: the indicators
and signals and so on.

In this connection, I wonder what thoughts from developers and users
about the future of GT. If you're happy with - this one, if the system
can (or need) to develop - is another.


So I have a question that developers think about the refusal of text
files and transfer all and everything in the DBMS. I wonder if they do
consider it useful? Most likely it one way or another will have a number
of issues on further development of GT and will require big changes.

> GT does work with various sql dbs, see:
>
> GETTING STARTED WITH GT:
> https://devel.geniustrader.org:8082/trac/wiki/GTFirstUse
> Points you to how to set beancounter up to work with GT. I reckon you should be able to populate your data using the same schema.
>
>   
Yes, beancounter.
Beancounter store data in seven tables :

<http://myadmin/db_structure.php?db=beancounter&token=485c2329b7e4de6f90dde5b864afd0c2>

Обзор beancounter
Обзор cash
Обзор fxprices
Обзор indices
Обзор portfolio
Обзор stockinfo
Обзор stockprices

That's all.
BTW, table stockprices has no field 'time' so you can only use D, W, Y TFs.
> You'll find some info regarding the GT config here:
> https://devel.geniustrader.org:8082/trac/wiki/ConfigFile
>
> I think that's pretty much it, for getting you set up. 
Yes, this is enough to set up the system. The problem is - there is a
feeling that it was made to trade on the daily TF.
> I'm currently using quotes from Yahoo, but have similar plans for working with better data, I would be very interested to hear how it turns out! 
>
>   

That I'm talking about - in beanconter URL and format of the data are
_hardcoded__, _it is not customizable. Accordingly, you have several
options:
- Throw with something external data in the beancounter table
- Make a patch of Finance::YahooQ for reading data source, which you need
- Make a new library Finance::MyHiResNoDelaySource ...

Cheers,
Alex.
> Cheers,
> Bjarne 
>  
>
>
>
>
> -----Original Message-----
> From: ext Alex Dubenetsky [mailto:ald...@di...] 
> Sent: 2. august 2010 14:34
> To: de...@ge...
> Subject: [GT] Are your have plans to switch from plain-text data to a DBMS? 
>
> Hi developers,
>
> 	While I started to write something similar, I've occasionally found the
> GT. I'd like to have trading system written with Perl, but it looks more
> reasonable for me to improve an existed soft then finish my own. So, I'd
> like to ask your to make a little more clear you vision of further
> development - do your have on objective to migrate to a DBMS (mysql, pg,
> any DBI::* ) in your to-do list? Or are your "happy" with plain text
> files? Not only prices, the whole system: indexes, signals, TS, SY and
> others...
>
> 	I'm asking since by my side it looks like after same quantity of
> sources (prices and time-frames) it could turn from plain-text to
> pain-text. (Just for example, I have 1Gb/year of data (and 2Gb/year with
> indexes) for one security (with ticks)...)
>
> Cheers,
> Alex.
>
> 	
>
>   

[GT] RE: Are your have plans to switch from plain-text data to a DBMS?

[<Bja...@no...>]

Hi Alex,
Your plans for developing a trading system sounds interesting. How far did you get and what are your plans for future development?

GT does work with various sql dbs, see:

GETTING STARTED WITH GT:
https://devel.geniustrader.org:8082/trac/wiki/GTFirstUse
Points you to how to set beancounter up to work with GT. I reckon you should be able to populate your data using the same schema.


You'll find some info regarding the GT config here:
https://devel.geniustrader.org:8082/trac/wiki/ConfigFile

I think that's pretty much it, for getting you set up. I'm currently using quotes from Yahoo, but have similar plans for working with better data, I would be very interested to hear how it turns out! 

Cheers,
Bjarne 
 




-----Original Message-----
From: ext Alex Dubenetsky [mailto:ald...@di...] 
Sent: 2. august 2010 14:34
To: de...@ge...
Subject: [GT] Are your have plans to switch from plain-text data to a DBMS? 

Hi developers,

	While I started to write something similar, I've occasionally found the
GT. I'd like to have trading system written with Perl, but it looks more
reasonable for me to improve an existed soft then finish my own. So, I'd
like to ask your to make a little more clear you vision of further
development - do your have on objective to migrate to a DBMS (mysql, pg,
any DBI::* ) in your to-do list? Or are your "happy" with plain text
files? Not only prices, the whole system: indexes, signals, TS, SY and
others...

	I'm asking since by my side it looks like after same quantity of
sources (prices and time-frames) it could turn from plain-text to
pain-text. (Just for example, I have 1Gb/year of data (and 2Gb/year with
indexes) for one security (with ticks)...)

Cheers,
Alex.

	

[GT] Are your have plans to switch from plain-text data to a DBMS?

[Alex D. <ald...@di...>]

Hi developers,

	While I started to write something similar, I've occasionally found the
GT. I'd like to have trading system written with Perl, but it looks more
reasonable for me to improve an existed soft then finish my own. So, I'd
like to ask your to make a little more clear you vision of further
development - do your have on objective to migrate to a DBMS (mysql, pg,
any DBI::* ) in your to-do list? Or are your "happy" with plain text
files? Not only prices, the whole system: indexes, signals, TS, SY and
others...

	I'm asking since by my side it looks like after same quantity of
sources (prices and time-frames) it could turn from plain-text to
pain-text. (Just for example, I have 1Gb/year of data (and 2Gb/year with
indexes) for one security (with ticks)...)

Cheers,
Alex.

	

[GT] Re: Hosting the project

[Robert A. S. <ra...@ac...>]

Bja...@no... wrote:

<snip>
> 
> RAS> clks github site is such a mess i can hardly tell what is what, but that may be due to the site, my old brower, git and my lack of understanding git.
> 
> I doubt it's a browser issue. GitHub works quite well, but I agree, the interface is not very user-friendly, which is a bit of a shame.  

well the stock mozilla 5.0 (shipped with solaris 10) doesn't work so well,
firefox at least doesn't crash and does support the pulldown selection lists ...

<snip>

> 
> RAS> an item of concern are the gt copyright notices -- don't know how that legally might come back to haunt someone 
> RAS> involved without a legal paper trail ...
> 
> Can you elaborate a bit? I'm not sure if I understood you right...

i refer to the bits in the book 'How to Run a Successful Free Software Project' by Karl Fogel
(http://producingoss.com) chapter 9.
it's not a problem until it becomes a problem, but by then it's too late ...
unless the legalisms are put in place before hand.

<snip>

> 
> Cheers,
> Bjarne 
> 
> 

[GT] Re: RE: Re: RE: Re: Hosting the project

[Raphael H. <ra...@ou...>]

Hi,

Le mardi 27 juillet 2010, Bja...@no... a écrit :
> Fair enough, I tend prefer Mercurial, but I can definitely live with
> Git, if that's preferred. I'm a bit curious, what makes you prefer Git
> over Mercurial?

Is use Git a lot and I have almost no experience with Mercurial. Also I
have the feeling that Git is the DVCS that is spreading the most quickly.

> Not to be picky, but do you think we can get
> http://github.com/geniustrader, or is it possible to point a domain to

That's a good question. Can we have a shared repository on github or is
there only personal repositories?

> the repo, i.e. code.geniustrader.org ->
> github.com/clkao/finance-geniustrader?

The domain is hosted at gandi.net and we can have some services for free
if needed:
- web redirection (your example above)
- a blog http://www.gandi.net/domaine/blog/
  with multiple persons allowed to maintain the blog
- email aliases (for the mailing list for example)
  http://www.gandi.net/domaine/mail#nav

> RH> Sourceforge is very generic and probably the best default choice if
> RH> there's no consensus on the forge to pick.
> 
> Ok, let's say we rule out Mercurial, which means googlecode is a no-go
> (unless we stick with SVN...). In that case we could also consider
> http://indefero.net/ (see http://geniustrader.indefero.net/p/gt/), I
> really like the simple layout and it has support for git and Subversion.

But the storage is limited to 10Mb. Sourceforge has no restriction
apparently:
http://sourceforge.net/apps/trac/sourceforge/wiki/Disk%20quotas

> RH> We also need to consider the website and the mailing lists at least.
> 
> Thomas has previously offered some hosting possibility, but I tend to
> agree with you Raphael, we should as far as possible avoid using private
> server for hosting, at least for hosting the code. However, I haven't
> come across free webhosting for opensource projects, where you can point
> your own domain name? I reckon need something similar to what we have
> today?

Sourceforge offer all of this:
http://sourceforge.net/apps/trac/sourceforge/wiki/Custom%20VHOSTs

That's for the "point your own domain name". BTW alioth.debian.org or any
other FusionForge-based forge will offer the same.

Other features:
http://sourceforge.net/register-project/features.php

I'm not a big sourceforge fan (too much centralized there and sourceforge
itself is not free software) but it's certainly one of the best offering
out there. I used it for geniustrader before I switched to my own server.

> Well, it seems that we have a few volunteers who are willing to chip-in
> for getting this going. I speak for myself; I would like to get involved
> because I would like to see this project gaining more momentum, and I
> definitely think it would be very sad if the project just fades away! I
> don’t have time for getting everything in place, plus maintaining and
> driving the project in future, but I very much like to take an active
> part in the work required! So I would vote for some kind of joint
> effort. 

Great to hear that!

Cheers,
-- 
Raphaël Hertzog ◈ Writer/Consultant ◈ Debian Developer ◈ [Flattr=20693]

Master Your Debian/Ubuntu System ▶ http://RaphaelHertzog.com (English)
                                 ▶ http://RaphaelHertzog.fr (Français)

[GT] Re: Re: Re: RE: Re: Hosting the project

[Thomas W. <we...@ms...>]

We had this discussion half a year ago. I believe the situation has not
changed since then so we should stick with our previous decision of
continuing with svn.

Th.

On 07/27/2010 05:26 PM, Chia-liang Kao wrote:
> Hi,
>
> On 27 July 2010 23:16, Raphael Hertzog <ra...@ou...> wrote:
>   
>> Hi,
>>
>> Le mardi 27 juillet 2010, Bja...@no... a écrit :
>>     
>>> Like mentioned previously, there are several choices when it comes to
>>> hosting the code, here are a few:
>>>       
>> We also need to consider the website and the mailing lists at least.
>>
>> If we switch VCS, I have a clear preference for Git so even if I'm not
>> active in the development, I'm happy to make the choice for the team if
>> you can't pick up a common preferred VCS.
>>     
> FYI, i actually have the repository already mirrored on github:
> http://github.com/clkao/finance-geniustrader
>
> this contains the svn mirror (trunk and cpan branch), as well as all
> my patches maintained in separate branches.  we can start from here,
> either clone the current mainline branches, or simply use it as the
> published repository.  for the latter, i'm willing to give commit bits
> to people and perhaps help merging patches if my time allows.
>
> Cheers,
> CLK
>
>   

[GT] RE: Re: RE: Re: Hosting the project

[<Bja...@no...>]

Hi,

RH> If we switch VCS, I have a clear preference for Git so even if I'm not
RH> active in the development, I'm happy to make the choice for the team if
RH> you can't pick up a common preferred VCS.

Fair enough, I tend prefer Mercurial, but I can definitely live with Git, if that's preferred. I'm a bit curious, what makes you prefer Git over Mercurial?


CLK> FYI, i actually have the repository already mirrored on github:
CLK> http://github.com/clkao/finance-geniustrader
CLK> 
CLK> this contains the svn mirror (trunk and cpan branch), as well as all
CLK> my patches maintained in separate branches.  we can start from here,
CLK> either clone the current mainline branches, or simply use it as the
CLK> published repository.  for the latter, i'm willing to give commit bits
CLK> to people and perhaps help merging patches if my time allows.

Cool! 
Not to be picky, but do you think we can get http://github.com/geniustrader, or is it possible to point a domain to the repo, i.e. code.geniustrader.org -> github.com/clkao/finance-geniustrader?

RAS> clks github site is such a mess i can hardly tell what is what, but that may be due to the site, my old brower, git and my lack of understanding git.

I doubt it's a browser issue. GitHub works quite well, but I agree, the interface is not very user-friendly, which is a bit of a shame.  

RH> But given my git preference and the ML requirement (which should be hosted on
RH> googlegroups.com then), I'm not convinced it's the best choice.

RH> Sourceforge is very generic and probably the best default choice if
RH> there's no consensus on the forge to pick.

Ok, let's say we rule out Mercurial, which means googlecode is a no-go (unless we stick with SVN...). In that case we could also consider http://indefero.net/ (see http://geniustrader.indefero.net/p/gt/), I really like the simple layout and it has support for git and Subversion.



RH> We also need to consider the website and the mailing lists at least.

Thomas has previously offered some hosting possibility, but I tend to agree with you Raphael, we should as far as possible avoid using private server for hosting, at least for hosting the code. However, I haven't come across free webhosting for opensource projects, where you can point your own domain name? I reckon need something similar to what we have today?

Thomas, the server you were referring to, would that we a server with high availability, e.g. hosted somewhere? Would you be able give i.e. ftp access, so we can have multiple admins?

I have access to a dedicated hosted server, where I could host the website as well. I can't give ssh/ftp etc access to anyone, but I can setup some kind of CMS with shared admin rights.  I can't provide mail server or mailing lists. There must be a free hosted mailing list server out there that we can use, any recommendations? Otherwise I guess we could use groups.google.com? 



RAS> an item of concern are the gt copyright notices -- don't know how that legally might come back to haunt someone 
RAS> involved without a legal paper trail ...

Can you elaborate a bit? I'm not sure if I understood you right...

RAS> one aspect of running (setting up and admin'ing) that i have yet to find are any metrics regarding how much time 
RAS> one might expect to expend for just maintaining the site. (any hints on a search phrase that might work for this?)

RH> We will also have to decide who is admin on the new forge. Does someone want to assume the leadership or shall we simply 
RH> make all committers co-admins?

Well, it seems that we have a few volunteers who are willing to chip-in for getting this going. I speak for myself; I would like to get involved because I would like to see this project gaining more momentum, and I definitely think it would be very sad if the project just fades away! I don’t have time for getting everything in place, plus maintaining and driving the project in future, but I very much like to take an active part in the work required! So I would vote for some kind of joint effort. 


Cheers,
Bjarne