geniustrader-devel — participate in geniustrader development, ask for help, suggest improvements, etc

Re: [GT] Error during checkout

[Robert A. S. <ra...@ac...>]

Evgeny Shragovich wrote:
> Greetings Robert,
> 
> I am setting up GT on a fresh CentOS system and I get the following error
> when trying to checkout the code:
> 
> [Eugene@localhost ~]$ svn checkout
> https://geniustrader.svn.sourceforge.net/svnroot/geniustrader/trunk/GTgeniustrader/GT
> svn: Can't read file 'geniustrader/GT/.svn/entries': End of file found
> 
> 
> Any idea why it happens?
> 
> Cheers!
> Eugene
> 

aloha evgeny

well checkout for Scripts worked for me:


$ mkdir /var/tmp/evgeny_17sep13
$ cd /var/tmp/evgeny_17sep13
$ mkdir gt
$ mkdir gt/Scripts
$ mkdir gt/GT
$ cd gt/Scripts
$ svn checkout https://geniustrader.svn.sourceforge.net/svnroot/geniustrader/trunk/Scripts .
A    display_system.pl
A    report_old.ash
A    backtest_multi.pl
A    graphic.pl
A    analyze_backtest.pl
A    Templates
A    Templates/backtest.mhtml
A    Templates/portfolio_historic.mhtml
A    Templates/analyze_backtest.mhtml
A    Templates/portfolio_positions.mhtml
A    Templates/backtest.mpl
A    Templates/portfolio_historic.mpl
A    Templates/analyze_backtest.mpl
A    Templates/portfolio_positions.mpl
A    backtest_many.pl
A    scan.pl
A    manage_portfolio.pl
A    anashell.pl
A    report_summary.ash
A    t
A    t/launch.sh
A    t/TODO
A    t/TODO.en
A    t/bin
A    t/bin/test_indicator.sh
A    t/bin/output_indicator.pl
A    t/bin/function.sh
A    t/data
A    t/data/alcatel.txt
A    t/.cvsignore
A    t/README
A    t/lists
A    t/lists/TEST.Indicators
A    OptimizeGT.pm
A    backtest.pl
A    select_combination.pl
A    display_indicator.pl
A    display_signal.pl
A    .cvsignore
A    test.xml
 U   .
Checked out revision 720.

did similar on for GT subdir -- it also worked

aha your command line is missing some whitespace here (but that could be the emailers)


https://geniustrader.svn.sourceforge.net/svnroot/geniustrader/trunk/GTgeniustrader/GT
                                                                     ^^^

[GT] Error during checkout

[Evgeny S. <shr...@gm...>]

Greetings Robert,

I am setting up GT on a fresh CentOS system and I get the following error
when trying to checkout the code:

[Eugene@localhost ~]$ svn checkout
https://geniustrader.svn.sourceforge.net/svnroot/geniustrader/trunk/GTgeniustrader/GT
svn: Can't read file 'geniustrader/GT/.svn/entries': End of file found


Any idea why it happens?

Cheers!
Eugene

[GT] gt mailing list and cci

[Robert A. S. <ra...@ac...>]

aloha delix

darned sf mailer didn't forward your query on cci (2013-05-28 06:34), so i didn't see it
until i went looking at the archive to see why the mailing list has been so quiet lately.

anyway, i will take a look-see at cci when i get a moment. meantime can you share any of
the parameters you are using when computing it?

i see way back in may 2005 when it might have been originated and/or modified there were
gt-devel messages about division by zero problems with it.

the cci pod i have suggests cci uses typical price, which isn't too well 'fixed' by the
ta community (at least as far as i recall). then it computes a mean deviation and there
is a 'fudge' factor constant (0.015) applied somewhere as well. all these factors are
places where 'differences' can get introduced.

so it is very likely different ta codes will yield somewhat different values, but i
would expect the indicators magnitude and line shape over time (direction, slope
change etc) should generally match.



regards

ras

[GT] CCI Indikator

[Delix <del...@t-...>]

is anybody using the CCI indicator ?
I tried to check the results with the data and parameters of the
following web site :
http://www.ariva.de/euro-dollar-kurs/chart?compare=None&idstring=x&box2=0&avgVal1=20&t=quarter&box3=0&avgVal2=0&end=27.10.2012&avgType1=EMA&resolution=auto&scale=lin&type=CandleStick&displayHighLow=1&box4=0&band=BB&avgType2=None&typ=&grid=1&show_ind_params=0&volume=1&boerse_id=130&antiAlias=0&go=Aktualisieren&savg=0&events=None&indicator=MACD&indicator=Aroon&indicator=CCI&indicator=SStoch&indicator=ROC&box5=0

GTs CCI always gives similar results but the values are too low (i.e.
-60 instead of -100)
has anybody an explanations for this ? most likely, it is a matter of
the formular but I'm not sure what it is finally...

-- 
Delix <lin...@t-...>

Re: [GT] genericdbi + sqlite : Date problem

[Robert A. S. <ra...@ac...>]

< snipped, but reference data should still work >

delix

i'm about out of things to do with this issue but i wanted
to mention (or reiterate) a few things before i put this away.

firstly you should *want* your database to be *correct* and
right now -- based on the database you sent me (well the list
but the list blocked it) (file test.sqlite) sqlite does *NOT*
consider your Datum tuple to be a *date*

   $  sqlite3 test.sqlite
   sqlite3> select strftime( '%Y-%m-%d', Datum ) as Datum from NZD_USD limit 1;

should print something like this sql query.

   sqlite3> select strftime( '%Y-%m-%d', 'now') as Datum;
   Datum = 2013-04-25





i'm unable to run your conversion-import process exactly on my
system -- because the command

   $  ssconvert -O 'separator=;' NZD-USD.gnumeric dummy.txt

does not work for me.
%    ( rm -f ./dummy.txt                      ; \
  setenv LD_LIBRARY_PATH /opt/csw/lib:/usr/openwin/lib/ ; \
  /opt/csw/bin/ssconvert             \
    -O 'separator=;'                 \
    NZD-USD.gnumeric                 \
    dummy.txt)
separator=;

** (gnumeric:16855): CRITICAL **: file workbook-view.c: line 650: assertion `IS_WORKBOOK_VIEW (wbv)' failed
** (gnumeric:16855): CRITICAL **: file workbook-view.c: line 73: assertion `IS_WORKBOOK_VIEW (wbv)' failed
(gnumeric:16855): GLib-GObject-CRITICAL **: file gobject.c: line 2370: assertion `G_IS_OBJECT (object)' failed



i've tried to resolve it, but haven't been able to find
a command line solution ...

i can run gnumeric directly using your input file NZD-USD.gnumeric,
but with my locale (C) the date format is the iso8601 standard yyyy-mm-dd.

so any export i do using the save-as menu including using the gui-ized
command line equivalent of --export-type=Gnumeric_stf:stf_assistant
which is what i think you are doing with ssconvert -O 'separator=;'
i can only get iso8601 dated or the raw gnumeric date epoch
(days since 1899-12-31).



some other observations you might want to look at in SQLimport_common.sh.
granted, i'm unable to produce dummy.txt the way you are but your
gnumeric database starts at 2008-04-14, but your database (test.sqlite)
starts at 2008/04/15.
   $   sqlite3 test.sqlite
   sqlite3> select * from NZD_USD order by Datum asc limit 1;

i think you are deleting the first data with this sed command:
    sed -e '1d' dummy.txt | \

assuming this directive was supposed to remove a header line
of some sort i'd somehow restrict the delete in the case where
the header line looked more like a data line.



here is how i convert raw gnumeric epoch dates into gt compatible
iso8601 date-time strings:

#!/bin/sh

perl -MDate::Manip -e'
  my @clean_file = <>; # read entire file
  while ( my $line = shift @clean_file ) {
     chomp $line;
     my ( $d, $rest ) = split /,/, $line, 2;
     my $secs = ( $d - 25569 ) * 86400;
     my $date = Date_ConvTZ( ParseDateString( "epoch $secs" ),"","GMT");
     print STDOUT UnixDate($date, "%Y-%m-%d %T"),",$rest\n";
  }
' NZD-USD.ras.csv > iso8601_file

i was able to create the file NZD-USD.ras.csv in two ways
  1) from within gnumeric using save-as selecting the gui-ized
     export customized .txt format selecting 'raw' for format
     to keep the gnumeric epoch date values
  2) using /opt/csw/bin/ssconvert:
     $   /opt/csw/bin/ssconvert                      \
         --import-encoding=Gnumeric_XmlIO:gnum_xml   \
         --export-type=Gnumeric_stf:stf_csv          \
         ./NZD-USD.gnumeric                          \
         ./NZD-USD.ras.csv

  



i think that about covers it [has beaten this dead horse to within an inch of it's life] ...

yell if you have questions or want to discuss any aspect of this further

aloha

ras

Re: [GT] Post install issues

[Robert A. S. <ra...@ac...>]

Evgeny Shragovich wrote:
> Hi ras,
> 
> Thank you for your comments!
> 
> Yes, I was able to locate the gt_files.pod under my GT/Docs directory.
> Previously I was looking in the Documentation section on GT site.

very good

> 
> I got the code as it is explained on GT site, here
> http://geniustrader.org/development.html:
> svn checkout
> https://geniustrader.svn.sourceforge.net/svnroot/geniustrader/trunk/GTgeniustrader/GT
                                                                      ^^^^^^^^^^^^^^^
                         i'm not too sure about this extra component  GTgeniustrader/
                          but that could be just a missing space here  ^ 
                          which would make the command line look just
                          about as i'd expect and show below.

i would expect the command line sequence you use to be more like

   $  mkdir /usr/local/geniustrader
   $  cd  /usr/local
   $  svn checkout https://geniustrader.svn.sourceforge.net/svnroot/geniustrader/trunk/GT         geniustrader/GT

> svn checkout
> https://geniustrader.svn.sourceforge.net/svnroot/geniustrader/trunk/Scriptsgeniustrader/Scripts

   $  svn checkout https://geniustrader.svn.sourceforge.net/svnroot/geniustrader/trunk/Scripts    geniustrader/Scripts

> 
> Is that the correct way?

if the gt Scripts and gt toolkit modules (GT) share the same parent directory
(/usr/local/geniustrader in the example)
then that is the recommended way to install

you should then be able to cd to Scripts (cd /usr/local/geniustrader/Scripts)
and run any of them by specifying the script name as first element on command line:

  $  graphic.pl ....
or if you do not have '.' in your $PATH
  $  ./graphic.pl ....
or if you do have sym-links and told Build.PL to create them
  $  ./graphic ....



aloha

ras

> 
> 
> Thanks!
> 
> Eugene
> 
> 


< snip >

Re: [GT] Post install issues

[Evgeny S. <shr...@gm...>]

Hi ras,

Thank you for your comments!

Yes, I was able to locate the gt_files.pod under my GT/Docs directory.
Previously I was looking in the Documentation section on GT site.

I got the code as it is explained on GT site, here
http://geniustrader.org/development.html:
svn checkout
https://geniustrader.svn.sourceforge.net/svnroot/geniustrader/trunk/GTgeniustrader/GT
svn checkout
https://geniustrader.svn.sourceforge.net/svnroot/geniustrader/trunk/Scriptsgeniustrader/Scripts

Is that the correct way?


Thanks!

Eugene


On Thu, Apr 18, 2013 at 11:53 PM, Robert A. Schmied <ra...@ac...> wrote:

> Evgeny Shragovich wrote:
> > Hi ras,
> >
> > I will be happy to help!
> > Please provide me the hacked version of the build.pl and I will run it
> on
> > my system.
> >
> >
> > Meanwhile, as I am going through the documentation I have few more
> > questions:
> >
> > 1. If I use DB::module Text, is it possible to specify more that one data
> > source paths?
> >     I have noticed that if I just put 2 path lines, the last one is being
> > used.
>
> only one directory path is supported. that directory can contain files for
> the
> same company (symbol) but with different time-frames too.
>
> if different directory paths are *extremely* important you can set the
> default
> value in your $HOME/.gt/options file and then pass an alternate path using
> the standard command line option
> --option='DB::text::directory=<alternate_path>'
> note that you must quote the key=value as i show in order to escape the '='
> from the perl argument processing module.
>
> >
> > 2. When going through the documentation, specifically
> > thehow_to_spec+debug_a_system.pod.html<
> http://geniustrader.org/doc/GT/Docs/how_to_spec+debug_a_system.pod.html>,
> > I have found the following list of documents recommended for initial
> > reading:
> >
> >     gt_sig-sys-desc.pod
> >     gt_files.pod
> >     backtest.pl
> >     GT/SystemManager.pm
> >     GT/CloseStrategy.pm
> >     GT/PortfolioManager.pm
> >     GT/TradeFilters.pm
> >     GT/OrderFactory.pm
> >     GT/MoneyManagement.pm
> >
> > However I was not able to locate the "gt_files.pod" file. Does it
> > exist/relevant?
> >
>
> humm -- well, i've got it in both my ras hack as well as in the svn repo
> but that doesn't necessarily mean it has actually been committed ...
> something like svn { status || log } ../svn_repo/GT/Docs/gt_files.pod might
> be the command i want ...
>
> well it does appear to be a old file, meaning it's been around since
> 2008-11-07
>
> now you are making me concerned what version of gt you've got.
>
> how did you download it? there are multiple branches -- we recommend
> using the trunk but the exp branch is also acceptable.
>
> i'm showing the trunk svn version as Revision: 720
>   %    svn info
>
> since the sourceforge conversion (13-14apr) the only way to download gt
> appears
> to be an svn repo clone copy ( see
> http://geniustrader.org/development.html for
> detailed instructions).
>
>
> a light might have just came on (in my head) -- the pod files are in
> GT/Docs -- did you look there?
>   %    find .. -name \*.pod
> or
>   %    ls -lFatr ../GT/Docs
>
>
> > 3. I have tried creating a graph using the gconf and ACTV.txt files.
> >     The graph was eventually created but again was looking funny and I
> have
> > also received the below error on the screen:
> >
> >    [Evgeny@localhost Scripts]$ ./graphic.pl --file screenshot_2.gconf
> --out
> > 'ACTV.png' ACTV
> >    MountainBand(Indicators::BOL/2 20 2.0,Indicators::BOL/3 15 2.0, \ is
> not
> > a valid object description at ./graphic.pl line 856.
>
> my guess is you failed to edit your cut+paste of screenshot_2.gconf to
> either
> remove entirely 'continuation' lines introduced by the '\' char being
> complained
> about or there are chars following the '\' char in that file.
>
> i recall that graphic.pl is sensitive to 'continuation' lines and cannot
> support them in some specific ways -- but the pod should explain those
> limits
> in detail ...
>
> using my svn repo and your options, shortened ADX data and a *correct*
> screenshot_2.gconf file
>   %   ./graphic.pl --file ../website/screenshots/screenshot_2.gconf \
>   --out /tmp/ADX_x.png ADX_x
> no errors/warnings and a correct chart ...
>
>
> aloha
>
> ras
>
>
> >    .
> >    .
> >    .
> >    Use of uninitialized value $func in pattern match (m//) at
> > ./graphic.plline 778.
> >    Use of uninitialized value $func in pattern match (m//) at
> > ./graphic.plline 782.
> >    VotingLine( Systems::Generic \ is not a valid object description at ./
> > graphic.pl line 856.
> >    Use of uninitialized value $func in pattern match (m//) at
> > ./graphic.plline 637.
> >    Use of uninitialized value $func in pattern match (m//) at
> > ./graphic.plline 648.
> >    .
> >    .
> >    .
> >    Use of uninitialized value $func in pattern match (m//) at
> > ./graphic.plline 782.
> >    MountainBand(Indicators::Generic::Eval \ is not a valid object
> > description at ./graphic.pl line 856.
> >    Use of uninitialized value $func in pattern match (m//) at
> > ./graphic.plline 637.
> >    Use of uninitialized value $func in pattern match (m//) at
> > ./graphic.plline 648.
> >
> >
> > Any idea what might cause these errors?
> >
> >
> > Thanks!
> > Eugene
> >
> >
>
> < snip >
>
>
>
> ------------------------------------------------------------------------------
> Precog is a next-generation analytics platform capable of advanced
> analytics on semi-structured data. The platform includes APIs for building
> apps and a phenomenal toolset for data science. Developers can use
> our toolset for easy data analysis & visualization. Get a free account!
> http://www2.precog.com/precogplatform/slashdotnewsletter
>



-- 
Best regards,

Shragovich       Evgeny
shr...@gm...

Re: [GT] genericdbi + sqlite : Date problem

[Robert A. S. <ra...@ac...>]

< snip >

> 
> Thanks Robert !
> As I wrote already in my answer to Thomas' hints (see the GT mail
> archive -- but not all messages are present there at the moment (?)  )

i don't see any messages from you to thomas and cc'ed in the gt-devel archive
so either the sf mailman doesn't like cc messages or you didn't cc the mail
list or ???

> I solved the problems by changing the --timeframe=day option so that
> the correct command is
> 
> ./graph --timeframe=year NZD_USD I:ADX

i'm thinking that doing that only 'side-steps' date-string format issue
in ../GT/DateTime.pm

> 
>  (Only) with this option it works and solves all error messages ! It is
> a similar thing as with the backtest moduls, which requires explicitely
> a
> --timeframe=day option to run.....
> 
> Concerning the population of the data base : I'm using a gnumeric
> spreadsheet with german data format (23.4.2013) as source and wrote a
> bash script which exports, cleans and imports the data to the sqlite3
> data base. The import command of sqlite3 checks the source data
> automatically, complains if the data of the specific field is not in
> an acceptable data format and converts it to the 2013/4/23 format
> without giving the user any choice.

ok, i've messed around with sqlite3 .import -- i've tried to enter
three date formats: YYYY/MM/DD,  YYYY-MM-DD and dd.mm.yyyy.

only YYYY-MM-DD is considered by sqlite3 to be a 'time-date' string.
the other forms are just text strings and cannot be converted using
the various date formatting provisions supported by sql and sqlite3.

the manual cut-paste commands i used to evaluate these are in the attached ...
and start near the bottom of the file after the line

hacking around with .import ...

you could still do the conversion in your version of ../GT/DB/generic.pm
on the prices arrays after they are output from the database ...

however, i think your best bet is to tweak your bash script to convert
your time-date strings into the YYYY-MM-DD format before you do the import.
you can easily do this with perl, sed and/or awk.
doing this will keep your GT code base consistent with the current version.

> So, I think the data format cannot be the reason for the timeframe
> problem

well it is actually -- if you look at the date decoding being done
just before line 285 in ../GT/DateTime.pm you will see the date is
split into 3 parts: day, month and year using the split character '-'.
in the timeframe 'day' mode when the split fails the month component
is undef, and -1 is passed to timegm as the month value which triggers
the error.

and before you start wanting to support other formats, we have
already evaluated this, and settled on using the ISO-8601 date format
standard as the primary date formats used internally.

we do however accommodate 'other' date formats (as recognized by primarily
Date::Manip) for command line inputs and similar (such as graphic
configuration files)

aloha

ras

 -- maybe its a matter of the localized versions my system
> uses as default for the date format (?).
> Anyway, I hope the problem is permanenly fixed by using the
> --timeframe=year option  ...
> Thx, delix
> 

Re: [GT] genericdbi + sqlite : Date problem

[Robert A. S. <ra...@ac...>]

Delix wrote:
> On Mon, 22 Apr 2013 15:29:03 -0700
> "Robert A. Schmied" <ra...@ac...> hatte geschrieben :
> Thanks Robert for the quick reply.
> Now follows the steps and results with the data and configs I used :
> 
> =================================
> the database part of my ./gt/options file (the other parts differ only
> in the paths for fonts and aliases from the provided example file) :
> 
> DB::module genericdbi
> DB::genericdbi::db             SQLite
> DB::genericdbi::dbname         /home/user/Kurse/Devisen.sqlite
> DB::genericdbi::name_sql       SELECT Kennung FROM stockinfo WHERE
> (Kennung="NZD_USD") 
> DB::genericdbi::prices_sql     SELECT
> Erster,Hoch,Tief,Schluss,Volumen,Datum FROM NZD_USD ORDER BY Datum DESC
> ==================================
> 
> the .schema NZD_USD output:
> CREATE TABLE [NZD_USD] ([Datum] DATE PRIMARY KEY NOT NULL UNIQUE,
> [Erster] REAL (7, 2) NOT NULL, [Hoch] REAL (7, 2) NOT NULL, [Tief] REAL
> (7, 2) NOT NULL, [Schluss] REAL (7, 2) NOT NULL, [Volumen] REAL (10,
> 2));
> ====================================
> 
> the data output of the NZD_USD table:
> 2008/04/15|0.7923|0.7945|0.7835|0.786|0.0
> 2008/04/16|0.786|0.7924|0.7828|0.7924|0.0
> 2008/04/17|0.7925|0.7932|0.7847|0.7885|0.0
> 2008/04/18|0.7885|0.794|0.7851|0.7894|0.0
> 2008/04/21|0.7905|0.797|0.7894|0.7936|0.0
> 2008/04/22|0.7936|0.7986|0.7901|0.7971|0.0
> ====================================
> 
> the output of the command
> ./display_indicator I:RSI NZD_USD:
> Calculating indicator RSI[14, {I:Prices CLOSE}] ...
> RSI[14, {I:Prices CLOSE}][2012/08/16] = 53.1250
> RSI[14, {I:Prices CLOSE}][2012/08/17] = 49.0814
> RSI[14, {I:Prices CLOSE}][2012/08/18] = 46.9945
> RSI[14, {I:Prices CLOSE}][2012/08/19] = 28.9855
> RSI[14, {I:Prices CLOSE}][2012/08/20] = 33.1034
> RSI[14, {I:Prices CLOSE}][2012/08/21] = 35.3333
> RSI[14, {I:Prices CLOSE}][2012/08/22] = 47.2222
> ====================================
> 
> the error message I get with the command ./graphic.pl --timeframe=day
> NZD_USD I:ADX
> 
> Use of uninitialized value $d in pattern match (m//)
> at ../GT/DateTime.pm line 284. Use of uninitialized value $m in
> subtraction (-) at ../GT/DateTime.pm line 285. Argument "2008/04/15"
> isn't numeric in numeric ge (>=) at /usr/share/perl/5.14/Time/Local.pm
> line 100. Month '-1' out of range 0..11 at ../GT/DateTime.pm line 285
> 
> ====================================
> So, you see the data set is not something special but very simple.
> What confuses is the successful run of the display_indicator command
> with the same data and settings. Maybe it is something else than a data
> format problem ?
> 
> 
> Thanks again for your help and patience !
> 

hey delix

the specific error Month '-1' out of range 0..11 at ../GT/DateTime.pm line 285

is occurring because of your original date-string format yyyy/mm/dd, if those '/'
chars were the gt default separator char '-' ../GT/DateTime.pm works as expected.
i don't see how to fix this issue except by
  1) using ras hack gt version (just ask)
  2) tweaking ../GT/DB/genericdbi.pm to convert '/' into '-' as mentioned elsewhere
  3) changing the way you store date (Datum) in your database

i'm recommending 3 ...

but that still leaves the issue with timeframe. by default it's supposed to be
'day' and that's what ../GT/DB/genericdbi.pm will presume unless you provide a
mapping configuration setting in accordance with the code in ../GT/DB/genericdbi.pm.
the pod is not very informative and the alternate version (i sent in a prior email)
is (or should be) very much similar to the trunk head version.

assuming you will want to eventually deal with week and month timeframes you
need to either create separate tables for each of these or add the timeframe
tuple for the data in the stockprices table.

as no one (except you) that i know of is using genericdbi.pm with a custom
database i'm not certain how or if larger timeframes (week, month, year)
can be automatically generated by GT based on day data, but GT can do that
with beancounter and ../GT/DB/bean.pm.

so you (we) are treading on uncharted territory.


> Concerning the population of the data base : I'm using a gnumeric
> spreadsheet with german data format (23.4.2013) as source and wrote a
> bash script which exports, cleans and imports the data to the sqlite3
> data base. The import command of sqlite3 checks the source data
> automatically, complains if the data of the specific field is not in
> an acceptable data format and converts it to the 2013/4/23 format
> without giving the user any choice.

can you give more specifics on 'import command of sqlite3'

is that
    sqlite3> .import FILE TABLE

or some other sqlite3 import utility i have yet to identify?

there isn't much documentation explaining how .import operates ...


i'm wondering if you have quote characters on the Datum field when
you create the FILE to import. that might explain why sqlite doesn't
seem to consider Datum to be a date -- at least in my testing here
it did not consider '2008/04/21' to be a date.

sqlite3>  select   strftime( '%Y-%m-%d %H:%M:%S', '2008/04/21' ) ;
strftime( '%Y-%m-%d %H:%M:%S', '2008/04/21' ) = 

but '2008-04-21' works fine:

sqlite3>  select   strftime( '%Y-%m-%d %H:%M:%S', '2008-04-21' );
strftime( '%Y-%m-%d %H:%M:%S', '2008-04-21' ) = 2008-04-21 00:00:00

is there any way you can modify your bash script to output the Datum
string as YYYY-MM-DD?  i would also suggest adding quotes around it when
writing FILE for sqlite3 to import.


> So, I think the data format cannot be the reason for the timeframe
> problem -- maybe its a matter of the localized versions my system
> uses as default for the date format (?).
> Anyway, I hope the problem is permanenly fixed by using the
> --timeframe=year option  ...

i don't think you've found an actual solution but just a set of
conditions for which some things are working differently ...


aloha

ras

Re: [GT] genericdbi + sqlite : Date problem

[Robert A. S. <ra...@ac...>]

Delix wrote:
> On Tue, 23 Apr 2013 09:59:13 -0700
> "Robert A. Schmied" <ra...@ac...> hatte geschrieben :
> 
> 
>>Delix wrote:
>>
>>>On Mon, 22 Apr 2013 15:29:03 -0700
>>>"Robert A. Schmied" <ra...@ac...> hatte geschrieben :
>>>Thanks Robert for the quick reply.
>>>Now follows the steps and results with the data and configs I used :
>>>
>>>=================================
>>>the database part of my ./gt/options file (the other parts differ only
>>>in the paths for fonts and aliases from the provided example file) :
>>>
>>>DB::module genericdbi
>>>DB::genericdbi::db             SQLite
>>>DB::genericdbi::dbname         /home/user/Kurse/Devisen.sqlite
>>>DB::genericdbi::name_sql       SELECT Kennung FROM stockinfo WHERE
>>>(Kennung="NZD_USD") 
>>>DB::genericdbi::prices_sql     SELECT
>>>Erster,Hoch,Tief,Schluss,Volumen,Datum FROM NZD_USD ORDER BY Datum DESC
>>>==================================
>>>
>>>the .schema NZD_USD output:
>>>CREATE TABLE [NZD_USD] ([Datum] DATE PRIMARY KEY NOT NULL UNIQUE,
>>>[Erster] REAL (7, 2) NOT NULL, [Hoch] REAL (7, 2) NOT NULL, [Tief] REAL
>>>(7, 2) NOT NULL, [Schluss] REAL (7, 2) NOT NULL, [Volumen] REAL (10,
>>>2));
>>>====================================
>>>
>>>the data output of the NZD_USD table:
>>>2008/04/15|0.7923|0.7945|0.7835|0.786|0.0
>>>2008/04/16|0.786|0.7924|0.7828|0.7924|0.0
>>>2008/04/17|0.7925|0.7932|0.7847|0.7885|0.0
>>>2008/04/18|0.7885|0.794|0.7851|0.7894|0.0
>>>2008/04/21|0.7905|0.797|0.7894|0.7936|0.0
>>>2008/04/22|0.7936|0.7986|0.7901|0.7971|0.0
>>>====================================
>>>
>>>the output of the command
>>>./display_indicator I:RSI NZD_USD:
>>>Calculating indicator RSI[14, {I:Prices CLOSE}] ...
>>>RSI[14, {I:Prices CLOSE}][2012/08/16] = 53.1250
>>>RSI[14, {I:Prices CLOSE}][2012/08/17] = 49.0814
>>>RSI[14, {I:Prices CLOSE}][2012/08/18] = 46.9945
>>>RSI[14, {I:Prices CLOSE}][2012/08/19] = 28.9855
>>>RSI[14, {I:Prices CLOSE}][2012/08/20] = 33.1034
>>>RSI[14, {I:Prices CLOSE}][2012/08/21] = 35.3333
>>>RSI[14, {I:Prices CLOSE}][2012/08/22] = 47.2222
>>>====================================
>>>
>>>the error message I get with the command ./graphic.pl --timeframe=day
>>>NZD_USD I:ADX
>>>
>>>Use of uninitialized value $d in pattern match (m//)
>>>at ../GT/DateTime.pm line 284. Use of uninitialized value $m in
>>>subtraction (-) at ../GT/DateTime.pm line 285. Argument "2008/04/15"
>>>isn't numeric in numeric ge (>=) at /usr/share/perl/5.14/Time/Local.pm
>>>line 100. Month '-1' out of range 0..11 at ../GT/DateTime.pm line 285
>>>
>>>====================================
>>>So, you see the data set is not something special but very simple.
>>>What confuses is the successful run of the display_indicator command
>>>with the same data and settings. Maybe it is something else than a data
>>>format problem ?
>>>
>>>
>>>Thanks again for your help and patience !
>>>
>>
>>aloha delix
>>
>>i am using sqlite3 ...
>>
>>i created 'delix_db', created and populated table NZD_USD therein ...
>>
>>and when i tried to convert tuple Datum i *immediately* found the
>>the problem. the data (date string) being stored into tuple Datum
>>is not a recognized/supported sqlite 'time-date' string.
>>
>>the recognized/support sqlite date string formats are:
>>
>>"A time string can be in any of the following formats:
>>    YYYY-MM-DD
>>    YYYY-MM-DD HH:MM
>>    YYYY-MM-DD HH:MM:SS
>>    YYYY-MM-DD HH:MM:SS.SSS
>>    YYYY-MM-DDTHH:MM
>>    YYYY-MM-DDTHH:MM:SS
>>    YYYY-MM-DDTHH:MM:SS.SSS
>>    HH:MM
>>    HH:MM:SS
>>    HH:MM:SS.SSS
>>    now
>>    DDDDDDDDDD
>>"
>>
>>so i changed the '/' used in all datum values to '-' and
>>recreated the delix_db database and NZD_USD table.
>>
>>sqlite3> SELECT Erster,Hoch,Tief,Schluss,Volumen,Datum FROM NZD_USD ORDER BY Datum DESC;
>> Erster = 0.7936
>>   Hoch = 0.7986
>>   Tief = 0.7901
>>Schluss = 0.7971
>>Volumen = 0.0
>>  Datum = 2008-04-22
>>
>> Erster = 0.7905
>>   Hoch = 0.797
>>   Tief = 0.7894
>>Schluss = 0.7936
>>Volumen = 0.0
>>  Datum = 2008-04-21
>>
>>.
>>.
>>.
>>
>>if you want the optional ' HH:MM:SS' attached to Datum
>>sqlite3> SELECT Erster,Hoch,Tief,Schluss,Volumen,
>>         strftime( '%Y-%m-%d %H:%M:%S', date(Datum)) as Datum 
>>         FROM NZD_USD ORDER BY Datum DESC;
>> Erster = 0.7936
>>   Hoch = 0.7986
>>   Tief = 0.7901
>>Schluss = 0.7971
>>Volumen = 0.0
>>  Datum = 2008-04-22 00:00:00
>>
>> Erster = 0.7905
>>   Hoch = 0.797
>>   Tief = 0.7894
>>Schluss = 0.7936
>>Volumen = 0.0
>>  Datum = 2008-04-21 00:00:00
>>
>>.
>>.
>>.
>>
>>
>>
>>couple of thoughts
>>
>>if you are inserting data into your database via code (a program)
>>that you can change i would enter the date string with date and
>>time values; if the data is end-of-day the time string should
>>be indicative of market close; but only you can determine what
>>market your NZD_USD close time represents (it could be any
>>end-of-market-day any place around the world), which suggests
>>the date-time field should be 'universal coordinated time',
>>but that is not something that is well integrated within GT,
>>and isn't supported directly by beancounter either for that matter.
>>
>>
>>using perl converting date strings in the form
>>    2008/04/22
>>into
>>    2008-04-22
>>is trivial
>>    $_ =~ tr@/@-@;
>>or with more provision for error detect use Date::Manip
>>    UnixDate(ParseDate( "$_" ), "%Y-%m-%d %T" );
>>
>>
>>now if your database is so large and difficult (costly) to recreate
>>you can continue to use it as is, but recognize the Datum tuple is
>>just a 'text string' that a human would recognize as a date but sqlite
>>does not. thus you need to hack GT::DB::genericdb::get_last_prices
>>to convert the Datum values into a gt compatible date string -- trivial
>>to fairly so -- as described immediately above and in a prior message.
>>
>>
>>hope this helps, good luck,
>>
>>
>>aloha
>>
>>ras
> 
> 
> Thanks Robert !
> As I wrote already in my answer to Thomas' hints (see the GT mail
> archive -- but not all messages are present there at the moment (?)  )
> I solved the problems by changing the --timeframe=day option so that
> the correct command is
> 
> ./graph --timeframe=year NZD_USD I:ADX
> 
>  (Only) with this option it works and solves all error messages ! It is
> a similar thing as with the backtest moduls, which requires explicitely
> a
> --timeframe=day option to run.....
> 
> Concerning the population of the data base : I'm using a gnumeric
> spreadsheet with german data format (23.4.2013) as source and wrote a
> bash script which exports, cleans and imports the data to the sqlite3
> data base. The import command of sqlite3 checks the source data
> automatically, complains if the data of the specific field is not in
> an acceptable data format and converts it to the 2013/4/23 format
> without giving the user any choice.
> So, I think the data format cannot be the reason for the timeframe
> problem -- maybe its a matter of the localized versions my system
> uses as default for the date format (?).
> Anyway, I hope the problem is permanenly fixed by using the
> --timeframe=year option  ...
> Thx, delix
> 


pod in ../GT/DB/genericdbi.pm [well in the pod i've got in my version]

  time frame mapping

    This version of genericdbi solves the problem of mapping database tables
    names to the timeframe which is being requested.

    Previously, genericdbi assumed that a GT timeframe code will be present
    in either a table name or a column value in the database.

    For instance, the GT code for hourly data is 60. If you have hourly
    data, genericdbi required that either your table be called EURUSD_60, or
    has have a column with value 60.

    This version of genericdbi provides a way to map whatever time frame
    name your database tables use to the appropriate GT timeframe codes via
    additional entries in ~/.gt/options.

  example

    if your database uses an hourly timeframe reference of 3600, you would
    map that to the GT hourly timeframe reference of 60 with this entry in
    your $HOME/.gt/options file.

    DB::genericdbi::tf_map::60 3600


i don't see a DB::genericdbi::tf_map in your options
and your table name does not indicate it is 'day' timeframe.

is this the issue???

while i do not see the statements in the pod above to be completely true,
after all things are working after a fashion without a GT timeframe
code tuple or via a table name, this could be an issue in the long run.
and graphic.pl does set the timeframe using
    $timeframe = GT::DateTime::name_to_timeframe($timeframe);

i don't have enough data to evaluate weekly or larger timeframes
and of course have nothing smaller than day.

ha!
 %     ./graphic.pl --no-volume --type=candle --start=2008/04/15 --end='today' --title='%n (%c) :: simple.gconf' --add='Set-scale(auto,logarithmic)' --out /var/tmp/delix_sqlite/NZD_USD_ntf.png NZD_USD
Month '-1' out of range 0..11 at ../GT/DateTime.pm line 285
 %     ./graphic.pl --no-volume --type=candle --timeframe=year  --start=2008/04/15 --end='today' --title='%n (%c) :: simple.gconf' --add='Set-scale(auto,logarithmic)' --out /var/tmp/delix_sqlite/NZD_USD_year.png NZD_USD

ok problem duplicated --
you need to incorporate somehow a GT timeframe factor into your custom database ...
either via a tuple (column) or into the table name
see the pod examples for the DB::genericdbi::prices_sql query


or you could use the attached ../GT/DB/genericdbi.pm which i think works around
this issue with your current database, but i don't know what other bad things
might occur with more data and or other untested indicator calculations ...
humm -- this might not solve your problem because it likely depends on loads
of other changes not included ...

aloha

ras

Re: [GT] genericdbi + sqlite : Date problem

[Robert A. S. <ra...@ac...>]

Delix wrote:
> On Tue, 23 Apr 2013 09:59:13 -0700
> "Robert A. Schmied" <ra...@ac...> hatte geschrieben :
> 
> 
>>Delix wrote:
>>
>>>On Mon, 22 Apr 2013 15:29:03 -0700
>>>"Robert A. Schmied" <ra...@ac...> hatte geschrieben :
>>>Thanks Robert for the quick reply.
>>>Now follows the steps and results with the data and configs I used :
>>>
>>>=================================
>>>the database part of my ./gt/options file (the other parts differ only
>>>in the paths for fonts and aliases from the provided example file) :
>>>
>>>DB::module genericdbi
>>>DB::genericdbi::db             SQLite
>>>DB::genericdbi::dbname         /home/user/Kurse/Devisen.sqlite
>>>DB::genericdbi::name_sql       SELECT Kennung FROM stockinfo WHERE
>>>(Kennung="NZD_USD") 
>>>DB::genericdbi::prices_sql     SELECT
>>>Erster,Hoch,Tief,Schluss,Volumen,Datum FROM NZD_USD ORDER BY Datum DESC
>>>==================================
>>>
>>>the .schema NZD_USD output:
>>>CREATE TABLE [NZD_USD] ([Datum] DATE PRIMARY KEY NOT NULL UNIQUE,
>>>[Erster] REAL (7, 2) NOT NULL, [Hoch] REAL (7, 2) NOT NULL, [Tief] REAL
>>>(7, 2) NOT NULL, [Schluss] REAL (7, 2) NOT NULL, [Volumen] REAL (10,
>>>2));
>>>====================================
>>>
>>>the data output of the NZD_USD table:
>>>2008/04/15|0.7923|0.7945|0.7835|0.786|0.0
>>>2008/04/16|0.786|0.7924|0.7828|0.7924|0.0
>>>2008/04/17|0.7925|0.7932|0.7847|0.7885|0.0
>>>2008/04/18|0.7885|0.794|0.7851|0.7894|0.0
>>>2008/04/21|0.7905|0.797|0.7894|0.7936|0.0
>>>2008/04/22|0.7936|0.7986|0.7901|0.7971|0.0
>>>====================================
>>>
>>>the output of the command
>>>./display_indicator I:RSI NZD_USD:
>>>Calculating indicator RSI[14, {I:Prices CLOSE}] ...
>>>RSI[14, {I:Prices CLOSE}][2012/08/16] = 53.1250
>>>RSI[14, {I:Prices CLOSE}][2012/08/17] = 49.0814
>>>RSI[14, {I:Prices CLOSE}][2012/08/18] = 46.9945
>>>RSI[14, {I:Prices CLOSE}][2012/08/19] = 28.9855
>>>RSI[14, {I:Prices CLOSE}][2012/08/20] = 33.1034
>>>RSI[14, {I:Prices CLOSE}][2012/08/21] = 35.3333
>>>RSI[14, {I:Prices CLOSE}][2012/08/22] = 47.2222
>>>====================================
>>>
>>>the error message I get with the command ./graphic.pl --timeframe=day
>>>NZD_USD I:ADX
>>>
>>>Use of uninitialized value $d in pattern match (m//)
>>>at ../GT/DateTime.pm line 284. Use of uninitialized value $m in
>>>subtraction (-) at ../GT/DateTime.pm line 285. Argument "2008/04/15"
>>>isn't numeric in numeric ge (>=) at /usr/share/perl/5.14/Time/Local.pm
>>>line 100. Month '-1' out of range 0..11 at ../GT/DateTime.pm line 285
>>>
>>>====================================
>>>So, you see the data set is not something special but very simple.
>>>What confuses is the successful run of the display_indicator command
>>>with the same data and settings. Maybe it is something else than a data
>>>format problem ?
>>>
>>>
>>>Thanks again for your help and patience !
>>>
>>
>>aloha delix
>>
>>i am using sqlite3 ...
>>
>>i created 'delix_db', created and populated table NZD_USD therein ...
>>
>>and when i tried to convert tuple Datum i *immediately* found the
>>the problem. the data (date string) being stored into tuple Datum
>>is not a recognized/supported sqlite 'time-date' string.
>>
>>the recognized/support sqlite date string formats are:
>>
>>"A time string can be in any of the following formats:
>>    YYYY-MM-DD
>>    YYYY-MM-DD HH:MM
>>    YYYY-MM-DD HH:MM:SS
>>    YYYY-MM-DD HH:MM:SS.SSS
>>    YYYY-MM-DDTHH:MM
>>    YYYY-MM-DDTHH:MM:SS
>>    YYYY-MM-DDTHH:MM:SS.SSS
>>    HH:MM
>>    HH:MM:SS
>>    HH:MM:SS.SSS
>>    now
>>    DDDDDDDDDD
>>"
>>
>>so i changed the '/' used in all datum values to '-' and
>>recreated the delix_db database and NZD_USD table.
>>
>>sqlite3> SELECT Erster,Hoch,Tief,Schluss,Volumen,Datum FROM NZD_USD ORDER BY Datum DESC;
>> Erster = 0.7936
>>   Hoch = 0.7986
>>   Tief = 0.7901
>>Schluss = 0.7971
>>Volumen = 0.0
>>  Datum = 2008-04-22
>>
>> Erster = 0.7905
>>   Hoch = 0.797
>>   Tief = 0.7894
>>Schluss = 0.7936
>>Volumen = 0.0
>>  Datum = 2008-04-21
>>
>>.
>>.
>>.
>>
>>if you want the optional ' HH:MM:SS' attached to Datum
>>sqlite3> SELECT Erster,Hoch,Tief,Schluss,Volumen,
>>         strftime( '%Y-%m-%d %H:%M:%S', date(Datum)) as Datum 
>>         FROM NZD_USD ORDER BY Datum DESC;
>> Erster = 0.7936
>>   Hoch = 0.7986
>>   Tief = 0.7901
>>Schluss = 0.7971
>>Volumen = 0.0
>>  Datum = 2008-04-22 00:00:00
>>
>> Erster = 0.7905
>>   Hoch = 0.797
>>   Tief = 0.7894
>>Schluss = 0.7936
>>Volumen = 0.0
>>  Datum = 2008-04-21 00:00:00
>>
>>.
>>.
>>.
>>
>>
>>
>>couple of thoughts
>>
>>if you are inserting data into your database via code (a program)
>>that you can change i would enter the date string with date and
>>time values; if the data is end-of-day the time string should
>>be indicative of market close; but only you can determine what
>>market your NZD_USD close time represents (it could be any
>>end-of-market-day any place around the world), which suggests
>>the date-time field should be 'universal coordinated time',
>>but that is not something that is well integrated within GT,
>>and isn't supported directly by beancounter either for that matter.
>>
>>
>>using perl converting date strings in the form
>>    2008/04/22
>>into
>>    2008-04-22
>>is trivial
>>    $_ =~ tr@/@-@;
>>or with more provision for error detect use Date::Manip
>>    UnixDate(ParseDate( "$_" ), "%Y-%m-%d %T" );
>>
>>
>>now if your database is so large and difficult (costly) to recreate
>>you can continue to use it as is, but recognize the Datum tuple is
>>just a 'text string' that a human would recognize as a date but sqlite
>>does not. thus you need to hack GT::DB::genericdb::get_last_prices
>>to convert the Datum values into a gt compatible date string -- trivial
>>to fairly so -- as described immediately above and in a prior message.
>>
>>
>>hope this helps, good luck,
>>
>>
>>aloha
>>
>>ras
> 
> 
> Thanks Robert !
> As I wrote already in my answer to Thomas' hints (see the GT mail
> archive -- but not all messages are present there at the moment (?)  )
> I solved the problems by changing the --timeframe=day option so that
> the correct command is
> 
> ./graph --timeframe=year NZD_USD I:ADX
> 
>  (Only) with this option it works and solves all error messages ! It is
> a similar thing as with the backtest moduls, which requires explicitely
> a
> --timeframe=day option to run.....
> 
> Concerning the population of the data base : I'm using a gnumeric
> spreadsheet with german data format (23.4.2013) as source and wrote a
> bash script which exports, cleans and imports the data to the sqlite3
> data base. The import command of sqlite3 checks the source data
> automatically, complains if the data of the specific field is not in
> an acceptable data format and converts it to the 2013/4/23 format
> without giving the user any choice.
> So, I think the data format cannot be the reason for the timeframe
> problem -- maybe its a matter of the localized versions my system
> uses as default for the date format (?).
> Anyway, I hope the problem is permanenly fixed by using the
> --timeframe=year option  ...
> Thx, delix
> 


aloha

well i took you at your word that your orginal database data strings
were failing, but i've recreated (to the extent i can) your database
with the formats   Datum = 2008/04/22

and a simple graphic.pl command does indeed work:
  ./graphic.pl --no-volume --type=candle --timeframe=day \
  --start=2008/04/15 --end='today' \
  --title='%n (%c) :: simple.gconf' \
  --add='Set-scale(auto,logarithmic)' \
  --out /var/tmp/delix_sqlite/NZD_USD.png NZD_USD

i don't have enough data to compute I:ADX ...

but i'm a bit concerned with your  --timeframe=year setting
that should cause a fairly significant change in your chart
unless you want a year timeframe or else i just don't understand
the problem yet.

just the closing price:
 %     display_indicator.pl I:Prices  NZD_USD
Calculating indicator Prices[] ...
Prices[]            [2008/04/15] = 0.7860
Prices[]            [2008/04/16] = 0.7924
Prices[]            [2008/04/17] = 0.7885
Prices[]            [2008/04/18] = 0.7894
Prices[]            [2008/04/21] = 0.7936
Prices[]            [2008/04/22] = 0.7971


sma with time-period of 2
%      display_indicator.pl I:SMA  NZD_USD 2 
Calculating indicator SMA[2, {I:Prices CLOSE}] ...
SMA[2, {I:Prices CLOSE}][2008/04/16] = 0.7892
SMA[2, {I:Prices CLOSE}][2008/04/17] = 0.7904
SMA[2, {I:Prices CLOSE}][2008/04/18] = 0.7890
SMA[2, {I:Prices CLOSE}][2008/04/21] = 0.7915
SMA[2, {I:Prices CLOSE}][2008/04/22] = 0.7954



ok I:ADX default tp is 14
so lets try 2

 %     display_indicator.pl I:ADX NZD_USD 2
Calculating indicator ADX[2, {I:Prices HIGH}, {I:Prices LOW}, {I:Prices CLOSE}] ...
+DMI[2, {I:Prices HIGH}, {I:Prices LOW}, {I:Prices CLOSE}][2008/04/17] = 4.4199
-DMI[2, {I:Prices HIGH}, {I:Prices LOW}, {I:Prices CLOSE}][2008/04/17] = 3.8674
DMI[2, {I:Prices HIGH}, {I:Prices LOW}, {I:Prices CLOSE}][2008/04/17] = 6.6667
ADX[2, {I:Prices HIGH}, {I:Prices LOW}, {I:Prices CLOSE}][2008/04/18] = 53.3333
+DMI[2, {I:Prices HIGH}, {I:Prices LOW}, {I:Prices CLOSE}][2008/04/18] = 9.1954
-DMI[2, {I:Prices HIGH}, {I:Prices LOW}, {I:Prices CLOSE}][2008/04/18] = 0.0000
DMI[2, {I:Prices HIGH}, {I:Prices LOW}, {I:Prices CLOSE}][2008/04/18] = 100.0000
ADX[2, {I:Prices HIGH}, {I:Prices LOW}, {I:Prices CLOSE}][2008/04/21] = 100.0000
+DMI[2, {I:Prices HIGH}, {I:Prices LOW}, {I:Prices CLOSE}][2008/04/21] = 23.0303
-DMI[2, {I:Prices HIGH}, {I:Prices LOW}, {I:Prices CLOSE}][2008/04/21] = 0.0000
DMI[2, {I:Prices HIGH}, {I:Prices LOW}, {I:Prices CLOSE}][2008/04/21] = 100.0000
ADX[2, {I:Prices HIGH}, {I:Prices LOW}, {I:Prices CLOSE}][2008/04/22] = 100.0000
+DMI[2, {I:Prices HIGH}, {I:Prices LOW}, {I:Prices CLOSE}][2008/04/22] = 28.5714
-DMI[2, {I:Prices HIGH}, {I:Prices LOW}, {I:Prices CLOSE}][2008/04/22] = 0.0000
DMI[2, {I:Prices HIGH}, {I:Prices LOW}, {I:Prices CLOSE}][2008/04/22] = 100.0000

without more data 3 fails ...

Re: [GT] genericdbi + sqlite : Date problem

[Robert A. S. <ra...@ac...>]

Delix wrote:
> On Mon, 22 Apr 2013 15:29:03 -0700
> "Robert A. Schmied" <ra...@ac...> hatte geschrieben :
> Thanks Robert for the quick reply.
> Now follows the steps and results with the data and configs I used :
> 
> =================================
> the database part of my ./gt/options file (the other parts differ only
> in the paths for fonts and aliases from the provided example file) :
> 
> DB::module genericdbi
> DB::genericdbi::db             SQLite
> DB::genericdbi::dbname         /home/user/Kurse/Devisen.sqlite
> DB::genericdbi::name_sql       SELECT Kennung FROM stockinfo WHERE
> (Kennung="NZD_USD") 
> DB::genericdbi::prices_sql     SELECT
> Erster,Hoch,Tief,Schluss,Volumen,Datum FROM NZD_USD ORDER BY Datum DESC
> ==================================
> 
> the .schema NZD_USD output:
> CREATE TABLE [NZD_USD] ([Datum] DATE PRIMARY KEY NOT NULL UNIQUE,
> [Erster] REAL (7, 2) NOT NULL, [Hoch] REAL (7, 2) NOT NULL, [Tief] REAL
> (7, 2) NOT NULL, [Schluss] REAL (7, 2) NOT NULL, [Volumen] REAL (10,
> 2));
> ====================================
> 
> the data output of the NZD_USD table:
> 2008/04/15|0.7923|0.7945|0.7835|0.786|0.0
> 2008/04/16|0.786|0.7924|0.7828|0.7924|0.0
> 2008/04/17|0.7925|0.7932|0.7847|0.7885|0.0
> 2008/04/18|0.7885|0.794|0.7851|0.7894|0.0
> 2008/04/21|0.7905|0.797|0.7894|0.7936|0.0
> 2008/04/22|0.7936|0.7986|0.7901|0.7971|0.0
> ====================================
> 
> the output of the command
> ./display_indicator I:RSI NZD_USD:
> Calculating indicator RSI[14, {I:Prices CLOSE}] ...
> RSI[14, {I:Prices CLOSE}][2012/08/16] = 53.1250
> RSI[14, {I:Prices CLOSE}][2012/08/17] = 49.0814
> RSI[14, {I:Prices CLOSE}][2012/08/18] = 46.9945
> RSI[14, {I:Prices CLOSE}][2012/08/19] = 28.9855
> RSI[14, {I:Prices CLOSE}][2012/08/20] = 33.1034
> RSI[14, {I:Prices CLOSE}][2012/08/21] = 35.3333
> RSI[14, {I:Prices CLOSE}][2012/08/22] = 47.2222
> ====================================
> 
> the error message I get with the command ./graphic.pl --timeframe=day
> NZD_USD I:ADX
> 
> Use of uninitialized value $d in pattern match (m//)
> at ../GT/DateTime.pm line 284. Use of uninitialized value $m in
> subtraction (-) at ../GT/DateTime.pm line 285. Argument "2008/04/15"
> isn't numeric in numeric ge (>=) at /usr/share/perl/5.14/Time/Local.pm
> line 100. Month '-1' out of range 0..11 at ../GT/DateTime.pm line 285
> 
> ====================================
> So, you see the data set is not something special but very simple.
> What confuses is the successful run of the display_indicator command
> with the same data and settings. Maybe it is something else than a data
> format problem ?
> 
> 
> Thanks again for your help and patience !
> 



delix

aloha

i don't have the complete picture of your custom database so i'm kinda
talking in the dark here, but ...

it appears you do not store a 'code' (or symbol) tuple in your prices
data table, but instead have a separate table for each code?

i base that on these two options key-value items:
  DB::genericdbi::name_sql       SELECT Kennung FROM stockinfo WHERE (Kennung="NZD_USD") 
  DB::genericdbi::prices_sql     SELECT Erster,Hoch,Tief,Schluss,Volumen,Datum FROM NZD_USD ORDER BY Datum DESC

this is likely going to be an issue *UNLESS* you will only ever what to monitor NZD_USD
or intend to change these two configuration options from the command line for every
currency pair of interest.

assuming neither of those are part of your overall plan i suggest you revise
your prices data table to include either a code/symbol tuple, or a reference
key to the symbol stored in stockinfo;

the revised DB::genericdbi::prices_sql value would be more like the generic
   SELECT Erster, Hoch, Tief, Schluss, Volumen, Datum \
    FROM stockprices WHERE symbol = '$code' \
    ORDER BY date DESC \
    LIMIT $limit;

and the table create statement more like
CREATE TABLE [stockprices] (
[Symbol]	varchar(12) not null default '',
[Datum]		date default null,
[Erster]	REAL (7, 2) NOT NULL,
[Hoch]		REAL (7, 2) NOT NULL,
[Tief]		REAL (7, 2) NOT NULL,
[Schluss]	REAL (7, 2) NOT NULL,
[Volumen]	REAL (10,2)
);
create unique index stockprices_pkey on stockprices (symbol, date);


if you are *NOT* doing this for paid work or to gain a university degree
i can help you. but you will need to send me a reasonably sized dump (say
less than a 0.250M byte) of your existing database. that might mean you
will have to truncate some of the data...
just provide me with sufficient info to recreate it here ...


aloha

ras

Re: [GT] genericdbi + sqlite : Date problem

[Delix <del...@t-...>]

On Tue, 23 Apr 2013 09:59:13 -0700
"Robert A. Schmied" <ra...@ac...> hatte geschrieben :

> Delix wrote:
> > On Mon, 22 Apr 2013 15:29:03 -0700
> > "Robert A. Schmied" <ra...@ac...> hatte geschrieben :
> > Thanks Robert for the quick reply.
> > Now follows the steps and results with the data and configs I used :
> > 
> > =================================
> > the database part of my ./gt/options file (the other parts differ only
> > in the paths for fonts and aliases from the provided example file) :
> > 
> > DB::module genericdbi
> > DB::genericdbi::db             SQLite
> > DB::genericdbi::dbname         /home/user/Kurse/Devisen.sqlite
> > DB::genericdbi::name_sql       SELECT Kennung FROM stockinfo WHERE
> > (Kennung="NZD_USD") 
> > DB::genericdbi::prices_sql     SELECT
> > Erster,Hoch,Tief,Schluss,Volumen,Datum FROM NZD_USD ORDER BY Datum DESC
> > ==================================
> > 
> > the .schema NZD_USD output:
> > CREATE TABLE [NZD_USD] ([Datum] DATE PRIMARY KEY NOT NULL UNIQUE,
> > [Erster] REAL (7, 2) NOT NULL, [Hoch] REAL (7, 2) NOT NULL, [Tief] REAL
> > (7, 2) NOT NULL, [Schluss] REAL (7, 2) NOT NULL, [Volumen] REAL (10,
> > 2));
> > ====================================
> > 
> > the data output of the NZD_USD table:
> > 2008/04/15|0.7923|0.7945|0.7835|0.786|0.0
> > 2008/04/16|0.786|0.7924|0.7828|0.7924|0.0
> > 2008/04/17|0.7925|0.7932|0.7847|0.7885|0.0
> > 2008/04/18|0.7885|0.794|0.7851|0.7894|0.0
> > 2008/04/21|0.7905|0.797|0.7894|0.7936|0.0
> > 2008/04/22|0.7936|0.7986|0.7901|0.7971|0.0
> > ====================================
> > 
> > the output of the command
> > ./display_indicator I:RSI NZD_USD:
> > Calculating indicator RSI[14, {I:Prices CLOSE}] ...
> > RSI[14, {I:Prices CLOSE}][2012/08/16] = 53.1250
> > RSI[14, {I:Prices CLOSE}][2012/08/17] = 49.0814
> > RSI[14, {I:Prices CLOSE}][2012/08/18] = 46.9945
> > RSI[14, {I:Prices CLOSE}][2012/08/19] = 28.9855
> > RSI[14, {I:Prices CLOSE}][2012/08/20] = 33.1034
> > RSI[14, {I:Prices CLOSE}][2012/08/21] = 35.3333
> > RSI[14, {I:Prices CLOSE}][2012/08/22] = 47.2222
> > ====================================
> > 
> > the error message I get with the command ./graphic.pl --timeframe=day
> > NZD_USD I:ADX
> > 
> > Use of uninitialized value $d in pattern match (m//)
> > at ../GT/DateTime.pm line 284. Use of uninitialized value $m in
> > subtraction (-) at ../GT/DateTime.pm line 285. Argument "2008/04/15"
> > isn't numeric in numeric ge (>=) at /usr/share/perl/5.14/Time/Local.pm
> > line 100. Month '-1' out of range 0..11 at ../GT/DateTime.pm line 285
> > 
> > ====================================
> > So, you see the data set is not something special but very simple.
> > What confuses is the successful run of the display_indicator command
> > with the same data and settings. Maybe it is something else than a data
> > format problem ?
> > 
> > 
> > Thanks again for your help and patience !
> > 
> 
> aloha delix
> 
> i am using sqlite3 ...
> 
> i created 'delix_db', created and populated table NZD_USD therein ...
> 
> and when i tried to convert tuple Datum i *immediately* found the
> the problem. the data (date string) being stored into tuple Datum
> is not a recognized/supported sqlite 'time-date' string.
> 
> the recognized/support sqlite date string formats are:
> 
> "A time string can be in any of the following formats:
>     YYYY-MM-DD
>     YYYY-MM-DD HH:MM
>     YYYY-MM-DD HH:MM:SS
>     YYYY-MM-DD HH:MM:SS.SSS
>     YYYY-MM-DDTHH:MM
>     YYYY-MM-DDTHH:MM:SS
>     YYYY-MM-DDTHH:MM:SS.SSS
>     HH:MM
>     HH:MM:SS
>     HH:MM:SS.SSS
>     now
>     DDDDDDDDDD
> "
> 
> so i changed the '/' used in all datum values to '-' and
> recreated the delix_db database and NZD_USD table.
> 
> sqlite3> SELECT Erster,Hoch,Tief,Schluss,Volumen,Datum FROM NZD_USD ORDER BY Datum DESC;
>  Erster = 0.7936
>    Hoch = 0.7986
>    Tief = 0.7901
> Schluss = 0.7971
> Volumen = 0.0
>   Datum = 2008-04-22
> 
>  Erster = 0.7905
>    Hoch = 0.797
>    Tief = 0.7894
> Schluss = 0.7936
> Volumen = 0.0
>   Datum = 2008-04-21
> 
> .
> .
> .
> 
> if you want the optional ' HH:MM:SS' attached to Datum
> sqlite3> SELECT Erster,Hoch,Tief,Schluss,Volumen,
>          strftime( '%Y-%m-%d %H:%M:%S', date(Datum)) as Datum 
>          FROM NZD_USD ORDER BY Datum DESC;
>  Erster = 0.7936
>    Hoch = 0.7986
>    Tief = 0.7901
> Schluss = 0.7971
> Volumen = 0.0
>   Datum = 2008-04-22 00:00:00
> 
>  Erster = 0.7905
>    Hoch = 0.797
>    Tief = 0.7894
> Schluss = 0.7936
> Volumen = 0.0
>   Datum = 2008-04-21 00:00:00
> 
> .
> .
> .
> 
> 
> 
> couple of thoughts
> 
> if you are inserting data into your database via code (a program)
> that you can change i would enter the date string with date and
> time values; if the data is end-of-day the time string should
> be indicative of market close; but only you can determine what
> market your NZD_USD close time represents (it could be any
> end-of-market-day any place around the world), which suggests
> the date-time field should be 'universal coordinated time',
> but that is not something that is well integrated within GT,
> and isn't supported directly by beancounter either for that matter.
> 
> 
> using perl converting date strings in the form
>     2008/04/22
> into
>     2008-04-22
> is trivial
>     $_ =~ tr@/@-@;
> or with more provision for error detect use Date::Manip
>     UnixDate(ParseDate( "$_" ), "%Y-%m-%d %T" );
> 
> 
> now if your database is so large and difficult (costly) to recreate
> you can continue to use it as is, but recognize the Datum tuple is
> just a 'text string' that a human would recognize as a date but sqlite
> does not. thus you need to hack GT::DB::genericdb::get_last_prices
> to convert the Datum values into a gt compatible date string -- trivial
> to fairly so -- as described immediately above and in a prior message.
> 
> 
> hope this helps, good luck,
> 
> 
> aloha
> 
> ras

Thanks Robert !
As I wrote already in my answer to Thomas' hints (see the GT mail
archive -- but not all messages are present there at the moment (?)  )
I solved the problems by changing the --timeframe=day option so that
the correct command is

./graph --timeframe=year NZD_USD I:ADX

 (Only) with this option it works and solves all error messages ! It is
a similar thing as with the backtest moduls, which requires explicitely
a
--timeframe=day option to run.....

Concerning the population of the data base : I'm using a gnumeric
spreadsheet with german data format (23.4.2013) as source and wrote a
bash script which exports, cleans and imports the data to the sqlite3
data base. The import command of sqlite3 checks the source data
automatically, complains if the data of the specific field is not in
an acceptable data format and converts it to the 2013/4/23 format
without giving the user any choice.
So, I think the data format cannot be the reason for the timeframe
problem -- maybe its a matter of the localized versions my system
uses as default for the date format (?).
Anyway, I hope the problem is permanenly fixed by using the
--timeframe=year option  ...
Thx, delix

-- 
Delix <lin...@t-...>

Re: [GT] genericdbi + sqlite : Date problem

[Robert A. S. <ra...@ac...>]

Delix wrote:
> On Mon, 22 Apr 2013 15:29:03 -0700
> "Robert A. Schmied" <ra...@ac...> hatte geschrieben :
> Thanks Robert for the quick reply.
> Now follows the steps and results with the data and configs I used :
> 
> =================================
> the database part of my ./gt/options file (the other parts differ only
> in the paths for fonts and aliases from the provided example file) :
> 
> DB::module genericdbi
> DB::genericdbi::db             SQLite
> DB::genericdbi::dbname         /home/user/Kurse/Devisen.sqlite
> DB::genericdbi::name_sql       SELECT Kennung FROM stockinfo WHERE
> (Kennung="NZD_USD") 
> DB::genericdbi::prices_sql     SELECT
> Erster,Hoch,Tief,Schluss,Volumen,Datum FROM NZD_USD ORDER BY Datum DESC
> ==================================
> 
> the .schema NZD_USD output:
> CREATE TABLE [NZD_USD] ([Datum] DATE PRIMARY KEY NOT NULL UNIQUE,
> [Erster] REAL (7, 2) NOT NULL, [Hoch] REAL (7, 2) NOT NULL, [Tief] REAL
> (7, 2) NOT NULL, [Schluss] REAL (7, 2) NOT NULL, [Volumen] REAL (10,
> 2));
> ====================================
> 
> the data output of the NZD_USD table:
> 2008/04/15|0.7923|0.7945|0.7835|0.786|0.0
> 2008/04/16|0.786|0.7924|0.7828|0.7924|0.0
> 2008/04/17|0.7925|0.7932|0.7847|0.7885|0.0
> 2008/04/18|0.7885|0.794|0.7851|0.7894|0.0
> 2008/04/21|0.7905|0.797|0.7894|0.7936|0.0
> 2008/04/22|0.7936|0.7986|0.7901|0.7971|0.0
> ====================================
> 
> the output of the command
> ./display_indicator I:RSI NZD_USD:
> Calculating indicator RSI[14, {I:Prices CLOSE}] ...
> RSI[14, {I:Prices CLOSE}][2012/08/16] = 53.1250
> RSI[14, {I:Prices CLOSE}][2012/08/17] = 49.0814
> RSI[14, {I:Prices CLOSE}][2012/08/18] = 46.9945
> RSI[14, {I:Prices CLOSE}][2012/08/19] = 28.9855
> RSI[14, {I:Prices CLOSE}][2012/08/20] = 33.1034
> RSI[14, {I:Prices CLOSE}][2012/08/21] = 35.3333
> RSI[14, {I:Prices CLOSE}][2012/08/22] = 47.2222
> ====================================
> 
> the error message I get with the command ./graphic.pl --timeframe=day
> NZD_USD I:ADX
> 
> Use of uninitialized value $d in pattern match (m//)
> at ../GT/DateTime.pm line 284. Use of uninitialized value $m in
> subtraction (-) at ../GT/DateTime.pm line 285. Argument "2008/04/15"
> isn't numeric in numeric ge (>=) at /usr/share/perl/5.14/Time/Local.pm
> line 100. Month '-1' out of range 0..11 at ../GT/DateTime.pm line 285
> 
> ====================================
> So, you see the data set is not something special but very simple.
> What confuses is the successful run of the display_indicator command
> with the same data and settings. Maybe it is something else than a data
> format problem ?
> 
> 
> Thanks again for your help and patience !
> 

aloha delix

i am using sqlite3 ...

i created 'delix_db', created and populated table NZD_USD therein ...

and when i tried to convert tuple Datum i *immediately* found the
the problem. the data (date string) being stored into tuple Datum
is not a recognized/supported sqlite 'time-date' string.

the recognized/support sqlite date string formats are:

"A time string can be in any of the following formats:
    YYYY-MM-DD
    YYYY-MM-DD HH:MM
    YYYY-MM-DD HH:MM:SS
    YYYY-MM-DD HH:MM:SS.SSS
    YYYY-MM-DDTHH:MM
    YYYY-MM-DDTHH:MM:SS
    YYYY-MM-DDTHH:MM:SS.SSS
    HH:MM
    HH:MM:SS
    HH:MM:SS.SSS
    now
    DDDDDDDDDD
"

so i changed the '/' used in all datum values to '-' and
recreated the delix_db database and NZD_USD table.

sqlite3> SELECT Erster,Hoch,Tief,Schluss,Volumen,Datum FROM NZD_USD ORDER BY Datum DESC;
 Erster = 0.7936
   Hoch = 0.7986
   Tief = 0.7901
Schluss = 0.7971
Volumen = 0.0
  Datum = 2008-04-22

 Erster = 0.7905
   Hoch = 0.797
   Tief = 0.7894
Schluss = 0.7936
Volumen = 0.0
  Datum = 2008-04-21

.
.
.

if you want the optional ' HH:MM:SS' attached to Datum
sqlite3> SELECT Erster,Hoch,Tief,Schluss,Volumen,
         strftime( '%Y-%m-%d %H:%M:%S', date(Datum)) as Datum 
         FROM NZD_USD ORDER BY Datum DESC;
 Erster = 0.7936
   Hoch = 0.7986
   Tief = 0.7901
Schluss = 0.7971
Volumen = 0.0
  Datum = 2008-04-22 00:00:00

 Erster = 0.7905
   Hoch = 0.797
   Tief = 0.7894
Schluss = 0.7936
Volumen = 0.0
  Datum = 2008-04-21 00:00:00

.
.
.



couple of thoughts

if you are inserting data into your database via code (a program)
that you can change i would enter the date string with date and
time values; if the data is end-of-day the time string should
be indicative of market close; but only you can determine what
market your NZD_USD close time represents (it could be any
end-of-market-day any place around the world), which suggests
the date-time field should be 'universal coordinated time',
but that is not something that is well integrated within GT,
and isn't supported directly by beancounter either for that matter.


using perl converting date strings in the form
    2008/04/22
into
    2008-04-22
is trivial
    $_ =~ tr@/@-@;
or with more provision for error detect use Date::Manip
    UnixDate(ParseDate( "$_" ), "%Y-%m-%d %T" );


now if your database is so large and difficult (costly) to recreate
you can continue to use it as is, but recognize the Datum tuple is
just a 'text string' that a human would recognize as a date but sqlite
does not. thus you need to hack GT::DB::genericdb::get_last_prices
to convert the Datum values into a gt compatible date string -- trivial
to fairly so -- as described immediately above and in a prior message.


hope this helps, good luck,


aloha

ras




database create details:

% rm delix_db
% sqlite3 delix_db

CREATE TABLE [NZD_USD] ([Datum] DATE PRIMARY KEY NOT NULL UNIQUE,
[Erster] REAL (7, 2) NOT NULL, [Hoch] REAL (7, 2) NOT NULL, [Tief] REAL
(7, 2) NOT NULL, [Schluss] REAL (7, 2) NOT NULL, [Volumen] REAL (10,2));

insert into NZD_USD ( Datum , Erster , Hoch , Tief, Schluss, Volumen )
             VALUES (
 '2008-04-15', 0.7923, 0.7945, 0.7835, 0.786, 0.0
);
insert into NZD_USD ( Datum , Erster , Hoch , Tief, Schluss, Volumen )
             VALUES (
 '2008-04-16', 0.786, 0.7924, 0.7828, 0.7924, 0.0
);
insert into NZD_USD ( Datum , Erster , Hoch , Tief, Schluss, Volumen )
             VALUES (
 '2008-04-17', 0.7925, 0.7932, 0.7847, 0.7885, 0.0
);
insert into NZD_USD ( Datum , Erster , Hoch , Tief, Schluss, Volumen )
             VALUES (
 '2008-04-18', 0.7885, 0.794, 0.7851, 0.7894, 0.0
);
insert into NZD_USD ( Datum , Erster , Hoch , Tief, Schluss, Volumen )
             VALUES (
 '2008-04-21', 0.7905, 0.797, 0.7894, 0.7936, 0.0
);
insert into NZD_USD ( Datum , Erster , Hoch , Tief, Schluss, Volumen )
             VALUES (
 '2008-04-22', 0.7936, 0.7986, 0.7901, 0.7971, 0.0
);

SELECT Erster,Hoch,Tief,Schluss,Volumen,Datum FROM NZD_USD ORDER BY Datum DESC;

SELECT Erster,Hoch,Tief,Schluss,Volumen, strftime( '%Y-%m-%d %H:%M:%S', date(Datum)) as Datum  from NZD_USD ORDER BY Datum ;

Re: [GT] genericdbi + sqlite : Date problem

[Thomas W. <tho...@un...>]

The code should work no matter what timeframe is selected, so there is
something wrong.

The timeframe just allows you to change what type of data you want to
look at in whatever routine you choose.

Th.


On 04/23/2013 11:24 AM, Delix wrote:
> On Tue, 23 Apr 2013 09:56:39 -0500
> Thomas Weigert <tho...@un...> hatte geschrieben :
>
>> Delix,
>>
>> I don't have the same version of the code you are running so I cannot
>> look at the actual file. But from the error messages you can see that
>> you are passing the string "2008/04/15" when a number is expected (in
>> fact, it seems that the month is expected there.)
>>
>> The first error message about the uninitialized value $d and $m may
>> indicate a problem or careless coding that does not result in a problem.
>> However, it appears that it might be related to the later problem so it
>> would be good to check those lines in DateTime.pm.
>>
>> Th.
>>
>> On 04/23/2013 04:15 AM, Delix wrote:
>>> the error message I get with the command ./graphic.pl --timeframe=day
>>> NZD_USD I:ADX
>>>
>>> Use of uninitialized value $d in pattern match (m//)
>>> at ../GT/DateTime.pm line 284. Use of uninitialized value $m in
>>> subtraction (-) at ../GT/DateTime.pm line 285. Argument "2008/04/15"
>>> isn't numeric in numeric ge (>=) at /usr/share/perl/5.14/Time/Local.pm
>>> line 100. Month '-1' out of range 0..11 at ../GT/DateTime.pm line 285
>>>
>>>
>  Thanks Thomas !
> After checking the mentioned code lines (these are the ones for
> splitting the date fpr the timeframe DAY) I tried the graphic.pl
> command with the --timeframe=year command. And it works and solves all
> errors ! Actually, I don't understand why I need the day timeframe for
> the Analysis line in the option file for the backtest,pl and the
> --timeframe=year option for the graphic modul -- but it seems it MUST
> be used this way.
> Thanks for the hint !

Re: [GT] genericdbi + sqlite : Date problem

[Thomas W. <tho...@un...>]

Delix,

I don't have the same version of the code you are running so I cannot
look at the actual file. But from the error messages you can see that
you are passing the string "2008/04/15" when a number is expected (in
fact, it seems that the month is expected there.)

The first error message about the uninitialized value $d and $m may
indicate a problem or careless coding that does not result in a problem.
However, it appears that it might be related to the later problem so it
would be good to check those lines in DateTime.pm.

Th.

On 04/23/2013 04:15 AM, Delix wrote:
>
> the error message I get with the command ./graphic.pl --timeframe=day
> NZD_USD I:ADX
>
> Use of uninitialized value $d in pattern match (m//)
> at ../GT/DateTime.pm line 284. Use of uninitialized value $m in
> subtraction (-) at ../GT/DateTime.pm line 285. Argument "2008/04/15"
> isn't numeric in numeric ge (>=) at /usr/share/perl/5.14/Time/Local.pm
> line 100. Month '-1' out of range 0..11 at ../GT/DateTime.pm line 285
>
>

Re: [GT] genericdbi + sqlite : Date problem

[Delix <del...@t-...>]

On Mon, 22 Apr 2013 15:29:03 -0700
"Robert A. Schmied" <ra...@ac...> hatte geschrieben :
Thanks Robert for the quick reply.
Now follows the steps and results with the data and configs I used :

=================================
the database part of my ./gt/options file (the other parts differ only
in the paths for fonts and aliases from the provided example file) :

DB::module genericdbi
DB::genericdbi::db             SQLite
DB::genericdbi::dbname         /home/user/Kurse/Devisen.sqlite
DB::genericdbi::name_sql       SELECT Kennung FROM stockinfo WHERE
(Kennung="NZD_USD") 
DB::genericdbi::prices_sql     SELECT
Erster,Hoch,Tief,Schluss,Volumen,Datum FROM NZD_USD ORDER BY Datum DESC
==================================

the .schema NZD_USD output:
CREATE TABLE [NZD_USD] ([Datum] DATE PRIMARY KEY NOT NULL UNIQUE,
[Erster] REAL (7, 2) NOT NULL, [Hoch] REAL (7, 2) NOT NULL, [Tief] REAL
(7, 2) NOT NULL, [Schluss] REAL (7, 2) NOT NULL, [Volumen] REAL (10,
2));
====================================

the data output of the NZD_USD table:
2008/04/15|0.7923|0.7945|0.7835|0.786|0.0
2008/04/16|0.786|0.7924|0.7828|0.7924|0.0
2008/04/17|0.7925|0.7932|0.7847|0.7885|0.0
2008/04/18|0.7885|0.794|0.7851|0.7894|0.0
2008/04/21|0.7905|0.797|0.7894|0.7936|0.0
2008/04/22|0.7936|0.7986|0.7901|0.7971|0.0
====================================

the output of the command
./display_indicator I:RSI NZD_USD:
Calculating indicator RSI[14, {I:Prices CLOSE}] ...
RSI[14, {I:Prices CLOSE}][2012/08/16] = 53.1250
RSI[14, {I:Prices CLOSE}][2012/08/17] = 49.0814
RSI[14, {I:Prices CLOSE}][2012/08/18] = 46.9945
RSI[14, {I:Prices CLOSE}][2012/08/19] = 28.9855
RSI[14, {I:Prices CLOSE}][2012/08/20] = 33.1034
RSI[14, {I:Prices CLOSE}][2012/08/21] = 35.3333
RSI[14, {I:Prices CLOSE}][2012/08/22] = 47.2222
====================================

the error message I get with the command ./graphic.pl --timeframe=day
NZD_USD I:ADX

Use of uninitialized value $d in pattern match (m//)
at ../GT/DateTime.pm line 284. Use of uninitialized value $m in
subtraction (-) at ../GT/DateTime.pm line 285. Argument "2008/04/15"
isn't numeric in numeric ge (>=) at /usr/share/perl/5.14/Time/Local.pm
line 100. Month '-1' out of range 0..11 at ../GT/DateTime.pm line 285

====================================
So, you see the data set is not something special but very simple.
What confuses is the successful run of the display_indicator command
with the same data and settings. Maybe it is something else than a data
format problem ?


Thanks again for your help and patience !

-- 
Delix <lin...@t-...>

Re: [GT] genericdbi + sqlite : Date problem

[Robert A. S. <ra...@ac...>]

Delix wrote:
> Hi Robert, it's me again...
> I'm still trying to use my own sqlite database as data source and solved
> most of the issue I got.
> But I need some help with solving the date format problem. It is the
> same as descibed in the BeancounterBug wiki page (now not availlable,
> but it simply advices to modify the return UnixDate format line, so
> the beancounter output is 22-10-2013 instead of 22/10/2013).
> 
> However, this modification has to be made in the Beancounter source
> code, not in any GT modul. So, in my case where Beancounter.pm is not
> called and included, the modification is still missing.
> Do you have a suggestion where I can put a similar modification in
> the genericdbi.pm or any other  GT modul to solve this problem ?
> Thanks again,
> delix 
> 


aloha delix

let me explain what i think you are doing (if anything is wrong please
supply correction(s)), and along the way show the issue you've run into
and then offer what i think might be solution candidates for you

-- but they might also depend on the version of sqlite you are using
(for arguments i will base my stuff on sqlite3 since that's the version
i've got installed).

1) you are using gt with a delix-custom database running by/on a sqlite engine.

2) you are therefore using the GT::DB::genericdbi interface module with
   unspecified sql query statements (in your $HOME/.gt/options file).

3) you are willing to modify any component in order to resolve the
   data-time issue you've run into.


*NOTES*
the now unavailable beancounter bug wiki page suggested a change to
stock beancounter: file Finance/BeanCounter.pm

sub GetDate {
  my ($value) = @_;
#  return UnixDate(ParseDate($value), "%Y%m%d");  # remove (comment out) this line
  return UnixDate(ParseDate($value), "%Y-%m-%d"); # add this line
}

the above suggests the date ordering is *different* from that in your message:
   specifically    year - month_number - day_of_month
          *not*    day_of_month - month_number - year


so if my understanding is basically correct, you want to somehow get sqlite
to output prices data with a 'date' field that is compatible with gt ...

using GT::DB::genericdbi your prices query 'DB::genericdbi::prices_sql'
might be something like this

        SELECT period_open, period_high, period_low, period_close, volume, date
        FROM stockprices
        WHERE symbol = '$code' \
          ORDER BY date DESC \
          LIMIT $limit;

    ---->>>> without having your database schema the tuple names are gonna be 'typical':
    ---->>>> without knowing exactly what you are storing for date and time i'm
             not going to able to give you specific answer, just generic answers.


if the date string yielded from the above sql query statement is NOT 

      "year-month_number-day_of_month HH:MM:SS"
                              where " HH:MM:SS" is optional
                              if using end-of-day or larger time-frame periods

then you can have the database engine convert the date string for you directly
or alter GT/DB/genericdbi.pm and make a perl module do the conversion.

sqlite only supports a limit number of time strings:

    YYYY-MM-DD
    YYYY-MM-DD HH:MM
    YYYY-MM-DD HH:MM:SS
    YYYY-MM-DD HH:MM:SS.SSS
    YYYY-MM-DDTHH:MM
    YYYY-MM-DDTHH:MM:SS
    YYYY-MM-DDTHH:MM:SS.SSS
    HH:MM
    HH:MM:SS
    HH:MM:SS.SSS
    now
    DDDDDDDDDD 

if you are storing something else then sqlite will not be able to do the
conversion for you.

assuming "date" being stored as a text string "YYYY-MM-DD" then to get
'yyyy-mm-dd hh:mm:ss' from sqlite something like this should/could/would work:

        SELECT period_open, period_high, period_low, period_close, volume,
          strftime( '%Y-%m-%d %H:%M:%S', date ) \
        FROM stockprices \
        WHERE symbol = '$code' \
          ORDER BY date DESC \
          LIMIT $limit;

if you are storing unix epoch seconds in date (DDDDDDDDDD) then something like
          datetime( date, 'unixepoch' ) \
might be all you need.

also see (http://www.sqlite.org/datatype3.html sec 1.2 Date and Time Datatype
and (http://www.sqlite.org/lang_datefunc.html)


if you forward your database schema details* i will be able to give you better
details on how to get the output format you want ... but you'll probably be
able to work it out yourself using the urls above too.

* 1) schema of prices table ( sqlite3> .schema stockprices )
  2) just 10 rows of price table data with at least two different datetime values
        ( sqlite3> select * from stockprices where symbol='SOME_SYMBOL' limit=10; )
  3) the sql prices query 'DB::genericdbi::prices_sql' value



on the other hand -- if you don't want to mess around with your database or
the sql query used to fetch the data you could modify ../GT/DB/genericdbi.pm
to do the date-string conversion.

again without specifics i'm just showing possibilities, but if the sql query
yields a time-date string that Date::Manip can recognize (and it does handle
a bunch) then you can tweak GT::DB::genericdbi::get_last_prices to add a conversion
to the date array element of @$array_ref before the return $q statement.

but if you are storing an unusual date-time string we might have to do
some perl string splitting and joining in order to get the string into
a recognizable date-time form.

if you want to go in this direction send me a 10 or so lines @$array_ref

something like:

    use Data::Dumper;
    for ( my $i=0; $i<=10; ++$i) {
      print Dumper $array_ref->[$i], "\n";
    }

just before the return statement in GT::DB::genericdbi::get_last_prices

aloha

ras

[GT] genericdbi + sqlite : Date problem

[Delix <del...@t-...>]

Hi Robert, it's me again...
I'm still trying to use my own sqlite database as data source and solved
most of the issue I got.
But I need some help with solving the date format problem. It is the
same as descibed in the BeancounterBug wiki page (now not availlable,
but it simply advices to modify the return UnixDate format line, so
the beancounter output is 22-10-2013 instead of 22/10/2013).

However, this modification has to be made in the Beancounter source
code, not in any GT modul. So, in my case where Beancounter.pm is not
called and included, the modification is still missing.
Do you have a suggestion where I can put a similar modification in
the genericdbi.pm or any other  GT modul to solve this problem ?
Thanks again,
delix 

-- 
Delix <lin...@t-...>

Re: [GT] Post install issues

[Robert A. S. <ra...@ac...>]

Evgeny Shragovich wrote:
> Hi ras,
> 
> I will be happy to help!
> Please provide me the hacked version of the build.pl and I will run it on
> my system.
> 
> 
> Meanwhile, as I am going through the documentation I have few more
> questions:
> 
> 1. If I use DB::module Text, is it possible to specify more that one data
> source paths?
>     I have noticed that if I just put 2 path lines, the last one is being
> used.

only one directory path is supported. that directory can contain files for the
same company (symbol) but with different time-frames too.

if different directory paths are *extremely* important you can set the default
value in your $HOME/.gt/options file and then pass an alternate path using
the standard command line option --option='DB::text::directory=<alternate_path>'
note that you must quote the key=value as i show in order to escape the '='
from the perl argument processing module.

> 
> 2. When going through the documentation, specifically
> thehow_to_spec+debug_a_system.pod.html<http://geniustrader.org/doc/GT/Docs/how_to_spec+debug_a_system.pod.html>,
> I have found the following list of documents recommended for initial
> reading:
> 
>     gt_sig-sys-desc.pod
>     gt_files.pod
>     backtest.pl
>     GT/SystemManager.pm
>     GT/CloseStrategy.pm
>     GT/PortfolioManager.pm
>     GT/TradeFilters.pm
>     GT/OrderFactory.pm
>     GT/MoneyManagement.pm
> 
> However I was not able to locate the "gt_files.pod" file. Does it
> exist/relevant?
> 

humm -- well, i've got it in both my ras hack as well as in the svn repo
but that doesn't necessarily mean it has actually been committed ...
something like svn { status || log } ../svn_repo/GT/Docs/gt_files.pod might
be the command i want ...

well it does appear to be a old file, meaning it's been around since 2008-11-07

now you are making me concerned what version of gt you've got.

how did you download it? there are multiple branches -- we recommend
using the trunk but the exp branch is also acceptable.

i'm showing the trunk svn version as Revision: 720
  %    svn info

since the sourceforge conversion (13-14apr) the only way to download gt appears
to be an svn repo clone copy ( see http://geniustrader.org/development.html for
detailed instructions).


a light might have just came on (in my head) -- the pod files are in
GT/Docs -- did you look there?
  %    find .. -name \*.pod
or
  %    ls -lFatr ../GT/Docs


> 3. I have tried creating a graph using the gconf and ACTV.txt files.
>     The graph was eventually created but again was looking funny and I have
> also received the below error on the screen:
> 
>    [Evgeny@localhost Scripts]$ ./graphic.pl --file screenshot_2.gconf --out
> 'ACTV.png' ACTV
>    MountainBand(Indicators::BOL/2 20 2.0,Indicators::BOL/3 15 2.0, \ is not
> a valid object description at ./graphic.pl line 856.

my guess is you failed to edit your cut+paste of screenshot_2.gconf to either
remove entirely 'continuation' lines introduced by the '\' char being complained
about or there are chars following the '\' char in that file.

i recall that graphic.pl is sensitive to 'continuation' lines and cannot
support them in some specific ways -- but the pod should explain those limits
in detail ...

using my svn repo and your options, shortened ADX data and a *correct*
screenshot_2.gconf file
  %   ./graphic.pl --file ../website/screenshots/screenshot_2.gconf \
  --out /tmp/ADX_x.png ADX_x
no errors/warnings and a correct chart ...


aloha

ras


>    .
>    .
>    .
>    Use of uninitialized value $func in pattern match (m//) at
> ./graphic.plline 778.
>    Use of uninitialized value $func in pattern match (m//) at
> ./graphic.plline 782.
>    VotingLine( Systems::Generic \ is not a valid object description at ./
> graphic.pl line 856.
>    Use of uninitialized value $func in pattern match (m//) at
> ./graphic.plline 637.
>    Use of uninitialized value $func in pattern match (m//) at
> ./graphic.plline 648.
>    .
>    .
>    .
>    Use of uninitialized value $func in pattern match (m//) at
> ./graphic.plline 782.
>    MountainBand(Indicators::Generic::Eval \ is not a valid object
> description at ./graphic.pl line 856.
>    Use of uninitialized value $func in pattern match (m//) at
> ./graphic.plline 637.
>    Use of uninitialized value $func in pattern match (m//) at
> ./graphic.plline 648.
> 
> 
> Any idea what might cause these errors?
> 
> 
> Thanks!
> Eugene
> 
> 

< snip >

Re: [GT] Post install issues

[Evgeny S. <shr...@gm...>]

2013-04-03,4.08,4.14,4.03,4.10,321900,4.10
2013-04-02,3.97,4.10,3.89,4.08,507000,4.08
2013-04-01,4.17,4.22,3.83,3.95,942400,3.95
2013-03-28,4.08,4.20,4.08,4.19,680200,4.19
2013-03-27,4.00,4.13,3.98,4.08,638800,4.08
2013-03-26,4.16,4.16,4.00,4.04,678600,4.04
2013-03-25,4.28,4.31,3.98,4.11,1332600,4.11
2013-03-22,4.25,4.31,4.21,4.25,309300,4.25
2013-03-21,4.28,4.34,4.24,4.25,877200,4.25
2013-03-20,4.57,4.60,4.27,4.32,1418200,4.32
2013-03-19,4.63,4.66,4.41,4.50,544200,4.50
2013-03-18,4.52,4.65,4.52,4.65,223900,4.65
2013-03-15,4.70,4.72,4.47,4.63,614700,4.63
2013-03-14,4.68,4.78,4.64,4.69,335700,4.69
2013-03-13,4.78,4.81,4.67,4.68,334600,4.68
2013-03-12,4.94,4.94,4.78,4.79,583500,4.79
2013-03-11,4.82,4.96,4.76,4.95,644800,4.95
2013-03-08,4.69,4.85,4.67,4.82,445300,4.82
2013-03-07,4.63,4.69,4.53,4.66,426600,4.66
2013-03-06,4.55,4.69,4.53,4.61,1040000,4.61
2013-03-05,4.54,4.60,4.49,4.54,500700,4.54
2013-03-04,4.53,4.59,4.47,4.54,1663900,4.54
2013-03-01,4.65,4.70,4.45,4.53,1334800,4.53
2013-02-28,4.81,5.00,4.66,4.69,469100,4.69
2013-02-27,4.78,4.88,4.75,4.82,355800,4.82
2013-02-26,4.80,5.02,4.70,4.76,599400,4.76
2013-02-25,4.85,5.01,4.76,4.81,542900,4.81
2013-02-22,4.82,4.90,4.70,4.84,525900,4.84
2013-02-21,4.97,5.04,4.73,4.75,707700,4.75
2013-02-20,5.29,5.38,4.99,4.99,945100,4.99
2013-02-19,5.40,5.41,5.07,5.30,976800,5.30
2013-02-15,5.04,5.44,5.00,5.37,2159600,5.37
2013-02-14,5.48,5.80,5.47,5.55,943500,5.55
2013-02-13,5.33,5.69,5.31,5.69,950800,5.69
2013-02-12,5.62,5.62,5.12,5.34,3636900,5.34
2013-02-11,6.03,6.03,5.52,5.61,1973800,5.61
2013-02-08,5.75,6.12,5.72,6.03,625000,6.03
2013-02-07,5.69,5.87,5.61,5.72,476800,5.72
2013-02-06,5.50,5.74,5.49,5.68,510300,5.68
2013-02-05,5.59,5.59,5.39,5.51,977500,5.51
2013-02-04,5.76,5.80,5.50,5.58,574400,5.58
2013-02-01,5.61,5.91,5.56,5.82,614800,5.82
2013-01-31,5.48,5.60,5.43,5.53,435400,5.53
2013-01-30,5.52,5.61,5.44,5.47,337400,5.47
2013-01-29,5.67,5.67,5.50,5.52,400400,5.52
2013-01-28,5.57,5.71,5.50,5.65,425000,5.65
2013-01-25,5.56,5.58,5.40,5.54,345600,5.54
2013-01-24,5.47,5.61,5.42,5.52,361700,5.52
2013-01-23,5.46,5.54,5.37,5.47,724100,5.47
2013-01-22,5.54,5.56,5.37,5.42,436900,5.42
2013-01-18,5.60,5.61,5.47,5.53,393900,5.53
2013-01-17,5.81,5.90,5.54,5.57,576400,5.57
2013-01-16,5.87,6.00,5.74,5.78,405900,5.78
2013-01-15,6.08,6.09,5.60,5.90,1032600,5.90
2013-01-14,5.75,6.30,5.67,6.16,1175000,6.16
2013-01-11,5.66,5.78,5.50,5.78,534100,5.78
2013-01-10,5.45,5.92,5.45,5.67,789300,5.67
2013-01-09,5.32,5.45,5.25,5.41,422800,5.41
2013-01-08,5.46,5.53,5.22,5.29,394000,5.29
2013-01-07,5.57,5.62,5.36,5.48,488000,5.48
2013-01-04,5.27,5.63,5.22,5.60,454600,5.60
2013-01-03,5.31,5.40,5.22,5.24,371600,5.24
2013-01-02,5.13,5.40,5.05,5.28,845700,5.28
2012-12-31,4.78,5.10,4.74,4.91,879700,4.91
2012-12-28,4.70,4.90,4.65,4.80,1484800,4.80
2012-12-27,4.77,4.78,4.66,4.70,509200,4.70
2012-12-26,4.69,4.81,4.63,4.78,1047800,4.78
2012-12-24,4.79,4.81,4.64,4.68,320100,4.68
2012-12-21,4.80,4.88,4.72,4.74,1002400,4.74
2012-12-20,4.99,4.99,4.82,4.88,697700,4.88
2012-12-19,5.17,5.18,4.97,4.99,528200,4.99
2012-12-18,5.00,5.27,4.99,5.09,782200,5.09
2012-12-17,4.99,5.01,4.91,4.97,403600,4.97
2012-12-14,4.98,5.04,4.87,4.96,751800,4.96
2012-12-13,5.10,5.14,4.99,5.01,650400,5.01
2012-12-12,4.95,5.32,4.81,5.09,2104400,5.09
2012-12-11,4.82,4.98,4.74,4.95,626600,4.95
2012-12-10,4.77,4.86,4.66,4.77,516000,4.77
2012-12-07,4.72,4.86,4.70,4.78,491300,4.78
2012-12-06,4.70,4.79,4.64,4.65,586800,4.65
2012-12-05,4.99,5.08,4.66,4.69,855800,4.69
2012-12-04,5.04,5.15,4.96,5.00,492800,5.00
2012-12-03,5.43,5.44,5.02,5.05,1175000,5.05
2012-11-30,5.63,5.66,5.32,5.36,2130100,5.36
2012-11-29,4.82,5.17,4.79,5.05,1203500,5.05
2012-11-28,4.70,4.75,4.67,4.67,580800,4.67
2012-11-27,4.80,4.80,4.69,4.71,488500,4.71
2012-11-26,4.87,4.94,4.74,4.79,681200,4.79
2012-11-23,4.83,4.85,4.60,4.83,338700,4.83
2012-11-21,5.04,5.04,4.78,4.78,359400,4.78
2012-11-20,5.00,5.00,4.83,4.91,676400,4.91
2012-11-19,5.02,5.18,4.96,5.03,564000,5.03
2012-11-16,4.98,5.07,4.67,4.94,1290800,4.94
2012-11-15,5.14,5.18,4.98,5.00,1056100,5.00
2012-11-14,5.34,5.34,5.09,5.12,1548700,5.12
2012-11-13,5.35,5.51,5.28,5.30,1100200,5.30
2012-11-12,5.71,5.75,5.30,5.34,721700,5.34
2012-11-09,5.71,5.84,5.63,5.74,869900,5.74
2012-11-08,5.72,5.87,5.65,5.67,790000,5.67
2012-11-07,5.83,5.83,5.50,5.75,1960000,5.75
2012-11-06,5.62,6.05,5.42,5.92,2773100,5.92
2012-11-05,5.33,5.67,5.28,5.61,3453100,5.61
2012-11-02,6.39,6.39,5.02,5.42,16220500,5.42
2012-11-01,8.63,9.49,8.46,9.29,2255600,9.29
2012-10-31,9.01,9.14,8.50,8.86,3130800,8.86
2012-10-26,9.19,9.20,8.94,8.97,984600,8.97
2012-10-25,9.27,9.28,8.97,9.21,908600,9.21
2012-10-24,9.55,9.61,9.13,9.19,480900,9.19
2012-10-23,9.59,9.63,9.31,9.49,522100,9.49
2012-10-22,9.94,9.94,9.54,9.63,568900,9.63
2012-10-19,10.04,10.12,9.88,9.95,535500,9.95
2012-10-18,9.94,10.21,9.90,10.16,654700,10.16
2012-10-17,10.33,10.33,9.58,9.92,1580600,9.92
2012-10-16,10.67,10.67,10.27,10.34,682900,10.34
2012-10-15,10.81,10.81,10.54,10.62,261600,10.62
2012-10-12,10.84,10.84,10.72,10.81,235000,10.81
2012-10-11,10.80,11.16,10.76,10.86,723400,10.86
2012-10-10,10.64,10.78,10.48,10.75,542500,10.75
2012-10-09,10.95,10.95,10.60,10.64,669600,10.64
2012-10-08,11.34,11.34,10.84,10.99,2320800,10.99
2012-10-05,11.60,11.60,11.36,11.38,497100,11.38
2012-10-04,11.79,11.82,11.49,11.60,633100,11.60
2012-10-03,12.00,12.05,11.73,11.78,656700,11.78
2012-10-02,12.14,12.18,11.90,11.95,715400,11.95
2012-10-01,12.56,12.63,12.05,12.06,1178100,12.06
2012-09-28,12.62,12.82,12.53,12.53,254900,12.53
2012-09-27,12.46,12.65,12.33,12.55,327100,12.55
2012-09-26,12.29,12.40,12.20,12.39,208000,12.39
2012-09-25,12.63,12.74,12.19,12.25,467900,12.25
2012-09-24,12.44,12.61,12.39,12.53,409500,12.53
2012-09-21,12.43,12.67,12.35,12.65,556900,12.65
2012-09-20,12.74,12.74,12.24,12.28,230900,12.28
2012-09-19,12.43,12.99,12.41,12.90,619800,12.90
2012-09-18,12.16,12.49,12.09,12.42,523500,12.42
2012-09-17,12.37,12.37,12.04,12.13,318100,12.13
2012-09-14,12.10,12.47,12.10,12.39,414200,12.39
2012-09-13,12.30,12.35,12.00,12.14,926800,12.14
2012-09-12,12.40,12.40,12.26,12.36,284200,12.36
2012-09-11,12.20,12.45,12.15,12.39,355400,12.39
2012-09-10,12.24,12.27,12.02,12.19,210700,12.19
2012-09-07,12.41,12.45,12.13,12.25,251400,12.25
2012-09-06,12.00,12.36,12.00,12.34,372500,12.34
2012-09-05,11.88,12.08,11.78,11.97,341500,11.97
2012-09-04,11.16,11.71,11.14,11.63,223300,11.63
2012-08-31,10.91,11.32,10.78,11.26,1355400,11.26
2012-08-30,11.03,11.05,10.78,10.80,238700,10.80
2012-08-29,11.00,11.14,10.99,11.07,214400,11.07
2012-08-28,11.09,11.14,10.99,11.01,260600,11.01
2012-08-27,11.08,11.18,11.00,11.04,357700,11.04
2012-08-24,10.93,11.06,10.90,10.99,324500,10.99
2012-08-23,11.25,11.25,10.94,10.99,343400,10.99
2012-08-22,11.35,11.48,11.28,11.30,212000,11.30
2012-08-21,11.28,11.63,11.27,11.41,291700,11.41
2012-08-20,11.46,11.67,11.24,11.25,356400,11.25
2012-08-17,11.50,11.63,11.40,11.50,378100,11.50
2012-08-16,11.50,11.73,11.43,11.54,718200,11.54
2012-08-15,11.43,11.68,11.40,11.50,424800,11.50
2012-08-14,11.26,11.54,11.26,11.45,334400,11.45
2012-08-13,11.25,11.26,11.03,11.22,199500,11.22
2012-08-10,11.51,11.62,11.22,11.24,282900,11.24
2012-08-09,11.49,11.63,11.42,11.60,435800,11.60
2012-08-08,11.25,11.53,11.25,11.49,393000,11.49
2012-08-07,11.32,11.48,11.03,11.30,417900,11.30
2012-08-06,11.82,11.92,11.15,11.29,462600,11.29
2012-08-03,13.37,13.50,11.43,11.75,1646200,11.75
2012-08-02,13.99,14.12,13.80,13.95,223500,13.95
2012-08-01,14.23,14.35,14.01,14.03,259300,14.03
2012-07-31,14.64,14.75,14.16,14.19,179500,14.19
2012-07-30,14.69,14.75,14.50,14.66,112400,14.66
2012-07-27,14.34,14.70,14.21,14.67,172400,14.67
2012-07-26,14.37,14.65,14.25,14.26,115200,14.26
2012-07-25,14.51,14.54,14.08,14.22,92000,14.22
2012-07-24,14.58,14.62,14.40,14.42,130800,14.42
2012-07-23,14.68,14.72,14.59,14.59,103300,14.59
2012-07-20,14.84,14.99,14.77,14.95,120200,14.95
2012-07-19,15.00,15.10,14.92,15.00,61100,15.00
2012-07-18,14.71,15.09,14.71,14.98,108300,14.98
2012-07-17,14.75,14.82,14.49,14.76,149400,14.76
2012-07-16,15.00,15.00,14.70,14.75,73200,14.75
2012-07-13,14.60,15.12,14.60,15.04,234300,15.04
2012-07-12,14.70,14.75,14.37,14.59,217900,14.59
2012-07-11,14.93,15.02,14.83,14.88,192700,14.88
2012-07-10,15.00,15.06,14.91,14.94,226100,14.94
2012-07-09,14.90,14.98,14.75,14.97,151700,14.97
2012-07-06,15.24,15.27,14.78,14.97,163800,14.97
2012-07-05,15.27,15.57,15.15,15.42,146000,15.42
2012-07-03,15.48,15.60,15.11,15.34,201700,15.34
2012-07-02,15.52,15.52,15.13,15.46,204800,15.46
2012-06-29,15.21,15.50,15.13,15.39,282900,15.39
2012-06-28,15.19,15.32,14.76,14.96,349700,14.96
2012-06-27,15.45,16.00,15.18,15.25,363700,15.25
2012-06-26,15.45,15.55,15.25,15.32,370300,15.32
2012-06-25,15.47,15.57,15.35,15.36,442100,15.36
2012-06-22,15.95,15.98,15.56,15.70,3436200,15.70
2012-06-21,16.17,16.17,15.71,15.91,659900,15.91
2012-06-20,15.93,16.32,15.81,16.24,533200,16.24
2012-06-19,15.84,15.94,15.66,15.93,379500,15.93
2012-06-18,15.66,15.95,15.42,15.71,413000,15.71
2012-06-15,15.20,15.80,15.20,15.80,394200,15.80
2012-06-14,15.25,15.34,15.10,15.20,452900,15.20
2012-06-13,14.86,15.34,14.82,15.23,855900,15.23
2012-06-12,14.70,14.95,14.58,14.95,401600,14.95
2012-06-11,14.60,14.86,14.42,14.62,508900,14.62
2012-06-08,14.61,14.62,14.44,14.59,392500,14.59
2012-06-07,14.80,14.89,14.51,14.60,657700,14.60
2012-06-06,14.30,14.77,14.30,14.70,435200,14.70
2012-06-05,13.87,14.27,13.81,14.27,816300,14.27
2012-06-04,13.82,14.03,13.67,13.90,380400,13.90
2012-06-01,13.83,13.93,13.70,13.80,305800,13.80
2012-05-31,14.05,14.18,13.95,14.04,454600,14.04
2012-05-30,14.13,14.14,13.97,14.05,464000,14.05
2012-05-29,13.95,14.25,13.91,14.23,468100,14.23
2012-05-25,13.76,13.96,13.71,13.88,300000,13.88
2012-05-24,13.10,13.80,13.08,13.80,621100,13.80
2012-05-23,13.09,13.17,12.70,13.06,183000,13.06
2012-05-22,13.20,13.35,13.00,13.15,259200,13.15
2012-05-21,12.98,13.18,12.88,13.18,164800,13.18
2012-05-18,12.62,13.10,12.44,12.93,458100,12.93
2012-05-17,13.09,13.09,12.18,12.64,1373000,12.64
2012-05-16,13.83,13.89,12.92,13.02,486000,13.02
2012-05-15,14.04,14.46,13.85,13.86,215000,13.86
2012-05-14,14.59,14.59,13.91,14.00,261100,14.00
2012-05-11,14.81,14.93,14.60,14.68,153000,14.68
2012-05-10,14.92,15.02,14.78,14.87,337900,14.87
2012-05-09,14.63,14.94,14.58,14.88,291200,14.88
2012-05-08,14.84,14.90,14.61,14.76,290000,14.76
2012-05-07,15.23,15.33,14.80,14.92,369000,14.92
2012-05-04,16.00,16.32,15.11,15.29,499800,15.29
2012-05-03,16.72,16.85,15.98,16.00,146000,16.00
2012-05-02,16.71,16.91,16.65,16.72,200000,16.72
2012-05-01,16.72,17.01,16.68,16.81,193700,16.81
2012-04-30,16.96,17.03,16.80,16.80,218000,16.80
2012-04-27,16.95,17.07,16.86,16.96,494600,16.96
2012-04-26,16.86,17.03,16.85,16.96,183600,16.96
2012-04-25,16.74,17.00,16.73,16.93,412800,16.93
2012-04-24,16.79,16.91,16.62,16.74,442100,16.74
2012-04-23,16.55,16.74,16.21,16.67,319900,16.67
2012-04-20,16.68,16.80,16.60,16.74,193100,16.74
2012-04-19,16.75,16.98,16.38,16.56,101700,16.56
2012-04-18,16.85,16.88,16.63,16.72,119900,16.72
2012-04-17,16.76,17.02,16.76,16.90,321600,16.90
2012-04-16,16.55,16.74,16.52,16.71,191100,16.71
2012-04-13,16.74,17.24,16.28,16.50,227200,16.50
2012-04-12,16.42,16.65,16.00,16.52,174000,16.52
2012-04-11,16.41,16.56,16.34,16.55,150300,16.55
2012-04-10,16.23,16.37,16.12,16.33,219500,16.33
2012-04-09,16.63,16.63,16.19,16.19,103400,16.19
2012-04-05,16.50,16.88,16.42,16.84,110200,16.84
2012-04-04,16.79,16.79,16.42,16.60,328800,16.60
2012-04-03,16.75,16.98,16.75,16.90,206100,16.90
2012-04-02,16.77,16.87,16.42,16.79,136600,16.79
2012-03-30,17.14,17.15,16.83,16.83,223400,16.83
2012-03-29,16.95,17.06,16.72,17.05,117700,17.05
2012-03-28,17.10,17.16,16.75,17.04,287000,17.04
2012-03-27,17.22,17.25,17.03,17.11,429500,17.11
2012-03-26,16.81,17.20,16.81,17.18,177300,17.18
2012-03-23,16.64,16.79,16.48,16.78,210000,16.78
2012-03-22,16.58,16.67,16.50,16.59,104700,16.59
2012-03-21,16.82,16.82,16.66,16.70,235200,16.70
2012-03-20,16.71,16.84,16.53,16.75,93600,16.75
2012-03-19,16.86,17.04,16.69,16.80,177400,16.80
2012-03-16,16.89,17.01,16.66,16.83,1014200,16.83
2012-03-15,17.30,17.39,16.63,16.96,290400,16.96
2012-03-14,17.70,17.70,17.12,17.34,242500,17.34
2012-03-13,17.00,17.74,16.94,17.74,402400,17.74
2012-03-12,16.64,17.00,16.45,16.99,460700,16.99
2012-03-09,16.30,16.98,16.16,16.69,670900,16.69
2012-03-08,16.00,16.46,15.64,16.39,865200,16.39
2012-03-07,15.67,15.91,15.50,15.87,329000,15.87
2012-03-06,15.70,15.81,15.42,15.65,126000,15.65
2012-03-05,15.88,16.05,15.84,15.86,310300,15.86
2012-03-02,15.94,16.05,15.82,15.97,642900,15.97
2012-03-01,16.01,16.04,15.88,16.00,564400,16.00
2012-02-29,16.00,16.06,15.85,16.00,308800,16.00
2012-02-28,15.83,16.12,15.78,15.80,292800,15.80
2012-02-27,15.92,16.03,15.52,15.86,621100,15.86
2012-02-24,15.50,16.48,15.50,16.00,793600,16.00
2012-02-23,15.71,15.74,15.44,15.49,136900,15.49
2012-02-22,15.97,16.00,15.59,15.75,206700,15.75
2012-02-21,16.15,16.15,15.88,15.97,289200,15.97
2012-02-17,15.82,16.17,15.77,16.16,155400,16.16
2012-02-16,15.74,15.92,15.63,15.76,169200,15.76
2012-02-15,15.59,15.76,15.54,15.69,124000,15.69
2012-02-14,15.47,15.50,15.30,15.50,48700,15.50
2012-02-13,15.49,15.50,15.43,15.49,138300,15.49
2012-02-10,15.13,15.42,15.13,15.38,92800,15.38
2012-02-09,15.15,15.31,15.06,15.31,223700,15.31
2012-02-08,15.17,15.21,15.07,15.15,78200,15.15
2012-02-07,15.15,15.21,15.10,15.20,66200,15.20
2012-02-06,15.28,15.30,15.05,15.21,73800,15.21
2012-02-03,15.00,15.36,14.93,15.36,488400,15.36
2012-02-02,14.78,14.90,14.74,14.83,67900,14.83
2012-02-01,14.75,15.00,14.69,14.73,219200,14.73
2012-01-31,14.65,14.94,14.50,14.88,86400,14.88
2012-01-30,14.39,14.65,14.37,14.58,51100,14.58
2012-01-27,14.52,14.59,14.50,14.53,55700,14.53
2012-01-26,14.55,14.60,14.47,14.58,69800,14.58
2012-01-25,14.44,14.61,14.40,14.52,79000,14.52
2012-01-24,14.22,14.59,14.16,14.48,73400,14.48
2012-01-23,14.17,15.35,14.01,14.28,154900,14.28
2012-01-20,14.88,14.88,14.31,14.59,261400,14.59
2012-01-19,15.01,15.01,14.75,14.93,97600,14.93
2012-01-18,15.01,15.09,14.78,15.00,141800,15.00
2012-01-17,15.00,15.28,14.79,15.00,490100,15.00
2012-01-13,14.89,15.02,14.75,14.99,325500,14.99
2012-01-12,14.16,15.05,14.09,15.00,710200,15.00
2012-01-11,13.54,14.36,13.52,14.12,1173600,14.12
2012-01-10,13.52,13.70,13.40,13.57,514100,13.57
2012-01-09,13.33,13.52,13.30,13.35,327700,13.35
2012-01-06,13.23,13.41,13.14,13.27,83200,13.27
2012-01-05,13.01,13.42,13.00,13.25,166300,13.25
2012-01-04,13.34,13.35,13.09,13.19,159300,13.19
2012-01-03,13.74,13.80,13.34,13.43,275400,13.43
2011-12-30,13.33,13.61,13.31,13.60,206000,13.60
2011-12-29,13.23,13.45,13.21,13.34,128200,13.34
2011-12-28,13.00,13.13,12.95,13.01,83400,13.01
2011-12-27,13.25,13.29,13.00,13.04,106300,13.04
2011-12-23,13.40,13.40,13.17,13.27,88300,13.27
2011-12-22,13.21,13.50,13.09,13.44,88500,13.44
2011-12-21,13.20,13.25,12.95,13.16,102000,13.16
2011-12-20,13.27,13.41,13.12,13.38,138000,13.38
2011-12-19,13.15,13.34,12.95,13.02,405900,13.02
2011-12-16,12.87,13.10,12.87,12.99,732800,12.99
2011-12-15,12.95,13.08,12.71,12.87,748600,12.87
2011-12-14,12.92,13.03,12.67,12.86,364200,12.86
2011-12-13,12.95,13.24,12.88,12.96,226600,12.96
2011-12-12,12.49,12.98,12.49,12.88,177600,12.88
2011-12-09,13.00,13.09,12.80,13.01,281500,13.01
2011-12-08,12.89,13.14,12.76,12.94,311400,12.94
2011-12-07,13.22,13.22,12.71,12.94,299900,12.94
2011-12-06,13.33,13.40,12.93,13.29,488600,13.29
2011-12-05,12.95,13.55,12.66,13.32,884100,13.32
2011-12-02,12.48,12.56,12.37,12.50,331600,12.50
2011-12-01,12.70,12.78,12.30,12.43,261100,12.43
2011-11-30,12.85,12.94,12.56,12.68,982500,12.68
2011-11-29,12.91,13.00,12.56,12.75,578700,12.75
2011-11-28,12.99,13.00,12.50,12.95,862000,12.95
2011-11-25,12.96,13.23,12.77,13.04,57200,13.04
2011-11-23,13.84,13.84,12.96,13.01,384500,13.01
2011-11-22,13.96,14.14,13.49,14.00,181500,14.00
2011-11-21,14.87,14.87,13.76,13.98,258200,13.98
2011-11-18,15.16,15.44,14.96,15.14,134400,15.14
2011-11-17,15.49,15.84,15.01,15.16,75200,15.16
2011-11-16,14.61,16.20,14.60,15.48,259700,15.48
2011-11-15,14.34,14.76,13.77,14.69,61500,14.69
2011-11-14,15.12,15.23,14.25,14.40,188400,14.40
2011-11-11,13.53,15.47,13.53,15.13,186700,15.13
2011-11-10,13.12,13.34,12.68,12.87,34900,12.87
2011-11-09,13.00,13.20,12.75,12.91,75000,12.91
2011-11-08,13.11,13.32,12.90,13.29,84000,13.29
2011-11-07,13.27,13.38,12.86,13.04,222000,13.04
2011-11-04,12.96,13.29,12.89,13.24,83200,13.24
2011-11-03,13.19,13.19,12.96,13.11,111500,13.11
2011-11-02,12.95,13.14,12.83,13.02,151900,13.02
2011-11-01,12.96,13.15,12.60,12.76,258600,12.76
2011-10-31,14.18,14.22,13.36,13.44,158300,13.44
2011-10-28,14.27,14.65,13.90,14.29,77200,14.29
2011-10-27,14.56,14.67,14.06,14.37,152600,14.37
2011-10-26,14.11,14.33,13.54,13.93,93200,13.93
2011-10-25,14.17,14.17,13.77,13.94,109800,13.94
2011-10-24,14.44,14.45,14.12,14.32,76400,14.32
2011-10-21,14.50,14.50,13.95,14.42,176300,14.42
2011-10-20,14.43,14.48,13.85,14.27,126000,14.27
2011-10-19,15.39,15.39,14.26,14.43,87800,14.43
2011-10-18,14.36,15.87,14.00,15.51,105700,15.51
2011-10-17,14.63,14.63,14.24,14.27,80700,14.27
2011-10-14,14.58,14.94,14.42,14.75,71900,14.75
2011-10-13,14.18,15.00,14.10,14.42,87100,14.42
2011-10-12,14.55,14.55,14.12,14.27,169600,14.27
2011-10-11,14.38,14.84,14.25,14.36,172500,14.36
2011-10-10,14.65,14.65,14.22,14.49,75000,14.49
2011-10-07,15.68,15.74,14.27,14.40,62600,14.40
2011-10-06,15.61,15.85,15.27,15.62,47000,15.62
2011-10-05,15.82,16.38,15.49,15.68,83100,15.68
2011-10-04,14.04,15.94,14.04,15.92,138000,15.92
2011-10-03,14.71,14.71,14.03,14.22,123900,14.22
2011-09-30,14.89,15.48,14.41,14.75,82500,14.75
2011-09-29,15.23,15.70,14.59,15.13,66800,15.13
2011-09-28,15.08,15.63,14.41,14.91,74100,14.91
2011-09-27,15.47,16.00,14.74,14.95,81500,14.95
2011-09-26,14.60,15.33,14.16,15.18,61400,15.18
2011-09-23,14.62,14.63,14.17,14.44,62300,14.44
2011-09-22,13.92,15.12,13.92,14.62,78000,14.62
2011-09-21,15.52,15.94,14.18,14.25,81400,14.25
2011-09-20,15.73,16.27,15.42,15.52,124100,15.52
2011-09-19,15.72,15.91,15.40,15.76,71000,15.76
2011-09-16,15.63,16.06,15.54,16.03,142900,16.03
2011-09-15,15.76,15.76,14.99,15.55,64200,15.55
2011-09-14,14.90,15.75,14.67,15.62,63600,15.62
2011-09-13,14.44,15.16,14.39,14.78,46800,14.78
2011-09-12,14.25,14.95,14.07,14.39,34500,14.39
2011-09-09,15.19,15.19,14.23,14.40,66500,14.40
2011-09-08,15.50,15.75,15.08,15.32,53800,15.32
2011-09-07,15.37,15.75,15.14,15.55,106800,15.55
2011-09-06,14.59,15.13,14.47,15.03,43300,15.03
2011-09-02,15.13,15.45,14.78,14.93,61300,14.93
2011-09-01,16.20,16.43,15.36,15.45,40400,15.45
2011-08-31,16.65,16.67,15.92,16.18,68100,16.18
2011-08-30,16.51,17.00,16.46,16.53,49300,16.53
2011-08-29,16.38,16.67,16.23,16.65,65200,16.65
2011-08-26,15.99,16.34,15.40,16.16,33000,16.16
2011-08-25,16.97,16.97,15.95,16.04,79400,16.04
2011-08-24,16.85,16.92,16.57,16.89,47300,16.89
2011-08-23,16.61,17.03,16.35,16.95,107300,16.95
2011-08-22,16.60,16.60,16.21,16.48,118200,16.48
2011-08-19,15.90,16.35,15.72,16.23,166000,16.23
2011-08-18,16.35,16.59,16.01,16.16,253100,16.16
2011-08-17,16.38,16.89,16.35,16.81,143100,16.81
2011-08-16,16.00,16.55,15.55,16.25,203100,16.25
2011-08-15,15.72,16.31,15.72,16.14,191100,16.14
2011-08-12,14.89,16.16,14.89,15.66,458100,15.66
2011-08-11,14.18,14.81,13.11,14.72,873300,14.72
2011-08-10,15.19,15.19,13.02,13.31,280100,13.31
2011-08-09,14.99,15.50,13.79,15.50,823800,15.50
2011-08-08,16.77,16.81,14.04,14.70,399800,14.70
2011-08-05,17.43,17.57,16.93,17.12,230100,17.12
2011-08-04,17.13,17.70,16.74,17.28,244000,17.28
2011-08-03,17.78,17.82,16.58,17.33,563300,17.33
2011-08-02,17.98,18.33,17.50,17.51,641400,17.51
2011-08-01,18.32,18.34,17.98,18.06,252200,18.06
2011-07-29,18.00,18.61,17.95,18.15,74900,18.15
2011-07-28,17.98,18.19,17.98,18.01,95300,18.01
2011-07-27,18.23,18.42,17.99,18.01,225800,18.01
2011-07-26,17.95,18.46,17.91,18.27,125700,18.27
2011-07-25,17.92,18.03,17.90,17.99,61300,17.99
2011-07-22,18.18,18.18,17.80,18.11,48800,18.11
2011-07-21,18.17,18.29,17.91,18.14,41700,18.14
2011-07-20,18.56,18.56,17.91,18.13,35000,18.13
2011-07-19,18.45,18.59,18.27,18.58,99900,18.58
2011-07-18,18.38,18.53,18.19,18.39,93800,18.39
2011-07-15,18.54,18.95,18.42,18.49,127900,18.49
2011-07-14,18.75,18.75,18.40,18.42,74400,18.42
2011-07-13,18.73,18.99,18.48,18.69,49500,18.69
2011-07-12,18.63,18.94,18.45,18.58,87300,18.58
2011-07-11,18.78,19.00,18.45,18.74,75900,18.74
2011-07-08,18.81,19.26,18.75,18.89,124000,18.89
2011-07-07,18.96,19.99,18.80,18.90,226000,18.90
2011-07-06,18.48,19.05,18.41,18.84,398000,18.84
2011-07-05,18.34,19.25,18.15,18.42,475200,18.42
2011-07-01,17.75,18.28,17.38,18.01,99200,18.01
2011-06-30,17.10,18.28,16.71,17.60,169800,17.60
2011-06-29,17.08,17.55,17.00,17.11,164400,17.11
2011-06-28,17.17,17.35,16.65,17.05,126300,17.05
2011-06-27,15.98,17.25,15.90,17.15,139100,17.15
2011-06-24,16.85,16.91,15.83,16.00,1813400,16.00
2011-06-23,17.86,17.99,16.63,16.90,381100,16.90
2011-06-22,18.35,18.96,17.93,18.01,308600,18.01
2011-06-21,18.22,18.43,18.00,18.40,100800,18.40
2011-06-20,18.09,18.23,18.00,18.18,67200,18.18
2011-06-17,18.73,18.90,17.88,18.21,234800,18.21
2011-06-16,18.90,19.19,18.32,18.72,57900,18.72
2011-06-15,18.93,19.11,18.53,18.99,98800,18.99
2011-06-14,19.00,19.35,18.74,19.00,127500,19.00
2011-06-13,18.88,19.47,18.88,19.08,190000,19.08
2011-06-10,19.08,19.70,17.82,18.64,266800,18.64
2011-06-09,19.20,19.50,19.00,19.29,252100,19.29
2011-06-08,19.27,19.49,19.00,19.41,199200,19.41
2011-06-07,19.00,19.89,18.68,19.40,210900,19.40
2011-06-06,18.00,19.44,18.00,18.96,253100,18.96
2011-06-03,18.05,18.54,18.02,18.25,171300,18.25
2011-06-02,18.09,18.71,17.41,18.25,176700,18.25
2011-06-01,18.00,18.50,17.80,18.28,259800,18.28
2011-05-31,18.26,18.83,17.80,18.35,225500,18.35
2011-05-27,16.05,18.80,15.90,18.80,410400,18.80
2011-05-26,15.81,16.89,15.30,16.28,611100,16.28
2011-05-25,15.00,17.11,14.75,15.90,7597600,15.90

Re: [GT] Post install issues

[Robert A. S. <ra...@ac...>]

Evgeny Shragovich wrote:
> Hi Ras,
> 
> Thank you for your help!

you're welcome! i appreciate your input in an effort to make
gt configuration and setup easier and more user friendly.

to that end i'm gonna add a hack or two to Build.PL to do some
better checking on a users prices data setup, especially if
it is DB::Text. and i would appreciate it if you could run it
and send me (or the list, your choice) the output ... thanks
but it's gonna take a bit to work it out so don't put anything
off waiting on it.

> 
> I have fixed the separator issue in my options file (I have accidentally
> including unneeded word in that line) and now it looking better:
> [Evgeny@localhost Scripts]$ ./display_indicator.pl I:RSI ADX
> Calculating indicator RSI[14, {I:Prices CLOSE}] ...
> RSI[14, {I:Prices CLOSE}][2012-06-14] = 47.9167
> RSI[14, {I:Prices CLOSE}][2012-06-15] = 53.7736
> RSI[14, {I:Prices CLOSE}][2012-06-18] = 51.4851
> RSI[14, {I:Prices CLOSE}][2012-06-19] = 60.0000
> RSI[14, {I:Prices CLOSE}][2012-06-20] = 57.1429
> RSI[14, {I:Prices CLOSE}][2012-06-21] = 60.0000
> 
> 
> 
> However I do get some odd "UTF-8" lines here:
> [Evgeny@localhost Scripts]$ ./backtest.pl 'TFS[30,7,7]' ADX
> error : string is not in UTF-8
> error : string is not in UTF-8

i don't immediately see where that message is being source from
so i have to assume it's from some perl module gt is using.

i also see that for some reason i did not have Build.PL print the
version (or other config data) of a users perl installation.
i'm gonna guess you are likely running a fairly advanced version
well beyond 5.10 even. so i'd guess that the 'better' UTF-8 support
is causing this, but i don't think it is a problem since you do
get results ...

have you tried backtest.pl with graphics and html output?
i think you will find that form of output more to your liking, but
you will have to 'view' it with an html browser.
  %     ./backtest.pl --start 2010-01-04 --end 2013-04-03 \
        --initial_value=100000 --output-directory=/tmp \
        --html --display-trades --graph /tmp/evgeny_ADX_bt.png \
        'TFS[30,7,7]' ADX > /tmp/evgeny_ADX_bt.html

humm -- it looks like --display-trades is not working in the svn version

> ## Analysis of SY:TFS 30 7|CS:SY:TFS 30|CS:Stop:Fixed 7|MM:Basic
> History of the portfolio :
> --------------------------
> Short position (0) on ADX
>   2012-12-24  Sell     952 at 10.5200
>   2012-12-28  Buy      952 at 10.4800
> Short position (1) on ADX
>   2013-02-26  Sell     893 at 11.1600
>   2013-03-01  Buy      893 at 11.3100
> Long position (2) on ADX
>   2013-03-01  Buy      858 at 11.3100
>   2013-04-03  Sell     858 at 11.6400
> ## Global analysis (full portfolio always invested)
> Analysis of the portfolio (2011-03-08 / 2013-04-03)
> -----------------------------------------------------
> Performance :  -0.5% ( -0.3%)      Buy & Hold :  15.8% (  7.6%) () => by
> year
> MaxDrawDown :        2.6%  B&H MaxDrawDown :    56.9%
> Best performance :   0.0%  Worst performance :  -2.6%
> Net gain :         -51.20  Gross gain :        187.27
> 
> Trades statistics
> Number of trades :      3  Trades/Year :         1.45
> Number of gains :       1  Number of losses :       2  Win. ratio :   33.3%
> Max consec. win :       1  Max consec. loss :       2  Expectancy :   -0.00
> Average gain :      2.11%  Average loss :      -1.29%  Avg. perf  :  -0.17%
> Biggest gain :      2.11%  Biggest loss :      -2.15%  Profit fac :    1.64
> Sum of gains :     205.14  Sum of losses :    -256.34  Risk of ruin : 100.0
> 
> 
> 
> 
> I have noticed the funny candles as well. I was using the "Adj. Close"
> value and as I've switched it to regular "Close", it became normal looking.
> However I'm still not sure how do I incorporate dividends and splits in
> backtesting if using regular close...

gt has no provision to utilize dividends or other corporate payments to
shareholders in any manner. partly (mostly) because there is no database
to maintain payment data and history. gt has very provisions to create
it's own databases, but it has never been leveraged into any generally
useful capability. for details refer to GT/MetaInfo.pm and GT/Serializable.pm.

beancounter (the stock standard) has limited provisions for current
dividend amounts and rate, but falls short on maintaining historical
data.

splits are in the same general category, meaning there is no direct
gt support, but the split problem is substantially more complex at
least i think it is. beancounter can handle splits, but in a manner
that i think is technically incorrect, in that it readjusts all past
history to reflect the current split data. while this keeps technical
analysis data consistent it completely screws up tax basis accounting.
and because there isn't a split history record one cannot back-out
the changes ...


either one of these two problems might make for an interesting
project for a summer internship ...


aloha

ras

> 
> 
> 
> 
> Thanks again!
> Eugene
> 
> 
> 

< SNIP >

Re: [GT] Post install issues

[Evgeny S. <shr...@gm...>]

Hi Ras,

Thank you for your help!

I have fixed the separator issue in my options file (I have accidentally
including unneeded word in that line) and now it looking better:
[Evgeny@localhost Scripts]$ ./display_indicator.pl I:RSI ADX
Calculating indicator RSI[14, {I:Prices CLOSE}] ...
RSI[14, {I:Prices CLOSE}][2012-06-14] = 47.9167
RSI[14, {I:Prices CLOSE}][2012-06-15] = 53.7736
RSI[14, {I:Prices CLOSE}][2012-06-18] = 51.4851
RSI[14, {I:Prices CLOSE}][2012-06-19] = 60.0000
RSI[14, {I:Prices CLOSE}][2012-06-20] = 57.1429
RSI[14, {I:Prices CLOSE}][2012-06-21] = 60.0000



However I do get some odd "UTF-8" lines here:
[Evgeny@localhost Scripts]$ ./backtest.pl 'TFS[30,7,7]' ADX
error : string is not in UTF-8
error : string is not in UTF-8
## Analysis of SY:TFS 30 7|CS:SY:TFS 30|CS:Stop:Fixed 7|MM:Basic
History of the portfolio :
--------------------------
Short position (0) on ADX
  2012-12-24  Sell     952 at 10.5200
  2012-12-28  Buy      952 at 10.4800
Short position (1) on ADX
  2013-02-26  Sell     893 at 11.1600
  2013-03-01  Buy      893 at 11.3100
Long position (2) on ADX
  2013-03-01  Buy      858 at 11.3100
  2013-04-03  Sell     858 at 11.6400
## Global analysis (full portfolio always invested)
Analysis of the portfolio (2011-03-08 / 2013-04-03)
-----------------------------------------------------
Performance :  -0.5% ( -0.3%)      Buy & Hold :  15.8% (  7.6%) () => by
year
MaxDrawDown :        2.6%  B&H MaxDrawDown :    56.9%
Best performance :   0.0%  Worst performance :  -2.6%
Net gain :         -51.20  Gross gain :        187.27

Trades statistics
Number of trades :      3  Trades/Year :         1.45
Number of gains :       1  Number of losses :       2  Win. ratio :   33.3%
Max consec. win :       1  Max consec. loss :       2  Expectancy :   -0.00
Average gain :      2.11%  Average loss :      -1.29%  Avg. perf  :  -0.17%
Biggest gain :      2.11%  Biggest loss :      -2.15%  Profit fac :    1.64
Sum of gains :     205.14  Sum of losses :    -256.34  Risk of ruin : 100.0




I have noticed the funny candles as well. I was using the "Adj. Close"
value and as I've switched it to regular "Close", it became normal looking.
However I'm still not sure how do I incorporate dividends and splits in
backtesting if using regular close...




Thanks again!
Eugene





On Wed, Apr 17, 2013 at 1:11 PM, Robert A. Schmied <ra...@ac...> wrote:

> aloha evgeny
>
> thanks for the abbreviated post.
>
> to repeat what i've already reported to you on these issues:
>
> add following line to your 'options' file so GT::DB::Text can correctly
> read your comma separated ADX.txt (and other) prices data file(s).
>
> DB::text::marker ,
>
> there is something else amiss with the 'options' file and the prices
> DB::text::fields settings because i'm seeing odd looking candles using
>    %  graphic.pl --timeframe=day \
>    --start=2012-11-05 --end 2013-04-03 --type=candle -volume \
>    --volume-height=150 --logarithmic ADX > /tmp/ADX_test.png
>
> this may (or may not) account for the errors i'm getting for your
> items 2 thru 4.
>
> i think i would diagnose the prices DB::text::fields setting issue
> using test cases based on these sort of command lines:
>
>    %   ./display_indicator.pl --start=2012-11-05 --end 2012-12-03
> I:Prices ADX OPEN
>    %   ./display_indicator.pl --start=2012-11-05 --end 2012-12-03
> I:Prices ADX HIGH
>    %   ./display_indicator.pl --start=2012-11-05 --end 2012-12-03
> I:Prices ADX LOW
>    %   ./display_indicator.pl --start=2012-11-05 --end 2012-12-03
> I:Prices ADX CLOSE
>
> i'm thinking these are how the price values are defined:
> > 2013-04-03,11.78,11.78,11.60,11.64,122100,11.64
>   ---date--- open  high  -low- close volume unknown (adjusted close???)
>        0       1     2     3     4      5
>
> and with the change for DB::text::fields::close the graphic.pl output
> looks 'better'.
> using the svn backtest.pl i get 'results' for ./backtest.pl TFS ADX
> and ./backtest.pl 'TFS[30,7,7]' ADX
>
>
> as for scan.pl you need to swap the two input files:
> SYNOPSIS
>     ./scan.pl [ options ] <market file> <date> <system file> [ <system
> file> ... ]
>
> where <market file> contains the list of symbols
>       <system file> contains the system or signal description
>
> i get (using svn scan.pl) and a date within the abbreviated price data:
>
>    %    ./scan.pl /tmp/evgeny_mkt 2013-04-03 /tmp/evgeny_sys.scan
>
>    Signal: S:G:And {S:G:Above {I:Prices } {I:EMA 30 {I:Prices CLOSE}}}
> {S:G:Above {I:Prices } {I:EMA 150 {I:Prices CLOSE}}}
>    ADX
>
> not real helpful, but none of the dates ( 2013-04-03 .. 2013-03-20 )
> produce a signal.
>
> hope this helps
>
> ras
>
>
>
> Evgeny Shragovich wrote:
> > Hi folks,
> >
> > I haven't been using GT for a while and now I'm trying to set it up again
> > from scratch.
> > It seems that I remembered how to install and apply the basic
> > configuration, however I couldn't make work after all..
> >
> >
> > I have attached the output of "Build.pl" script, "options" configuration
> > file and the ADX stock file.
> >
> >
> > I've tried the tests as described in the FAQ
> >
> > 1. ./display_indicator.pl I:RSI ADX
> > .
> > .
> > Use of uninitialized value in subtraction (-) at ../GT/Indicators/RSI.pm
> > line 67.
> > Use of uninitialized value in subtraction (-) at ../GT/Indicators/RSI.pm
> > line 67.
> > Use of uninitialized value in subtraction (-) at ../GT/Indicators/RSI.pm
> > line 67.
> > Use of uninitialized value in subtraction (-) at ../GT/Indicators/RSI.pm
> > line 67.
> > Use of uninitialized value in subtraction (-) at ../GT/Indicators/RSI.pm
> > line 67.
> > Use of uninitialized value in subtraction (-) at ../GT/Indicators/RSI.pm
> > line 67.
> > Use of uninitialized value in subtraction (-) at ../GT/Indicators/RSI.pm
> > line 67.
> > RSI[14, {I:Prices CLOSE}][2012-06-14,10.23,10.34,10.22,10.30,67300,9.71]
> =
> > 0.0000
> > RSI[14, {I:Prices
> CLOSE}][2012-06-15,10.33,10.42,10.32,10.42,125600,9.82] =
> > 0.0000
> > RSI[14, {I:Prices
> CLOSE}][2012-06-18,10.40,10.47,10.35,10.44,126700,9.84] =
> > 0.0000
> > RSI[14, {I:Prices
> CLOSE}][2012-06-19,10.47,10.57,10.47,10.55,155700,9.95] =
> > 0.0000
> > RSI[14, {I:Prices
> CLOSE}][2012-06-20,10.58,10.58,10.46,10.52,109800,9.92] =
> > 0.0000
> > RSI[14, {I:Prices
> CLOSE}][2012-06-21,10.51,10.55,10.32,10.33,130800,9.74] =
> > 0.0000
> > RSI[14, {I:Prices CLOSE}][2012-06-22,10.35,10.40,10.34,10.39,92300,9.80]
> =
> > 0.0000
> > .
> > .
> >
> > 2. ./backtest.pl TFS ADX
> > Illegal division by zero at ../GT/Indicators/MaxDrawDown.pm line 52.
> >
> > 3. ./backtest.pl 'TFS[30,7,7]' ADX
> > Illegal division by zero at ../GT/Indicators/MaxDrawDown.pm line 52.
> >
> >
> > 4. I've also tried to run the following scan(as explained in FAQ):
> > scan.pl signal_file YYYY-MM-DD stocklist_file
> >
> > *I have created the following files*:
> > *signal_file*, which consists of:
> > S:G:And {S:G:Above {I:Prices} {I:EMA 30}} {S:G:Above {I:Prices} {I:EMA
> 150}}
> >
> > and *stocklist_file*, which consists of only one symbol which is located
> > according to the configuration in the options file
> > (/home/username/Market/data/exchanges/NYSE)
> >
> > When I try to run it, I get the following error:
> > [username@localhost Scripts]$ ./scan.pl signal_file 2010-01-01
> > stocklist_file
> > Can't locate GT/ADX.pm in @INC (@INC contains: .. /usr/local/lib/perl5
> > /usr/local/share/perl5 /usr/lib/perl5/vendor_perl
> > /usr/share/perl5/vendor_perl /usr/lib/perl5 /usr/share/perl5 .) at (eval
> > 14) line 1, <> line 1.
> > BEGIN failed--compilation aborted at (eval 14) line 1, <> line 1.
> >
> >
> >
> > I assumed that I've misconfigured something related to the data format in
> > the ADX.txt file but after checking few times I didn't find any problem.
> >
> >
> >
> > Thanks!
> > Eugene
> >
> >
> >
> > ------------------------------------------------------------------------
> >
> > 2013-04-03,11.78,11.78,11.60,11.64,122100,11.64
> > 2013-04-02,11.77,11.78,11.72,11.74,83500,11.74
> > 2013-04-01,11.75,11.77,11.66,11.75,200800,11.75
> > 2013-03-28,11.76,11.78,11.70,11.74,221300,11.74
> > 2013-03-27,11.70,11.75,11.68,11.73,305200,11.73
> > 2013-03-26,11.71,11.74,11.68,11.72,125100,11.72
> > 2013-03-25,11.71,11.74,11.63,11.64,84800,11.64
> > 2013-03-22,11.72,11.76,11.65,11.68,176000,11.68
> > 2013-03-21,11.67,11.73,11.66,11.68,77800,11.68
> > 2013-03-20,11.70,11.75,11.66,11.71,91300,11.71
> > 2013-03-19,11.71,11.74,11.59,11.69,133100,11.69
> > 2013-03-18,11.63,11.70,11.63,11.68,126300,11.68
> > 2013-03-15,11.70,11.75,11.69,11.69,105300,11.69
>
> < snip>
>
>
> ------------------------------------------------------------------------------
> Precog is a next-generation analytics platform capable of advanced
> analytics on semi-structured data. The platform includes APIs for building
> apps and a phenomenal toolset for data science. Developers can use
> our toolset for easy data analysis & visualization. Get a free account!
> http://www2.precog.com/precogplatform/slashdotnewsletter
>



-- 
Best regards,

Shragovich       Evgeny
shr...@gm...

Re: [GT] Post install issues -- Build.PL commentary

[Robert A. S. <ra...@ac...>]

Evgeny Shragovich wrote:
> Hi folks,
> 
> I haven't been using GT for a while and now I'm trying to set it up again
> from scratch.
> It seems that I remembered how to install and apply the basic
> configuration, however I couldn't make work after all..
> 
> 
> I have attached the output of "Build.pl" script, "options" configuration
> file and the ADX stock file.
> 
> 

< snip >

the following is Evgenys Build.PL terminal output
> Build.PL release: 1.25
> GT release is: not available
> 
> Module::Build release is: 0.4004

very advanced -- i'm only at 0.2805.

> 
> 
> Aloha Evgeny.
> 
> The intent of this utility is to assess your platforms perl installation and
> add-on modules with respect to the needs of GT and the primary perl modules
> GT depends on. It is in no way intended to be a complete dependency
> installer for GT and its' dependencies dependencies.  Its output should be
> used as a guide, along with information on the various webpages
> (http://geniustrade.org) for you to determine what prerequisites might still
> be missing and to manually install them yourself.
> 
> There are very good, but never-the-less complex dependency discussions in
> the Install & Setup page and in the Users Wiki pages. All shoould be at
> least looked at, if not studied carefully, with respect to your platform,
> your perl, how you plan to provide/manage market price data for GT and how
> you plan to display GT results. All play a role in the dependencies you will
> need for a successful GT setup.
> 
> While it was never the intent that this utility be able to automagically
> install missing dependencies, if it does that so much the better. Instead,
> it was created to aid you in identifying what you have already and what
> might be still missing. Armed with this information you can then use
> whatever your standard installation method is to install them.
> 
> The last parts of this script will take a look at your GT configuration file
> ($HOME/.gt/options) and check to see if it contains the minimal required
> entries and that they are, to the extent possible, correct and valid.
> 
> The final part will, under your consent, tweak each GT script app sharpbang
> line to be consistent with your perls' notion of it (usually not
> necessary).  Lastly it will create symlinks to these scripts if your
> platform groks symlinks.
> 
> 

this stuff is provided for informational purposes --
there is nothing here that you need unless you try to use
beancounter or some other prices database engine.

> Checking prerequisites...
>   recommends:
>     *  Image::Magick is not installed
>     *  Log::Log4perl is not installed
>     *  PostScript::Simple is not installed
>     *  SVG is not installed
> 
> Checking optional features...
> beancounter............disabled
>   requires:
>     ! Finance::BeanCounter is not installed
> beancounter_support....disabled
>   requires:
>     ! Finance::BeanCounter is not installed
>     ! Finance::YahooQuote is not installed
>     ! Statistics::Descriptive is not installed
> mysql_support..........disabled
>   requires:
>     ! DBD::mysql is not installed
> odbc_support...........disabled
>   requires:
>     ! DBD::ODBC is not installed
> pg_support.............disabled
>   requires:
>     ! DBD::Pg is not installed
> sqlite2_support........disabled
>   requires:
>     ! DBD::SQLite2 is not installed
> sqlite_support.........disabled
>   requires:
>     ! DBD::SQLite is not installed
> 

i'm unable to control the following stuff -- but not to worry,
there is nothing here that is a problem

> ERRORS/WARNINGS FOUND IN PREREQUISITES.  You may wish to install the versions
> of the modules indicated above before proceeding with this installation
> 
> Run 'Build installdeps' to install missing prerequisites.
> 
> No 'module_name' was provided and it could not be inferred
> from other properties.  This will prevent a packlist from
> being written for this file.  Please set either 'module_name'
> or 'dist_version_from' in Build.PL.

if i really understood what Module::Build was complaining about
i might be able to fix the above 'module_name' and or 'dist_version_from'
bitching ...

> 
> yea! you have the perl module DBI, which is absolutely needed for GT if you
> intend to use any sql based price database. let's see which DBI/DBD drivers
> you have available ...
> 
> 
> 
> CSV
>   humm: you have perl modules needed to use the GT CSV driver.
>   THIS IS TOTALLY EXPERIMENTAL that means that you are on your own with this
>   -- refer to pod in GT/DB/CSV.pm.
> 
> 
> 
> checking on Evgeny's GT configuration file
>  "/home/Evgeny/.gt/options"
> 
> 
> 
> Congrats!!! you have correctly configured these font faces:
> Path::Font::Times
> Path::Font::Courier
> Path::Font::Arial
> 
> 
> you are configured for the Text prices database module.
> with these database module key settings:
>     DB::text::directory /home/Evgeny/Market/data/exchanges/NYSE
>     DB::text::format 0
>     DB::text::fields::datetime 0
>     DB::text::fields::open 1
>     DB::text::fields::low 3
>     DB::text::fields::high 2
>     DB::text::fields::close 6
>     DB::text::fields::volume 5
> 

here is where more could should be done. especially since you
had a mis-configured options file based on the separator character.

but in order to do more the user would have to provide the file
name of at least one prices data file ...

> 
> your GT configuration file looks ok
> 
> 
> 
> 
> excellent! your platform supports symlinks, so if you want i can create
> symlinks for each of the gt application scripts without that damned .pl
> extension ... it saves a bit of typing the command lines.
> 
> do you want the symlinks? y/[n] 
> 

ras

Re: [GT] Post install issues

[Robert A. S. <ra...@ac...>]

aloha evgeny

thanks for the abbreviated post.

to repeat what i've already reported to you on these issues:

add following line to your 'options' file so GT::DB::Text can correctly
read your comma separated ADX.txt (and other) prices data file(s).

DB::text::marker ,

there is something else amiss with the 'options' file and the prices
DB::text::fields settings because i'm seeing odd looking candles using
   %  graphic.pl --timeframe=day \
   --start=2012-11-05 --end 2013-04-03 --type=candle -volume \
   --volume-height=150 --logarithmic ADX > /tmp/ADX_test.png 

this may (or may not) account for the errors i'm getting for your
items 2 thru 4.

i think i would diagnose the prices DB::text::fields setting issue
using test cases based on these sort of command lines:

   %   ./display_indicator.pl --start=2012-11-05 --end 2012-12-03 I:Prices ADX OPEN
   %   ./display_indicator.pl --start=2012-11-05 --end 2012-12-03 I:Prices ADX HIGH
   %   ./display_indicator.pl --start=2012-11-05 --end 2012-12-03 I:Prices ADX LOW
   %   ./display_indicator.pl --start=2012-11-05 --end 2012-12-03 I:Prices ADX CLOSE

i'm thinking these are how the price values are defined:
> 2013-04-03,11.78,11.78,11.60,11.64,122100,11.64
  ---date--- open  high  -low- close volume unknown (adjusted close???)
       0       1     2     3     4      5

and with the change for DB::text::fields::close the graphic.pl output looks 'better'.
using the svn backtest.pl i get 'results' for ./backtest.pl TFS ADX
and ./backtest.pl 'TFS[30,7,7]' ADX


as for scan.pl you need to swap the two input files: 
SYNOPSIS
    ./scan.pl [ options ] <market file> <date> <system file> [ <system file> ... ]

where <market file> contains the list of symbols
      <system file> contains the system or signal description

i get (using svn scan.pl) and a date within the abbreviated price data:

   %    ./scan.pl /tmp/evgeny_mkt 2013-04-03 /tmp/evgeny_sys.scan

   Signal: S:G:And {S:G:Above {I:Prices } {I:EMA 30 {I:Prices CLOSE}}} {S:G:Above {I:Prices } {I:EMA 150 {I:Prices CLOSE}}}
   ADX

not real helpful, but none of the dates ( 2013-04-03 .. 2013-03-20 )
produce a signal.

hope this helps

ras



Evgeny Shragovich wrote:
> Hi folks,
> 
> I haven't been using GT for a while and now I'm trying to set it up again
> from scratch.
> It seems that I remembered how to install and apply the basic
> configuration, however I couldn't make work after all..
> 
> 
> I have attached the output of "Build.pl" script, "options" configuration
> file and the ADX stock file.
> 
> 
> I've tried the tests as described in the FAQ
> 
> 1. ./display_indicator.pl I:RSI ADX
> .
> .
> Use of uninitialized value in subtraction (-) at ../GT/Indicators/RSI.pm
> line 67.
> Use of uninitialized value in subtraction (-) at ../GT/Indicators/RSI.pm
> line 67.
> Use of uninitialized value in subtraction (-) at ../GT/Indicators/RSI.pm
> line 67.
> Use of uninitialized value in subtraction (-) at ../GT/Indicators/RSI.pm
> line 67.
> Use of uninitialized value in subtraction (-) at ../GT/Indicators/RSI.pm
> line 67.
> Use of uninitialized value in subtraction (-) at ../GT/Indicators/RSI.pm
> line 67.
> Use of uninitialized value in subtraction (-) at ../GT/Indicators/RSI.pm
> line 67.
> RSI[14, {I:Prices CLOSE}][2012-06-14,10.23,10.34,10.22,10.30,67300,9.71] =
> 0.0000
> RSI[14, {I:Prices CLOSE}][2012-06-15,10.33,10.42,10.32,10.42,125600,9.82] =
> 0.0000
> RSI[14, {I:Prices CLOSE}][2012-06-18,10.40,10.47,10.35,10.44,126700,9.84] =
> 0.0000
> RSI[14, {I:Prices CLOSE}][2012-06-19,10.47,10.57,10.47,10.55,155700,9.95] =
> 0.0000
> RSI[14, {I:Prices CLOSE}][2012-06-20,10.58,10.58,10.46,10.52,109800,9.92] =
> 0.0000
> RSI[14, {I:Prices CLOSE}][2012-06-21,10.51,10.55,10.32,10.33,130800,9.74] =
> 0.0000
> RSI[14, {I:Prices CLOSE}][2012-06-22,10.35,10.40,10.34,10.39,92300,9.80] =
> 0.0000
> .
> .
> 
> 2. ./backtest.pl TFS ADX
> Illegal division by zero at ../GT/Indicators/MaxDrawDown.pm line 52.
> 
> 3. ./backtest.pl 'TFS[30,7,7]' ADX
> Illegal division by zero at ../GT/Indicators/MaxDrawDown.pm line 52.
> 
> 
> 4. I've also tried to run the following scan(as explained in FAQ):
> scan.pl signal_file YYYY-MM-DD stocklist_file
> 
> *I have created the following files*:
> *signal_file*, which consists of:
> S:G:And {S:G:Above {I:Prices} {I:EMA 30}} {S:G:Above {I:Prices} {I:EMA 150}}
> 
> and *stocklist_file*, which consists of only one symbol which is located
> according to the configuration in the options file
> (/home/username/Market/data/exchanges/NYSE)
> 
> When I try to run it, I get the following error:
> [username@localhost Scripts]$ ./scan.pl signal_file 2010-01-01
> stocklist_file
> Can't locate GT/ADX.pm in @INC (@INC contains: .. /usr/local/lib/perl5
> /usr/local/share/perl5 /usr/lib/perl5/vendor_perl
> /usr/share/perl5/vendor_perl /usr/lib/perl5 /usr/share/perl5 .) at (eval
> 14) line 1, <> line 1.
> BEGIN failed--compilation aborted at (eval 14) line 1, <> line 1.
> 
> 
> 
> I assumed that I've misconfigured something related to the data format in
> the ADX.txt file but after checking few times I didn't find any problem.
> 
> 
> 
> Thanks!
> Eugene
> 
> 
> 
> ------------------------------------------------------------------------
> 
> 2013-04-03,11.78,11.78,11.60,11.64,122100,11.64
> 2013-04-02,11.77,11.78,11.72,11.74,83500,11.74
> 2013-04-01,11.75,11.77,11.66,11.75,200800,11.75
> 2013-03-28,11.76,11.78,11.70,11.74,221300,11.74
> 2013-03-27,11.70,11.75,11.68,11.73,305200,11.73
> 2013-03-26,11.71,11.74,11.68,11.72,125100,11.72
> 2013-03-25,11.71,11.74,11.63,11.64,84800,11.64
> 2013-03-22,11.72,11.76,11.65,11.68,176000,11.68
> 2013-03-21,11.67,11.73,11.66,11.68,77800,11.68
> 2013-03-20,11.70,11.75,11.66,11.71,91300,11.71
> 2013-03-19,11.71,11.74,11.59,11.69,133100,11.69
> 2013-03-18,11.63,11.70,11.63,11.68,126300,11.68
> 2013-03-15,11.70,11.75,11.69,11.69,105300,11.69

< snip>