Learn how easy it is to sync an existing GitHub or Google Code repo to a SourceForge project! See Demo


Support screening of ETFs

  • yccheok

    Dear mdespain,

    I really appreciate your effort of submitting bugs and feature implementation. Thank you.

    For your bug submission, I will review it ASAP as it is mid-night right now. Possible make it into next coming release if everything is fine after reviewing (1.0.5n)

    For ETF support, in current, we do have a feature where we can add stock/ scan stock based on the stock's industry. There should be ETF category for US stock market. However, current database for US market is very incomplete. A better database is Malaysia. (Although is still not good enough). You may try to change the country to Malaysia, then go to Edit -> Add Stocks… you will get what I mean.

    Hence, instead of providing a country named Country.UnitedStateETF, I prefer to have a long term solution. We will make JStock able to retrieve a more complete stock database, by hosting the stock database information in our own server (Currently, I am using Google App Engine. Currently, the upcoming China stock market is using this mythology.

    http://jstock-static.appspot.com/stocks_information/china/stocks.csv. Currently, it only contain symbol, code and name information. It should contains Board and Industry as well in upcoming release (1.0.5p)

    Regarding on generating CSV file, we will primary write Perl script to go to the best destination to grab the information. Luckily we have a Perl programmer in our team right now.

    I have a very brief to-do-list on this.


    Let me know if you are interested ;)


  • Mark DeSpain
    Mark DeSpain

    Hi!  Thanks for responding.

    I completely agree that  having a UnitedStateETF is not the desirable long term solution. 

    I implemented it yesterday based on what I have figured out about the codebase so far.  After having become familiar with how jStock pulls Yahoo finance and after realizing that the same Yahoo URLs could be used for pulling ETF historical data and charts, I came up with the "UnitedStateETF" approach. In the short term, it certainly works for me, and I was able to use it to find some trading opportunities.  I thought it was good enough to at least offer back to you as an idea, and to show how valuable I think this feature is.

    Maybe more research is needed to convince yourselves, but adding robust support for ETFs might be an opportunity get jStock additional exposure.  When I looked at the freely available tools, there weren't any that had ETF filtering capabilities to the level that jStock is capable of.

    Thinking about an ideal, long term solution, I haven't put a whole lot of thought into how to properly categorize ETFs, beyond simply recognizing that jStock shouldn't represent ETFs as a separate country in the long term.  For myself, I've primarily leveraged technical analysis to find candidate ETFs for further investigation.  Coming up which an ETF categorization scheme may be somewhat of a challenge, though maybe someone else has already tackled that. 
    * Industry: Some ETFs can be associated with an Industry (e.g. FAZ), while others can represent a more general indexes that will have a mixture of stocks from different industries (e.g. QQQQ).  And some of them can track things other than stocks (e.g. SMB). 
    * Board: This information isn't within the CSV from which the patch downloads the list of ETFs.  However, I suspect it won't be too hard to get it from some place else.

    I tried checking out what you said about the Malaysia database, but when I select that country jStock shows "Network Error" for the status message at the bottom of the window.  After that, when I go to Edit -> Add Stocks, a dialog is shown saying "We haven't connected to Stock server".  This is happen both for my local build and for the 1.0.5.m release.  If I can find I way to get past that, I'll check out what you've described.

  • Mark DeSpain
    Mark DeSpain

    Here is an example NPE stack trace for what I believe as a bug that I fixed.  

    Exception in thread "pool-13-thread-5" java.lang.NullPointerException
    at org.yccheok.jstock.analysis.StockRelativeHistoryOperator.calculate(StockRelativeHistoryOperator.java:281)
    at org.yccheok.jstock.analysis.OperatorIndicator.setStockHistoryServer(OperatorIndicator.java:111)
    at org.yccheok.jstock.gui.IndicatorScannerJPanel.update(IndicatorScannerJPanel.java:685)
    at org.yccheok.jstock.gui.IndicatorScannerJPanel.access$8(IndicatorScannerJPanel.java:653)
    at org.yccheok.jstock.gui.IndicatorScannerJPanel$10.update(IndicatorScannerJPanel.java:700)
    at org.yccheok.jstock.gui.IndicatorScannerJPanel$10.update(IndicatorScannerJPanel.java:1)
    at org.yccheok.jstock.engine.Subject.notify(Subject.java:49)
    at org.yccheok.jstock.engine.StockHistoryMonitor$StockHistoryRunnable.run(StockHistoryMonitor.java:203)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
    at java.lang.Thread.run(Thread.java:636)

    This happened for the ETF symbol BSCD.  The number of values for the history was 55, but the period was 200.  This caused TechnicalAnalysis.createEMA() to return null, which in turn lead to an NPE due to attempting to assign null to a double primitive.

    I believe this can also happen for stock as well (e.g. a IPO for some arbitrary stock).  All that seems to be required is to have less historical information than is needed for a technical indicator implemented one of the methods from the TechnicalAnalysis class.  In that scenario, those methods will return null, which will cause an NPE in the StockHistoryOperator and StockRelativeHistoryOperator classes.

  • yccheok

    You need to install released version, either Linux or Windows as they come with database.zip. Execute them, they will extract out the XML files and locate them in to your <user folder>/.jstock/…

    After that, you may execute your development version.

  • Mark DeSpain
    Mark DeSpain

    I see what you're referring to regarding Malaysia…  ETF appears as an Industry option for that country. 

    For my needs, having ETFs appear similarly for the United States would be fine.