#60 Faster Satellite File Operations

Undecided
open
nobody
5
2013-02-04
2011-09-23
No

Operations that touch the entire satellite file database can be slow on some machines especially with large databases.

As an example, my EEE pc with 1.6Ghz Atom and 2 Gigs of ram, a tle update can take several minutes and opening the new module or module configuration dialog can take an uncomfortably long time. (Yes, this machine is a little slow these days.)

This should be addressed before importing the satellite catalog from celestrak is implemented as the catalog can add launch date and decay date information on everything NORAD has published publicly. This would instantly bloat most users databases to the same size of extra large.

The correct solution is unclear. A single key file with all satellite data might make sense. Using another database may as well.

The ostrich algorithm and moore's law may also apply.

Discussion

  • Charles Suprin

    Charles Suprin - 2012-01-02

    I am attaching a patch for changes to the satellite selector. This is intended to speed up operations creating new modules and editing existing ones. It gets rid of the multiple models uses to date and creates instead a series of hashs that store the membership of each category. These hashes are then used in the filtering of the selector model. The hashes are intended for quick lookup of membership of a particular satellite in the category.

    The primary advantage of this approach is that it eliminates the need to load the satellite data for every group the satellite belongs.

    Initial tests on a machine with 15000+ satellite in 75+ groups with 575000+ entries total in all the groups reduced the time to open the new module window from 3 minutes 10 seconds to 10 seconds.

     
  • Alexandru Csete

    Alexandru Csete - 2012-01-04

    Hi Charles,
    This speedup looks interesting. I'll take a look at it as soon as I can.

     
  • Charles Suprin

    Charles Suprin - 2012-01-05

    No worries. It seems there are many ways to fix this problem so finding "the best" is probably a conversation.

    Just to let you know how the directory became so cluttered, space-track files have a day string in the file name. I keep throwing them into a directory and then do a local update on that directory on a regular basis. Whenever there is a new satellite in one of the files, it creates the category. The next update it loads all the satellites in that file into the category. This makes a large number of categories with a large number of satellites in each category.

    I am wondering if it is worth creating a --clean-cat command line option to purge these categories.

     
  • Alexandru Csete

    Alexandru Csete - 2013-02-04
    • ****: 3 --> 5
     

Log in to post a comment.

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:

JavaScript is required for this form.





No, thanks