dspatial-develop — News, announcements and updates for developers using DSpatial

[Dspatial-develop] DSpatial version 0.3 released

[Patrick v. L. <pat...@te...>]

Version 0.3.0 of the DSpatial kernel was released today on Sourceforge.
While there is not much change on the surface, a number of quite
important changes to the kernel and kernel access have been implemented.
First of all, the locking mechanism has completely changed. Locking is
now initiated at the "user" level (that is, outside of the kernel),
where the kernel generates a token suitable for the requested access.
The application needs to explicitly drop the token when done. This
dramatically reduces the number of locks that needs to be made, because
an application can access multiple data set attributes or perform
multiple operations before dropping the token. On the down side, a badly
written application can lock a data set indefinitely. Secondly, the
kernel is now cross-platform compatible. Compiler directives have been
added to enable compilation on D5, D7 and Kylix (anybody any experience
with D6?).

The most important change is the improved memory management. It is now
possible to access data sources of arbitrary size, and the kernel will
automatically limit the amount of data that is held in memory. This is
controlled with the FMaxDataSetSize member of the DSpatial object. You
can change this size in the constructor of the TDSpatial class in the
DSpGlobals unit. Any data set that is smaller than the indicated value
will be held in memory in its entirety, which provides the greatest
access speeds. Large data sets that would quickly use all virtual memory
are only partially loaded, and this is completely transparent for the
application developer. This design lays the foundation for the gridmath
module and other data access mechanisms, which will be provided in an
upcoming release.

The gridmath module also sees the light of day with this release.
Unfortunately it is still a very meagre offering, with only a few
identity operations available. These are necessary to use the SDTS
format drivers. On the up side, the development effort is now focused on
this module and incremental updates are expected on a regular basis.

The documentation of DSpatial is also starting in earnest now. Attention
is given to a detailed document on programming with the DSpatial kernel
and the extension modules, and on how to develop additional raster
drivers.

Best regards,
The DSpatial Development Team

[Dspatial-develop] DSpatial update

[Patrick V. L. <pva...@us...>]

Version 0.3 is well underway, although it will still be a while before
it can be released. This upcoming release will see a number of quite
important changes to the kernel and kernel access. You can check the
status of these and other changes from our Code status page.

First of all, the locking mechanism has completely changed. Locking is
now initiated at the "user" level (that is, outside of the kernel),
where the kernel generates a token suitable for the requested access.
The application needs to explicitly drop the token when done. This
dramatically reduces the number of locks that needs to be made, because
an application can access multiple data set attributes or perform
multiple operations before dropping the token. On the down side, a badly
written application can lock a data set indefinitely.

Secondly, the first analysis functions on rasters will be made
available. Basic functions (reading and writing of raster data bands,
data type conversion) will be part of the kernel, while basic arithmetic
on raster bands will be provided through the gridmath module, which will
be released together with kernel release 0.3.

Thirdly, the kernel will be cross-platform compatible. Compiler
directives have been added to enable compilation on D5, D7 and Kylix
(anybody any experience with D6?). If you want to get started with
DSpatial on Kylix right away, you can now download the code from the CVS
at SourceForge. Note that the HDF format driver does not yet work with
Kylix.

Lastly, user level data access will be more sophisticated, where the
application can request specific areas of a raster band, or use a filter
that represents a number of rows in a raster band and which also
supports relative pixel addressing.

Version 0.3 is scheduled for release sometime in February.

On the administrative side, we welcome Fred Esberg as a new member of
the developers team. Fred has extensive and broad experience with
Delphi, particularly in database access, statistics, and image
progressing. Fred is responsible for the development of the orbit module
for image processing.

All the best for the New Year,
DSpatial Development Team

[Dspatial-develop] DSpatial release v0.2.1

[Patrick v. L. <pat...@te...>]

Hi all,
 
Yesterday DSpatial version 0.2.1 was made available on SourceForge. This
minor update has a number of interesting new features and enhancements,
and you should upgrade to this version at your earliest convenience.
 
The PFA has seen some minor improvements, mainly to reduce the risk of
thread collisions and to improve the life-time management of data
sources. A new method, IdleProcessing, has been added which is called
when the application goes into idle state, and it is this method that
deletes unused data source objects. This requires you to assign
DSpatial.IdleProcessing to the Application.OnIdle event handler. You can
do this in two ways: (1) use the TApplicationEvents component; and (2)
make the assignment in your dpr file before the line Application.Run. If
you do not do this, the data sources you open will remain in memory
until your application quits. The actual data of the data sets in the
source will not be loaded until required and it is purged when all
references to the data set in question are released, so the memory
requirements of such an unused source are not particularly high, but the
file system may place locks on any files the source driver accesses so
you are better off getting rid of them by enabling the IdleProcessing
mechanism.
 
The rendering pipeline has been optimized by placing the renderer map at
the level of the renderer instead of the raster band. This reduced the
memory requirement of the rendering pipeline and the number of
operations per Paint operation is reduced as well.
 
New drivers have been added for the Idrisi raster format, and for HDF
files. The HDF driver is available as a separate download, because it is
quite big due to the two necessary DLLs and you may not be using HDF
files. There are tons of free satellite images in HDF format from the
NASA EOS web sites, so you might want to check it out anyway. The raw
binary driver has been enhanced to support ESRI FLT binary grid files.
Most drivers now record minimum and maximum values in a raster band, and
this information is used in the renderer to set the bounds on the colour
range (instead of the default [0..1000]).
 
A histogram component has been added which is highly customizable, and
it will take any kind of data, raster band or not. As long as you know
the data type and the number of elements of your data (chunk), you can
display it. Data can be added in chunks, so even ridiculously big data
sets can be analyzed, as long as the number of elements in any
particular bin does not exceed 2^32. This means that you can probably
churn your whole 80Gb hard disk through the component and see how many
bytes you have with the value of, say, 177, if the disk is kinda full at
least. Check out the screenshot on the home page. The histogram
component and the TDSpViewer component are now packaged in
DSpComponents.bpl, so you can install them in the IDE on the DSpatial
page, making application development that much easier.
 
For the next release the development focus is on enabling the PFA and
the current crop of drivers to save new data sets. This is in
preparation of the launch of the analysis modules, two of which are
already being developed. The first module is called gridmath and it will
contain a full suite of standard arithmetic, logarithmic, and
trigonometric operators for raster band operations. The second module is
called terra and it will contain terrain-based operations, such as
slope, aspect, and visibility. A third module, called orbit, for the
analysis of satellite imagery, is on the drawing boards.
 
Have fun,
The DSpatial development team