Installing PDL Manually

From pdl

(Difference between revisions)

Revision as of 04:33, 30 November 2009

Overview

Whereas Installing PDL the quick and easy way covers installing PDL using package management tools, here I will assume that you want to install PDL from source. This page covers the how and why of doing just that.

Why install PDL Manually?

That's a good question, and I'm glad you asked. You might install PDL manually for a number of reasons, some of which probably include:

you have an operating system or flavor of Linux that does not contain a pre-packaged version of PDL, or which contains an old version of PDL;
you want to test the new PDL release on some of your scripts before installing it;
you want to contribute to the development of PDL (thanks!);
you're trying to debug problems with an installation or you want your own, customized build of PDL, and the manual installation gives you more diagnostics and control

Whatever your reasons, this page should get you through the configuration and installation phase of installing PDL manually.

Installing Necessary and Recommended Dependencies

PDL can be very tricky to install, and the primary reason it gives such trouble is because it has a number of dependencies. Although the development team is working towards automating the whole installation process, we're not there yet. So, this portion of this guide should tell you which dependencies you'll need to install to get a nicely running PDL.

You can find the full list of dependencies in the DEPENDENCIES file in the root of the PDL tarball. To see this file without downloading and unwrapping that tarball, take a look at the DEPENDENCIES file in PDL's GIT repository.

At the moment, only OpenGL is required, but many are recommended. In what follows, we try to give instructions for installing the necessary libraries on your OS. If yours is not listed, we would love to add it. Please send a list of installation instructions to the PDL mailing list. Thanks!

All Dependencies

Each dependency is discussed below, but here's a single command (or set of commands) to get them all installed:

Ubuntu:
sudo apt-get install libxi-dev libxmu-dev freeglut3-dev libgsl0-dev libnetpbm10-dev libplplot-dev pgplot5 libterm-readline-perl-perl

Ubuntu Dapper:
sudo apt-get install libxi-dev libxmu-dev freeglut3-dev libgsl0-dev libnetpbm10-dev libplplot-dev pgplot5 slatec-dev libterm-readline-perl-perl

X Development Files

If you use the X window system on an OS that primarily distributes precompiled libraries, you will likely need to install the X development libraries in order to get OpenGL to work. (This is arguably the most annoying part of the process for me.)

Ubuntu:
#> sudo apt-get install libxi-dev libxmu-dev

OpenGL, FreeGLUT

PDL can perform 3D visualization with the TriD package, which requires OpenGL. Technically, PDL doesn't explicitly require OpenGL, but it does depend on POGL (Perl's OpenGL), which does depend on OpenGL.

Ubuntu:
#> sudo apt-get install freeglut3-dev

GSL

GSL provides a complete scientific numerical library. PDL does not currently implement wrappers for all of GSL's functionality, but it's a goal of ours, and the stuff that is implemented is handy and highly recommended. Here's how to install GSL for various operating systems.

Ubuntu:
#> sudo apt-get install libgsl0-dev

Netpbm

Netpbm is useful for converting between image formats. If you can't find installation instructions for you OS here, you can hopefully find them at their Sourceforge project page: http://netpbm.sourceforge.net/

Ubuntu:
#> sudo apt-get install libnetpbm10-dev

PLplot

PDL has two (three if you include OpenGL, four if you consider SDL, five if you count Karma) different libraries for visualizing your data. PLplot is a well written and currently supported library written in C.

Ubuntu:
#> sudo apt-get install libplplot-dev

PGPLOT

PGPLOT is a fortran-based plotting library that has gotten a lot of use with PDL. It has a well-developed interface for PDL, so if you have a fortran compiler or can get your hands on a pre-compiled copy, it is a great way to graph your data.

Ubuntu
#> sudo apt-get install pgplot5

Slatec

Slatec is an ancient (circa '95) fortran library for matrix operations and general math. However, it is well tested and fast.

Ubuntu Dapper
#> sudo apt-get install slatec-dev

Term-Readline

You'll want this for a nice perldl command-line. It will also enable the command-line history of the cpan shell, if that's not working for you.

Ubuntu
#> sudo apt-get install libterm-readline-perl-perl

Arch Linux
#> sudo pacman -S perl-term-readline

Acquiring and Installing PDL

When it comes to acquiring the source code for PDL, you have at least three options: you can install PDL through the CPAN shell, you can manually download the files from Sourceforge, or you can pull the sources from the GIT repository. Before running the actual installation, be sure you have installed the dependencies (which are discussed below at the moment). If you have never installed PDL before, we highly recommend that you install PDL using CPAN because that will automate the installation of a few important Perl dependencies. Once you have done the CPAN-based installation once, you can use Sourceforge or Git if you want even more up-to-date editions of PDL than you can get through CPAN.

CPAN

Begin by firing up the CPAN shell. You'll want to do this with root/administrative privileges because it will eventually install files into some of your system folders. To start CPAN, issue a command like one of these:

$> cpan       # as root/administrator
$> sudo cpan  # if your systems uses sudo, like with Ubuntu

Side Note: Starting CPAN for the first time

If you have never used CPAN before, it will begin with a rather verbose message, finishing with a question, like this:

CPAN is the world-wide archive of perl resources. It consists of about 300 sites that all replicate the same contents around the globe. Many countries have at least one CPAN site already. The resources found on CPAN are easily accessible with the CPAN.pm module. If you want to use CPAN.pm, lots of things have to be configured. Fortunately, most of them can be determined automatically. If you prefer the automatic configuration, answer 'yes' below.

If you prefer to enter a dialog instead, you can answer 'no' to this question and I'll let you configure in small steps one thing after the other. (Note: you can revisit this dialog anytime later by typing 'o conf init' at the cpan prompt.) Would you like me to configure as much as possible automatically? [yes]

Just hit ENTER (which makes the default selection of yes). Don't say no unless you actually know what you're doing (in which case you wouldn't need to read this section anyway). If you go for automatic configuration - which we highly recommend - volumes of text will pass by on the screen and you will hopefully be presented with the following prompt:

cpan[1]>

Now we are ready to issue our installation commands.

Get POGL

We won't begin immediately with PDL because first we need to get Perl's OpenGL module, aka POGL. For many modules, it is sufficient to tell CPAN to install the module, but OpenGL can run into some tricky snags and it'll be easier to take it one step at a time. So issue this command at the cpan prompt:

cpan[1]> get OpenGL

You will see a few lines of text, ending with this question:

Is it OK to try to connect to the Internet? [yes]

CPAN needs to connect to the internet to get the latest repository list. Hit ENTER (or type 'yes' and ENTER).

Side Note: Updating CPAN

At this point, if you see a message like this:

New CPAN.pm version (v1.9402) available.

go ahead and install the new CPAN. It says you'll want to issue two commands, but I find that the first one spits out so much text that I can't find the second command because it's pushed out of my screen's buffer. So I'll list them both for you here:

cpan> install CPAN    # tons of text will follow this command
cpan> reload cpan     # Notice the difference in capitalization!

And now you should be running the latest CPAN module.

Making, testing, and installing POGL

After getting the source code for POGL, we'll want to run the make command by issuing this:

cpan> make OpenGL

If all goes well, you will have Perl's OpenGL bindings compiled and you will be ready to move on to the next stage. However, if you get errors, chances are pretty good that you don't have the OpenGL libraries or one of their dependencies installed on your system. Check the external dependencies above. If you still have trouble, send an email to the PDL mailing list.

You've compiled POGL, but you haven't tested or installed it (yet). To test POGL, issue this command:

cpan> test OpenGL

At some point, if all goes well, you'll have a graphic of a floating tea-pot. You'll be able to play around with the graphic using your keyboard, using these commands:

Hold down arrow keys to rotate, 'r' to reverse, 's' to stop.
Page up/down will move cube away from/towards camera.
Use first letter of shown display mode settings to alter.
Press 'g' to toggle fullscreen mode (not supported on all platforms).
Press 'c' to capture/save a RGBA targa file.
'q' or [Esc] to quit; OpenGL window must have focus for input.

Be sure to type q when you're done. Finally, you should install OpenGL:

cpan> install OpenGL

Again, if you have any trouble with this, send an email to the PDL mailing list.

Installing Dependencies

The Inline module seems to give PDL trouble, so you should install it before moving onto installing PDL. While you're at it, you may as well install the Astro::FITS::Header now, too:

cpan> install Inline
cpan> install Astro::FITS::Header

Installing PDL

We should now be ready to install PDL. Try one of these commands:

cpan> install PDL            # If you've got all of the dependencies installed, this will get, make, test, and install in one shot

-or -

cpan> get PDL
cpan> make PDL
cpan> test PDL
cpan> install PDL

If you run into trouble, it's sometimes easier to do things manually, which we do with this command:

cpan> look PDL

Then you can issue the commands manually:

#> perl Makefile.PL
#> make
#> make test
#> make install

THIS SHOULD BE BETTER DOCUMENTED.

Tweaking PDL

Your PDL installation can be tweaked by editing the perldl.conf file

Sourceforge

You can find the latest official releases on sourceforge at [1]. The files are distributed as a gzipped tarball. To extract them, use your archive manager software, or (if you've got a Unix-like system) at the prompt type go to the directory where you saved the downloaded archive and issue this command:

tar -xzf PDL-2.4.5.tar.gz

This will create a folder called PDL-2.4.5 in your current directory. Change into the directory and continue below with GIT and Sourceforge concluding steps.

GIT

If you download PDL from the GIT repository, you will have the bleeding-edge edition of PDL. Obviously, you'll need to have GIT installed on your machine before proceeding.

Install GIT on your system, if you don't already have it.
Open a prompt and cd into whatever directory you would use for these sorts of projects. For example, on my machine I keep code bases that I want to hold onto in the folder /home/david/packages/
Run the command: git clone git://pdl.git.sourceforge.net/gitroot/pdl/pdl PDL

Then continue with the next section

GIT and Sourceforge Concluding Steps

At this point, I will proceed with a basic installation process. You can get much more detailed installation options by looking at the INSTALL file in the source code that you have just downloaded. If you want to tweak the compilation process, have a look at perldl.conf.

perl Makefile.PL

This command will spit out lots of lines saying, writing Makefile for .... The most important part of this step is to check for anything that says, "Not building..." or "Will skip...", which indicate either that you've told perldl.conf to skip that part or you don't have a dependency installed. For example, the first time I ran this I was told my PDL wouldn't have GSL support or FFTW. Skip down to the Dependencies section for details on how to fix that. Install whatever dependencies you think you need, tweak the perldl.conf file, and run it again. Once everything seems to be working the way you want run the makefile:

make

This step will probably take a while. You may get many warnings, but as long as you get no errors, you can continue. When it's finished, assuming all went well, run

make test

This will also take some time. The program should present a summary when it's done, and if any particular test files gave trouble, you can run them separately with

prove -vb t/test_file.t

Doing so will give you more precise info to send to the mailing list when you run into trouble. Once all the tests pass, or you've convinced yourself it's OK that some have not passed, build the help documentation database by issuing:

make doctest

Once that's finished, test the shell with this command:

perl -Mblib perldl

While in the shell, you can check any documentation by typing help readfraw, or similarly for the function or item you want to know more about. Finally, once you've convinced yourself it's all working, run as root (or using sudo):

make install

And you should be all set!

Running into Trouble

Based loosely on BUGS.

Most of the steps discussed in the installation give voluminous output, which can make it hard to track down precisely what is giving you trouble. However, if you run into trouble while trying to do a manual installation, the best place to start is by examining the output of the command that gave you trouble.

Problems with perl Makefile.PL

First, make sure that what you see with this command are in fact errors. You will often get output saying something like, Not building... or Will skip build of.... These are not errors: you should be able to get PDL to build and install despite these warnings.

If you do run into an error, or if you get a warning about a module that you want, then your problem is almost surely a dependency problem, or there's a bug in the makefile. Check the #Dependencies section (above). If everything looks right but you're still having trouble, #Send an Email to the PDL Mailing List.

Problems with make

When you run make, you will get lots of output. Although it's not obvious that make succeeded (Unfortunately, make doesn't say "Everything's all done!"), it's usually abundantly clear when make failed. First, the compilation step that failed will usually be filled with many lines indicated 'error', and second, the last line of make should say something like, "make: *** [test.o] Error 1" with test.o replaced by whatever file make was attempting to create, but could not because of trouble. (Note - make will not give you the name of the file it choked on: that's the compiler's job, because it's really the compiler that choked.)

You really shouldn't run into these sorts of errors, so if you do, you should file a bug report, discussed below.

Problems with make test

Although make test, like all the other steps, will give lots of output, it will give a summary at the end of which tests failed. If you like, you can get lots of details on what worked and what didn't by running the individual tests with prove, like this:

prove -vb t/fastraw.t

This is not strictly necessary, but it might tell you exactly what you need to know. If you're as ambitious as you are good looking, you will dig into the source code and try to fix the problem. If you do that, be sure that you work with the most recent GIT version (see #GIT, above), and when you've fixed the problem, be sure to submit a patch.

If you're not so ambitious, you should file a bug report.

Problems with make install

If you ran into trouble at this point, you probably didn't perform the install with root permissions.