Menu ▾ ▴

Home

Jan Wagner

User-space file system for Conduant StreamStor

A read-only filesystem for the Conduant StreamStor disk recorders. These recorders are used in Mark5, one of the several types of astronomic and geodetic VLBI data storage computers.

This file system provides normal file access to the data stored on the recorder diskpack(s). It works in normal kernel 2.4 and kernel 2.6 on normal Linux distros. On a Mark5 the file system makes any 'disk2file' step obsolete, as you no longer have to copy scans out onto disk before you can access them. In addition scans become accessible to normal software for FTP, Tsunami UDP, Grid-FTP, UDT, NFS or cluster export, data processing such as correlation, Matlab, and so on.

Of the Haystack recorders Mark5A, Mark5B(+) and now Mark5C are supported. Haystack has also another fuseMk5 that provides Mark5C support and it comes preinstalled on the Mark5C.

The file system is implemented via FUSE. FUSE is part of every usual Linux distro. It is a generic library and kernel module that makes writing and debugging file systems very easy. It is used by many other user-space file systems such as GridFS, sshfs, ZFS for Linux and iFuse.

Source code access

Currently no binaries are distributed, because the Conduant StreamStor library licensing scheme is not clear on binary redistribution. For this reason you have to compile fuseMk5 from the sources. Please use the git repository to access source code.

You can check out source code with

git clone git://git.code.sf.net/p/fusemk5a/git fusemk5a-git

and can update an already checked-out copy later with

cd fusemk5a-git
git pull

Compiling

The instructions for compiling fuseMk5 and its utilities can be found in the README together with the source code. Basically,

edit Makefile  # check Conduant driver path SDK_DIR, check library version is correct (e.g. libwdapi1150.so)
make -j
make install

Programs

fuseMk5 : file system emulation for Mark5A/B/B+/C disk modules
maxspeed : test the maximum read performance from module
rawread : try to read a broken disk module without user directory
diskpack2scanlist : try to detect scans from a disk module without user directory
difxdirfile2userdir.py : recover a user directory via existing DiFX directory file
fslog2userdir.py : recover a user directory via FieldSystem log
writeuserdir : write binary user directory file onto module in bank A

Known limitations and issues

  1. Read-only access, can't write to the diskpack nor erase scans. Haystack and Conduant are possibly going to provide write support in the future. Others can of course volunteer.
  2. Conduant SDK 9.x firmware no longer plays back modules with broken/missing disks; the fill pattern insertion feature is non-functional in 9.4, but might still work in 9.3.
  3. Files do not have real timestamps because StreamStor does not use timestamps.
  4. StreamStor SDK versioning is somewhat confusing. Makefile LDFLAGS must be updated manually. For SDK9.x Beta, use -lwdapi921, for 9.x use -lwdapi910, for 7.x/8.x use -lwdapi801. Best to figure this out using 'locate libwdapi'...
  5. The StreamStor design allows only one application to use a diskpack bank at any time. This means you should stop 'Mark5A' before mounting the diskpack, and umount the diskpack before you start 'Mark5A'. The Mark5A program also resets the card so that it is unlikely it would be possible to read from one fuseMk5'ed bank while 'Mark5A' writes to the other.
  6. Beginning with v1.0.3, multiple files can be opened and read, which is good when using NFS. But the StreamStor has very poor seek performance, nowhere close to software RAID, so don't say you weren't warned... Try to avoid reading from more than two files simultaneously.
  7. You can export the fuseMk5A file system with sshfs or userspace NFS. You should export not as root but as the user who mounted the diskpack. In /etc/fstab or in the nfs mount options use rsize=16384 or larger for best performance. Verify that cat /proc/sys/net/core/rmem_default is a few hundred kByte.
  8. NFS clients are sometimes prone to freeze with no timeouts in situations when the NFS server disappears but the server computer still runs. E.g. with an /etc/init.d/nfs stop -terminated nfs server. The problem happens not just with fuse. Resolution unknown. Maybe 'nolock' option, maybe not.
  9. The fuse-utils 2.5.x/2.6.x contains a bug when you unmount any fuse file system (sshfs, obexfs, fuseMk5a) that was mounted into the background. This happens with earlier fuseMk5A versions (<=1.0.3) started without '--verbose' or with any (sshfs, ... etc) FUSE file systems mounted from /etc/fstab. Unmounting the file system appears successfull, but in reality the file system provider process may still be running ('ps axuf|grep fuseMk5A'). Unfortunately a 'kill -9 <fusemk5a pid="">' is not really effective. For this reason, unless you can upgrade to fuse-utils 2.7.1, you should start all fuse filesystems (sshfs and others) into the foreground.</fusemk5a>

Development ideas

In principle, the StreamStor PCI-X board could be replaced with 4 x PCI Express IDE/PATA cards that support two devices (master, slave)
on a single IDE cable. This would bypass the 1.4 Gbit/s limitation of the PCI-X bus and should increase playback speed to 3+ Gbit/s.

This would additionally restore the capability for fill-pattern insertion and allow playback of partially broken modules again.

At JIVE the disks of a module were found to contain 65536-byte blocks. The first 8 byte of each block are an unused 32-bit
integer, followed by a 28-bit block sequence number (for de-striping / gather) and a 4-bit filler. The user directory is found at the end
of every disk at 11272704 bytes before the end of the disk.


Project Members:

MongoDB Logo MongoDB