VastSky is a linux-based cluster storage system, which provides logical volumes (linux block devices) to users by aggregating disks over a network.
A VastSky system consists of three kinds of servers;
- storage manager
- head servers
- storage servers
VastSky assumes that the servers are connected each other by a mostly-flat network. (e.g. an ethernet segment)
A storage manager maintains a database which describes physical and logical resources in a system, e.g. servers, disks, logical volumes. It serves XML-RPC based API that operates on resources, e.g. create and attach logical volumes.
Normally there should be only one storage manager on a VastSky system. VastSky currently does not provide any redundancy for the storage manager itself. It might be a good idea to consider to use external HA solutions for this.
Head servers run user applications or virtual machines which actually use VastSky logical volumes.
Storage servers have physical disks which are used to store user data. They are exported over the network and used to provide logical volumes on head servers. Anything which Linux recognizes as a block device can be used as vastsky physical disks.
Redundancy, fault detection and recovery
VastSky mirrors user data to three storage servers by default and all of them are updated synchronously. VastSky attempts to detect hardware failures including broken disks. On a failure, the storage manager attempts to reconfigure mirrors by allocating new extents from other disks automatically.
VastSky can be configured to use two networks (e.g. two independent ethernet segments) for redundancy. When one network is down VastSky will attempt to use the other network transparently to users.
The storage manager periodically checks if each head and storage servers are responsive. Disks on unresponsive storage servers are considered broken and the mirror reconfiguration mentioned above will happen.
VastSky provides XML-RPC based API, which allows external monitoring programs report resource failures.
Physical to Logical Volume Mapping
VastSky gives good performance on both read/write I/O operations and recovery operations for mirrored devices.
A logical volume is a set of several mirrored disks, each of which consists of several physical disk chunks on different servers. Actually, one 1TB logical volume will have a hundred mirrored disks in it. With this architecture, all I/O operations, including read, write and even recovery operations for a VastSky logical volume will be distributed to all the physical disks. Although the I/O loads of logical volumes are usually unbalanced, with VastSky's approach, the loads will be equalized across the physical disks, which leads that it utilizes the I/O bandwidth of them. (Fig. Load balancing of read/write operations)
I/O operations to rebuild mirrored devices are also distributed across many physical disks. If one physical disk happens to get broken, a lot of recovery operations for the mirrored devices which have a mirror on the disk will start. These recovery operations can run independently because they will allocate a spare from different physical disks.
VastSky supports an active-active redundant network, with which the bandwidth will be fully utilized.
Most of cluster file-systems and storage systems which have a meta-data control node have a scalability problem. The control node tends to easily become a bottleneck since that node has to control all operations. On the other hand, VastSky doesn't have this problem since once a logical volume is set up, all I/O operations will be done only through Linux drivers without any storage manager interactions.
The latest version of VastSky can be downloaded from the project on sf.net.
- This document shows how to install VastSky v3.0.
- This document shows how to make use of a logical volume on VastSky v3.0.
- This is a command reference manual.
- Features that will be implemented in a few months.
- OpenStack integration.
- A lot of other stuff. (click here)
Click on the following image to upload a new version of the PNG logo image for your project: