Technical Documentation
From giantdisc
Technical Documentation
MP3 File Naming Scheme
#ws_mp3_file_naming_scheme All mp3 files have to follow a strict naming scheme. The syntax is one of the following specifications:
- mp3filename ::= "tr0x" cddbID "-" tracknum ".mp3" (or ".ogg")
- mp3filename ::= "trxx" hexID ["-" tracknum] ".mp3" (or ".ogg")
- mp3filename ::= "http://" host [path]
Where cddbID is the unique CD hex number to identify CDs. cddbID and hexID are both 8-places hex numbers. tracknum is the number of the track in the CD.
If the CDDB-ID of the originating CD is known, the first scheme should be used (for more information about the CDDB-ID go to http://www.cddb.com or http://www.freedb.org). If the CDDB-ID is unknown, the second scheme should be used.
Examples:
- tr0x2f09d004-2.mp3 (CDDB-ID and track number are known) tr0x2f09d004-2.ogg
- trxx000000a2.mp3 (CDDB-ID and track number unknown)
- trxx000000a3-4.ogg (CDDB-ID unknown)
- http://aud-one.kpfa.org:8000 (streaming source)
Note 1: The GiantDisc system actually does not depend on the above naming scheme. Other naming schemes would work as well. These are the mandatory requirements for audio file names:
- the audio filename (without path) must be unique
- the length of the filename should not exceed 30 characters
All GiantDisc programs and scripts automatically generate filenames according the above scheme except for gdimporttri.pl and gdimportcdi.pl. If for some reasons, a different naming scheme is required, these two import scripts should be used to import track and album data.
Note 2: The mp3 files are stored in the directories ~music/00, ~music/01, .... The path, where a specific mp3 file is located is not stored in the database. This is not necessary, because the mp3 filenames are guaranteed to be unique. The command ls ~music/[0-9][0-9]/mp3filename will always list exactly one or zero files. A nice consequence of this is that mp3 files can be freely moved around between the directories ~music/00, ~music/01, ... and the system will still be able to find and play them.
Cover Image Naming Scheme
- filename ::= "img0x" cddbID "-" imgnum ".jpg"
- imgnum defines the order of the images and is always 2 digits
- example img0x55ff99cc-04.jpg
- The front cover:
- the front cover is always imgnum=00
- if booklet images are assigned, a front cover must always exist
- the front cover image is assumed to be square
- supported image type: jpg
- any image resolution, size and aspect ratio is accepted. Images will be scaled to the size how they are needed in the palm or other client programs. It is usually sufficient to scan images at 150dpi, roughly resulting in a 710x710 px image for a typical front cover. Higher resolutions can be used if necessary.
#Import_Export_File_Specifications
Import/Export File Specifications
Track Info Files (*.tri)
Track info files can represent all track related informations. TRI files are used to import, export and backup track data.
This is an example of the TRI file tr0x2f09d004-2.tri:
TRACK=2 TARTIST=Herbie Hancock TTITLE=Watermelon man TGENRE1=lf TGENRE2=l TYEAR=1974 TLANG=- TTYPE=1 TRATING=3 TLENGTH=394 TSOURCE=0 TSOURCEID=2f09d004 TCONDITION=0 TVOLADJUST=0 TLENGTHFRM= TSTARTFRM= TBPM= TCREATED=1999-01-01 TMODIFIED=2000-05-04
The tag syntax is tagname=tagcontent. A tag ends with the line end and can therefore not span multiple lines. The order of the tags within one TRI file is not significant.
Album Info Files (*.cdi)
Album, or CD info files can represent all album related informations. CDI files are used to import, export and backup album data.
This is an example of the CDI file cd0x2f09d004.cdi:
AARTIST=Herbie Hancock ATITLE=Headhunters CDDBID=2f09d004
The tag syntax is tagname=tagcontent. A tag ends with the line end and can therefore not span multiple lines. The order of the tags within one CDI file is not significant.
Genres List (genres.txt)
Languages List (languages.txt)
Music Types List (musictypes.txt)
Sources List (sources.txt)
The Database Structure
Table tracks
| Field | Type | Description |
| artist | varchar(255) | |
| title | varchar(255) | |
| composer | varchar(255) | |
| genre1 | char(10) | Genre code (see table genre) |
| genre2 | char(10) | Genre code (see table genre) |
| year | smallint unsigned | |
| lang | char(4) | |
| type | tinyint unsigned | |
| rating | tinyint unsigned | 0:'-', 1:'0', 2:'+', 3:'++' |
| length | smallint unsigned | track length in seconds |
| source | tinyint unsigned | cd, vinyl, radio etc. |
| sourceid | char(20) | cd-id, or other unique id Ex: '2e0c5214'. This is also the link to an album record |
| tracknb | tinyint unsigned | |
| mp3file | varchar(255) | either name of audio file (without path)or the URL of an audio stream |
| condition | tinyint unsigned | 0:"ok", 1:"noisy", 2:"damaged" |
| voladjust | smallint | adjustment of volume in percent (not used yet) |
| lengthfrm | mediumint | length of track in frames (not used yet) |
| startfrm | mediumint | start frame (not used yet) |
| bpm | smallint | beats per minute (not used yet) |
| lyrics | mediumtext | raw ASCII or HTML |
| mediumtext | mediumtext | raw ASCII or HTML |
| bitrate | char(9) | format of the audio file. examples: 'mp3 64', 'mp3 128', 'ogg 112', 'ogg 160', 'flac 44', 'flac 48' (see supported file formats) |
| created | date | |
| modified | date | any changes of track/info |
| backup | tinyint unsigned | 0:"notsaved", 1:"saved" (not used yet) |
| id | int not null |
Table album
| Field | Type | Description |
| artist | varchar(255) | |
| title | varchar(255) | |
| composer | varchar(255) | |
| cddbid | char(20) | cddb-ID or other unique ID without prefix '0x' |
| coverimg | varchar(255) | If this field is not empty, a front cover image file exists for this album Booklet image file naming scheme: see above. (Although filenames are built with the cddbid, this field is not redundant. It is used to efficiently determine, if a booklet image is available for this album). |
| covertxt | mediumtext | raw ASCII or HTML (not used yet) |
| modified | date | date of last change |
| genre | varchar(10) | genre of album. This is dynamically calculated based on the tracks that are associated to the album |
Table genre
| Field | Type | Description |
| id | char(10) | Genre code (see below) |
| id3genre | smallint | id3 genre code |
| genre | varchar(255) | |
| freq | int | how many tracks are associated to this genre (see more about this) |
Genres are hierachically organized in GiantDisc. Therefore, the genre id has to be constructed in a way that allows to locate a specific genre in the hierarchy.
- all id's of top-level genres consist of a single lower case letter (Example: 'd')
- the subgenres of a genre are specified by an id string. The prefix of the string consists of the id string of the supergenre and the suffix is again a single lower case letter (Example: 'ddgh' is (the id of) a subgenre of (a genre having th id) 'ddg')
This representation of a hierarchical structure allows to efficiently perform all important genre operations in the GiantDisc system.
Table language
| Field | Type | Description |
| id | char(4) | ISO 639 code |
| language | varchar(40) | |
| freq | int | how many tracks are associated to this language (see more about this) |
Table musictype
| Field | Type | Description |
| musictype | varchar(40) | |
| id | tinyint unsigned |
Table source
| Field | Type | Description |
| source | varchar(40) | |
| id | tinyint unsigned |
Table playlist
| Field | Type | Description |
| title | varchar(255) | The name of the saved playlist |
| author | varchar(255) | The creator/owner of the playlist (not used yet) |
| note | varchar(255) | (not used yet) |
| length | integer | total length in seconds (not used yet) |
| modified | timestamp(14) | |
| id | int unsigned |
Table playlistitem
| Field | Type | Description |
| playlist | int | link to playlist id |
| tracknumber | mediumint | n-th track in list |
| trackid | int | link to track |
Table playlog
| Field | Type | Description |
| trackid | int | The id of the track that was played |
| played | date | The date when the track was played |
| id | tinyint unsigned |
Table player
| Field | Type | Description |
| ipaddr | varchar(255) not null | address/hostname of sounddevice |
| uichannel | varchar(255) not null | communication channel to user interface, i.e. /dev/ttyS1 |
| resumetracknb | int | playlist item, where resume should begin after a server restart |
| resumeplaytime | int | playtime, where resume should begin after a server restart |
| resumeshufflepar | int | resume shuffle |
| resumeshufflestat | int | resume shuffle |
| id | int not null |
For each instance of the server script gdd.pl there is exactly one player record. Usually there is only one GD server running on a machine and hence one single player record exists.
Table playerstate
| Field | Type | Description |
| playerid | int not null | |
| playertype | int not null | sound output id 0:audio0, 1:audio1, 10:speaker, 11:monitor, 20: lan attached mp3 streaming device |
| snddevice | varchar(255) | device attached to sound output, i.e. /dev/audio |
| currtracknb | int | tracknb on top of playlist |
| state | varchar(4) | pl':play, 'st':stop, 'in':pause, 'ff':ff, 'Ff':faster ff, 'rw':rew, 'Rw':faster rew |
| mode | varchar(255) | playlist mode. Possible values: '0': stop at end of playlist, '1': repeat playlist, '2': when reaching the end of the playlist, randomize its order and restart, '3': stop after current track, '4': repeat current track |
| shufflepar | varchar(255) | If this string is not empty, then the player is in shuffle state. The string contains all parameters that define the track set to be played (parameters of the palm's search dialog, tab-separated) |
| shufflestat | varchar(255) | Contains a statistics string about the track set that is played in shufle mode. Ex: "94 Tracks [9h13m]" |
| framesplayed | int | current playtime (44100/1152=38.281 frames per sec) |
| framestotal | int |
Basically, multiple sound outputs can be addressed by a player (a player = a GD server gdd.pl). For each sound output a playerstate record exists. But currently only a single sound out of type 0 is supported.
The player parameters are copied from the config file into the database for efficiency reasons - starting a player needs to be as fast as possible.
Table tracklistitem
| Field | Type | Description |
| playerid | int not null | |
| listtype | smallint not null | 0:playlist, 1:riplist, 2:compressionlist, 3:cdrwlist |
| tracknb | int not null | |
| trackid | int not null |
Table parameter
GiantDisc consists of numerous scripts and programs. For efficiency reasons, program parameters are read only once per server instance from the command line and configuration file. They are then stored in this heap-table where they can be retrieved when the db-host and the player-id (server instance number) is known.
| Field | Type | Description |
| playerid | int | |
| name | varchar(50) | name of the parameter |
| value | varchar(255) | value of the parameter |
The mysql console
The mysql console can sometimes be useful to fix small problems or for debugging purpose. It is started from a shell with the command mysql.
Just some examples:
| Command | Description |
| use GiantDisc; | make GiantDisc the current database |
| show tables; | lists all tables in the GiantDisc database |
| describe tracks; | shows the field definitions of the tracks table |
| select * from musictype; | lists all music types |
| select title from tracks where artist like "%michael%"; | lists all titles of tracks with the word "michael" in the artist field (% is a wildcard) |
| exit; | quit the mysql console |
For a complete list of commands refer to the mysql manual or mysql tutorial.
#Playing_Music_in_a_Concurrent_Process
Playing Music in a Concurrent Process
The server script should always be ready to reply to requests of the Palm. It should therefore never be blocked for more than 5-10 seconds.
When music is playing, this happens in a separate process.
IPC Message routines make sure that a stop operation really kills all player processes.
Method:
- the server starts the player 'gdplayd.pl' in background
- right after starting gdplayd.pl, the server waits for a message of gdplayd.pl
- in the init phase of gdplayd.pl, it writes it's own process-id to the database and sends then a message to the message queue
- the server receives the message and can continue to process requests from the palm
- when the server should now stop playing, it is 100% guaranteed, that the correct process-id of the playing process is in the db. Killing it (and it's subprocesses) stops playing.
More tech notes
- #More_tech_notes_tablefreq The field freq of the genre and language tables contains the number of tracks, that are associated to a genre/language. Its values are recalculated whenever tracks are recorded or imported by the script gdtablefreq.pl.
Communication Protocol (API)
This API is used by clients that communicate with the server over socket or serial line connections.
- The communication between the Client and the server is stateless
- An interaction is always initiated by the Client
- There are two types of interactions: instructions and requests
- Instruction protocol: The Client sends exactly one textline (a string terminated by CR/LF) and continues without waiting for a response
- Request protocol: 1) the Client sends exactly one textline (a string terminated by CR/LF), 2) the Client waits for the server to respond, 3) the server responds by sending back one or more textlines (strings terminated by CR/LF), 4) the Client stops receiving textlines when the server sends an empty line
