jazzplusplus-devel

[jazzplusplus-devel] It's a night. Reduced trackwin's global variable dependence

[Dave F. <dav...@co...>]

All,

Working on that middle layer.  :)  I reduced some of trackwin's dependence on 
global variables tonight, but it ain't pretty.  Tomorrow I'll tackle handling 
selection, which should make it possible to clean up everything I did 
tonight.  :)  Mostly I changed a lot of calls to separate global variables to 
calls within gProject.  It's not quite the nice API I'm trying to build yet, 
though.  I might have to implement a track interface at that time, which will 
be good to get rid of the tSong class.

I hope I didn't break anything with this commit, but I probably did.  :(  It 
still builds, though.  :)

Anyway, on handling selection.  Here's what I'm shooting for and the steps I'm 
planning on taking to get there.

I'm shooting to have a jppTrack class that stores track data for each track in 
the song and eliminating the tSong class with it.  Most of tSong will get 
factored into jppProject and jppTrack.  Anyway, I want jppTrack to store 
selection data, and I want some GUI widget to share a pointer (probably use 
reference counting) to the internal jppTrack that both jppProject and the 
mysterious GUI widget know about.  I want trackwin to dynamically create a 
new GUI track for each track in the project, and allow for creating new 
tracks.  So I want each jppTrack to store its own selection data.  :)

So, I figure I'll make an interface to selection data in jppProject as a 
stepping-stone, and I'll switch trackwin to use that.  That will allow me to 
later work on the jppTrack class and rewrite the selection implementation in 
jppProject to use the new jppTrack when it's ready.  I'll also change the 
Open and Save methods (now stored in jppProject) to create new jppTracks 
instead of sticking it in a song.  I figure the existing GUI will just ask 
jppProject what's selected in order to draw it, and when the user makes a new 
selection it will tell jppProject about the new selection.  Ultimately I want 
the GUI to be able to ask the tracks themselves for selection data, so having 
a selection interface in jppProject is just a steppingstone and will become 
deprecated as soon as there's a jppTrack class.  :)

The main thing about selection is that there's a lot of middle-layer logic in 
tTrackWin::MousePlay that uses selection data to figure out where to start 
playing, what to record, and so forth.  I consider selection to be data 
shared between the middle layer and the GUI layer, so I figure I'll move it 
to the middle layer and give the GUI strong access to it (like having its own 
pointer and using reference counting or some method like that).

Eliminating the tSong class and the corresponding Song global variable is 
going to affect pretty much everything, near as I can tell.  That's ok, 
though.  :)  If you want to work on eliminating the Song global variable, you 
can add your 'extern gProject' line to any file currently using the Song 
variable and change every reference to Song to gProject->Song.  That, at 
least, brings all the global variables into one place.  It won't help to 
eliminate the class, though.  What's needed is an interface in gProject to 
the tracks themselves that's independent of the backend storage facility, and 
the tSong class turned into the backend storage facility.  So jppProject 
would be a light layer between the GUI and tSong, but that interface to the 
tracks themselves is needed to be able to make the change in the backend 
storage facility.

Ultimately I'd like to have Jazz using its own project file format and reading 
midi files will be done by importing the file.  There are bad reasons to do 
this, and good ones.  Mainly I want to be able to embed wav data in a project 
file, and have a good open project file format that will allow easily adding 
patterns, templates, and stuff, and easily support multiple midi devices in 
the studio (similar to RoseGarden, in fact as a proof-of-concept when we get 
there, someone (me?) should write an importer for RoseGarden projects).

Dave

-- 
Visit my website!
http://www.davefancella.com/?event=em

A fool's brain digests philosophy into folly, science into superstition, and
art into pedantry.  Hence University education.
		-- G. B. Shaw

Re: [jazzplusplus-devel] It's a night. Reduced trackwin's global variable dependence

[Joakim V. <jo...@ve...>]

>Ultimately I'd like to have Jazz using its own project file format and reading 
>midi files will be done by importing the file.  There are bad reasons to do 
>this, and good ones.  Mainly I want to be able to embed wav data in a project 
>file, and have a good open project file format that will allow easily adding 
>patterns, templates, and stuff, and easily support multiple midi devices in 
>the studio (similar to RoseGarden, in fact as a proof-of-concept when we get 
>there, someone (me?) should write an importer for RoseGarden projects).
>  
>
Yes, Jazz would need its own format at some point.

Something I did with Jazz was to add the facility to play tracks from 
other tracks,
a playtrack event. This was a feature in a sorely-missed amiga sequencer 
I used, Music-X.

This doesnt entirely fit with the normal midi-file standard(although 
there are sequence play midi events),
so i kind of kludged it in in Jazz midi format :-P

Anyway, these kind of features would be easier with a new format.

I would advocate looking at existing formats before inventing something, 
like
music-xml or something(music-xml has taken some flak, but it can't be 
all bad can it?)

An xml based aproach would open up for many cool concepts, like using 
XSLT transforms
for midi event processing, which could be re-used in other projects.

One of my ideas for Jazz is to open up for plugins and so on that 
manipulates event data,
like GIMP:s python, scheme, and perl interfaces.


Cheers,
Joakim

>Dave
>
>  
>

Re: [jazzplusplus-devel] It's a night. Reduced trackwin's global variable dependence

[Dave F. <dav...@co...>]

On Friday 26 March 2004 09:47 am, Joakim Verona wrote:
> >Ultimately I'd like to have Jazz using its own project file format and
> > reading midi files will be done by importing the file.  There are bad
> > reasons to do this, and good ones.  Mainly I want to be able to embed wav
> > data in a project file, and have a good open project file format that
> > will allow easily adding patterns, templates, and stuff, and easily
> > support multiple midi devices in the studio (similar to RoseGarden, in
> > fact as a proof-of-concept when we get there, someone (me?) should write
> > an importer for RoseGarden projects).
>
> Yes, Jazz would need its own format at some point.
>
> Something I did with Jazz was to add the facility to play tracks from
> other tracks,
> a playtrack event. This was a feature in a sorely-missed amiga sequencer
> I used, Music-X.

Heh, never used Music-X.  MeD was a lot of fun, though.  ;)  (DMCS also had an 
unbeated staff editor, the likes of which I haven't seen again but has 
probably been bested)

> This doesnt entirely fit with the normal midi-file standard(although
> there are sequence play midi events),
> so i kind of kludged it in in Jazz midi format :-P
>
> Anyway, these kind of features would be easier with a new format.
>
> I would advocate looking at existing formats before inventing something,
> like
> music-xml or something(music-xml has taken some flak, but it can't be
> all bad can it?)

Actually, the specific format I was interested in is adapting Audacity's 
project format.  Audacity, being a sound editor, has serious problems dealing 
with memory and the raw amount of data it processes.  But they've solved a 
number of problems relevant here, like having unlimited undo across saves.  
Anyway, we don't have the same memory problems with midi, since a midi file 
is small enough that we can probably put a hundred in memory at one time.  
Anyway, I want to adapt their blockfile approach.

So, Audacity has a data directory for the project, and a project file.  The 
project file is xml and it tells what order the files in the data directory 
go in, what tracks they belong to, and so forth.  I think the project file 
also stores the history queue.

So, what I want to do with Jazz is store blocks of midi data in midi files, 
one track to a file, and organized in blocks of data.  We don't need this for 
the RAM, but it would be useful for pattern support and unlimited undo.  I 
want to use their exact project file format, extended to support midi 
blockfiles, and derive from their BlockFile class to create midi support.

It has the disadvantage of having the project spread out into a bunch of small 
files.  We can optimize for that, since midi files are so small we can still 
load the whole project into memory, and just clear out memory when something 
gets pushed to the history queue.  There might be other ways to approach the 
problem of having all these files strewn about a directory hierarchy, but I 
think the advantages far outweigh the disadvantages and am fully prepared to 
defend this idea.  :)

Dave

> An xml based aproach would open up for many cool concepts, like using
> XSLT transforms
> for midi event processing, which could be re-used in other projects.
>
> One of my ideas for Jazz is to open up for plugins and so on that
> manipulates event data,
> like GIMP:s python, scheme, and perl interfaces.
>
>
> Cheers,
> Joakim
>
> >Dave
>
> -------------------------------------------------------
> This SF.Net email is sponsored by: IBM Linux Tutorials
> Free Linux tutorial presented by Daniel Robbins, President and CEO of
> GenToo technologies. Learn everything from fundamentals to system
> administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
> _______________________________________________
> jazzplusplus-devel mailing list
> jaz...@li...
> https://lists.sourceforge.net/lists/listinfo/jazzplusplus-devel

-- 
Visit my website!
http://www.davefancella.com/?event=em

You now have Asian Flu.

Re: [jazzplusplus-devel] It's a night. Reduced trackwin's global variable dependence

[Patrick E. <pa...@pa...>]

On Thursday 25 March 2004 19:26, Dave Fancella wrote:
> So, what I want to do with Jazz is store blocks of midi data in
> midi files, one track to a file, and organized in blocks of data. 
> We don't need this for the RAM, but it would be useful for pattern
> support and unlimited undo.  I want to use their exact project file
> format, extended to support midi blockfiles, and derive from their
> BlockFile class to create midi support.

Does that mean there would be a file for each track in a song?  That 
doesn't impress me much at all.  I know I'll have many songs, and 
having to manage each track sounds like a real annoyance.

I would rather that for midi data at least, the "song" files contain 
all of the track data.  Moving individual tracks between song files 
would be a useful operation, and you could always copy the history 
data along with the track itself.  Tracks could still be 
self-contained, but they needn't use multiple filesystem entries.

Blocking midi data also doesn't make a lot of sense to me.  In an 
audio file, what you have is a sequence of random bytes, so dividing 
it up into smaller pieces can help with management.  However, with 
midi, everthing is well defined, so it makes some sense to define 
history events in terms of the midi data itself.  For example, you 
can say something like:

Note On C4 added at location 2035.
Note Off C4 added at location 2055.

Section 1034 to 1500 deleted (previous data: XX XX XX XX XX)

> It has the disadvantage of having the project spread out into a
> bunch of small files.  We can optimize for that, since midi files
> are so small we can still load the whole project into memory, and
> just clear out memory when something gets pushed to the history
> queue.  There might be other ways to approach the problem of having
> all these files strewn about a directory hierarchy, but I think the
> advantages far outweigh the disadvantages and am fully prepared to
> defend this idea.  :)

Okay, let's hash it out. :)  My opinion is that it's good if midi 
tracks are self contained entities and there are multiple tracks 
within a single song file.

         Patrick

Re: [jazzplusplus-devel] It's a night. Reduced trackwin's global variable dependence

[joakim v. <jo...@ve...>]

Patrick Earl wrote:

>>It has the disadvantage of having the project spread out into a
>>bunch of small files.  We can optimize for that, since midi files
>>are so small we can still load the whole project into memory, and
>>just clear out memory when something gets pushed to the history
>>queue.  There might be other ways to approach the problem of having
>>all these files strewn about a directory hierarchy, but I think the
>>advantages far outweigh the disadvantages and am fully prepared to
>>defend this idea.  :)
>>    
>>
>
>Okay, let's hash it out. :)  My opinion is that it's good if midi 
>tracks are self contained entities and there are multiple tracks 
>within a single song file.
>
>         Patrick
>

Just an idea:

You could store the entire projectfile in an archive file.

This is a well known and useful practice in the web world

For Yazz, it would have the added benefit that other tools could easily 
manipulate the
Yazz file data(like copying tracks from a project to another, XSLT 
transform them and so on)

Im all for a more transparent format(but lets get what we got working ok 
first)

Cheers,
/Joakim


>
>
>-------------------------------------------------------
>This SF.Net email is sponsored by: IBM Linux Tutorials
>Free Linux tutorial presented by Daniel Robbins, President and CEO of
>GenToo technologies. Learn everything from fundamentals to system
>administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
>_______________________________________________
>jazzplusplus-devel mailing list
>jaz...@li...
>https://lists.sourceforge.net/lists/listinfo/jazzplusplus-devel
>  
>

Re: [jazzplusplus-devel] It's a night. Reduced trackwin's global variable dependence

[Patrick E. <pa...@pa...>]

On Friday 26 March 2004 09:41, joakim verona wrote:
> Just an idea:
>
> You could store the entire projectfile in an archive file.
>
> This is a well known and useful practice in the web world
>
> For Yazz, it would have the added benefit that other tools could
> easily manipulate the
> Yazz file data(like copying tracks from a project to another, XSLT
> transform them and so on)

As for XSLT transforms, if it's XML, it's easily transformable... 
doesn't really matter how many files there are.  The archive idea is 
fine with me.  It's probably slightly easier than cutting and pasting 
from a file like this:
    <project>
        <song>
            <track>...</track>
            </track>...<track>
        </song>
    </project>

> Im all for a more transparent format(but lets get what we got
> working ok first)

Indeed! :)

        Patrick

Re: [jazzplusplus-devel] It's a night. Reduced trackwin's global variable dependence

[Matt K. <ra...@ch...>]

> 
> All,
> 
> Working on that middle layer.  :)  I reduced some of trackwin's dependence on 
> global variables tonight, but it ain't pretty.  Tomorrow I'll tackle handling 
> selection, which should make it possible to clean up everything I did 
> tonight.  :)  Mostly I changed a lot of calls to separate global variables to 
> calls within gProject.  It's not quite the nice API I'm trying to build yet, 
> though.  I might have to implement a track interface at that time, which will 
> be good to get rid of the tSong class.
> 
> I hope I didn't break anything with this commit, but I probably did.  :(  It 
> still builds, though.  :)

Yup, it's broke.  :)

When starting JAZZ, it dumps core at Project.cpp:32.  It's trying to 
access mRecInfo->Track, but mRecInfo points to nothing.  This is in the
contructor, so, not sure where mRecInfo should be created.  I just put
a quick "mRecInfo = new tRecordInfo;" at the start of the
constructor func  and it now runs.

Matt

Re: [jazzplusplus-devel] It's a night. Reduced trackwin's global variable dependence

[Dave F. <dav...@co...>]

On Friday 26 March 2004 06:16 pm, Matt Kelly wrote:
> Yup, it's broke.  :)
>
> When starting JAZZ, it dumps core at Project.cpp:32.  It's trying to
> access mRecInfo->Track, but mRecInfo points to nothing.  This is in the
> contructor, so, not sure where mRecInfo should be created.  I just put
> a quick "mRecInfo = new tRecordInfo;" at the start of the
> constructor func  and it now runs.

Thanks matt, apparently I missed a few things there.  :(  I'll get 'em 
tonight.

Dave

> Matt
>
>
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by: IBM Linux Tutorials
> Free Linux tutorial presented by Daniel Robbins, President and CEO of
> GenToo technologies. Learn everything from fundamentals to system
> administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
> _______________________________________________
> jazzplusplus-devel mailing list
> jaz...@li...
> https://lists.sourceforge.net/lists/listinfo/jazzplusplus-devel

-- 
Visit my website!
http://www.davefancella.com/?event=em

A waist is a terrible thing to mind.
		-- Ziggy

Re: [jazzplusplus-devel] It's a night. Reduced trackwin's global variable dependence

[Dave F. <dav...@co...>]

On Friday 26 March 2004 04:19 pm, Patrick Earl wrote:
> On Thursday 25 March 2004 19:26, Dave Fancella wrote:
> > So, what I want to do with Jazz is store blocks of midi data in
> > midi files, one track to a file, and organized in blocks of data.
> > We don't need this for the RAM, but it would be useful for pattern
> > support and unlimited undo.  I want to use their exact project file
> > format, extended to support midi blockfiles, and derive from their
> > BlockFile class to create midi support.
>
> Does that mean there would be a file for each track in a song?  That
> doesn't impress me much at all.  I know I'll have many songs, and
> having to manage each track sounds like a real annoyance.

Yes, there'd be a file for each track in the song, and you won't have to 
manage each track, jazz would manage it for you.  The xml file describes how 
to assemble the blocks into a song, and jazz does the assembling.  It 
simplifies the backend (with some upfront implementation overhead) by 
allowing us to modify certain sections of a track, create a new blockfile for 
it, update the project in memory, and then when we write it to xml it's done.  
And we'd need some method of pushing the project state after each edit to 
maintain history.  History in this setup is entire project history and there 
isn't any way to deal with individual track history, but we could certainly 
work something out to do it, if we really wanted to.

> I would rather that for midi data at least, the "song" files contain
> all of the track data.  Moving individual tracks between song files
> would be a useful operation, and you could always copy the history
> data along with the track itself.  Tracks could still be
> self-contained, but they needn't use multiple filesystem entries.
>
> Blocking midi data also doesn't make a lot of sense to me.  In an
> audio file, what you have is a sequence of random bytes, so dividing
> it up into smaller pieces can help with management.  However, with
> midi, everthing is well defined, so it makes some sense to define
> history events in terms of the midi data itself.  For example, you
> can say something like:
>
> Note On C4 added at location 2035.
> Note Off C4 added at location 2055.
>
> Section 1034 to 1500 deleted (previous data: XX XX XX XX XX)

Blocking midi data itself isn't that useful, admittedly.  What I'm thinking is 
that when we have segment support (or patterns), we'd store one segment of 
music in one file.  The project file will define which midi channel and 
instrument to play it with.  The project file will also define when to play 
that segment.  Presumably the user will use a segment multiple times in the 
song, and will even use the same segment on multiple tracks (I do that a 
lot).  Then we have the option when the user edits that segment to either 
copy-on-write, or edit the base segment.  Copy-on-write means only the 
segment in the specific location in the song gets edited, leaving all other 
instances of the segment unaltered.  Editing the base segment means every 
instance of that segment in the song gets edited.

If a track isn't split into segments, then the track can exist within one 
single blockfile.  Blocking midi data doesn't make sense in this case and 
having the whole track in one file makes perfect sense.  (Mind you, it's 
common for an audacity project to become greater than 1GB in size.  That 
should *never* happen with midi data.  I'll be impressed to see a project > 1 
MB of midi data)

We still have to deal with mixing, but this setup allows us cascading-style 
volume and pan controls, volume and pan envelopes (since the blockfiles 
wouldn't necessarily contain that information), and other things I haven't 
even thought of.  Surround-sound?  In each of these cases, it's just a matter 
of making entries in the xml file that describes the project and the 
underlying midi data doesn't get affected in any way.  One place where 
RoseGarden specifically is very weak is in handling track-level pan and 
volume, and then dealing with segment-level pan and volume (mostly it looks 
like UI issues, but there's so many bugs in that program it's hard to 
distinguish what they intended to happen from what is happening).  In this 
fashion we'd define track-level pan and volume as applicable for the whole 
track, and changes would be notated by position in the song and independent 
of song layout.

We also get some added stability, or at least recovery from error.  Currently, 
it seems that midi sequencers just load all the midi data into memory, which 
is fine because it's so small that it's reasonable to do so.  Then, if the 
program crashes, or the power goes out, or something, you lose *everything* 
since your last save.  Autosave deals with this to some extent, but autosave 
is built-in to the blockfile approach.  Since we push onto the history queue 
after every change, recovery from error is just a matter of checking the 
timestamp on the project file and reading the history queue after that 
timestamp to rebuild the project in-memory.  We've now spread our dependence 
out over several files.

And, of course, in the long-term we get interoperability with Audacity, and 
there is a fair amount of complaining going on that there's no free as in 
speech midi sequencer/audio editor while all the proprietary offerings are 
combining midi sequencing and audio editing.

We do pay the price of having our own special project format that no other 
application knows.  We can easily port the code to audacity, though.  One of 
Jazz's strengths right now is that it works with midi files rather than using 
it's own special file format, so no import/export step is needed.  We also 
pay a price in the directory hierarchy by having many filesystem entries when 
before we'd've had only one.  Filesystems are generally fast enough that this 
shouldn't be an issue, though, but we can certainly have some sort of attic 
like cvs to stick old files in.

But I think the system is inherently more extensible and more powerful than 
any single-file format we could design, and I'd prefer to build knowing that 
what I'm building will allow many additions in features to the program with 
very little change in the backend storage facility.  Hell, the code I write 
in the next few weeks might go untouched for years (ha!).

Dave

> > It has the disadvantage of having the project spread out into a
> > bunch of small files.  We can optimize for that, since midi files
> > are so small we can still load the whole project into memory, and
> > just clear out memory when something gets pushed to the history
> > queue.  There might be other ways to approach the problem of having
> > all these files strewn about a directory hierarchy, but I think the
> > advantages far outweigh the disadvantages and am fully prepared to
> > defend this idea.  :)
>
> Okay, let's hash it out. :)  My opinion is that it's good if midi
> tracks are self contained entities and there are multiple tracks
> within a single song file.
>
>          Patrick
>
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by: IBM Linux Tutorials
> Free Linux tutorial presented by Daniel Robbins, President and CEO of
> GenToo technologies. Learn everything from fundamentals to system
> administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
> _______________________________________________
> jazzplusplus-devel mailing list
> jaz...@li...
> https://lists.sourceforge.net/lists/listinfo/jazzplusplus-devel

-- 
Visit my website!
http://www.davefancella.com/?event=em

Robert Tappen Morris, Jr., got six months in jail for crashing 10% of the
computers that Bill Gates made $100 million crashing last weekend.

Re: [jazzplusplus-devel] It's a night. Reduced trackwin's global variable dependence

[Patrick E. <pa...@pa...>]

On Friday 26 March 2004 06:16, Dave Fancella wrote:
> Yes, there'd be a file for each track in the song, and you won't
> have to manage each track, jazz would manage it for you.

I meant managing the files on my file system.  They could all be 
placed in a single directory, but if you wanted to stick a song on a 
web page or something, you'd need to tar them up.

I'd rather have a little dialog that allows you to copy segments 
between projects.  Could be as easy as dragging them around.  Select 
one song on one side, another song on the other side, drag the 
appropriate segements.  If you don't copy the segment, you might even 
accidentally change a segment that was used in another song.  Having 
to think about all those independent segments doesn't sound like a 
lot of fun to me.

In the dialog approach, it would be helpful to have a "play" button to 
see which segment you're dealing with.  I don't really want to have 
to come up with memorable names for all the little pieces of my 
music.  In fact, I rarely want to come up with names for any of my 
music. :)

> We still have to deal with mixing, but this setup allows us
> cascading-style volume and pan controls, volume and pan envelopes
> (since the blockfiles wouldn't necessarily contain that
> information), and other things I haven't even thought of. 
> Surround-sound?  In each of these cases, it's just a matter of
> making entries in the xml file that describes the project and the
> underlying midi data doesn't get affected in any way.  One place
> where RoseGarden specifically is very weak is in handling
> track-level pan and volume, and then dealing with segment-level pan
> and volume (mostly it looks like UI issues, but there's so many
> bugs in that program it's hard to distinguish what they intended to
> happen from what is happening).  In this fashion we'd define
> track-level pan and volume as applicable for the whole track, and
> changes would be notated by position in the song and independent of
> song layout.

I like the idea of having segments that can do relative volume 
adjustments, loops, and whatnot.  However, I still don't believe each 
segment deserves its own file.

> We also get some added stability, or at least recovery from error. 
> Currently, it seems that midi sequencers just load all the midi
> data into memory, which is fine because it's so small that it's
> reasonable to do so.  Then, if the program crashes, or the power
> goes out, or something, you lose *everything* since your last save.
>  Autosave deals with this to some extent, but autosave is built-in
> to the blockfile approach.  Since we push onto the history queue
> after every change, recovery from error is just a matter of
> checking the timestamp on the project file and reading the history
> queue after that timestamp to rebuild the project in-memory.  We've
> now spread our dependence out over several files.

How are history storage, journaling, and block files not orthogonal 
issues?  I don't really see the advantage of many small files unless 
you write a poor history system and it crashes in the middle of 
writing to the large file.

> And, of course, in the long-term we get interoperability with
> Audacity, and there is a fair amount of complaining going on that
> there's no free as in speech midi sequencer/audio editor while all
> the proprietary offerings are combining midi sequencing and audio
> editing.

I don't think many midi segment files are needed to be interoperable 
with Audacity.  I personally don't even think that Audacity's track 
handling extends well to the midi world.

> But I think the system is inherently more extensible and more
> powerful than any single-file format we could design, and I'd
> prefer to build knowing that what I'm building will allow many
> additions in features to the program with very little change in the
> backend storage facility.  Hell, the code I write in the next few
> weeks might go untouched for years (ha!).

I'd say a good XML format would be easier to manipulate and more 
extensible than either a dir full of files or an archived set of 
files.  There's nothing tricky about sticking a bunch of <segment> 
things in a midi file.  XML also allows recursion, so nesting isn't a 
problem.  Segments could even be given names, just like in the file 
system.  If somebody wanted to manipulate the file, they wouldn't 
have to find an archiver or read a directory listing of files along 
with another project file.  They could use a tool as simple as a text 
editor or as complex as XSL.

        Patrick

Re: [jazzplusplus-devel] It's a night. Reduced trackwin's global variable dependence

[Dave F. <dav...@co...>]

On Saturday 27 March 2004 07:20 am, Patrick Earl wrote:
> On Friday 26 March 2004 06:16, Dave Fancella wrote:
> > Yes, there'd be a file for each track in the song, and you won't
> > have to manage each track, jazz would manage it for you.
>
> I meant managing the files on my file system.  They could all be
> placed in a single directory, but if you wanted to stick a song on a
> web page or something, you'd need to tar them up.

Only if you wanted people to work with the project you created.  You could 
otherwise export it to a single file.

> I'd rather have a little dialog that allows you to copy segments
> between projects.  Could be as easy as dragging them around.  Select
> one song on one side, another song on the other side, drag the
> appropriate segements.  If you don't copy the segment, you might even
> accidentally change a segment that was used in another song.  Having
> to think about all those independent segments doesn't sound like a
> lot of fun to me.

This dialog and the blockfile approach aren't mutually exclusive.  We'd just 
have to define the default behavior or just decide that this case counts as a 
write and the segment should be copied even if it hasn't changed.

> In the dialog approach, it would be helpful to have a "play" button to
> see which segment you're dealing with.  I don't really want to have
> to come up with memorable names for all the little pieces of my
> music.  In fact, I rarely want to come up with names for any of my
> music. :)

I'd like some way to be able to look at a segment and know what it is, but I 
agree with the naming thing here.  :)  It'd be nice if a segment just said 
"verse" or something, and Jazz knew enough about song structure to either ask 
you what the thing is, or just guess and guess right.

> > We still have to deal with mixing, but this setup allows us
> > cascading-style volume and pan controls, volume and pan envelopes
> > (since the blockfiles wouldn't necessarily contain that
> > information), and other things I haven't even thought of.
> > Surround-sound?  In each of these cases, it's just a matter of
> > making entries in the xml file that describes the project and the
> > underlying midi data doesn't get affected in any way.  One place
> > where RoseGarden specifically is very weak is in handling
> > track-level pan and volume, and then dealing with segment-level pan
> > and volume (mostly it looks like UI issues, but there's so many
> > bugs in that program it's hard to distinguish what they intended to
> > happen from what is happening).  In this fashion we'd define
> > track-level pan and volume as applicable for the whole track, and
> > changes would be notated by position in the song and independent of
> > song layout.
>
> I like the idea of having segments that can do relative volume
> adjustments, loops, and whatnot.  However, I still don't believe each
> segment deserves its own file.

As long as it can be managed well, but consider that you could be looking at 
50+ segments each with its own settings separate from the track settings.  I 
can't hold that much information in my head (well, I can, but when I'm 
working on a song I'd rather not have to hold that information in my head at 
all).  I'd prefer an envelope instead, it's much simpler and easier to grok 
visually, and you don't have to hold any information in your head because the 
envelope is clearly visible in the GUI.

> > We also get some added stability, or at least recovery from error.
> > Currently, it seems that midi sequencers just load all the midi
> > data into memory, which is fine because it's so small that it's
> > reasonable to do so.  Then, if the program crashes, or the power
> > goes out, or something, you lose *everything* since your last save.
> >  Autosave deals with this to some extent, but autosave is built-in
> > to the blockfile approach.  Since we push onto the history queue
> > after every change, recovery from error is just a matter of
> > checking the timestamp on the project file and reading the history
> > queue after that timestamp to rebuild the project in-memory.  We've
> > now spread our dependence out over several files.
>
> How are history storage, journaling, and block files not orthogonal
> issues?  I don't really see the advantage of many small files unless
> you write a poor history system and it crashes in the middle of
> writing to the large file.

Well, around here, in high winds, brief losses of electricity are not 
uncommon.  In fact, just a month ago or so we lost power for two hours or so.  
I can't even count how many hours I've wasted rebuilding lost work when the 
power went out.  Whether that's because of poorly-written history or just bad 
design that prevents a well-written history, I don't really know.  (I am, of 
course, talking about several apps spread across a variety of purposes)  So 
naturally I'm worried about loss of data quite a bit.  :)  (RoseGarden 
crashes frequently and its autosave feature isn't adequate to the task)

So basically I don't want to have the whole project stored in memory at all 
times unless it's done solely for optimization.  If the project is an xml 
file, frankly I don't see how it can be dealt with efficiently and still save 
its state to disk with every edit, because for every save we'd have to 
compose a new xml file that doesn't resemble the old one a lot.  With the 
blockfile approach, we can have a history stored in its own file that we can 
open for append, push state, and close (I'm willing to let the OS handle 
flushing the IO buffer at this point).  We create our new blockfile, keeping 
a copy of the old blockfile, and we only have to save the xml project file 
when the user asks to save it.  We can rebuild the project file from the 
history file, and all the actual musical data is preserved as well.

I'm also not sure I answered your question.  :(  I think I did, though.  :)

> > And, of course, in the long-term we get interoperability with
> > Audacity, and there is a fair amount of complaining going on that
> > there's no free as in speech midi sequencer/audio editor while all
> > the proprietary offerings are combining midi sequencing and audio
> > editing.
>
> I don't think many midi segment files are needed to be interoperable
> with Audacity.  I personally don't even think that Audacity's track
> handling extends well to the midi world.

It may not.  Mind you, Audacity is dealing with much larger files than we are, 
and their approach is excellent for it.  WIth other programs that store the 
entire project in one huge interleaved wav file, or each track in its own wav 
file, when you apply an effect to a selection the program has to load the 
whole file and loop through it doing nothing to get to the selection, then 
loop through it applying the effect, and then loop through it again doing 
nothing but saving the old data.  A wav file for one track can easily be 
20MB, and with 8-9 tracks (average for me), you quickly exceed available RAM 
and swap space combined.  We don't have that problem here at all.  :)

> > But I think the system is inherently more extensible and more
> > powerful than any single-file format we could design, and I'd
> > prefer to build knowing that what I'm building will allow many
> > additions in features to the program with very little change in the
> > backend storage facility.  Hell, the code I write in the next few
> > weeks might go untouched for years (ha!).
>
> I'd say a good XML format would be easier to manipulate and more
> extensible than either a dir full of files or an archived set of
> files.  There's nothing tricky about sticking a bunch of <segment>
> things in a midi file.  XML also allows recursion, so nesting isn't a
> problem.  Segments could even be given names, just like in the file
> system.  If somebody wanted to manipulate the file, they wouldn't
> have to find an archiver or read a directory listing of files along
> with another project file.  They could use a tool as simple as a text
> editor or as complex as XSL.

Yes and no.  The main problems I see with using one single file are history 
and having the whole project in RAM.  Yes, we can carefully work out the 
format to put history in it, but in order to write the history we have to 
write the whole project, so with every edit (to have my file-based history) 
we'd have to write the whole project.  That may not be as annoying as I'm 
thinking it will be, though.  The way I see it, either one or the other needs 
to be on-disk.  All the other benefits I'm touting could probably be achieved 
another way, I'm not doubting that at all.

Now, I'm pretty sleepy right now, so I won't be surprised if I read this 
tomorrow and it doesn't make any sense.  ;)  So I figure we should probably 
cut it off, I don't have anything new to add to the discussion that wouldn't 
end up being arguing just for the sake of being right and proving I'm an ass.  
As Joakim pointed out, we're a ways from doing anything with the project 
format anyway, so we can let it cook for awhile and see what comes up.  :)  I 
promise not to implement anything for the project file without bringing the 
subject back up for discussion.  ;)  Normally I say "Write it however I want, 
it's easier to look at it and say what's wrong and fix it than it is to write 
something that's perfect the first time", but this is an area we need to try 
to get perfect the first time.  Users (me, especially) hate it when you have 
to change the project format for any reason.

Dave

>         Patrick
>
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by: IBM Linux Tutorials
> Free Linux tutorial presented by Daniel Robbins, President and CEO of
> GenToo technologies. Learn everything from fundamentals to system
> administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
> _______________________________________________
> jazzplusplus-devel mailing list
> jaz...@li...
> https://lists.sourceforge.net/lists/listinfo/jazzplusplus-devel

-- 
Visit my website!
http://www.davefancella.com/?event=em

Suaviter in modo, fortiter in re.
Se non e vero, e ben trovato.