Re: [Lmuse-developer] existing audio caching stuff

[Florian J. <flo...@we...>]

Hi

this is going to be a very long mail, but please fully read and
understand it before answering. (and look at the diagram :) )



Am 04.02.2013 09:47, schrieb Tim E. Real:

> No such methods. You probably want WaveEventBase::readAudio(WavePart* ...)
> That's where I put my processing code.

did i get this right: WaveEventBase::readAudio does not need to be hard
real time capable, because its output is not directly sent to the
speakers, but first fed into a some-seconds-long buffer (by the
AudioPrefetch stuff), and data from this buffer is then read by the
Audio thread and sent to the speakers?

please correct me if i'm wrong.



> (The code is all commented and was migrated into AudioConverter class, 
>  but if you look you can mostly see the way it was when it was functioning.
> You should have a look for some tips.) 
> 
> I already pass a WavePart* all the way from Event::readAudio to
>  WaveEventBase::readAudio().

i don't like this design (passing information about the parent to the
childs. IMHO, this should be a one-way-direction, children do not need
to know about their parents.)



i'd like to redesign it as follows

please have a look at the following epicly drawn diagram ;)
http://ente.hawo.stw.uni-erlangen.de/~ki08jofa/diagram.jpg

(i left out MIDI stuff, Wave stuff is the relevant)

first some definitions:
Event means also EventBase
WaveFile==SndFile
frame-to-frame-map = f2fmap = stretch profile is a map<int,int>, which
maps some samples of the original wave file to the sample positions of
the desired output.
i.e., for (i=0;i<10000;i+=100) f2fmap[i] = i*i/10000; causes the wave to
be extremely stretched at the beginning, then getting and faster and
faster until it's faster than the original in the end.
the total duration will be the same.

The green stuff is the current-is state, the red additions are what i'd
like to change sooner or later.


my point is, that the calls and queries shall only go from parent to
children, never should a parent give information about itself to its
child. (i.e., Event shall not need to access parentPart->pos()!)

Instead, i'd like to assign every Event the frame-to-frame-map it needs,
so it can just pass this to the SndFile and expect SndFile to process it
appropriately (SndFile may or may not doing caching stuff, this is
irrelevant to Event,Part,Track.)

Whenever a Part is moved, or the tempomap is changed, Parts are getting
informed about this. They will need to iterate through all of their
events then and adjust the f2fmaps of the events.


I understand that my design is incompatible with our clone parts, using
shared EventLists; because then, we'd have the SAME Event (with the same
f2fmap) in multiple Parts; i.e., one Event may have more parents!




There are two ways of resolving this immediate problem:
1) Change the way how clone parts are implemented: don't share the
   EventList, but each Clone has its own EventList. Whenever one of
   the clones changes its EventList, this is communicated to all other
   clones, which then need to adopt their private EventLists accordingly

or

2) Events do not contain the f2fmap. Instead, Part is passing this
   f2fmap [1] to Event::getData() on every call.
   Not only this defeats the concept of self-contained objects, which
   know everything, and only what they need, but it also introduces a
   second problem: how can the part find out the f2fmap it needs for
   an event? It could recalculate it for every getData() call, which
   would be dead slow. It cannot store it next to the Event, because
   the Events live in a (shared) Eventlist.
   It could store it in a map<EventID,f2fmap>, but lookup there is in
   O(log(number of events)), which makes this also dead slow.

My point is, every variant of 2) sucks. We cannot avoid the need of
redesigning the clone parts.


1) might sound like a heavy performance impact, if we need to propagate
all event list changes to all clone parts; but it isn't. We do not
substitute the whole EventList (which would require a full rebuild of
all other Parts' eventlists), but only issuing requests like "please get
the event at $position and change it". We can easily execute this
request for all clone parts, because Events are lightweight:
MIDI-Events only contain some bytes (midi data is small), and
Wave-Events also only contain some bytes (they only contain the
*pointer* to the SndFile).


okay. i hope you can agree with me so far. the clone design worked fine
as long everything an Event could do was not influenced by its parent
part. But now it is, and now it explodes.


[1] Well, i lied in the above text. actually we're not passing around
f2fmaps, neither is an Event storing a full f2fmap<int,int>, not is
SndFile::getData accepting one.
Rather, SndFile::create_stretch_profile() accepts a full f2fmap, and
does the following:
  1. in all f2fmaps we have stored so far, look if that one is existent.
  2. if it's not, add this f2fmap together with a unique key to our list
     and return the generated key (the key is an int)
  3. if it is, just return the key.
  (4. maybe inform our background worker that he might want to start
      calculating things)
Oppositely, SndFile::drop_stretch_profile() accepts a key, and removes
the stored map from its list (and possibly clears the associated cache,
if any.)
And finally SndFile::get_data() accepts "from", "to" and "key", which
might or might now use any cached material (which might have been
calculated by some background thread before), or retrieves the actual
f2fmap plus context from our internal, private list and does the processing.


You might have noticed all these "might or might nots". I'm writing
this, because i want *abstraction*. It doesn't matter HOW SndFile gives
us the stretched data. That way we can easily make it cache, or make it
not cache stuff, we easily can add new algorithms. We'll only need to
change one source file then.




What do you think of it? Might be some work, but do you consider this
design clean?

(I really don't want to build the stretching feature upon a bad design.
And as i have pointed out, i consider the current design bad for this
purpose.)





> Your new cache is just for stretching?

yes. it might or might not exist at all :)

> Maybe some overlap of functionality with AudioPrefetch.
> Maybe you can tap into the audio prefetch.

no, because...

> These caches are meant to be semi-small. So they should not eat much memory. 

that's the reason :)
My cache is meant to be large or huge. It shall contain the whole
stretched wave file.

> MusE (supposedly) prevents disk memory swapping by locking all current and 
>  future memory at start. I think it's working but I guess at some point it
>  must swap.

My cache is meant to be treated as any other audio file. It would be
cool if we don't need to swap it, but we would swap it if neccessary.

greetings
flo