sub-sonic-development

[Sub-sonic-development] Time to get things moving

[Douglas G. <do...@ss...>]

Hello everyone, it's been just under 3 years since my last post to this list
and I imagine most people have totally forgot about ever subscribing. In
just under a week I will have finished my University degree, during this
time I have been working on SUB-SONIC on and off.

During the time I built on the originally base work that was done as a
group, but over time I also rewrote the core to learn more about component
development myself. Over the past few years I have written numerous plugins
for SUB-SONIC and it's not too shabby a system though is still far from
perfect.

In the next few weeks I intended to release a package containing the entire
code base of SUB-SONIC "as is" to finally achieve a decent public release of
the software that I never stopped working on. This release will be under the
LGPL license unless otherwise restated.

The software contains support for the following formats (may not be
complete):
- Ogg Vorbis
- MP3 and MP2
- AAC
- CD audio
- Apple AIFF
- Sun AU
- Microsoft Wave (raw only)
- Westwood AUD
- Electronic Arts various formats
- IMF (MOD, S3M, XM, MTM, etc)

Some additional components included:
- Win32 compatible software core (shouldn't be hard to port to Linux)
- File input reader plugin
- HTTP input reader plugin
- DirectSound output plugin
- Wave writer output plugin (can't remember if it works or not)

If any of you are still interested I'd love to hear back, feel free to post,
remember this software hasn't be uploaded publically as yet, I will send out
another e-mail in the coming two weeks hopefully to announce the release on
SF.net

I also intend to update my personal website shortly to provide further
information.

Douglas Gore
Project Manager of SUB-SONIC
MSN: mig...@ho...

[Sub-sonic-development] SUB-SONIC updates/info

[<Do...@as...>]

Hello, I'm writing this e-mail to welcome on the new members of our
development team and get the current ones up to speed on the progress of
SUB-SONIC.

Since Saturday, concentrated development of SUB-SONIC is in progress. We're
hoping to provide you all with the core system on which to work on over the
next week. I've been working on a few plugins which I'll be trying to get
onto the CVS and released soon.

Ok what I'm going to do now is give a good break down of the system and
what's going on with it so that everyone is up to speed and has a basic
understanding of SUB-SONIC.

SUB-SONIC General
-----------------
The general idea behind SUB-SONIC is that there is a core system and a
series of plugins. The core system provides the architecture of the entire
system, it must handle audio input, process it with any effects requested
and then send it to it's output destination. The plugins are the actual
pieces of code which perform their unique part of each stage. There are
input, output and effect plugins. A input plugin will take a source usually
a file or data stream and turn it into a stream of uncompressed audio data
for the core to use. The effect plugin will apply a real-time effect on an
incoming audio source before it goes to the output destination. Finally the
output plugin takes the final mixed audio and passes it to the output device
which can be physical or computer based e.g. soundcard or disk file.

SUB-SONIC core
--------------
This component has mostly been developed by Dean Bilotti and Lourens Veen, I
have provided advice and an explanation on how to implement the audio side
of things but they have designed and produced the code. This component is
the most system dependent part of the system. Although we've coded it to be
as portable as possible, inevitably each platform's multi-threading works in
a different way as well as other features, therefore a certain amount of
porting will be needed of the core for each platform. A certain portion of
the efficiency and power of the overall SUB-SONIC system lies within the
core therefore there is always plenty of scope for improvement there.

As it stands presently SUB-SONIC is an entirely raw audio data based system,
this covers most music systems but some types of music systems currently
can't be implemented very well. For example MIDI can only be implemented as
a software synthesiser as there is no way for SUB-SONIC to get music data to
output devices, therefore a data layer would be required for that.

Input plugins
-------------
Input plugins in SUB-SONIC have more of a say in the way they want to
operate in compared to other APIs like Winamp. We've designed them to be
flexible so that an author isn't forced to write to a constrained
architecture. If I recall from looking at Winamp's API you had to provide
audio data in the same way an MP3 would generate it, which I found
potentially troublesome if the music system did not lend itself to that form
of data output.

The input plugins function by specifying their output audio format (e.g.
44.1Khz, stereo, 16-bit) and then providing a stream of audio data in the
specified format at the request of the SUB-SONIC core. At the moment the
input plugins data passing system is going for a bit of a redesign.
Currently we expect input plugins to produce a time length of data as their
input into the core but we are extending this so that plugins can push out
an arbitrary quantity of audio data which will then be arranged into
suitable blocks of audio by SUB-SONIC.

Output plugins
--------------
These will vary between portable and system dependent. Output plugins
usually deal with taking the mixer output and relaying it to the soundcard
either directly or via the appropriate API for whichever platform. The
plugins can choose whether or not to provide their own mixing code to
replace SUB-SONIC's built in mixer. This is designed mainly to allow for
hardware mixing. Each platform that SUB-SONIC gets ported to is going to
need at least one plugin of some sort to get the output of SUB-SONIC to the
computers audio device.

Effect plugins
--------------
As it stands the details of these are still a little sketchy. Effect plugins
are designed to allow you to apply audio effects (aka DSP effects) to a
stream of audio therefore adjusting the final audio in some way. As it
stands you will apply effects to the individual streams of audio and as yet
there are still design issues with applying effects to the mixed audio.
Effect plugins are our least needed area of development at the moment as
it's not an overly critical part of the system for the time being.

-------------------------------------------------

Hopefully the above summary should give you all an idea of roughly how
things work under the present design, if you wish to add design comments
feel free to do so. The actual development state of SUB-SONIC varies.

The core has been through an R&D stage, we had several trial versions
developed. After a few days discussion with Dean we are now working on what
we hope should become the final version of the core.

I've been working on 3 input plugins based on an old R&D version of
SUB-SONIC. They are as follows:

IMF - Intermediate Module Format
This is a multi-format module player. Presently only a few formats are
supported and a few of the pattern effects are missing but the system works.
This plugin will be needing a lot of further development and I hope to
document it in detail soon. The source code is very well commented.

Wave plugin
This just plays a standard raw PCM data Microsoft wave file. Further
enhancements should either use the ACM with running on Windows or we should
implement our own version of some of the ACM formats so that they can be
played on other platforms (e.g. MP3, ADPCM).

EA Audio plugin
This is a plugin written to play back music in Electronic Arts own audio
format found in many of their games. Presently we've got the uncompressed
audio working fine but the ADPCM code doesn't seem to work properly.

AAC plugin
I've started a bit of work on trying to implement the MPEG2 AAC audio format
as plugin. This code does not function yet as I've only done some initial
coding.

--------------------
So now you all know about SUB-SONIC I guess your wondering where do we go
from here. Well that all depends on what you the development team want to
do. This is a free project done in what I assume is everyone's spare time
therefore you contributions should be what interest you the most.

On the plugins side we have many formats to support. I'm not sure what
approach to take with things like MP3 as there are several choices. We can
either based the plugins on an existing project's decoding library such as
LAME, write our own based on the ISO sources or write our own from scratch
using the actual ISO standard. The amount of work require of course varies
greatly amongst those options. Some plugins are going to need only a little
bit of effort, others are huge jobs. IMF for example is almost like a
project within itself.

The core will need porting as soon as something solid and stable is ready
and there are always design obstacles that need to be addressed and overcome
with the core system itself.

Feel free to ask questions, suggest things to do etc. I hope we can all get
into the development of SUB-SONIC soon.

Doug

------------------------------------------------
Douglas Gore (do...@as...) ICQ: 15620154
------------------------------------------------
SUB-SONIC project manager
http://www.sourceforge.net/projects/sub-sonic/
------------------------------------------------

Re: [Sub-sonic-development] My introduction

[Lourens V. <js...@dd...>]

Hi everyone,

Well, I'm not really a developer anymore, but I'll introduce myself
anyway for reasons soon to become apparent.

My name is Lourens Veen, 18 years old, and I live in Enschede, Holland
where I study computer science at the University of Twente. Programming
interest are wide-ranging, but I'm more interested in "solving puzzles"
than in the actual coding. In other words I can spend hours figuring out
a working algorithm for some problem, then quit halfway implementing is
as soon as it's apparent that it works, or fails miserably, as then it's
not longer interesting to me. Audio has never really been my area of
expertise, as I'm a very visually oriented person. I'm currently
attempting to do things with video compression

I wrote the plugin parameter system as it's currently implemented in
Sub-Sonic, so if you have questions about that I'm here to answer. I
don't have time to really work on SS in the form of writing code, but
I'll stick around the mailinglist with wanted and unwanted advice, which
doesn't require me to spend a lot of time but still keeps me up to date
on how you're doing.

I don't have Windows, and I don't code for it, but general problems as
well as Linux questions are things I may be able to help out with.

Cheers,

Lourens

[Sub-sonic-development] My introduction

[<Do...@as...>]

Hi everyone, I'm just writing an e-mail to help introduce you all to
SUB-SONIC and inform you about what's going on etc. While I do that I think
it would be a good idea if everyone got to know each other a bit first to
start the team building process off.

You don't have to participate but I feel that a strong team comes from good
communications and relationships. I'll start this process off by introducing
myself and hopefully the rest of you will follow by introducing yourselves
on the mailing list.

I'm Douglas Gore, 19 years old, living in Somerset, England. I have been
programming for the last 10 years. I'm experienced in BASIC, VB and C++.

My programming interests mainly lie in multimedia, particularly audio. I
love sound and music and have always felt it to be an important part of
computing and therefore I enjoy learning about it. I love learning as much
information about computers and how they work and therefore have a wide
ranging knowledge in computing.

I've spend several years writing my own original music on the computer using
a variety of technologies, some of which I've been implementing into
SUB-SONIC as plugins.

This September I will be going to University in Swansea, Wales where I'll be
studying Computer Science for the next 3 years.

I'm frequently on the Internet and am a heavy user of ICQ and IRC. If you
are on ICQ then feel free to add me, my number is 15620154 (just mention
your name or something on the authorisation request). I have set up a
SUB-SONIC channel on the IRC network I use, the network is small and
friendly therefore I encourage those of you with IRC access to come and
visit.

There are three IRC servers you can access to get onto this particular IRC
network. They are:

tucny.thew3rd.co.uk
cs.thew3rd.co.uk
chex.thew3rd.co.uk

Once on join #sub-sonic. Feel free to also drop into #good which is where
most of general conversation/fun goes on.

If any of you are on other Internet paging services similar to ICQ like MSN
Messenger, Yahoo! Messenger, AOL, etc mention them as I can connect to them
as well.

I find that online communication is useful for direct discussion of how to
implement something. E-mail is good as well, but you can't directly
ask/explain something.

I'll get on with writing an e-mail to bring you all up to speed on SUB-SONIC
development so that everyone knows where we are and where we are going. Also
feel free to post questions to the mailing list.

Doug

------------------------------------------------
Douglas Gore (do...@as...) ICQ: 15620154
------------------------------------------------
SUB-SONIC project manager
http://www.sourceforge.net/projects/sub-sonic/
------------------------------------------------

[Sub-sonic-development] An update

[<Do...@as...>]

Well it's been a long time since I've posted to this list but I feel it's
about time I e-mailed everyone involved to tell you how SUB-SONIC has been
going recently.

Although coding has been slow recently we've started to make some major
progress. The concept IMF system has been release to the public in the form
of a source code distribution and BeOS distribution (available via BeBits
and SourceForge). Although by itself the concept IMF system isn't too much
use, it provides some basic code for the public to see and one or two might
find a use for it (making wave files out of MODs or maybe using the code as
the basis for some plugin to an audio system).

A few days I undertook the task of getting IMF to run as a SUB-SONIC plugin,
it's been a long time coming due to the many issues of converting a
structured music system to a polled audio stream generator. Well I finally
did it after a few days of pointer errors. Although the sound is quite poor
at the moment due to some buffer size issues it should be working fine in a
week or so.

Programming using the SUB-SONIC architecture has been a great way to test
how well it works and where improvements/changes can be made. It was fairly
straightforward although various API changes are probably going to be needed
before it's ready for public use.

I've been working through our project page getting it up to date and trying
to put some useful stuff onto it. I added a few tasks, etc anyone else (I
might need to give you permissions) who wants add stuff like tasks, bug
reports, documentation, etc. I'm sure this stuff is useful and it's there so
why not use it. Btw I've created a new mailing list purely for announcements
so if you know anyone interested in SUB-SONIC who might want to find out the
latest info about it and it's releases without reading through lots of
developer mail can join that list.

I'm trying to get the website sorted out the moment as we have 2 versions of
it (informative, ugly and non-informative, pretty) so I hope to be bringing
those together soon. We also want to give it some PHP magic to help make it
easy to maintain and keep up to date with the latest SUB-SONIC happenings.

I recently advertised a programming position for SUB-SONIC and have had an
applicant, so hopefully we will be welcoming a new coder onto the team soon.
They are a Windows/Linux C++ programmer and have a multimedia background, so
that sounds good.

Finally I'd like to thank you all for your efforts so far no matter how
small, I think we're really get some results now and that SUB-SONIC can make
a difference when we get it into the real world. SUB-SONIC has attracted
some interest, we've had something like 541 project page views and around 80
downloads of the IMF concept program.

As we like to say...
"Lets make some noise"

------------------------------------------------
Douglas Gore (do...@as...) ICQ: 15620154
------------------------------------------------
Windows/BeOS programmer
SUB-SONIC project manager
http://www.sourceforge.net/projects/sub-sonic/
------------------------------------------------

Re: [Sub-sonic-development] oops. Meant to send to mailing list. :)

[David O. <da...@ga...>]

On Thu, 06 Jul 2000, Dean Bilotti wrote:
> Yep I tried rotation too. 3 clocks. :)
> 
> I did think about the Pentium superpipelines and superscaling. That's why I 
> did some tests and timed them. :) The difference in clock counts for a 486 
> processor seemed to be very much in line with real life Pentium 
> Performance.

Did you try breaking out loops and interleaving use of different
registers? Instructions will have to stall until they get their
input data, but you can usually squeeze in some other operation in
the meantime. A good rule is to do as many things at once as possible
without running out of registers, and interleave operations to wait
as long as possible before requiring the result of an instruction.
That usually makes quite a difference, and it's also the reason why
it's not trivial to hack better code than a good compiler - this
coding style tends to generate very messy code...

Pipelining totally relies on the code being physically possible to
parallelize within the pipeline deepth, as CPUs unfortunately still
lack TTUs. (Time Travel Units) ;-)

BTW, MMX, KNI and co is even more sensitive to this, as their cores
are usually better optimized for pipelining than the normal CPU core.
(It gets easier when conditional code is normally handled with
true/false masks rather than skips.) As an example, Intel MMX
multiplications can be fully pipelined, as opposed to the standard
ALU version. They take 3 cycles to finish (just as the normal MUL),
but you can start a new one every cycle, as long as the data is
available. (Can't figure out how you'd feed like that continously, but
at least, it eliminates one or two stall cycles every now and then.)


//David


.- M u C o S --------------------------------. .- David Olofson ------.
|          A Free/Open Multimedia            | |     Audio Hacker     |
|      Plugin and Integration Standard       | |    Linux Advocate    |
`------------> http://www.linuxdj.com/mucos -' | Open Source Advocate |
.- A u d i a l i t y ------------------------. |        Singer        |
|  Rock Solid Low Latency Signal Processing  | |      Songwriter      |
`---> http://www.angelfire.com/or/audiality -' `-> da...@li... -'

[Sub-sonic-development] oops. Meant to send to mailing list. :)

[Dean B. <dea...@bt...>]

Yep I tried rotation too. 3 clocks. :)

I did think about the Pentium superpipelines and superscaling. That's why I 
did some tests and timed them. :) The difference in clock counts for a 486 
processor seemed to be very much in line with real life Pentium 
Performance.

Re: [Sub-sonic-development] Swapping Endians

[David O. <da...@ga...>]

On Fri, 30 Jun 2000, Dean Bilotti wrote:
> I've been playing with assembly speeding up my BSP compiler over the last 
> few days and I came accross a problem where I had to flip BCD bytes in a 
> register. This is similar to the Endian Swap you have to do for the mod's.
> 
> I basically had to Endian Swap 32-bit integers.
> 
> The way I saw it there were 2 viable methods for the 16 bit integer:
> 
> mov ax, Value	; 1 clock
> xor ah, al	; 1 clock
> xor al, ah	; 1 clock
> xor ah, al	; 1 clock
> 
> and
> 
> mov ax, Value	; 1 clock
> xchg ah, al	; 3 clocks

Hmm... Is rotation by 8 bits 3 clocks as well?

Anyway, have you considered what these clock counts are in real life,
with the pipeline and multiple ALUs in action?


//David


.- M u C o S --------------------------------. .- David Olofson ------.
|          A Free/Open Multimedia            | |     Audio Hacker     |
|      Plugin and Integration Standard       | |    Linux Advocate    |
`------------> http://www.linuxdj.com/mucos -' | Open Source Advocate |
.- A u d i a l i t y ------------------------. |        Singer        |
|  Rock Solid Low Latency Signal Processing  | |      Songwriter      |
`---> http://www.angelfire.com/or/audiality -' `-> da...@li... -'

[Sub-sonic-development] Swapping Endians

[Dean B. <dea...@bt...>]

I've been playing with assembly speeding up my BSP compiler over the last 
few days and I came accross a problem where I had to flip BCD bytes in a 
register. This is similar to the Endian Swap you have to do for the mod's.

I basically had to Endian Swap 32-bit integers.

The way I saw it there were 2 viable methods for the 16 bit integer:

mov ax, Value	; 1 clock
xor ah, al	; 1 clock
xor al, ah	; 1 clock
xor ah, al	; 1 clock

and

mov ax, Value	; 1 clock
xchg ah, al	; 3 clocks

so a total of 4 cycles for each.

I needed to do this with 32-bit registers though. :(
And as you well know there are only al, ah, ax, and eax so you can't really 
use the 2 methods above.

So I used the following slow method for the meantime and was gonna come 
back and speed the bugger up at a later date.

mov eax, InValue		; 1 clock
mov edx, InValue		; 1 clock
shr edx, 16			; 3 clocks
xchg al, ah			; 3 clocks
xchg dl, dh			; 3 clocks
shl eax, 16			; 3 clocks
mov ax, dx			; 1 clock

which is a total of 15 cycles. :(((((

When I came back to speed it up I stumbled accross a beautifull little 
instruction that is available on 486+ processors called bswap. :)

It allowed me to do this:

mov eax, InValue	; 1 clock
bswap eax		; 1 clock

This took the clock cycles for this code from 15 to 2!!!!!!!!
And as this is in one of the main loops involved in the compliation process 
you wouldn't believe the speed increase that this bad boy has given me. :)

I was thinking that we should be using this sucker in the mod loader but 
then I discovered that it only works with the entire 32-bits of the 
register. :(
So for it to work for 16 bit registers you have to:

mov eax, InValue	; 1 clock
bswap eax		; 1 clock
shr eax, 16		; 3 clocks

which is 5 clocks in total so it's best sticking with xchg. :(

So it's faster to Endian swap 32-bit values than 16-bit values.

Just thought you guys might be interested in this stuff. :)

laterz.

Re: [Sub-sonic-development] Interpolation figured out

[Lourens V. <js...@dd...>]

> For some reason I got the idea that in the module code I had been examining
> interpolation was a post process which doesn't make sense really, seeing as
> interpolation is about creating points.
> 
> I decided to re-exam what was going on and figured it out well and truely
> this time. What it does mean is that we 100% must go over the integer based
> mixing now, we cannot stick with floating point, there is no advantage and
> if it stays floating point our code will be extremely slow, I can easily see
> losses of several hundread percent.

That article is a bit misleading indeed! Glad my instinct was right
about this anyway (:

> Also it doesn't take long before you realise how well this code could work
> in assembler. The potential benefits are immense.

Yep. Looks dandy!

>The only problem in the above example is that SamplePos is moved by
>floating-point step, as F2/F1 is a floating-point value. To solve this, we
>can do simple linear interpolation, that is, separately add integral step
>and fractional step, like this: 

Huh? That is fixed point, has nothing to do with lerping, right?
 
> One final point and this is what I need you guys to help me on, see this
> line of code:
> 
> IntSamplePos += IntStep + CY;
> 
> CY from what I can work out represents the carry flag, it would appear this
> is something that isn't present in Windows compilers yet I believe you would
> find it in Borland compilers. I could really do with finding or simulating
> the carry counter (or maybe even the overflow flag) for my ANSI C++ part of
> SUB-SONIC. Once this is done I can actually get on and write the asm code
> and watch the CPU usage go down dramatically.

Well, I don't think you're going to get direct access to the carry flag
from ANSI C, after all there may not even be a carry flag on a certain
platform. Anyway, I suggest doing it the way you're doing it already,
add FracStep, and check if FracSamplePos is now smaller. If so,
increment IntSamplePos.

> So I hope I've cleared those points up for you (and me) now. I will play
> around with my mixing code a little more, it looks extremely ugly having two
> sorts of mixing code in the same module with 50% of commented out.

What happened to #ifdef?

> Doug

#Lourens

[Sub-sonic-development] Interpolation figured out

[<Do...@as...>]

Something had been bugging me for a while about my understanding of mixing
and today I finally worked it out. This stuff directly fits in with what
Lourens has been doing so it should make an interesting read.

For some reason I got the idea that in the module code I had been examining
interpolation was a post process which doesn't make sense really, seeing as
interpolation is about creating points.

I decided to re-exam what was going on and figured it out well and truely
this time. What it does mean is that we 100% must go over the integer based
mixing now, we cannot stick with floating point, there is no advantage and
if it stays floating point our code will be extremely slow, I can easily see
losses of several hundread percent.

The reason?

Read this bit from an article I pointed some of you to on IRC.

------------- Begin extract from HUGI #19 -----------------

If you have ever used a tracker, you know that samples can have different
frequencies depending on the current note, and they can also have different
volumes. If the sound buffer we are going to fill has frequency F1, and a
sample has frequency F2, we can simply copy the sample into the buffer,
moving in the sample with step F2/F1, and scaling sample data by volume.
Example: 


        for (i=0; i<SoundBufferSize; i++) {
           SoundBuffer[i] = Sample[SamplePos] * SampleVolume / 0x40;
           SamplePos += F2/F1;
        }

In the above example, I assume that sample volume varies between 0 (0%) and
0x40 (100%), like in typical modules. 

The only problem in the above example is that SamplePos is moved by
floating-point step, as F2/F1 is a floating-point value. To solve this, we
can do simple linear interpolation, that is, separately add integral step
and fractional step, like this: 


        IntStep = long(F2/F1);
        FracStep = long((F2<<32)/F1); // 64-bit / 32-bit division
        for (i=0; i<SoundBufferSize; i++) {
           SoundBuffer[i] = Sample[IntSamplePos] * SampleVolume >> 6;
           FracSamplePos += FracStep;
           IntSamplePos += IntStep + CY;
        }

In the above example, integral step is added to integral sample position,
along with carry from fractional step addition. 

-----------------------------------------------------------

By using fixed point arithmetic using two integer values we get a fractional
and integral part. The integral part represents the original data values and
each loop in which we are incrementing the fractional part is a new data
value we're creating, which in this case is duplicating the original byte.

At this point we can start to introduce the interpolation code to improve
the sound quality output.

The problem with floating point incrementing is that we don't get seperate
integer and fractional parts, we could extract them but that would be
extremely inefficient and unneeded when you consider how well the integer
technique works.

Also it doesn't take long before you realise how well this code could work
in assembler. The potential benefits are immense.

One final point and this is what I need you guys to help me on, see this
line of code:

IntSamplePos += IntStep + CY;

CY from what I can work out represents the carry flag, it would appear this
is something that isn't present in Windows compilers yet I believe you would
find it in Borland compilers. I could really do with finding or simulating
the carry counter (or maybe even the overflow flag) for my ANSI C++ part of
SUB-SONIC. Once this is done I can actually get on and write the asm code
and watch the CPU usage go down dramatically.

So I hope I've cleared those points up for you (and me) now. I will play
around with my mixing code a little more, it looks extremely ugly having two
sorts of mixing code in the same module with 50% of commented out.

Doug

RE: [Sub-sonic-development] Dean's design test thingy - the hello pants mother one

[Dean B. <dea...@bt...>]

Cheers Lourens. That means a lot buddy. :)

To be honest I couldn't remember the source you were talking about at first 
so I had to have a look back. I remember now though. :)

I'm no music expert but I think this is definately the way to go also. I 
just can't seem to find any flaws in the system. It will be easy to code 
and it's easy to understand what's going on even though it's semi-complex 
in design. To me it seemed very logical. I just love the idea of the linked 
lists. Everything seems to fit into place. It seems perfect for having 
multiple effects, perfect for multiple input streams, perfect for an 
accumulation buffer type mixer.

My English was especially bad today. Sorry. :)

cya round guys.

good luck with the code Lourensio.


-----Original Message-----
From:	Lourens Veen [SMTP:js...@dd...]
Sent:	23 June 2000 11:09
To:	sub...@li...
Subject:	[Sub-sonic-development] Dean's design test thingy - the hello 
pants mother one

Had another look at it today, realized it was outputting garbage because
of unix case-dependent filesystem (it tried to open Hello.txt while the
file was named hello.txt). Fixed now. Also found out how to use GNU
recode to kill those annoying 013's in the sources you guys send me.
Anyway, this is definitely the way to go, very neat coding dean!

I'm going to eat an egg now, and then work on some more mixer research.

Lourens

_______________________________________________
Sub-sonic-development mailing list
Sub...@li...
http://lists.sourceforge.net/mailman/listinfo/sub-sonic-development

[Sub-sonic-development] Dean's design test thingy - the hello pants mother one

[Lourens V. <js...@dd...>]

Had another look at it today, realized it was outputting garbage because
of unix case-dependent filesystem (it tried to open Hello.txt while the
file was named hello.txt). Fixed now. Also found out how to use GNU
recode to kill those annoying 013's in the sources you guys send me.
Anyway, this is definitely the way to go, very neat coding dean!

I'm going to eat an egg now, and then work on some more mixer research.

Lourens

[Sub-sonic-development] IMF milestone 1 reached

[<Do...@as...>]

Over the past few days I have been implementing S3M support into the IMF
system. To ensure that the output is good I have also been implementing many
of the extra effects that are needed by many of my test tunes. Most of this
has now been finished and I am pleased to announce that reasonable S3M
support is now in.

A few stability issues have arisen and many bug fixes have been put in
place. It will still take a few days for me to make the system stable after
which I will upload the IMF module to SourceForge as a preview test version
in the hope that I can get people to start testing IMF for problems and try
to find some areas that really need some work.

XM support may start getting implemented soon as well. Once this is in IMF
will be useful to developers. IT is still far to complex to implement at the
moment and until MOD, S3M and XM are running without problems and with
accurate audio output IT will not be implemented (at least that's what I'm
saying at the moment).

Alternative formats may get working implementations over the next few weeks
but no promises. It depends on the quality of their documentation and
whether or not I have to adjust my internal architecture to accomodate them.

As IMF evolves some issues have started to arise:
 * Compatibility - ensuring what I've implemented works
 * Stability - ensuring that no valid music file can crash IMF
 * Accuracy - ensuring that the audio matches the output of the original
player
 * Speed - making things run faster

Speed has become a particular concern of mine as it results in high CPU use
for the end developer and makes testing slower as I have to wait longer for
file generation.

Mixing is going to be a universal issue so it would be useful if we could
all research and develop this code together. I would be grateful for a few
days help  on optimizing the IMF mixing core. One of my main problems is
moving the ANSI C code over to an alternative mixing system so that it can
share the same tracking variables as the asm MMX code I will be putting in.

Generating output audio of 44.1Khz, 16-bit, mono, filtered for INSIDE.S3M an
8 channel S3M module I get the follow results in terms of estimated CPU
usage for real-time playback from two compilers:
	
Visual C++ compiler - 32%
Intel C++ compiler - 22%

This is very poor performance and totally unusable by the intended audience
and needs some serious optimizaitons.

Anyway I must go now.

Doug

[Sub-sonic-development] website grapics

[Lourens V. <js...@dd...>]

I've fiddled with it some more, latest shots on

http://sub-sonic.sourceforge.net/lourens/ver1new.jpg
and
http://sub-sonic.sourceforge.net/lourens/ver1new2.jpg

Dean's design thingy compiles but doesn't work, I'm working on it.

Lourens

[Sub-sonic-development] SS tech and SS design

[Dean B. <dea...@bt...>]

There are 2 zips included. One tech one and one design one.
The tech one is the technology that is needed to stream sounds through the 
system. Circular buffers, notify points, threads, etc.
The design one is the design of the system.

Both are rough at the moment but have potential.


DESIGN SOURCE
=============

You have 2 main linked lists. One for StreamObjects and one for Effect 
plugins.
Each StreamObject has a pointer to an input plugin and a pointer to another 
linked list. The link list tied to the StreamObject points to Effects held 
in the main Effects linked list.
So StreamObjects can have one Input plugin but many effect plugins.
The System can have many StreamObjects.
You give the output plugin the StreamObject Linked List and a Mixer.
It is not the case in this test but the Mixer would be full of virtual 
functions so the output can override certain functions if it wished to.
The Output Plugin uses the Mixer to mix the Input buffers.
Each Input Plugin has it's own buffer and the Output Plugin has it's own 
buffer. They could inheret these from a base buffer class at some point.
At the moment the Mixer is in control of the reading from input plugins and 
applying effects. This may not be the way to do it but it means everything 
is in one loop.
The Output Plugin can ignore the mixer and do it's own hardware mixing or 
it can overload certain parts of the class it's given. (just need some 
virtual funcs)
It is very easy to add or remove playing streams from the system as you 
just remove or add them to the playing linked list.
Adding or removing effects is just as easy.
Once the mixer is done the output plugin takes over.

This is just a test and things will obviously be different in the real 
thing but not much.
We just have to combine the design with the technology.
Plugins need to have pure abstract bases, Buffers can be inhereted from a 
buffer base, Mixer needs virtual funcs, etc, etc.

The actual program takes 9 bytes out of 2 source files called Hello.txt and 
Pant.txt.
And creates a new file called Mother.txt.
SORRY ABOUT THE WEIRD NAMES. :)
The mixer just concats the strings together at the moment.
An UpperCase effect is applied to Hello.txt.
The resulting file (Mother.txt) is one containing the first 9 chars from 
Hello.txt then the first 9 chars from Pant.txt + a NULL terminator.

The buffers have fixed sizes at the moment but that can be easily changed 
in the future.


IT SHOULD COMPILE ON LINUX. Can't be sure what it will do though. :)


TECHNOLOGY SOURCE
==================

This is the latest Win32 build with source. Sorry Linux guys. :)

Basically this program takes sound data from an input plugin pulls it 
through the system applying the effect and outputs it to an output circular 
buffer.
The Input Plugin is allowed to change the size of the buffer.
I have allowed the output plugin to control things like Sound Volume, 
Panning and Speed.

The plugin system allows many modules to exist in one dll. For instance, a 
.voc and a .wav input plugin could be held in the same dll.

I have put all of SS's functions into a C dll which can be called from any 
language.
Just make sure you use cdecl. I just found out today that VB might not 
support cdecl. May have to change it to stdcall. Can a VB guy tell me if 
that is true?

I've included source to a Delphi/Pascal Frontend that I wrote in about 10 
mins.
I know it's shit but don't blame me blame Pascal. I hate the language. :)
For some reason the position of the frequency bar isn't set properly.
Still have to work that one out. :)

By the way the effect really is total shit. I just blank out half of every 
block. :)

The plugin names are stored in the registry here:

HKEY_LOCAL_MACHINE\Software\GOOD\Sub Sonic

If you want a debugging console window in the C++ front end then just 
#define I_WANT_A_CONSOLE in WinMain.cpp.

I've compared the performance to WinAmp. In WinAmp I turned off all effects 
and visualization stuff to make it a bit farer. Playing the same file, this 
technology test uses less processing power.

It continues playing the buffer till it stops at the end. This causes some 
old sound to be played at the end of some wav files depending on where the 
buffer pointer is.
PLEASE TEST THIS WITH BIG!!!!!!! WAV FILES. Write an MP3 to WAV or 
something like that.



I've probably forgotten to say a few things.
I haven't really given the code a good check yet either.

OH WELL.
HAVE FUN GUYS!



  

Re: [Sub-sonic-development] Sub Sonic

[Lourens V. <js...@dd...>]

attachment? or is it on sourceforge somewhere?

Dammit, that makes everything I've done today and yesterday
superfluous...

[Sub-sonic-development] Sub Sonic

[Dean B. <dea...@bt...>]

Sorry for not mailing you guys. Sorry Lourens. :(
I've been kinda busy.

Here is my source. It's mostly very rough stuff. Very unportable. I just 
wanted to get a feel for streaming music and this is what I wrote. Now we 
have to work on a proper format.

There are Input/Output/Effect plugins. They all work through a generic 
interface and one dll can contain many modules, eg, you could have a .voc 
and a .wav input classes in the same dll.

Re: [Sub-sonic-development] IMF to be released

[Lourens V. <js...@dd...>]

> I will be optimizing the mixing code soon in order to yield greater
> performance. Very soon I will be putting in the SUB-SONIC interface info IMF
> so that I can start testing it in a real-time environment and 


Ehm, what SS interface stuff would that be? I mean I am working on
something, dean is working on something totally different (I think??).

???

Lourens

[Sub-sonic-development] IMF to be released

[<Do...@as...>]

Just to keep you all informed I will be making the first official internal
test of IMF availible over the next day or so hopefully via the SourceForge
website.

Yesterday I set up a live IMF directory with some webspace which you can
access at http://imf.goodev.com/ I will try to update this everyday with the
latest source release, history logs and Win32 exe. But bare in mind the live
release means it may function correctly, only zip up releases will be
working correctly.

I'm still working on the GCC port of IMF, I need to work out how to get a
kepress in an ANSI C way as getch() is a conio.h function and therefore is
not present on non-Windows/DOS OSes. So far all the other functions I've
tried have given my very poor results and have not functioned how I want
them to.

I've fixed several bugs and added support for a few more effects. I am
hoping to get some comprehensive IMF testing going soon.

S3M support is giving me some problems at the moment so that will still be a
while coming but I'm hoping to get it going soon as S3M will be a major
milestone in the usefulness factor of IMF.

I will be optimizing the mixing code soon in order to yield greater
performance. Very soon I will be putting in the SUB-SONIC interface info IMF
so that I can start testing it in a real-time environment and hopefully see
what the performance of IMF is like, my mathematical estimates show that IMF
takes up roughly 7.8% CPU use on a 433Mhz Celeron which is far from ideal.
The mixing code accounts for nearly all that CPU use therefore even the
smallest of improvements in the mixing code will yield much better results.
There is some pretty awful stuff in there. Also I'm yet to introduce the MMX
optimized assembly code which should really bring the use down. But for now
I need to get rid of the floating point calculations to prepare for the MMX
code.

I've begun to start including extra documentation with my code now. I've got
3 files now:

history.txt - This documents the changes I've made from version to version
codelog.txt - An explanation of what each code module does and various notes
readme.txt  - General notes for people who download and use IMF


On another note, I've started on sorting out the state of the SUB-SONIC
project on SourceForge. We should now be listed again as I have filled in
the TROVE cateogorization information now.

Doug

RE: [Sub-sonic-development] buffers

[<Do...@as...>]

This would be standard practice in all effects. You can't really write back
over the incoming buffer in case you still need to refer to the data your
over writing and in any case this would probably be inefficient. In IMF I
have at least 2 internal buffers in operation at anyone time so I don't see
a problem with what your doing.

Doug

-----Original Message-----
From: Lourens Veen [mailto:js...@dd...]
Sent: 30 May 2000 15:27
To: sub...@li...
Subject: [Sub-sonic-development] buffers


Hi,

I'm experimenting with some SS stuff, and in my current design an effect
plugin would always take data from one buffer, do something to it, and
write it to another one. Is this bad? For example for an echo effect you
need these buffers anyway. Ofcourse there is some overhead but it makes
for a more logical design in my eyes.

Opinions?

L.

_______________________________________________
Sub-sonic-development mailing list
Sub...@li...
http://lists.sourceforge.net/mailman/listinfo/sub-sonic-development

[Sub-sonic-development] MTM now supported

[<Do...@as...>]

Yep that's right after some late night coding and half a day of pure
confusion I've been able to add near perfect (I think I need to add the fix
for the bug in the format) MTM support to IMF. This process taught me a lot
about the way in which I should load module files making future format
implementations much easier. Of course MTM isn't really the format most of
you are intereted in seeing supported so I'll carry on with the others so
that IMF can start looking like a serious multi-format module system.

I have defined three major format milestones that will ensure good support
and a robust music playing core. The 3 formats are in the chronological
order in which I plan to implement them and are as follows:

1. S3M - Scream Tracker 3 - A now obsolete format this format use to be as
popular as MOD in it's heyday, as it borrows largely from the MOD format and
is sample based it should be the first major format to be implemented after
MOD.

2. XM - Fast Tracker 2 - This format is still popular but it is not modelled
on MOD as much and therefore differs in many areas. It will bring about a
reworking in the player internals due to linear frequency tables and much
more advanced instruments, but it paves the way for the other more advanced
formats.

3. IT - Impulse Tracker - The most advanced tracker to date, I personally
haven't studied it too much but I am well aware of the problems it presents
to traditional module players planning to integrate it. It introduces the
concept of polyphony where you can have more than one note playing on one
channel at the same time which is going to be a bit of a nightmare to
implement. The other thing is there appears to be filters (listen to the
tune Drifting Onwards included with the tracker for an example of them)
which means lots of extra shit to program in.

Obviously there will be all the other less known formats to implement as I
go along but most of them are a lot more trival to implement unless they try
'ground breaking' new concepts.

If IMF goes the way I want it to there will be about 60 formats to implement
in total.

Doug

[Sub-sonic-development] buffers

[Lourens V. <js...@dd...>]

Hi,

I'm experimenting with some SS stuff, and in my current design an effect
plugin would always take data from one buffer, do something to it, and
write it to another one. Is this bad? For example for an echo effect you
need these buffers anyway. Ofcourse there is some overhead but it makes
for a more logical design in my eyes.

Opinions?

L.

Re: [Sub-sonic-development] Quick IMF update

[Lourens V. <js...@dd...>]

I've been trying to add some plugin stuff to Deans copier, but I can't
really get my head around it. Probably some leftovers from the exams in
there. Oh well, I'll try some more, and then I'll fix it tomorrow night
(have to play tennis tomorrow).

Lourens

[Sub-sonic-development] Quick IMF update

[<Do...@as...>]

Just to keep you all informed.

My IMF system should now compile under GCC (hasn't been tested though). I've
added a NOASM define to the global header, just enable the define and all
asm will be substituted for C code.

I will improve that factor over time to be a little cleaner.

Right here is some info to know...

Most MODs play without problems now, some effects are missing but the
majority of standard MODs are working.

You can now select playback rates, 8 or 16 bit, still no choice between mono
or stereo.

The mixing is now 32-bit reduced down the the required output quality (8 or
16 bit). At the moment it's still floating point, but it should become
integer soon ready for MMX optimizations.

I will be talking to Dean about optimizing his central mixer code using some
of the techniques I'm implementing in my mixer so that we get optimal all
round mixing.

I'm hoping to get more effects in and implement S3M support over the next
week.

Doug