[ glame-Bugs-871787 ] MP3 importer is desperately slow 20MB/6h on 500MHz/Intel

[SourceForge.net <no...@so...>]

Bugs item #871787, was opened at 2004-01-06 17:35
Message generated for change (Comment added) made by richi
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=101627&aid=871787&group_id=1627

Category: Midlayer
Group: normal
>Status: Closed
>Resolution: Wont Fix
Priority: 5
Submitted By: Emanoil Kotsev (deloptes)
Assigned to: Richard Guenther (richi)
Summary: MP3 importer is desperately slow 20MB/6h on 500MHz/Intel 

Initial Comment:
MP3 importer is desperately slow 20MB/6h on 500MHz/Intel

I use debian and glame0.7 1.1 from CVS

What could I do to speed it up?

----------------------------------------------------------------------

>Comment By: Richard Guenther (richi)
Date: 2005-01-09 14:55

Message:
Logged In: YES 
user_id=7575

Won't fix this, if it is still a problem.  Closed.

----------------------------------------------------------------------

Comment By: Emanoil Kotsev (deloptes)
Date: 2004-04-18 19:24

Message:
Logged In: YES 
user_id=944112

sounds reasonable to me but right now I don't have time to
test it sorry. I'll write when I am so far. 
I have experienced real desaster with 2.6 so I don't think
I'll use it as base system but I'm thinking of putting it
into a VM.
regards

----------------------------------------------------------------------

Comment By: Emanoil Kotsev (deloptes)
Date: 2004-04-18 19:23

Message:
Logged In: YES 
user_id=944112


sounds reasonable to me but right now I don't have time to
test it sorry. I'll write when I am so far. 
I have experienced real desaster with 2.6 so I don't think
I'll use it as base system but I'm thinking of putting it
into a VM.
regards

----------------------------------------------------------------------

Comment By: Richard Guenther (richi)
Date: 2004-04-18 13:26

Message:
Logged In: YES 
user_id=7575

I just remembered that the slowdown could be starting at the
point where we are forced to start unmapping cached clusters
of the destination file.   Unmapping will cause a msync() on
the mapping and possibly disturb the otherwise fine
writeback operation of the kernel.  2.6 may be more immune
to this problem.

----------------------------------------------------------------------

Comment By: Emanoil Kotsev (deloptes)
Date: 2004-03-06 07:47

Message:
Logged In: YES 
user_id=944112

Hi

I've got a new experience with glame and my problem that it
was slowing down and guess what?!
It's not only my problem but it relates to everybody using
kernel 2.4.X and compiling on ext3.
I don't know why but it's not working properly some people
mentioned so I compiled glame on ext2  and now a slow down
apperas at about X=1000 before was about X=250
I tried also using swap on ext2 but it makes no difference

do have any useful advice except reconcidering comilations
on ext3?

similar problems
http://www.ussg.iu.edu/hypermail/linux/kernel/0212.3/0112.html

converting ext3 to ext2
http://www.ussg.iu.edu/hypermail/linux/kernel/0212.3/0115.html

----------------------------------------------------------------------

Comment By: Emanoil Kotsev (deloptes)
Date: 2004-02-07 22:00

Message:
Logged In: YES 
user_id=944112

Hi, richi,
I wanted just to report, that I have installed  kernel
2.4.24 and full alsa support , though nothing changes. I
still have this slow down!

If you hear something about this problem pls, forward it to
me - I can not believe I am the onlyone with this kind of
problem.

I also dont have any idea what could be the issu. I notice
that it slows down as it writes to disk (it's normal) but
afterwords it goes on with the same slow speed (doesn't get
back to the speed before). That's strange, cause I don't see
reason to go on reading on lower speed.

There is the last performance picture atached too.

regards

regards

----------------------------------------------------------------------

Comment By: Richard Guenther (richi)
Date: 2004-01-16 22:44

Message:
Logged In: YES 
user_id=7575

Sorry, I'm out of ideas.  Its very hard to tell what is
going wrong as long as I cannot reproduce the problem. 
Maybe it's a kernel I/O scheduler problem - I never imported
such a large file before using 2.4.23.

Richard.

----------------------------------------------------------------------

Comment By: Emanoil Kotsev (deloptes)
Date: 2004-01-14 01:42

Message:
Logged In: YES 
user_id=944112

Hi Richi,

me again. I've expanded my memory to 643388 and tested glame
on a ext2 partition with 7.2GB free space with the same result.
The file I tried was 70104609B 

In each case it slows down when x is about 200 what did you
say x was - the time I think it would be nice to have the
status of some other parameters in this case. What I
followed up was the CPU time. When the speed is modal it's
about 1/2 of the time that we need to work a frame out.
In the beginning it was permanently going down. Now it
starts to slow down at a brake point. So the question is
what is happening at this point. 
You could see that the CPU time starts to grow. What could
be consuming this CPU time?

I can't believe that it happens only on my machine.
I may try it in my office (but don't tell anybody :-).
The machine there is AMD7 1800. 
I have here only Pentium III Intel 500 on a legendary ASUS
P3BF mainboard - 
I'm sorry but I don't know the transfer rate of the HD

I use for now the standard version of the kernel 2.4.18-686
that comes along with the debian stable distro but I have
also 2.4.20 compiled and installed. Unfortunately I didn't
have time to test it. I don't think I'll be able to try it
until end of january.

So what do next to help glame?

regards

regards

Neither in dmsg nor in the sysem logs I see some evidence
for unusual events.

I hope this information can help you to decide what we
should next.
Have you ever heard of somebody having similar problem, when
working on such large files?

I think the best solution to split my big mp3's is to write
a script for it, or do you know somoe other way to do it?
I can write the script its no problem -but it would be nice
to can edit the samples in a gui like glame.
I am a bit disapointed that I spent about two hours on
looking for a program with similiar capabilities to Cubase
for linux, but I couldn't find any.
That's how the story started :-)





----------------------------------------------------------------------

Comment By: Richard Guenther (richi)
Date: 2004-01-12 09:54

Message:
Logged In: YES 
user_id=7575

Ok, I see, in your case it's slowing down a lot.  But as I
can't reproduce such dramatic slowdown it's hard for me to
track this down.  Rather than looking for a bug in glame,
I'd look at differences of our systems, like
- what journalling mode are you running your ext3
filesystem? (I'm running in ordered mode)
- how much space is left on the partition the swapfile
resides on (Its about 50% full for me, about 20Gb left)
- which kernel are you running? (I'm running 2.4.23)

because the graph looks like it's doing I/O very slowly at
the end.  Can you also check 'dmesg' for any kernel output
on maybe errors on you IDE bus?

Oh, there is no mp3 exporter, because we don't have very
much time to spend on glame.

Thanks,
Richard.

----------------------------------------------------------------------

Comment By: Emanoil Kotsev (deloptes)
Date: 2004-01-12 02:19

Message:
Logged In: YES 
user_id=944112

I dont need words now better look at the picterus I have
uploaded.

I have 4 files
1.8
2.8
9.
and
70 MB
I upload the plot from the last one ...

please respond

----------------------------------------------------------------------

Comment By: Emanoil Kotsev (deloptes)
Date: 2004-01-10 21:58

Message:
Logged In: YES 
user_id=944112

I see well unless I have some more free time I could spent
on glame can you tell me why there is no mp3 export option I
think it won't be  that difficult to implement as you can
import (ie you have already functions)?

I am excited if it will work (with plot too :-)

tanx

----------------------------------------------------------------------

Comment By: Richard Guenther (richi)
Date: 2004-01-10 19:01

Message:
Logged In: YES 
user_id=7575

Just ignore the v1_0 stuff, CVS HEAD is ok, too. But as
usually, anonymous CVS access on sourceforge lags behind 24
hours, I think... :/ So you just need to wait some time
until the changes to importexport.c propagate.

gnuplot is easy to use, just start it and type
plot "performance"

Richard.

----------------------------------------------------------------------

Comment By: Emanoil Kotsev (deloptes)
Date: 2004-01-10 18:41

Message:
Logged In: YES 
user_id=944112

yes exactly - I copy-pasted and something went wrong, sorry,
but that was the point
with this correction it seems to work a bit faster now, but
not really after the sample size goes up

I didn't get what you meant by updateing teh v1_0 version ? 

I downloaded from cvs, compiled and nothing regarding
performance is saved in the performance file after I run, 

gui/glame | tee performance

I compile with
	--prefix=/opt/glame 	--enable-dependency-tracking 	--enable-debug 	--enable-swapfiledebug 	--disable-ladspa

and I have libmad 15 in opt/glame, 

Besides I can not use gnuplot :-((

What should I do I couldn't follow you!

----------------------------------------------------------------------

Comment By: Richard Guenther (richi)
Date: 2004-01-10 18:15

Message:
Logged In: YES 
user_id=7575

You are partly right, but it should read
GLAME_IO_BUFSIZE*(1+(SAMPLE_SIZE*(ie->mad_pos+pcm->length))/GLAME
_IO_BUFSIZE));

Anyway, I removed that code from CVS.

----------------------------------------------------------------------

Comment By: Emanoil Kotsev (deloptes)
Date: 2004-01-10 18:05

Message:
Logged In: YES 
user_id=944112

I fixed your code and let it work - I'll try the new one
later I thing you've just forgotten to multiply my
SAMPLE_SIZE when truncating.
		sw_ftruncate(ie->mad_swfds[i],
		
GLAME_IO_BUFSIZE*(1+(ie->mad_pos+(SAMPLE_SIZE*pcm->length))/GLAME_IO_BUFSIZE));
it was
GLAME_IO_BUFSIZE*(1+(ie->mad_pos+pcm->length)/GLAME_IO_BUFSIZE));
Thats why the part in the middle of the audio  stream was
missing

regards


----------------------------------------------------------------------

Comment By: Richard Guenther (richi)
Date: 2004-01-10 16:45

Message:
Logged In: YES 
user_id=7575

Oh, forgot that x-axis is the importing time in seconds and
y-axis is the number of samples imported sofar. So it's 10
minutes for a 100mb mp3 resulting in a 2.2Gb swapfile (so a
writing speed of ~3.8MB/s which is not great, but not too
bad either compared to a raw linear disk writing speed of
22MB/s).

----------------------------------------------------------------------

Comment By: Richard Guenther (richi)
Date: 2004-01-10 16:38

Message:
Logged In: YES 
user_id=7575

I checked in a merge to the v1_0 branch version of the
importer (removes the truncate stuff) and added a
performance monitor.  Running the importer on a 100MB MP3
file (283.229.568 samples) and plotting the resulting data
you can see nearly perfectly constant performance:
http://www.tat.physik.uni-tuebingen.de/~rguenth/performance.png

please produce a similar graph for your case, I was using
debug enabled glame and produced the graph with

> ./gui/glame | tee performance
(edit performance to only contain the data)
use gnuplot for the graph.

Richard.

----------------------------------------------------------------------

Comment By: Emanoil Kotsev (deloptes)
Date: 2004-01-10 06:15

Message:
Logged In: YES 
user_id=944112

I wrote some pice of code into the importexort.c file that I
would like you to see.
It tracks the time consumed by the cpu and the samples
worked out.

If you try it you would see what I meen and you can follow
me up. The time consumed by the cpu seems to get higher as
higher the sample size is.

After size of appr.30 it takes 10% more time to work the
samples out as in the beginning - why?

The reason why you didn't find it is because it is not
tested with that large files/sample sizes I guess.
It starts with about 250000 samples per second or what ever
mad_pos is standing for. Some wher at about 10418688 slows
down to 190000 and at about 35000000 it is very slow.
Averige cpu time is increesing (per second). Please have a
look at the atached file and try it out you can use it for
some kind of feed back unless the algorythm I used is wrong,
in this case forive .

I think in the name of all of of us somebody should do
something. as you can see from my code I am not a C
profecional but there are many way to do things . some of
them are slower than others with the same effect :-))

I have a Intel 500MHz with about 380MB. I set the size of
the memmory/swap at 128 and the audio buffer to 2048
I dont know if it makes sense to you but tehre is definitely
a bug
 
Please responde!

----------------------------------------------------------------------

Comment By: Emanoil Kotsev (deloptes)
Date: 2004-01-10 05:36

Message:
Logged In: YES 
user_id=944112

Ok, so there is no danger if I say something like value as
argument to this function?

----------------------------------------------------------------------

Comment By: Clinton Ebadi (unknown_lamer)
Date: 2004-01-09 17:59

Message:
Logged In: YES 
user_id=13218

The scm_c_export(sym,...) /* nothing */ means that you are running 
Guile 1.4 which does not use scm_c_export so I defined it to a macro 
which does nothing. It worked for me when I was testing my 
modifications to get GLAME to run with both Guile 1.4 and 1.6 a long 
time ago. I was using GCC 3.1 at the time. Variable argument macros 
were a new feature added to C99 and I don't think GCC 2.95 has 
support for them. 
 
GLAME doesn't use scm_c_export with more than two args (at least not 
now) so redefining the macro as: 
 
#define scm_c_export(sym,something) /* not used */ 
 
should work. 

----------------------------------------------------------------------

Comment By: Clinton Ebadi (unknown_lamer)
Date: 2004-01-09 17:54

Message:
Logged In: YES 
user_id=13218

The scm_c_export(sym,...) /* nothing */ means that you are running 
Guile 1.4 which does not use scm_c_export so I defined it to a macro 
which does nothing. It worked for me when I was testing my 
modifications to get GLAME to run with both Guile 1.4 and 1.6 a long 
time ago. I was using GCC 3.1 at the time. Variable argument macros 
were a new feature added to C99 and I don't think GCC 2.95 has 
support for them. 
 
GLAME doesn't use scm_c_export with more than two args (at least not 
now) so redefining the macro as: 
 
#define scm_c_export(sym,something) /* not used */ 
 
should work. 

----------------------------------------------------------------------

Comment By: Emanoil Kotsev (deloptes)
Date: 2004-01-09 12:33

Message:
Logged In: YES 
user_id=944112

Well, sorry, but it didn't help much.
It is now faster, as I don't have to wait that  long for
files upto 27154000 samples.
briefly it works better but not perfectly. The only
difference is, that the limit when it starts to slow down
has gone up. 
I don't know why but for some reason on larger files (more
samples) some when after sample 65000000 it starts to work
very slow resulting in the time intervals shown below. And
this time intervals are growing too as I said. after 2min.
glame has reached the mentioned size of 65000000samples and
starts working 10X slowlier as before so it takes not
minutes but hours to do the rest of the samples (

What did you done, because now it's a bit faster, at least
in the beginning  - can you try improve it much more.

I thinkg the sample size and numbers are relevant for this
issue, but as I said I am not a programmer professional I
could be wrong.

Can you try with sample numbers in the size and above 70000000

And please do something for this sw_truncation it breaks the
imported files It is importing few seconds in the beginning
and few seconds at the end. 
'Here is the piece of code that I commented out so it can work.

1.
===============================
	/* Expand file, if we're going into the next IO sized chunk. */
		if (ie->mad_pos/GLAME_IO_BUFSIZE
			< (ie->mad_pos+pcm->length)/GLAME_IO_BUFSIZE ||
ie->mad_pos == 0) {
/* didn't work	sw_ftruncate(ie->mad_swfds[i],
		
GLAME_IO_BUFSIZE*(1+(ie->mad_pos+pcm->length)/GLAME_IO_BUFSIZE));
*/
			now = 1;
		}

2.
===== importexport.c output
Status :: mad_pos=124531200; pcm_size=1152;dif. was 2;
tot.::32156 sec;
Status :: mad_pos=124532352; pcm_size=1152;dif. was 1;
tot.::32157 sec;
Status :: mad_pos=124533504; pcm_size=1152;dif. was 1;
tot.::32158 sec;

3
======== vstat
   procs                      memory    swap          io   
 system         cpu
 r  b  w   swpd   free   buff  cache  si  so    bi    bo  
in    cs  us  sy  id
 2  0  0 401204   4376  39732 181460   0   0     0     0 
106   388  51  49   0
 3  0  0 401204   4368  39732 181468   0   0     0     0 
105   383  48  52   0
 2  0  0 401204   4356  39740 181472   0   0     0   996
2097   358  50  50   0
 3  0  0 401204   4352  39740 181476   0   0     0     0 
105   353  51  49   0
 2  0  1 401204   4340  39740 181488   0   0     0     0 
106   403  55  45   0
 3  0  0 401204   4460  39732 181380   0   0     0     0 
105   382  50  50   0
 1  0  0 401204   4452  39732 181388   0   0     0     0 
106   380  46  54   0
10  0  0 401204   4444  39740 181392   0   0     0  1140
2423   473  51  48   1
 4  0  0 401204   4436  39740 181396   0   0     0     0 
153   535  65  35   0
 6  0  0 401204   4428  39740 181404   0   0     0     0 
107   718  48  52   0
 5  0  0 401204   4424  39740 181408   0   0     0     0 
105   400  52  48   0
 5  0  1 401204   4424  39752 181416   0   0     0  1048
2239   516  45  55   0
 2  0  0 401204   4396  39752 181424   0   0     0     4 
151   643  53  47   0
 2  0  0 401204   4392  39752 181428   0   0     0     0 
182   770  54  46   0

Status :: mad_pos=124534656; pcm_size=1152;dif. was 1;
tot.::32159 sec;
Status :: mad_pos=124535808; pcm_size=1152;dif. was 1;
tot.::32160 sec;
Status :: mad_pos=124536960; pcm_size=1152;dif. was 2;
tot.::32162 sec;
Status :: mad_pos=124538112; pcm_size=1152;dif. was 1;
tot.::32163 sec;
Status :: mad_pos=124539264; pcm_size=1152;dif. was 1;
tot.::32164 sec;
Status :: mad_pos=124540416; pcm_size=1152;dif. was 1;
tot.::32165 sec;
Status :: mad_pos=124541568; pcm_size=1152;dif. was 2;
tot.::32167 sec;
Status :: mad_pos=124542720; pcm_size=1152;dif. was 1;
tot.::32168 sec;
Status :: mad_pos=124543872; pcm_size=1152;dif. was 1;
tot.::32169 sec;
Status :: mad_pos=124545024; pcm_size=1152;dif. was 1;
tot.::32170 sec;
Status :: mad_pos=124546176; pcm_size=1152;dif. was 1;
tot.::32171 sec;
Status :: mad_pos=124547328; pcm_size=1152;dif. was 2;
tot.::32173 sec;
Status :: mad_pos=124548480; pcm_size=1152;dif. was 1;
tot.::32174 sec;
Status :: mad_pos=124549632; pcm_size=1152;dif. was 2;
tot.::32176 sec;
Status :: mad_pos=124550784; pcm_size=1152;dif. was 2;
tot.::32178 sec;

----------------------------------------------------------------------

Comment By: Richard Guenther (richi)
Date: 2004-01-08 20:31

Message:
Logged In: YES 
user_id=7575

Ok, I checked with a 60MB mp3 file and added progress report
and cancel ability to the stable glame version.  Performance
of mp3 import is not great, but it is constant speed and not
degrading.  Every some time there is HD activity and the
import stalls for a few seconds, but then goes on with
initial speed.  As I would expect (this is ext3, too, with
ordered journal).  Ogg import is about twice as fast as mp3
import (the library has a cleaner interface and is easier to
do efficient), and of course wav beats them all.

Note that with a 128kbit 60MB mp3 you need to transfer
roughly 1.4GB of data to disk!

Richard.

----------------------------------------------------------------------

Comment By: Richard Guenther (richi)
Date: 2004-01-08 13:43

Message:
Logged In: YES 
user_id=7575

Uh, the truncate was to improve performance by
pre-allocating on the filesystem.  If that makes performance
worse, it's clearly strange...
I'll try to check performance with such large mp3 files
tonight - there must be something else going wrong.
You may try running a "vmstat 1" while importing to see
filesystem activity of the kernel.  How much memory do you
have? You should set the swapfile parameter to not larger
than 3/4 of your memory, i.e. use 64 for 128MB ram, 196 for
256MB, etc..

Now I still need to create such large mp3... ;)

Richard.

----------------------------------------------------------------------

Comment By: Emanoil Kotsev (deloptes)
Date: 2004-01-08 13:35

Message:
Logged In: YES 
user_id=944112

I am sorry to disappoint you but it still slows down after
mad_pos size hits some limit
I add some time statistics and now I can see whats going
wrong I still don't know why and when but it still works
fine with files up to 30MB
I'll try with different sizes between 30 and 60 because by
18 or 30 it takes 30 sec to convert it. by 60 I hit the
cancel button after the 3minute ;-)

Any Idea?

now another thing what did you do with this truncation there
- did you see my snapshot?

I commented it out! It doesnt improves the speed much but
the progress bar work better now. i added something that it
can start imediately.

here some  - it looks like this now
~~~~~~~~~~~~~~~~~~~~~~~
	/* Expand file, if we're going into the next IO sized chunk. */
		if (ie->mad_pos/GLAME_IO_BUFSIZE
			< (ie->mad_pos+pcm->length)/GLAME_IO_BUFSIZE ||
ie->mad_pos == 0) {
/*			sw_ftruncate(ie->mad_swfds[i],
		
GLAME_IO_BUFSIZE*(1+(ie->mad_pos+pcm->length)/GLAME_IO_BUFSIZE));
*/
			now = 1;
		}
~~~~~~~~~~~~~~~~~~~


And improves the speed by files the size I mentioned.

please do something 

and thanks

----------------------------------------------------------------------

Comment By: Richard Guenther (richi)
Date: 2004-01-08 09:27

Message:
Logged In: YES 
user_id=7575

I think this all looks like issues of a full filesystem
maybe?  Also changing the swapfile settings from 256 to 512
doesnt change the swapfile size on disk, but rather the
amount of memory used for caching.

And the "badly punctuated parameter list..." I never saw -
what compiler are you using?

Otherwise from your postings can I conclude that performance
is now fine ?

Thanks,

Richard.

----------------------------------------------------------------------

Comment By: Emanoil Kotsev (deloptes)
Date: 2004-01-08 06:35

Message:
Logged In: YES 
user_id=944112

I had to rever to the older version of
src/gui/importexport.c see the picture attached. I use now 
v1.30 with the time functions I see
ie_import_mp3_output: mad_pos=129962880; pcm=1152; 0 sec;
tot.: 1426 sec;
time consumed is below 0sec. I wonder how I can see the
miliseconds? didn't have time to do this

here is the difference
~~~~~~~~~~~~~~~ DIFF OUTPUT
3c3
<  * $Id: importexport.c,v 1.30 2003/07/08 20:29:39 richi Exp $
---
>  * $Id: importexport.c,v 1.32 2004/01/06 23:16:51 richi Exp $
45d44
< #include <mad.h>
49,51c48,51
< /* char m;
< double to; */
< //EKO
---
> char m;
> double to;
> //EKO */
> #include <mad.h>
59c59
<
---
>
61c61
< static char *sflabel[] = { "16 bit signed", "24 bit
signed", "32 bit signed",
---
> static char *sflabel[] = { "16 bit signed", "24 bit
signed", "32 bit signed",
115a116,119
>       int mad_channels;
>       swfd_t *mad_swfds;
>       int mad_bufsize;
>       char *mad_buf;
211c215
<                                   GLAME_BULK_BUFSIZE,
---
>                                   GLAME_IO_BUFSIZE,
259c263
<                                                          
struct mad_frame *frame)
---
>                   struct mad_frame *frame)
283,284d286
<
<
290d291
<       SAMPLE *buf;
293,294d293
<       //EKO
<       t1 = time(NULL);
298,299d296
<               /* EKO */
<               DPRINTF("ooch error mad_pos=%i",ie->mad_pos);
303c300
<       //      DPRINTF("mad_pos=%i \n",ie->mad_pos);
---
>
306a304,305
>               ie->mad_swfds = malloc(pcm->channels *
sizeof(swfd_t));
>               ie->mad_channels = pcm->channels;
318a318,319
>                       ie->mad_swfds[i] =
sw_open(gpsm_swfile_filename(swfile), O_RDWR);
>                       sw_ftruncate(ie->mad_swfds[i],
GLAME_IO_BUFSIZE);
322c323,330
<       buf = malloc(SAMPLE_SIZE*pcm->length);
---
>       if (!ie->mad_buf
>           || ie->mad_bufsize < pcm->length) {
>               if (ie->mad_buf)
>                       free(ie->mad_buf);
>               DPRINTF("mad buffer size %i samples\n",
pcm->length);
>               ie->mad_buf = malloc(SAMPLE_SIZE*pcm->length);
>               ie->mad_bufsize = pcm->length;
>       }
326,330c334,342
<               SAMPLE *b = buf;
<               gpsm_swfile_t *file =
gpsm_find_swfile_vposition((gpsm_grp_t *)ie->item, NULL, i);
<               swfd_t fd;
<               fd = sw_open(gpsm_swfile_filename(file),
O_RDWR);
<               sw_lseek(fd, 0, SEEK_END);
---
>               SAMPLE *b = (SAMPLE *)ie->mad_buf;
>
>               /* Expand file, if we're going into the next
IO sized chunk. */
>               if (ie->mad_pos/GLAME_IO_BUFSIZE
>                   <
(ie->mad_pos+pcm->length)/GLAME_IO_BUFSIZE) {
>                       sw_ftruncate(ie->mad_swfds[i],
>                                   
GLAME_IO_BUFSIZE*(1+(ie->mad_pos+pcm->length)/GLAME_IO_BUFSIZE));
>                       now = 1;
>               }
335,351c347,348
<               sw_write(fd, buf, SAMPLE_SIZE*pcm->length);
<               sw_close(fd);
<               now = 1;
<       }
<       free(buf);
<
<               //EKO
<       t2 = time(NULL);
<
<       td = t2 - t1;
<       tt = t2 - ts;
<       //EKO
<       ie->mad_pos += pcm->length;
<
<     DPRINTF("mad_pos=%i; pcm=%i; %s sec; tot.: %s sec;\n",
<                       ie->mad_pos,pcm->length,char
*(td),char *(tt));
<       //EKO
---
>               sw_write(ie->mad_swfds[i], ie->mad_buf,
SAMPLE_SIZE*pcm->length);
>       }
358d354
<
368c364
<       int fd;
---
>       int fd, i;
370d365
<       char *buf;
373,374d367
<       //EKO
<       tt = td = 0;
389a383,384
>       ie->mad_buf = NULL;
>       ie->mad_bufsize = 0;
402c397,404
<       munmap(buf, s.st_size);
---
>       for (i=0; i<ie->mad_channels; ++i) {
>               sw_ftruncate(ie->mad_swfds[i],
sw_lseek(ie->mad_swfds[i], 0, SEEK_CUR));
>               sw_close(ie->mad_swfds[i]);
>       }
>       if (ie->mad_buf)
>               free(ie->mad_buf);
>
>       munmap(ie->mad_buffer, s.st_size);
462,463d463
<               // EKO stat time that was consumed for importing
<               ts = time(NULL);
575c575
<         while (gtk_events_pending())
---
>               while (gtk_events_pending())
623c623
< static void ie_update_plabels(struct imp_s *ie)
---
> static int ie_update_plabels(struct imp_s *ie)
642,643c642,643
<       } else
<               sprintf(buffer, "nan");
---
>       } else
>               return -1;
686a687,688
>
>       return 0;
758c760
<         while (gtk_events_pending())
---
>               while (gtk_events_pending())
830a833,839
>               ie->gotstats = 0;
>              
gtk_label_set_text(GTK_LABEL(ie->fi_plabel[0]), "unknown");
>               for(i=1; i<MAX_PROPS; i++)
>                      
gtk_label_set_text(GTK_LABEL(ie->fi_plabel[i]), "-");
>               gtk_widget_set_sensitive(ie->checkresample,
FALSE);
>               gtk_widget_set_sensitive(ie->rateentry, FALSE);
>               gtk_widget_set_sensitive(ie->getstats, FALSE);
837c846,847
<       if (strcmp(mimetype, "audio/x-mp3") == 0) {
---
>       if (strcmp(mimetype, "audio/x-mp3") == 0
>           || strcmp(mimetype, "audio/mpeg") == 0) {
848c858,859
<       /* check if its an mp3 */
---
>
>       /* check if its an ogg */
850c861,862
<           || strcmp(mimetype, "application/x-ogg") == 0) {
---
>           || strcmp(mimetype, "application/x-ogg") == 0
>           || strcmp(mimetype, "application/ogg") == 0) {
860a873
>       g_free(mimetype);
862c875,881
<       if (strncmp(mimetype, "audio/", 6) != 0) {
---
>       /* check if it's not recognized by the audio_in
plugin */
>       fparam =
filterparamdb_get_param(filter_paramdb(ie->readfile),
"filename");
>       filterparam_set(fparam, &(ie->filename));
>       ie_update_plabels(ie);
>       if (ie_update_plabels(ie) == -1) {
>               ie->gotfile = 0;
>               ie->gotstats = 0;
868d886
<               g_free(mimetype);
871d888
<       g_free(mimetype);
873,876d889
<       fparam =
filterparamdb_get_param(filter_paramdb(ie->readfile),
"filename");
<       filterparam_set(fparam, &(ie->filename));
<
<       ie_update_plabels(ie);
1375c1388
<         while (gtk_events_pending())
---
>               while (gtk_events_pending())


----------------------------------------------------------------------

Comment By: Emanoil Kotsev (deloptes)
Date: 2004-01-08 03:38

Message:
Logged In: YES 
user_id=944112

What does scm_c_export(sym, ...) /* nothing */ means?

I've got error 
In file included from ../../src/glmid/glscript.h:31,
                 from ../../src/glmid/glconfig.h:31,
                 from main2.cpp:49:
../../src/include/glame_guile_compat.h:78: badly punctuated
parameter list in `#define'
make[4]: *** [main2.o] Error 1


----------------------------------------------------------------------

Comment By: Emanoil Kotsev (deloptes)
Date: 2004-01-08 00:25

Message:
Logged In: YES 
user_id=944112

I've changed back to
	if (now)
gtk_progress_bar_pulse(gnome_appbar_get_progress(GNOME_APPBAR(ie->appbar)));

	while (gtk_events_pending())
		gtk_main_iteration();

I think it was some other kind of problem.
I tried deleting the swapdirectory and let it create again,
so now the speed is constant - still not that fast but at
least constant

I also changed the swapfile settings to 512 it was 256
before and the output buffer for the audio I/O settings to 4096

I added a time checking function and so my presumptions
could be proved - I respect numbers :-)

And yes it starts with time from 0.00-0.01 then goes to
0.01-0.02 and so on. I think it the block pasted below or
perhaps the mpr_input function no idea...
when mad_pos is at about 1000000 it takes 0.04-0.05sec cpu time

I'll start believing in ghost - could it be that my cpu
clock has something to do with it.
I found out when I'm ripping my clock has delay upto 1/2 hour.

thanks, regards
	//EKO
	t1 = clock();
	for (i=0; i<pcm->channels; ++i) {
		unsigned int nsamples = pcm->length;
		mad_fixed_t const *data = pcm->samples[i];
		SAMPLE *b = (SAMPLE *)ie->mad_buf;

		/* Expand file, if we're going into the next IO sized
chunk. */
		if (ie->mad_pos/GLAME_IO_BUFSIZE
		    < (ie->mad_pos+pcm->length)/GLAME_IO_BUFSIZE) {
			sw_ftruncate(ie->mad_swfds[i],
				    
GLAME_IO_BUFSIZE*(1+(ie->mad_pos+pcm->length)/GLAME_IO_BUFSIZE));
			now = 1;
		}

		while (nsamples--)
		    *b++ = mad_f_todouble(*data++);

		sw_write(ie->mad_swfds[i], ie->mad_buf,
SAMPLE_SIZE*pcm->length);
	}
	//EKO
	t2 = clock();
	td = (t2-t1)/(double)CLOCKS_PER_SEC;
	tt += td;
	//EKO
	ie->mad_pos += pcm->length;
	//EKO
    DPRINTF("mad_pos=%i; pcm=%i; %.4lf sec; tot.: %.4lf sec;\n",
             ie->mad_pos,pcm->length,td,tt);
	//EKO


The filesystem type 

----------------------------------------------------------------------

Comment By: Daniel Kobras (nold)
Date: 2004-01-07 16:33

Message:
Logged In: YES 
user_id=7832

I've recently read about other audio software experiencing a 
significant performance hit on ext3. In order to eliminate this 
variable in the game, could you try with a swapfile on ext2?

----------------------------------------------------------------------

Comment By: Emanoil Kotsev (deloptes)
Date: 2004-01-07 16:02

Message:
Logged In: YES 
user_id=944112

I don't know what's going on!
My libmad is libmad-0.15.0b
My filesystem is ext3, where the swapfile is
Should I make my swapfile bigger it's 256MB now.

What could be reasonable time to import let's say 60MB mp3
on Intel 500MHz and what could be reasonable swapfilesize
for working with few files of that size?

P.S.
The mp3 importer starts well but when mad_pos reaches value
60000 and something it starts to slow down and remains on
some level till it's done.



----------------------------------------------------------------------

Comment By: Richard Guenther (richi)
Date: 2004-01-07 14:46

Message:
Logged In: YES 
user_id=7575

Ok, that is certainly interesting.  What makes it slower
after a while is of course the operating system starting to
write out things into the swapfile.  I changed allocation
policy somewhat to speed that up.  But it was nowhere near
6h for 20MB mp3, rather about 30sec for a 8MB mp3 on my
600MHz iBook.  It may be a disk speed issue (what filesystem
is your swapfile residing on? It's on ext2 for me).

The progress bar update change was a quick hack - it was
updated every mp3 frame which means very much too often,
I'll need to add logic to update only, say, every 1/2 second
(and also check for gtk events only then).

You can also try profiling (I did so, yesterday, but only
the mp3 importing routine showed up, and I didn't bother
setting up oprofile or something like that).

Richard.

----------------------------------------------------------------------

Comment By: Emanoil Kotsev (deloptes)
Date: 2004-01-07 14:03

Message:
Logged In: YES 
user_id=944112

What makes you think it's the gtk_progress_bar_pulse

It is not working as what you are doing can be specified as
"cosmetic change"
you are updating the progress bar less frequently. Well, I
don't mind. If you could see it you should think what I
thought. Something is incrementally slowing the program
down. One indicator was the progress bar and I added another one

I did following after the line
	ie->mad_pos += pcm->length;
I added
        DPRINTF("mad_pos=%i \n",ie->mad_pos);
I found out that if I disable the gt_progress_bar there is
no change in the way the mad_pos is printed out. It starts
fast and goes slowlier afterwords.
Then I commented out
        //while (gtk_events_pending())
mad_pos is growing with stable speed and is faster than if
this line is not commented out if I comment out the next line
       //	gtk_main_iteration();
it's even faster but at some point it starts to grow slowier
and slowier.
This way I don't see the updates of the gui but the program
achieves the best results I think.

Another indicator is the led for the HDD activity - it is
indicating activity more frequently after I run the importer
and less frequently afterwords.

So the quetsion is: What could it be thats making the
program run slowlier.

Can it be that I need a bigger swap my one is 256 or that
there might be errors in the stream

regards & thanks

----------------------------------------------------------------------

Comment By: Richard Guenther (richi)
Date: 2004-01-06 23:00

Message:
Logged In: YES 
user_id=7575

The
ie_import_mp3_error: MAD error pos=0: lost synchronization
is harmless - it's ignored (should be an ID3 tag or the like).

Swap debug output is not needed here, also, what libmad
version do you have? I have
ii  libmad0          0.15.0b-3        MPEG audio decoder library

Also I have now committed a little optimization to CVS, can
you update and try again?

Thanks,

Richard.

----------------------------------------------------------------------

Comment By: Emanoil Kotsev (deloptes)
Date: 2004-01-06 21:29

Message:
Logged In: YES 
user_id=944112

I use debian woody stable and you are right glame is using
gnome2
And the glame package is from the CVS, I downloaded
yesterday - I type 
emanoil@bendida:~/tmp$ /opt/glame/bin/glame --version
Gnome glame0.7 1.1CVS
I tried to import a mp3 I created with xmms (it's working
there fine) but glame says
ie_import_mp3_error: MAD error pos=0: lost synchronization
Unfortunately I can convert the MP3-files into wav's and
import them with no delay, but it takes time and place to
convert them, and afterwords I have to wait exporting them
again into MP3's

I compiled glame with debug and swap debug options so I have
a lot of ouput. How should I submit this information to the
glame team?
It is the first time I am taking serious advantage of the
bug-tracking of sf.
I think I'll read the docs?

Thanks for the fast response, I opreciate it!

Regards

----------------------------------------------------------------------

Comment By: Richard Guenther (richi)
Date: 2004-01-06 21:06

Message:
Logged In: YES 
user_id=7575

I changed this to be a bug, but I like to know more about
your system. Is this debian unstable? And what exactly is
glame0.7 1.1 from CVS? I suppose this is CVS HEAD with
gnome2? Then I can imagine what is going wrong here...

Can you try commenting out the gtk_progress_bar_pulse()
calls in src/gui/importexport.c and look if this helps?

Thanks,

Richard.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=101627&aid=871787&group_id=1627