njbfs-devel — Development List - (High Traffic)

Re: block size (Re: [njbfs-devel] branches, bearshare, where to go next)

[Ben O. <be...@gi...>]

Should we publish a 0.2.1 update with the source tree as it stands?  The
track listings and the 2.4.9 compilation fixes seem pretty important to
me...

-ben


On Fri, 15 Feb 2002, [iso-8859-15] J=F6rg Prante wrote:

> Danish Qadri wrote:
> > One of our biggest
> > problems is how slow the USB interface. BTW, a separate question... if =
you
> > had an external USB hard drive, is the transfer rate fast enough to
> > support Winamp/XMMS directly? I will investigate this issue though, and
> > see if an intelligent read ahead is possible.
>
> It works: I stream directly into xmms. But there are the buffer underruns=
 I
> mentioned.
>
> I just looked in the 0.2 sources, which I already dropped, and I didn't
> publich the fix for the speed. It took me a long time to tune it. The fix=
 for
> this is to increase the block size for read/write.
>
> Have a look at njbfs.h
>
> #define NJBFS_BLOCKSIZE 1024
> #define NJBFS_BLOCKSIZE_BITS 10
>
> If you change this to
>
> #define NJBFS_BLOCKSIZE 4096
> #define NJBFS_BLOCKSIZE_BITS 12
>
> which is the maximum page size for I/O in Linux, the speedup is significa=
nt.
>
> J=F6rg
>
> _______________________________________________
> njbfs-devel mailing list
> njb...@li...
> https://lists.sourceforge.net/lists/listinfo/njbfs-devel
>

block size (Re: [njbfs-devel] branches, bearshare, where to go next)

[<joe...@gm...>]

Danish Qadri wrote:
> One of our biggest
> problems is how slow the USB interface. BTW, a separate question... if you
> had an external USB hard drive, is the transfer rate fast enough to
> support Winamp/XMMS directly? I will investigate this issue though, and
> see if an intelligent read ahead is possible.

It works: I stream directly into xmms. But there are the buffer underruns I 
mentioned.

I just looked in the 0.2 sources, which I already dropped, and I didn't 
publich the fix for the speed. It took me a long time to tune it. The fix for 
this is to increase the block size for read/write.

Have a look at njbfs.h

#define NJBFS_BLOCKSIZE 1024
#define NJBFS_BLOCKSIZE_BITS 10

If you change this to

#define NJBFS_BLOCKSIZE 4096
#define NJBFS_BLOCKSIZE_BITS 12

which is the maximum page size for I/O in Linux, the speedup is significant.

Jörg

Re: [njbfs-devel] branches, bearshare, where to go next

[Ben O. <be...@gi...>]

On Fri, 15 Feb 2002, Danish Qadri wrote:

>
>
> > -Writes, esp with consideration to creating files, auto-recognizing id3
> > tags, and creating a more transparent writing system.  Jorge, you had
> > mentioned (in some documentation somewhere) that you were working on a
> > system to cache the file in the kernel when it's written and commit it
> > when its done... any progress on that?
>
> I was definitely thinking that the above is a good idea. It will
> definitely increase our memory usage (if we want to buffer the entire
> file, then significantly), but otherwise would be a useful feature.

If the nomad is as good with writes as it is with reads, we may have to
buffer the entire file for even a small change =)

Another, simpler option might be to not allow file changes with the
exception of changes to the mp3 tag (which we would translate into nomad
tag changes).  We could allow create & delete but not modify.  Although
I'm not sure that's possible with VFS.


> >From what you mentioned before, the NJB doesn't like seeking, so I'm not
> sure what possibilities we have for a read ahead. One of our biggest
> problems is how slow the USB interface. BTW, a separate question... if you
> had an external USB hard drive, is the transfer rate fast enough to
> support Winamp/XMMS directly? I will investigate this issue though, and
> see if an intelligent read ahead is possible.

Yeah, I've been thinking about this.  The only solution I've come up with
is:

-VFS requests a read
-We do the read as requested, and set a timer.
-The timer calls a read ahead function that gets some blocks and stores
them in a buffer.
-The next time VFS requests a read, it's already in our buffer, so we can
just return that without requesting another read from the jukebox.


the problem with this is, I don't know if you can get to user context
(where we can do a usb read without blocking the kernel) with a timer.
There also looks like there's some hope in usb "urbs" (looks like
asynchronous usb i/o) but I don't fully understand how they work yet.

> >From what I've seen (visually anyway), if there is a bulk error, then the
> jukebox slows down to about 5 or 6 K/sec, and kicks back up to the full
> speed with the next file (possibly because of another
> njb_usb_handshake? If so, then we can make it force a new handshake only
> if the previous fn had an error)

I haven't come accross the bulk error condition yet.  I just know that a
bulk timeout sometimes takes a looong time.  Maybe it's just 2.4.9.

> I would say that we should focus on the xmms problem, since it's a much
> more widely used app. I'm also assuming that this issue might extend to
> other software mp3 players?

I haven't come across it with xmms/musicmatch/winamp, but maybe something
I did fixed it.  ;)


Where are you from, anyway?

-ben

Re: [njbfs-devel] branches, bearshare, where to go next

[Danish Q. <dx...@nj...>]

> -Writes, esp with consideration to creating files, auto-recognizing id3
> tags, and creating a more transparent writing system.  Jorge, you had
> mentioned (in some documentation somewhere) that you were working on a
> system to cache the file in the kernel when it's written and commit it
> when its done... any progress on that?

I was definitely thinking that the above is a good idea. It will
definitely increase our memory usage (if we want to buffer the entire
file, then significantly), but otherwise would be a useful feature. 
 
> -Improving multi-file access, to make it somewhat bearable.  Maybe a
> scheduling mechanism as Danish suggested, maybe some kind of intelligent
> read-ahead.

From what you mentioned before, the NJB doesn't like seeking, so I'm not
sure what possibilities we have for a read ahead. One of our biggest
problems is how slow the USB interface. BTW, a separate question... if you
had an external USB hard drive, is the transfer rate fast enough to
support Winamp/XMMS directly? I will investigate this issue though, and
see if an intelligent read ahead is possible.

> -looking into making the jukebox timeout quicker... on my system it takes
> almost a minute to timeout if a bulk read fails... which basically makes
> the whole filesystem lock up.

From what I've seen (visually anyway), if there is a bulk error, then the
jukebox slows down to about 5 or 6 K/sec, and kicks back up to the full
speed with the next file (possibly because of another
njb_usb_handshake? If so, then we can make it force a new handshake only
if the previous fn had an error)

> -looking more at seeks and seeing if there's a way to force the nomad to
> give the offset we want, not the one it wants to give us.

> Any suggestions on where to spend time?  If I could reproduce either the
> midnight commander error or the xmms error, I'd look trying to fix them.

I would say that we should focus on the xmms problem, since it's a much
more widely used app. I'm also assuming that this issue might extend to
other software mp3 players?


Sorry for my lack of input/participation recently. It seems like time is
moving much too fast for me :).


- Danish

[njbfs-devel] question

[Ben O. <be...@gi...>]

Is there a reason we're calling a njb_usb_handshake on every call to
njb_usb_open?  The handshake functions seems to add about 1-3 seconds onto
every call.

-ben

[njbfs-devel] branches, bearshare, where to go next

[Ben O. <be...@gi...>]

Hi guys,

I've got the njbfs to a reasonably stable place on my system, so I think
I'm going to go ahead and branch the source tree (I know, overkill) to get
some revision control happening before I start hacking on the next thing.

Currently I've got bearshare (windows gnutella client) running... it does
a hash of every file on the jukebox, and has to read in fully every file
on the jukebox.  Sharing files was actually working, but because its in
the hashing phase, totally inefficient.

Here's the few things I've been thinking of working on:

-Writes, esp with consideration to creating files, auto-recognizing id3
tags, and creating a more transparent writing system.  Jorge, you had
mentioned (in some documentation somewhere) that you were working on a
system to cache the file in the kernel when it's written and commit it
when its done... any progress on that?

-Improving multi-file access, to make it somewhat bearable.  Maybe a
scheduling mechanism as Danish suggested, maybe some kind of intelligent
read-ahead.

-looking into making the jukebox timeout quicker... on my system it takes
almost a minute to timeout if a bulk read fails... which basically makes
the whole filesystem lock up.

-looking more at seeks and seeing if there's a way to force the nomad to
give the offset we want, not the one it wants to give us.


Any suggestions on where to spend time?  If I could reproduce either the
midnight commander error or the xmms error, I'd look trying to fix them.

-Ben

[njbfs-devel] xmms

[Ben O. <be...@gi...>]

After a long day/night playing with xmms, I've figured out a few things
about xmms.

1)  xmms uses the exact same mechanism that I've been using in file seeks;
instead of just doing an lseek to the offset it needs, it reads up until
the point it needs to get to, thus getting it into the linux page buffer.
I think this is why seeks were succeeding in xmms; they weren't actually
seeks to us, but sequential reads.

2) I'm not too fond of xmms. =)

3) There seems to be no way to fool xmms into not opening a file when you
add it to the playlist... it always wants to read the mp3 header, not just
the id3 header.  So adding a file to the playlist while playing a track
will always cause interruptions (unless we return -EBUSY, but I'm not sure
this is the best thing to do, seeing how applications leave files open all
the time).  Coaxing linux into doing bigger read-aheads could help this
problem, maybe.

4) I was unable to reproduce J's buffer underruns at the beginning of
files.  There's a bit of fuzziness sometimes when I play, but that's it.

5) I don't think seeks work right, still.  I think that xmms gets the
correct tag from the id3v2 header, and ignores the bad data it gets when
it tries to seek to the end.   I'll try to test this theory tommorow.

-Ben

Re: [njbfs-devel] njb issues

[Danish Q. <dx...@nj...>]

You could always create another branch in CVS to play with. If you haven't
played much with branches, there's more info in the docs at cvshome.org. I
can try creating it but I've never really had to branch code before, so it
would be new for me as well.

- Danish



- Danish


On Tue, 12 Feb 2002, [iso-8859-15] J=F6rg Prante wrote:

> Ben Osheroff wrote:
> > My current source is attached;  I'm wondering if there's a better way t=
o
> > share code that you don't want to check into cvs yet?
>
> Thank you, Ben. I will have a look at it the next days. If you have acces=
s to
> a Web site or FTP it would be great to upload your files there and mail a
> link. Or, if you know diff, create a patch against current njbfs sources.
>
> Cheers
> J=F6rg
>
> _______________________________________________
> njbfs-devel mailing list
> njb...@li...
> https://lists.sourceforge.net/lists/listinfo/njbfs-devel
>

Re: [njbfs-devel] njb issues

[<joe...@gm...>]

Ben Osheroff wrote:
> My current source is attached;  I'm wondering if there's a better way to
> share code that you don't want to check into cvs yet?

Thank you, Ben. I will have a look at it the next days. If you have access to 
a Web site or FTP it would be great to upload your files there and mail a 
link. Or, if you know diff, create a patch against current njbfs sources.

Cheers
Jörg

Re: [njbfs-devel] njb issues (fwd)

[Ben O. <be...@gi...>]

> Ben Osheroff wrote:
>
> > -the njb doesn't like reads at arbitrary locations
>
> Are you sure? Well playing with file seeks, I found the njb is just working
> with high latency at random seeks, but it works - xmms gets the right ID3
> tags at random file positions.

Yeah, I've noticed that behavior too... it seems to seek to the end of the
file and read there just fine, but when some arbitrary block in the middle
of the file is requested, a song just starts over from the beginning.
Did you notice that behavior?

> I do not recommend bypassing the VFS. It saves a lot of code that we would
> have to write then on our own,  e.g. dealing with swapping pages. To manage
> the page cache, you could look at the code in other file systems how they
> managed that situation. It should be safe to let the Linux VM allocate pages
> of 4K for our file buffer operations, delegate the low level stuff to the VFS
> and let njbfs deal with higher level operations.

Yup, I've looked into the nfs code.  I don't wanna go there =).  I have
figured out a way to simply call back into the VFS and request reads,
using do_generic_file_read and some tricky lock bypassing.  It seems to
make seeks work real nice for me... current source is attached if you want
to check it out.

>
> The njb can only be addressed by a single host via USB. I doubt if it can
> manage multiple tracks - opening two files locked up the device. The policy I
> tried to implement is: capture the device, open exactly one file, read from
> it - write to it, close this file, release the device. If there are more
> accesses, they should be explicitly rejected with "device busy" (-EBUSY), not
> with a lock, which only delays I/O access.
>

The only problem I can see with this approach is that some programs
(musicmatch, for instance) will simply require lots of open files at one
time... when you are playing a file, if you add another to the queue, it
will attempt to open that file and read in the id3 tags.  The ID3 tag
faking alleviated this somewhat.

The policy in the code that I've sent is this:
-only request the track from the jukebox when an read is requested that
needs the jukebox.
-if another file is currently being read from the jukebox, close it down
and begin the transfer of the second file

I'm not sure if this is a sane policy... if you're playing a file and
attempt to 'cp' another one off, each will just slowly crawl along,
opening and closing the other file.  I think today I'm going to attempt
reading ahead about 300k into the file being closed to see if I can get
adequate performance... and maybe try sharing the drive on gnutella to see
what happens =)

> Another unfixed problem I fight with, maybe it can be reproduced: xmms
> performs heavy buffer underruns when starting playing an MP3 track from the
> njb directly, resulting in 20-30 seconds missing at the beginning of a song.
> The underrun results because the I/O is asynchronous, I have not found out
> how to synchronize buffer flushes. It seems when xmms has filled the internal
> xmms buffer completely, it will start to play but at the wrong position. When
> reaching the end of time, it will play silence - weird. This can also be a
> buffer problem.

hmmm, weird. I'll have to get my soundcard up under linux and try it out.

-ben

Re: [njbfs-devel] njb issues

[Ben O. <be...@gi...>]

My current source is attached;  I'm wondering if there's a better way to
share code that you don't want to check into cvs yet?

> Ben Osheroff wrote:
>
> > -the njb doesn't like reads at arbitrary locations
>
> Are you sure? Well playing with file seeks, I found the njb is just working
> with high latency at random seeks, but it works - xmms gets the right ID3
> tags at random file positions.

Yeah, I've noticed that behavior too... it seems to seek to the end of the
file and read there just fine, but when some arbitrary block in the middle
of the file is requested, a song just starts over from the beginning.
Did you notice that behavior?

> I do not recommend bypassing the VFS. It saves a lot of code that we would
> have to write then on our own,  e.g. dealing with swapping pages. To manage
> the page cache, you could look at the code in other file systems how they
> managed that situation. It should be safe to let the Linux VM allocate pages
> of 4K for our file buffer operations, delegate the low level stuff to the VFS
> and let njbfs deal with higher level operations.

Yup, I've looked into the nfs code.  I don't wanna go there =).  I have
figured out a way to simply call back into the VFS and request reads,
using do_generic_file_read and some tricky lock bypassing.  It seems to
make seeks work real nice for me... current source is attached if you want
to check it out.

>
> The njb can only be addressed by a single host via USB. I doubt if it can
> manage multiple tracks - opening two files locked up the device. The policy I
> tried to implement is: capture the device, open exactly one file, read from
> it - write to it, close this file, release the device. If there are more
> accesses, they should be explicitly rejected with "device busy" (-EBUSY), not
> with a lock, which only delays I/O access.
>

The only problem I can see with this approach is that some programs
(musicmatch, for instance) will simply require lots of open files at one
time... when you are playing a file, if you add another to the queue, it
will attempt to open that file and read in the id3 tags.  The ID3 tag
faking alleviated this somewhat.

The policy in the code that I've sent is this:
-only request the track from the jukebox when an read is requested that
needs the jukebox.
-if another file is currently being read from the jukebox, close it down
and begin the transfer of the second file

I'm not sure if this is a sane policy... if you're playing a file and
attempt to 'cp' another one off, each will just slowly crawl along,
opening and closing the other file.  I think today I'm going to attempt
reading ahead about 300k into the file being closed to see if I can get
adequate performance... and maybe try sharing the drive on gnutella to see
what happens =)

> Another unfixed problem I fight with, maybe it can be reproduced: xmms
> performs heavy buffer underruns when starting playing an MP3 track from the
> njb directly, resulting in 20-30 seconds missing at the beginning of a song.
> The underrun results because the I/O is asynchronous, I have not found out
> how to synchronize buffer flushes. It seems when xmms has filled the internal
> xmms buffer completely, it will start to play but at the wrong position. When
> reaching the end of time, it will play silence - weird. This can also be a
> buffer problem.

hmmm, weird. I'll have to get my soundcard up under linux and try it out.

-ben

[njbfs-devel] njb issues

[<joe...@gm...>]

Hi Ben and Danish,

Welcome to the project Ben!

Ben Osheroff wrote:

> -the njb doesn't like reads at arbitrary locations

Are you sure? Well playing with file seeks, I found the njb is just working 
with high latency at random seeks, but it works - xmms gets the right ID3 
tags at random file positions.

> -thus, when a user seeks forward in a file we should just continuing
> 	reading sequentially from the file until we get to the right
> 	offset.
>
> I've got one part working right, and it does seem to fix forward seeks.

I will try your code with xmms.

> However, I haven't figured out a way to get the data that I've read into
> the linux pageacache.

I do not recommend bypassing the VFS. It saves a lot of code that we would 
have to write then on our own,  e.g. dealing with swapping pages. To manage 
the page cache, you could look at the code in other file systems how they 
managed that situation. It should be safe to let the Linux VM allocate pages 
of 4K for our file buffer operations, delegate the low level stuff to the VFS 
and let njbfs deal with higher level operations.

The njb can only be addressed by a single host via USB. I doubt if it can 
manage multiple tracks - opening two files locked up the device. The policy I 
tried to implement is: capture the device, open exactly one file, read from 
it - write to it, close this file, release the device. If there are more 
accesses, they should be explicitly rejected with "device busy" (-EBUSY), not 
with a lock, which only delays I/O access.

The Midnight Commander can mount /unmount devices dynamically as you enter 
directories. This could be a reason for errors, if it silently unmounts the 
njb without notice. But I don't use mc, so this is just a guess.

Also a big fat warning, it is not supported that a just uploaded track will 
appear immediately in the directory. Why? Well to read all track headers from 
the njb again, it took about 30 sec on my machine. Imagine entering an njb 
track directory, or uploading a track, with additional 30 seconds each time! 
That is really annoying. So I commented out the cache update. To get the 
correct directory status, a njb remount should be done.

Another unfixed problem I fight with, maybe it can be reproduced: xmms 
performs heavy buffer underruns when starting playing an MP3 track from the 
njb directly, resulting in 20-30 seconds missing at the beginning of a song. 
The underrun results because the I/O is asynchronous, I have not found out 
how to synchronize buffer flushes. It seems when xmms has filled the internal 
xmms buffer completely, it will start to play but at the wrong position. When 
reaching the end of time, it will play silence - weird. This can also be a 
buffer problem.

I'm happy the project is speeding up. Keep up the good work.

Jörg

Re: [njbfs-devel] Multiple operations

[Ben O. <be...@gi...>]

On Sun, 10 Feb 2002, Danish Qadri wrote:

> So I guess we need to work at some sort of scheduling or time slicing
> algorithm? The developers of other file systems must have solved this
> problem, so I imagine we could find some cluse it in their sources.


Yeah, I imagine they must have.  It would be great to give about a second
or 2 to each open file, then switch..

I think the trick is going to be figuring out how to get the nomad to read
at an offset (instead of what it seems to do normally, which is just give
you a linear progression of the whole file.

-ben

[njbfs-devel] Multiple operations

[Danish Q. <dx...@nj...>]

I think the issue with Midnight Commander is related to the multiple
operation issue that you were mentioning Ben. I've been running a straight
cp ocmmand and it's been going good (except for when I tapped my PC's HD,
but that's a different story).

An issue I did have is that if I had cp running in the background, and
then I tried to get a directory listing, both operations would freeze, and
I'd have to kill both of them.


So I guess we need to work at some sort of scheduling or time slicing
algorithm? The developers of other file systems must have solved this
problem, so I imagine we could find some cluse it in their sources.



 - Danish

[njbfs-devel] Lockups with Midnight Commander

[Danish Q. <dx...@nj...>]

I had another crash again using MC. The logs showed nothing special, and I
was able to copy many files by hand including the one it had just finished
when it crashed, and the one after it. I'm going to try letting some other
programs run to see if it's a MC specific thing (maybe it's doing some
sort of weird seeking), or in our kernel module.


Feb 10 19:08:41 mantis kernel: dir.c: dname:Dance Dance Revolution - Dance
Dance Revolution - In the Navy '99 (XXL Disaster Remix) (Full).mp3
Feb 10 19:08:41 mantis last message repeated 2 times
Feb 10 19:08:41 mantis kernel: proc.c: open: opening file, file id=293504
Feb 10 19:08:41 mantis kernel: njb_usb.c: attempting to open 'Dance Dance
Revolution - In the Navy '99 (XXL Disaster Remix) (Full)'..
Feb 10 19:08:43 mantis kernel: njb_usb.c: get_library_counter: lsl=27,
msl=0
Feb 10 19:08:43 mantis kernel: njb_usb.c: njb_usb_set_library_counter: 28
0
Feb 10 19:08:43 mantis kernel: njb_usb.c: get_library_counter: lsl=28,
msl=0
Feb 10 19:08:43 mantis kernel: njb_usb.c: njb_usb_set_library_counter: 27
0
Feb 10 19:08:43 mantis kernel: njb_usb.c: get_library_counter: lsl=27,
msl=0
Feb 10 19:08:43 mantis kernel: njb_usb.c: open: 'request track' command
ok, track id = 293504
Feb 10 19:08:55 mantis kernel: proc.c: close: closing file, file id=293504
Feb 10 19:08:55 mantis kernel: njb_usb.c: attempting to close "Dance Dance
Revolution - In the Navy '99 (XXL Disaster Remix) (Full)"
Feb 10 19:08:55 mantis kernel: njb_usb.c: close: jukebox closed
Feb 10 19:22:33 mantis syslogd 1.4-0: restart.
Feb 10 19:22:33 mantis kernel: klogd 1.4-0, log source = /proc/kmsg
started.

Re: [njbfs-devel] Re: njbfs (fwd)

[Ben O. <be...@gi...>]

On Sun, 10 Feb 2002, Danish Qadri wrote:

>
>
> > Has either of y'all ever done any work with the linux pagecache?  I'm
> > trying to add a page into it manually, and it doesn't seem to be easy.
>
> I've never played with it before, but are we both talking about the
> functions that are mentioned in /usr/src/linux/include/linux/pagemap.h ?
> I'm not sure what you mean by "manually adding them".

Yup, those are the ones.

by "manually adding them"... check out this sequence

1. we're reading along merrily from the njb at offset 8192
2. we get a request to read a block at 10000
3. we know we were only at 8192 last time, and the njb doesn't like
	arbitrary reads... so we have to grab blocks 8192-10000
	from the njb.
4. we read in 8192-10000, and get to the correct offset... but we
	could have shoved the blocks that we read back into the
	linux filesystem cache, which would enable backwards seeking.
>
>
> > -the njb doesn't like reads at arbitrary locations
> > -thus, when a user seeks forward in a file we should just continuing
> > 	reading sequentially from the file until we get to the right
> > 	offset.
> >
> > I've got one part working right, and it does seem to fix forward seeks.
> > However, I haven't figured out a way to get the data that I've read into
> > the linux pageacache.
>
> I'll go through the kernel src, to see if I can pick up anything.
>

Yeah, I poked around a bit, and found some code here that should work:
http://www.moses.uklinux.net/patches/lki-4.html

only problem is that add_to_page_cache comes up as an undefined symbol in
2.4.9... I'm guessing that they don't want module developers screwing
around with the cache, maybe?

-ben

Re: [njbfs-devel] Re: njbfs (fwd)

[Danish Q. <dx...@nj...>]

> Has either of y'all ever done any work with the linux pagecache?  I'm
> trying to add a page into it manually, and it doesn't seem to be easy.

I've never played with it before, but are we both talking about the
functions that are mentioned in /usr/src/linux/include/linux/pagemap.h ?
I'm not sure what you mean by "manually adding them".


> -the njb doesn't like reads at arbitrary locations
> -thus, when a user seeks forward in a file we should just continuing
> 	reading sequentially from the file until we get to the right
> 	offset.
>
> I've got one part working right, and it does seem to fix forward seeks.
> However, I haven't figured out a way to get the data that I've read into
> the linux pageacache.

I'll go through the kernel src, to see if I can pick up anything.


- Danish

Re: [njbfs-devel] oh

[Danish Q. <dq...@nj...>]

I'm using vim for my editing, so I say go for it :).

I didn't see too many changes in going to K&R style, so it can't hurt.


- Danish


On Sun, 10 Feb 2002, Ben Osheroff wrote:

>
> would anyone mind if I ran the code through indent -kr -i8?  I dunno which
> editors you guys use, so I'll have to defer to that...
>
> -ben
>
>
> _______________________________________________
> njbfs-devel mailing list
> njb...@li...
> https://lists.sourceforge.net/lists/listinfo/njbfs-devel
>

[njbfs-devel] oh

[Ben O. <be...@gi...>]

would anyone mind if I ran the code through indent -kr -i8?  I dunno which
editors you guys use, so I'll have to defer to that...

-ben

[njbfs-devel] Re: njbfs (fwd)

[Ben O. <be...@gi...>]

try this again..

Ok, we're over on the mailing list now.  I went ahead and commited the two
tiny little patches.

Has either of y'all ever done any work with the linux pagecache?  I'm
trying to add a page into it manually, and it doesn't seem to be easy.

This comes as a result for an idea for seeks on the njb:

-the njb doesn't like reads at arbitrary locations
-thus, when a user seeks forward in a file we should just continuing
	reading sequentially from the file until we get to the right
	offset.

I've got one part working right, and it does seem to fix forward seeks.
However, I haven't figured out a way to get the data that I've read into
the linux pageacache.

-ben



On Sun, 10 Feb 2002, Danish Qadri wrote:

>
>
> Well, by the time I joined the project, Jorg was just starting the
> audiotracks framework, so we never really talked about how to setup the
> development stuff.
>
> We do have a public development mailing list,
> njb...@li... to which I see you're already
> subscribed. Why don't we move all further correspondence onto there, so
> future users/developers can have access to them through the archives?
>
> There is also a CVS commit mailing list, which is helpful when to know
> when to update your local copy. I'm still trying to fully flesh out how we
> can use the SF bug tracking & feature request system in the best
> way for our project.
>
> I would say that the only semi-hard rule for CVS commits is that try to
> make sure that the code you're committing can compile okay. It's not 100%
> necessary, but it will help I think. Any comments on this?
>
>
> BTW, Ben, you now have admin and developer access to the project, so you
> can go ahead and commit your patches. Your njb_usb.c patch for the short
> reads worked on my box, and now I'm seeing all my songs. I still get hard
> lockups with MC, but that's something else that I need to look at the logs
> for.
>
>
> Thing we are also looking at was moving to the autoconf/automake build
> system. It sort of started to make sense to go there after we started
> adding tools into the project. I do believe that Jorg added that to his
> audiotracks setup, so we can tune that later when Jorg's commits are in.
>
>
>
>
>
> - Danish
>
>
>
> On Sun, 10 Feb 2002, Ben Osheroff wrote:
>
> > Cool!
> > I'd like to send you a bigger set of patches soon; my jukebox seems much
> > more stable now, and I'd like to see how they do on another system.
> >
> > My SF username is ben_osheroff.
> >
> > to be honest, I've never worked on a sourceforge project before; how have
> > you guys been running the project so far?
> >
> > On Sat, 9 Feb 2002, Danish Qadri wrote:
> >
> > >
> > > Hey Ben!
> > >
> > >
> > > I'm sorry as well for not replying at once. I'm a senior college student,
> > > and school just started, I got bogged down in work.
> > >
> > > I'm going to try patching my copy of njbfs and recompiling, as I am also
> > > afflicted by a short list of songs. My experience is that I also hit hard
> > > locks when copying the entire set of tracks out of the NJB using Midnight
> > > Commander. The track header problem may also be causing this and I'll let
> > > you know of the results.
> > >
> > > Jorg is working on porting the code to a new framework, but he's not
> > > completely done with it so far. We have several areas in which we can
> > > cleanup bugs as well as add features in the meantime. Hopefully porting to
> > > the new architecture will not be so hard. So I suggest that while we're
> > > waiting for Jorg to import the new audiotracks arch into CVS, we can
> > > continue developing njbfs.
> > >
> > > In other news, I've read that there is now a rudimentary KIO Slave for the
> > > njbfs, which allows KDE's konqueror to implement browsing on the NJB.
> > >
> > > Just let me know of your SF username, and we'll add you to the
> > > admin/developer list and begin coding anew (we've been pretty quiet for a
> > > while :).
> > >
> > > - Danish
> > >
> >
> >
>

[njbfs-devel] Re: njbfs (fwd)

[Ben O. <be...@gi...>]

Ok, we're over on the mailing list now.  I went ahead and commited the two
tiny little patches.

Has either of y'all ever done any work with the linux pagecache?  I'm
trying to add a page into it manually, and it doesn't seem to be easy.

This comes as a result for an idea for seeks on the njb:

-the njb doesn't like reads at arbitrary locations
-thus, when a user seeks forward in a file we should just continuing
	reading sequentially from the file until we get to the right
	offset.

I've got one part working right, and it does seem to fix forward seeks.
However, I haven't figured out a way to get the data that I've read into
the linux pageacache.

-ben



On Sun, 10 Feb 2002, Danish Qadri wrote:

>
>
> Well, by the time I joined the project, Jorg was just starting the
> audiotracks framework, so we never really talked about how to setup the
> development stuff.
>
> We do have a public development mailing list,
> njb...@li... to which I see you're already
> subscribed. Why don't we move all further correspondence onto there, so
> future users/developers can have access to them through the archives?
>
> There is also a CVS commit mailing list, which is helpful when to know
> when to update your local copy. I'm still trying to fully flesh out how we
> can use the SF bug tracking & feature request system in the best
> way for our project.
>
> I would say that the only semi-hard rule for CVS commits is that try to
> make sure that the code you're committing can compile okay. It's not 100%
> necessary, but it will help I think. Any comments on this?
>
>
> BTW, Ben, you now have admin and developer access to the project, so you
> can go ahead and commit your patches. Your njb_usb.c patch for the short
> reads worked on my box, and now I'm seeing all my songs. I still get hard
> lockups with MC, but that's something else that I need to look at the logs
> for.
>
>
> Thing we are also looking at was moving to the autoconf/automake build
> system. It sort of started to make sense to go there after we started
> adding tools into the project. I do believe that Jorg added that to his
> audiotracks setup, so we can tune that later when Jorg's commits are in.
>
>
>
>
>
> - Danish
>
>
>
> On Sun, 10 Feb 2002, Ben Osheroff wrote:
>
> > Cool!
> > I'd like to send you a bigger set of patches soon; my jukebox seems much
> > more stable now, and I'd like to see how they do on another system.
> >
> > My SF username is ben_osheroff.
> >
> > to be honest, I've never worked on a sourceforge project before; how have
> > you guys been running the project so far?
> >
> > On Sat, 9 Feb 2002, Danish Qadri wrote:
> >
> > >
> > > Hey Ben!
> > >
> > >
> > > I'm sorry as well for not replying at once. I'm a senior college student,
> > > and school just started, I got bogged down in work.
> > >
> > > I'm going to try patching my copy of njbfs and recompiling, as I am also
> > > afflicted by a short list of songs. My experience is that I also hit hard
> > > locks when copying the entire set of tracks out of the NJB using Midnight
> > > Commander. The track header problem may also be causing this and I'll let
> > > you know of the results.
> > >
> > > Jorg is working on porting the code to a new framework, but he's not
> > > completely done with it so far. We have several areas in which we can
> > > cleanup bugs as well as add features in the meantime. Hopefully porting to
> > > the new architecture will not be so hard. So I suggest that while we're
> > > waiting for Jorg to import the new audiotracks arch into CVS, we can
> > > continue developing njbfs.
> > >
> > > In other news, I've read that there is now a rudimentary KIO Slave for the
> > > njbfs, which allows KDE's konqueror to implement browsing on the NJB.
> > >
> > > Just let me know of your SF username, and we'll add you to the
> > > admin/developer list and begin coding anew (we've been pretty quiet for a
> > > while :).
> > >
> > > - Danish
> > >
> >
> >
>

[njbfs-devel] Re: njbfs

[<joe...@gm...>]

> Well, just let me know. I'm still doing some work on it. I only spent a
> couple hours using njbfs/writing the tool. Its still evolving :)
>
> Feedback or ideas for new improovments is welcomed.

It works and it's useful. Your tool is similar to the NJB userspace programs 
I found at John Mechalas' good site libnjb.sourceforge.net

I added your tool to the njbfs CVS repository in a new directory named 
'tools'. Also you will find a C program which can extract ID3 info from MP3 
files and prints them in a format compatible to the njbfs file name parser 
format. I think there will be many more tools in the future.

Njbfs was designed to almost automatically upload MP3 files. But most times, 
user interaction is needed. The biggest weakness of njbfs is that the MP3 
file size is unknown and must be given by hand. This will change in the near 
future, I experiment with a buffer mechanism, which will read a complete file 
and evaluate the file size before uploading. This will cause a small delay 
and needs some megabytes of memory. Another way would be finding the TSIZ tag 
which is defined in ID3v2 and make use if it as soon as it appears in the 
data stream. With an auto-tagging tool, a large buffer memory won't be needed.

Also, extensive testing of missing is missing. E.g. uploading of several MP3 
files with a single 'cp' command. I really don't know if this works or not.

Thanks again, and keep up your good work,

Jörg

[njbfs-devel] Fwd: njbfs

[<joe...@gm...>]

----------  Weitergeleitete Nachricht  ----------

Subject: njbfs
Date: Tue, 1 Jan 2002 16:14:06 -0500
From: Mental <Me...@Ne...>
To: joe...@gm...

Hey there! I just found your njbfs driver on sourceforge. What a great
utility! As I was playing around with it, I decided I needed a way to
help with the uploading of files to it. So I wrote a quick script.

So far it will walk a directory full of mp3's and let you edit their
id3tags, then copy them over to the Jukebox. If you think its at all
useful, feel free to include it as a proof of concept utility. I wrote it
in perl and used the MP3::Info module on cpan. The code is here:
http://www.neverlight.com/njbutil.pl

I'm only just getting started with your driver, but it has a lot of
potential. The autofs module you wrote was great too.



--
Mental (Me...@Ne...)

I can levitate birds.  No one cares.
--Steven Wright

GPG public key: http://www.neverlight.com/Mental.asc

-------------------------------------------------------

[njbfs-devel] Status of GNU build system (autoconf/automake)

[Danish Q. <dx...@nj...>]

I'm following some examples from various programs (one of them being
CVS), and it's going along slowly.

Right now, I'm trying to add in good support for the automount stuff with
a --enable-automount option. 

Looking at CVS and some other tools, they break down their directory
structure as follows:

. - README/ChangeLog/configure stuff
include - the all inclusive config.h file that has what options were
	selected
src - This has all the source files and header files for the software

With this setup, if you decide to distribute a tool along with your
software, you can just throw it in at the top level in another directory,
and adjust the build system to build it as well. Take a look at the CVS
sources, and how they include zlib among other things. I was reading
through the mp3lib docs, and they recommend distributing it along with the
software as well.

I was thinking of using the above for njbfs as well, what do you think?


- Danish

[njbfs-devel] Possible solution to file transfer hoopla

[Danish Q. <dx...@nj...>]

Quick question... what's wrong with running a stat on the file before
the transfer? File size is reported in one of the fields of the returned
struct.

I'll test it and see if it works. There is also a small program which
calls the C function. I ran it on a track from the NJB and it returned the
correct information, which means we should be able to utilize it within
the copy command. I'll try that tomorrow if I find time.

Happy Holidays!


- Danish