I haven't looked at the patch file in detail, but I believe that is the type of change that is needed to make the module compile and work in a later (but not too late) version of Samba. I note that the patch file doesn't show any changes to the function signatures, so I assume the only change is to the set of parameters used in the macros used to pass the data down to the next VFS layer. I also note that the utime function is commented out at the bottom and assume this is because the set of VFS callbacks has changed and no longer includes the utime function.
Philip
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
And coupled with what I looked at this morning and I might have a better idea about how to get it working, and keep it working as samba development progresses in it's strange way
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Huzzah, I've not successfully written my very own vfs layer to prevent non-jpg files being written to a share, and I understand a little more about how the whole thing works
media_harmony now compiles fine, and even appears to work.
I created msmMMOB.mdb in //localhost/SHARE/Avid MediaFiles, and it created msmMMOB.mdb_127.0.0.1_nobody
I then created msmMMOB.mdb in //extip/SHARE/Avid MediaFiles, and it created msmMMOB.mdb_192.168.18.18_nobody
doing an ls on the share doesn't reveal the existance of the "other" msmMMOB.mdb (I appreciate you can get it if you really want)
I finally created foo.txt in //localhost/SHARE/Avid MediaFiles, and it appeared at //extip/SHARE/Avid MediaFiles/foo.txt
I had to do a minor change to the Makefile as well
I'm happy with the fact the connection_struct is a no-impact removal, as the only time the connection information is used is in fcat_client_suffix and pcat_client_suffix, which was using the vfs_handle_struct anyway!
pstrcat(path, handle->conn->client_address);
However I have two questions/comments
1) SMB_VFS_NEXT_OPEN has changed, it now has an additional parameter, a files_struct. Does this need to change? I guess experimenting will tell
for a complete tarball of the modules, including .o and .so files, built under ubuntu 7.10 (samba 3.0.26a)
The (samba VFS guy) thinks I don't have to worry about the fsp that's arrived in the _open function, at least for media_harmony. There are possibly a couple of other gotchas
I plonked mxf_harmony on a share that had previously been used for an avid using media_harmony. I browsed it with FCP on a mac, but there was nothing obvious -- I'm guessing I messed up with editing mxf_harmony.c. The module seems to load, but the only files the FCP understood were the .aifs
There are some virtual in the file structure, and there's evidence the vfs module is doing something in the syslog:
Apr 2 23:27:39 newsjtclap99 smbd[10267]: Added mxf virtual file 27
Apr 2 23:27:39 newsjtclap99 smbd[10267]: Removed mxf virtual file 27
Apr 2 23:27:39 newsjtclap99 smbd[10267]: Added mxf virtual file 27
Apr 2 23:27:39 newsjtclap99 smbd[10267]: Removed mxf virtual file 27
Apr 2 23:27:39 newsjtclap99 smbd[10267]: Added mxf virtual file 27
Apr 2 23:27:39 newsjtclap99 smbd[10267]: Removed mxf virtual file 27
And the file structure reveals some "tngA02......mxf._v_.pcm" files, about 15-20% shorter than the tngA02....mxf file. It doesn't reveal any video MXF files, but they do exist if I get them directly. Known feature, or am I missing something?
smb: \Avid MediaFiles\MXF\1\> ls tng15A04V01.47EBF484.03B2D0.mxf
tng15A04V01.47EBF484.03B2D0.mxf A 1049185 Thu Mar 27 19:24:53 2008
33638 blocks of size 1048576. 17563 blocks available
smb: \Avid MediaFiles\MXF\1\> ls *._v_.*
tngA02.47EC0449_47E47EC0449.mxf._v_.pcm R 1290240 Thu Mar 27 20:24:50 2008
tng15A04A01.47EBF484.03B1F0.mxf._v_.pcm R 26882 Thu Mar 27 19:24:54 2008
tngA01.47EC0460_47E47EC045E.mxf._v_.pcm R 1056000 Thu Mar 27 20:25:10 2008
tng15A04A01.47EBF740.AEB040.mxf._v_.pcm R 1935410 Thu Mar 27 19:37:03 2008
tngA02.47EC045F_47E47EC045E.mxf._v_.pcm R 1056000 Thu Mar 27 20:25:10 2008
tngA01.47EC044A_47E47EC0449.mxf._v_.pcm R 1290240 Thu Mar 27 20:24:49 2008
tng15A04A02.47EBF4B7.105730.mxf._v_.pcm R 1186592 Thu Mar 27 19:26:08 2008
tng15A04A01.47EBF4B7.105820.mxf._v_.pcm R 1186592 Thu Mar 27 19:26:07 2008
tng15A04A02.47EBF484.03B1E0.mxf._v_.pcm R 26882 Thu Mar 27 19:24:54 2008
tng15A04A02.47EBF740.AEAF50.mxf._v_.pcm R 1935410 Thu Mar 27 19:37:03 2008
33638 blocks of size 1048576. 17563 blocks available
smb: \Avid MediaFiles\MXF\1\> ls tng15A04V01.47EBF484.03B2D0.mxf._v_.dv
tng15A04V01.47EBF484.03B2D0.mxf._v_.dv R 655360 Thu Mar 27 19:24:53 2008
33638 blocks of size 1048576. 17563 blocks available
media_harmony, at least, appears to work perfectly with streaming multiple video files off my laptop's with a couple of avids. Poor laptop...
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
The virtual files sizes should be less than the original MXF file because the virtual file is only exposing the essence data blob wrapped in the MXF file and it doesn't include all the metadata and file structure information.
Do you also not see the ..._v_.dv file if you do a "ls"?
I'm surprised the virtual DV file size is not a multiple of 144000 for PAL DV-25 or 288000 for PAL DV-50. Although I seem to remember that Avid sometimes leaves junk at the end of the essence data blob after ingesting (and ideally the MXF harmony should check the MXF metadata and ignore the junk at the end).
Philip
ps. just to warn you that the media_many and media_link modules are experimental and could have some unexpected behaviour.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
FYI to anyone looking to install on a 32 bit ubuntu 7.10 box
mkdir /tmp/mh
cd /tmp/mh
wget http://195.50.87.86/~npf/08040223.tar.gz
sudo cp *.so /usr/lib/samba/vfs/
To build on ubuntu 7.10 (32 or 64 bit)
get samba source, stick in /tmp/samba
export SAMBA_DIR=/tmp/samba
apt-get install build-essential linux-headers-generic libreadline5-dev libacl1-dev libkrb5-dev libldap2-dev
make && sudo cp *.so /usr/lib/samba/vfs/
Then you edit your smb.conf file
[MH]
comment = A media_harmony testbed
writable = yes
locking = yes
path = /mnt/mh
public = yes
vfs object = media_harmony
I think you need to do a /etc/init.d/samba restart to pick up the vfs object
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I'm having issues with mediaharmony. I attempt to compile using
gcc -c -g -O2 -Iinclude -I/home/paul/ingex/work/samba/samba-3.0.26a/source/include -I/home/paul/ingex/work/samba/samba-3.0.26a/source/popt -I/home/paul/ingex/work/samba/samba-3.0.26a/source/ubiqx -I/home/paul/ingex/work/samba/samba-3.0.26a/source/smbwrapper -I/home/paul/ingex/work/samba/samba-3.0.26a/source/tdb/include/ -I/home/paul/ingex/work/samba/samba-3.0.26a/source/lib/replace -I/home/paul/ingex/work/samba/samba-3.0.26a/source/librpc -I/home/paul/ingex/work/samba/samba-3.0.26a/source -I. -I/home/paul/ingex/work/samba/samba-3.0.26a/source -fPIC media_harmony.c
And get
media_harmony.c:222:70: error: macro "SMB_VFS_NEXT_STATVFS" passed 4 arguments, but takes just 3
Now MH likes 4 arguments to these VFS macros
../MediaHarmony/media_harmony.c: return SMB_VFS_NEXT_STATVFS(handle, conn, clientPath, statbuf);
../MediaHarmony/media_harmony.c: return SMB_VFS_NEXT_STATVFS(handle, conn, path, statbuf);
../MediaHarmony/media_link.c: return SMB_VFS_NEXT_STATVFS(handle, conn, realVFSPath, statbuf);
../MediaHarmony/mxf_harmony.c: return SMB_VFS_NEXT_STATVFS(handle, conn, realPath, statbuf);
However Samba 3.0.26a expects 3 arguments
../samba/samba-3.0.26a/examples/VFS/skel_transparent.c: return SMB_VFS_NEXT_STATVFS(handle, path, statbuf);
../samba/samba-3.0.26a/source/include/vfs_macros.h:#define SMB_VFS_NEXT_STATVFS(handle, path, statbuf) ((handle)->vfs_next.ops.statvfs((handle)->vfs_next.handles.statvfs, (path), (statbuf)))
../samba/samba-3.0.26a/source/modules/vfs_full_audit.c: result = SMB_VFS_NEXT_STATVFS(handle, path, statbuf);
Now it's true that, at least, this copy of 3.0.23 used 4 arguments (handle and conn seem to have been replaced by handle alone)
https://stage.maemo.org/svn/maemo/projects/connectivity/samba/tags/samba-3.0.23c-1osso5/examples/VFS/skel_transparent.c
static int skel_statvfs(struct vfs_handle_struct *handle, struct connection_struct *conn, const char *path, struct vfs_statvfs_struct *statbuf)
{
return SMB_VFS_NEXT_STATVFS(handle, conn, path, statbuf);
}
Is it a simple change to move to the new way?
I've managed to get media_harmony_patch to compile
http://195.50.87.86/~npf/media_harmony_patch.gz
I think I understand how the module works now, and I'm sure it's not correct, but it this the right kind of thing to be doing?
I haven't looked at the patch file in detail, but I believe that is the type of change that is needed to make the module compile and work in a later (but not too late) version of Samba. I note that the patch file doesn't show any changes to the function signatures, so I assume the only change is to the set of parameters used in the macros used to pass the data down to the next VFS layer. I also note that the utime function is commented out at the bottom and assume this is because the set of VFS callbacks has changed and no longer includes the utime function.
Philip
That's pretty much it. I really have no idea about this sort of thing, however I've found a vfs modules for dummies guide
http://www.geocities.com/orville_torpid/papers/vfs_tutorial.html
And coupled with what I looked at this morning and I might have a better idea about how to get it working, and keep it working as samba development progresses in it's strange way
Huzzah, I've not successfully written my very own vfs layer to prevent non-jpg files being written to a share, and I understand a little more about how the whole thing works
media_harmony now compiles fine, and even appears to work.
I created msmMMOB.mdb in //localhost/SHARE/Avid MediaFiles, and it created msmMMOB.mdb_127.0.0.1_nobody
I then created msmMMOB.mdb in //extip/SHARE/Avid MediaFiles, and it created msmMMOB.mdb_192.168.18.18_nobody
doing an ls on the share doesn't reveal the existance of the "other" msmMMOB.mdb (I appreciate you can get it if you really want)
I finally created foo.txt in //localhost/SHARE/Avid MediaFiles, and it appeared at //extip/SHARE/Avid MediaFiles/foo.txt
I had to do a minor change to the Makefile as well
I'm happy with the fact the connection_struct is a no-impact removal, as the only time the connection information is used is in fcat_client_suffix and pcat_client_suffix, which was using the vfs_handle_struct anyway!
pstrcat(path, handle->conn->client_address);
However I have two questions/comments
1) SMB_VFS_NEXT_OPEN has changed, it now has an additional parameter, a files_struct. Does this need to change? I guess experimenting will tell
2) SMB_VFS_OP_UTIME has been replaced, I think it's been replaced by "ntimes", from the svn change below, but I could be wrong
http://svn.netlabs.org/samba/changeset/22?new_path=trunk%2Fsamba%2Fexamples
New patches (practically the same as before) are
http://195.50.87.86/~npf/media_harmony.patch.gz
and
http://195.50.87.86/~npf/Makefile.patch.gz
Thanks for the patch files. I've added then the CVS: http://ingex.cvs.sourceforge.net/ingex/ingex/MediaHarmony/samba-3.0.26a-patch/
Philip
I've got all the modules to compile
Download
http://195.50.87.86/~npf/patch_08040223.gz
For the patch, and
http://195.50.87.86/~npf/08040223.tar.gz
for a complete tarball of the modules, including .o and .so files, built under ubuntu 7.10 (samba 3.0.26a)
The (samba VFS guy) thinks I don't have to worry about the fsp that's arrived in the _open function, at least for media_harmony. There are possibly a couple of other gotchas
I plonked mxf_harmony on a share that had previously been used for an avid using media_harmony. I browsed it with FCP on a mac, but there was nothing obvious -- I'm guessing I messed up with editing mxf_harmony.c. The module seems to load, but the only files the FCP understood were the .aifs
There are some virtual in the file structure, and there's evidence the vfs module is doing something in the syslog:
Apr 2 23:27:39 newsjtclap99 smbd[10267]: Added mxf virtual file 27
Apr 2 23:27:39 newsjtclap99 smbd[10267]: Removed mxf virtual file 27
Apr 2 23:27:39 newsjtclap99 smbd[10267]: Added mxf virtual file 27
Apr 2 23:27:39 newsjtclap99 smbd[10267]: Removed mxf virtual file 27
Apr 2 23:27:39 newsjtclap99 smbd[10267]: Added mxf virtual file 27
Apr 2 23:27:39 newsjtclap99 smbd[10267]: Removed mxf virtual file 27
And the file structure reveals some "tngA02......mxf._v_.pcm" files, about 15-20% shorter than the tngA02....mxf file. It doesn't reveal any video MXF files, but they do exist if I get them directly. Known feature, or am I missing something?
smb: \Avid MediaFiles\MXF\1\> ls tng15A04V01.47EBF484.03B2D0.mxf
tng15A04V01.47EBF484.03B2D0.mxf A 1049185 Thu Mar 27 19:24:53 2008
33638 blocks of size 1048576. 17563 blocks available
smb: \Avid MediaFiles\MXF\1\> ls *._v_.*
tngA02.47EC0449_47E47EC0449.mxf._v_.pcm R 1290240 Thu Mar 27 20:24:50 2008
tng15A04A01.47EBF484.03B1F0.mxf._v_.pcm R 26882 Thu Mar 27 19:24:54 2008
tngA01.47EC0460_47E47EC045E.mxf._v_.pcm R 1056000 Thu Mar 27 20:25:10 2008
tng15A04A01.47EBF740.AEB040.mxf._v_.pcm R 1935410 Thu Mar 27 19:37:03 2008
tngA02.47EC045F_47E47EC045E.mxf._v_.pcm R 1056000 Thu Mar 27 20:25:10 2008
tngA01.47EC044A_47E47EC0449.mxf._v_.pcm R 1290240 Thu Mar 27 20:24:49 2008
tng15A04A02.47EBF4B7.105730.mxf._v_.pcm R 1186592 Thu Mar 27 19:26:08 2008
tng15A04A01.47EBF4B7.105820.mxf._v_.pcm R 1186592 Thu Mar 27 19:26:07 2008
tng15A04A02.47EBF484.03B1E0.mxf._v_.pcm R 26882 Thu Mar 27 19:24:54 2008
tng15A04A02.47EBF740.AEAF50.mxf._v_.pcm R 1935410 Thu Mar 27 19:37:03 2008
33638 blocks of size 1048576. 17563 blocks available
smb: \Avid MediaFiles\MXF\1\> ls tng15A04V01.47EBF484.03B2D0.mxf._v_.dv
tng15A04V01.47EBF484.03B2D0.mxf._v_.dv R 655360 Thu Mar 27 19:24:53 2008
33638 blocks of size 1048576. 17563 blocks available
media_harmony, at least, appears to work perfectly with streaming multiple video files off my laptop's with a couple of avids. Poor laptop...
I've added your patch to the CVS.
The virtual files sizes should be less than the original MXF file because the virtual file is only exposing the essence data blob wrapped in the MXF file and it doesn't include all the metadata and file structure information.
Do you also not see the ..._v_.dv file if you do a "ls"?
I'm surprised the virtual DV file size is not a multiple of 144000 for PAL DV-25 or 288000 for PAL DV-50. Although I seem to remember that Avid sometimes leaves junk at the end of the essence data blob after ingesting (and ideally the MXF harmony should check the MXF metadata and ignore the junk at the end).
Philip
ps. just to warn you that the media_many and media_link modules are experimental and could have some unexpected behaviour.
FYI to anyone looking to install on a 32 bit ubuntu 7.10 box
mkdir /tmp/mh
cd /tmp/mh
wget http://195.50.87.86/~npf/08040223.tar.gz
sudo cp *.so /usr/lib/samba/vfs/
To build on ubuntu 7.10 (32 or 64 bit)
get samba source, stick in /tmp/samba
export SAMBA_DIR=/tmp/samba
apt-get install build-essential linux-headers-generic libreadline5-dev libacl1-dev libkrb5-dev libldap2-dev
make && sudo cp *.so /usr/lib/samba/vfs/
Then you edit your smb.conf file
[MH]
comment = A media_harmony testbed
writable = yes
locking = yes
path = /mnt/mh
public = yes
vfs object = media_harmony
I think you need to do a /etc/init.d/samba restart to pick up the vfs object
Thanks for the compiled files ISOROX!!!!
They worked a treat on Ubuntu 7.1.