You can subscribe to this list here.
2006 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
(7) |
Oct
|
Nov
(3) |
Dec
|
---|---|---|---|---|---|---|---|---|---|---|---|---|
2007 |
Jan
|
Feb
|
Mar
(10) |
Apr
(24) |
May
|
Jun
|
Jul
|
Aug
(18) |
Sep
(1) |
Oct
(1) |
Nov
|
Dec
|
2008 |
Jan
|
Feb
|
Mar
(13) |
Apr
(7) |
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
(2) |
Nov
|
Dec
|
2010 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
(3) |
Dec
|
From: Stef B. <st...@bo...> - 2010-11-10 19:48:11
|
On 11/10/2010 07:49 PM, Allan Clark wrote: > On Wed, Nov 10, 2010 at 07:48, Stef Bon<st...@bo...> wrote: >> Hello, >> >> has somebody tried to create a lowlevel version of mysqlfs? > Hi Stef; > > What do you mean by a low-level version? It's currently a FUSEfs, how > much lower do you mean? Well, fuse offers two interfaces you can make use of to make your fs. A highlevel, where the path is an argument, and a lowlevel one, where the inode is an argument. I've started to program the highlevel interface, which is easier, and switched to the lowlevel, cause I want to implement aio with my fs, and that is easier with the low level interface. See: http://fuse.sourceforge.net/doxygen/structfuse__lowlevel__ops.html So there are different level of "lowness"... Stef |
From: Allan C. <al...@ch...> - 2010-11-10 18:49:20
|
On Wed, Nov 10, 2010 at 07:48, Stef Bon <st...@bo...> wrote: > Hello, > > has somebody tried to create a lowlevel version of mysqlfs? Hi Stef; What do you mean by a low-level version? It's currently a FUSEfs, how much lower do you mean? Allan -- al...@ch... "金鱼" http://linkedin.com/in/goldfish please, no proprietary attachments (http://tinyurl.com/cbgq) |
From: Stef B. <st...@bo...> - 2010-11-10 16:08:11
|
Hello, has somebody tried to create a lowlevel version of mysqlfs? Stef |
From: Michal L. <mi...@lo...> - 2008-10-22 23:44:50
|
Blakey wrote: > Hi Everyone, > > I'm just trying to compile mysqlfs, but have a minor issue, ./configure > seems to bomb out once it gets to the pthreads testing section. IIRC it's got something to do with MySQL or FUSE libraries. Although it says pthreads lib is missing it actually means libmysql-dev or libfuse-dev (or how are these called on Ubuntu) are missing. Make sure you've got them installed and re-run configure. It should work. Michal |
From: Blakey <bl...@bo...> - 2008-10-22 22:41:33
|
Hi Everyone, I'm just trying to compile mysqlfs, but have a minor issue, ./configure seems to bomb out once it gets to the pthreads testing section. I've had a search around and it would appear that the pthreads library is installed. Does anyone have any ideas / suggestions? I'm running x86 xubuntu 8.04 if thats of any use... Thanks, Craig |
From: Smart W. G. F. W. <f.w...@sm...> - 2008-04-14 08:12:23
|
Hi, Shane Spencer schrieb: > So, I set up mysql master to slave replication as a test. It works > really stupidly well with mysqlfs for distributing an instant backup > offsite somewhere. I'm hoping to use mysqlfs for some awesome, > pointless, evil.. > > Any chance you could aid my effort by allowing two DB connections if > requested? One for insert/update and one for all selects? This would > allow changes to go upstream (as well as local w/ good locking). I > know this probably makes your brain twist a bit, but c'mon. It'll be > unsafe, unprotected byte-sex. You know you wanna do it. You can just use mysql-ndbcluster for that purpose and mount mysqlfs readonly on the second client. -- Mit freundlichen Grüßen, Florian Wiessner Smart Weblications GmbH Martinsberger Str. 1 D-95119 Naila fon.: +49 700 762 789 32 - 0,12 EUR/Min* fax.: +49 700 762 789 32 - 0,12 EUR/Min* 24/7: +49 900 311 886 00 - 1,99 EUR/Min* http://www.smart-weblications.de -- Sitz der Gesellschaft: Naila Geschäftsführer: Florian Wiessner HRB-Nr.: HRB 3840 Amtsgericht Hof *aus dem dt. Festnetz, ggf. abweichende Preise aus dem Mobilfunknetz |
From: Andrew R. <ros...@gm...> - 2008-04-14 06:35:19
|
On 14/04/2008, Shane Spencer <sh...@bo...> wrote: > So, I set up mysql master to slave replication as a test. It works > really stupidly well with mysqlfs for distributing an instant backup > offsite somewhere. I'm hoping to use mysqlfs for some awesome, > pointless, evil.. > > Any chance you could aid my effort by allowing two DB connections if > requested? One for insert/update and one for all selects? This would > allow changes to go upstream (as well as local w/ good locking). I > know this probably makes your brain twist a bit, but c'mon. It'll be > unsafe, unprotected byte-sex. You know you wanna do it. Hi Shane Why not use mysql proxy (http://forge.mysql.com/wiki/MySQL_Proxy) first just to be sure what your doing actually requires "unprotected byte-sex" ;) - Andrew Rose http://andrew-rose.blogspot.com |
From: Shane S. <sh...@bo...> - 2008-04-14 05:48:45
|
So, I set up mysql master to slave replication as a test. It works really stupidly well with mysqlfs for distributing an instant backup offsite somewhere. I'm hoping to use mysqlfs for some awesome, pointless, evil.. Any chance you could aid my effort by allowing two DB connections if requested? One for insert/update and one for all selects? This would allow changes to go upstream (as well as local w/ good locking). I know this probably makes your brain twist a bit, but c'mon. It'll be unsafe, unprotected byte-sex. You know you wanna do it. - Shane |
From: Smart W. G. F. W. <f.w...@sm...> - 2008-04-08 10:06:11
|
Hi, Stef Bon schrieb: >> Yes, the query grows bigger, and it is executed every time a lstat() or >> lstat64() syscall happens - which makes performance really bad on >> ndbcluster with distributed clients on different locations. We should >> cache it somewhere (either in mysqlfs_getattr, or as cache somewhere in >> query_inode_full. >> >> the cache must be cleared for the inode, if write/rename or chown/chmod >> is executed so that query_inode_full always returns proper values. >> >> i thought about that inode-cache - it shouldn't be more than an array >> containing values which has to be checked before the query is run. >> > Yes, a cache should will things up. The construction now does a lot of > lookups and never uses > data it has found before. For example, if you want to read a file > /a/b/c/d/testfile, it will check you've permissions to > that file, and walk the path to it: > check / [check...] > check /a/b/c/d/testfile > > to determine the inode of say /a/b/c, it does not use the information it > just found before. > > So a cache is a smart thing to do. > There are other fuse fs's using a cache(sshfs, fusesmb). My idea is to > look first there how it is solved there. > My first impression is that fuse offers possibilities to use a cache, > but I'm not sure. > >> as for the locking-problem, what about creating a table metadata to >> store information on which inode is currently locked? metadata could >> also be stored with ndbcluster-engine, and thus we would easily be able >> to implement file locking. >> >> > Why not create an extra field in the inodes table, with a information > about the locking? > An extra table is making things extra complicated, I think. > ok, i'll look at other fuse-fs's how cache is implemented there. one extra field in the inode table is sufficient if mysqlfs is not run on a cluster, but i don't know if it would be smarter to store this kind of metainfo in a seperate table "metadata" - e.g. which mysql-host has locked which inode etc. but i think we should first implement the cache, then we can solve the locking problem... -- Mit freundlichen Grüßen, Florian Wiessner Smart Weblications GmbH Martinsberger Str. 1 D-95119 Naila fon.: +49 700 762 789 32 - 0,12 EUR/Min* fax.: +49 700 762 789 32 - 0,12 EUR/Min* 24/7: +49 900 311 886 00 - 1,99 EUR/Min* http://www.smart-weblications.de -- Sitz der Gesellschaft: Naila Geschäftsführer: Florian Wiessner HRB-Nr.: HRB 3840 Amtsgericht Hof *aus dem dt. Festnetz, ggf. abweichende Preise aus dem Mobilfunknetz |
From: Stef B. <st...@bo...> - 2008-04-08 06:53:17
|
Smart Weblications GmbH, Florian Wiessner wrote: > Hi, > > Stef Bon schrieb: > > >> This query does grow bigger as the path is longer (deeper). >> >> > Yes, the query grows bigger, and it is executed every time a lstat() or > lstat64() syscall happens - which makes performance really bad on > ndbcluster with distributed clients on different locations. We should > cache it somewhere (either in mysqlfs_getattr, or as cache somewhere in > query_inode_full. > > the cache must be cleared for the inode, if write/rename or chown/chmod > is executed so that query_inode_full always returns proper values. > > i thought about that inode-cache - it shouldn't be more than an array > containing values which has to be checked before the query is run. > Yes, a cache should will things up. The construction now does a lot of lookups and never uses data it has found before. For example, if you want to read a file /a/b/c/d/testfile, it will check you've permissions to that file, and walk the path to it: check / check /a check /a/b check /a/b/c check /a/b/c/d check /a/b/c/d/testfile to determine the inode of say /a/b/c, it does not use the information it just found before. So a cache is a smart thing to do. There are other fuse fs's using a cache(sshfs, fusesmb). My idea is to look first there how it is solved there. My first impression is that fuse offers possibilities to use a cache, but I'm not sure. > as for the locking-problem, what about creating a table metadata to > store information on which inode is currently locked? metadata could > also be stored with ndbcluster-engine, and thus we would easily be able > to implement file locking. > > Why not create an extra field in the inodes table, with a information about the locking? An extra table is making things extra complicated, I think. Stef Bon |
From: Smart W. G. F. W. <f.w...@sm...> - 2008-04-08 00:09:13
|
hi all, i have done the following setup: server1: amd athlon 64 x2, 2gb ram, 2x250gb hdd software raid1 server2: intel p4 1.8ghz, 256mb ram, 1x20gb ide disk the servers are connected via 100mbit, but with async routing over 7 hops with about 10ms latency. i have mounted a mysqlfs with ndb-cluster and put my website on it (www2.smart-weblications.de is now dns-round-robin with mysqlfs underneath), to benchmark the performance: http://www2.smart-weblications.de/mysqlfs-strace.txt it takes about 6-8 seconds to render the page - most of the time the system waits on lstat64() (which is query_inode_full() in our case...) i also copied files from and to the mysqlfs mountpoint: time cp mysqlfs-strace.txt /tmp/foo real 0m0.394s user 0m0.002s sys 0m0.001s time cp /tmp/strace4 mysqlfs-strace.txt real 0m5.677s user 0m0.001s sys 0m0.002s which means that reading from the fs is quite fast (approx. 700kb/sek), writing is awful slow (approx. 10kb/sek)... http://www2.smart-weblications.de/mysqlfs-strace-write.txt so somehow we should cache write() -- Mit freundlichen Grüßen, Florian Wiessner Smart Weblications GmbH Martinsberger Str. 1 D-95119 Naila fon.: +49 700 762 789 32 - 0,12 EUR/Min* fax.: +49 700 762 789 32 - 0,12 EUR/Min* 24/7: +49 900 311 886 00 - 1,99 EUR/Min* http://www.smart-weblications.de -- Sitz der Gesellschaft: Naila Geschäftsführer: Florian Wiessner HRB-Nr.: HRB 3840 Amtsgericht Hof *aus dem dt. Festnetz, ggf. abweichende Preise aus dem Mobilfunknetz |
From: Smart W. G. F. W. <f.w...@sm...> - 2008-04-07 09:59:42
|
Hi, Stef Bon schrieb: >> >> now I can check the flow of what happens (by activating debugging). It's >> much more clear now. >> >> But when I look at query_inode_full, I do not understand what happens >> there at all. >> The purpose of this function is to translate a path into a inode, right? >> But it does much more things. >> It "returns" a table nlinks, the inode, the parent. (and more?) >> >> Can someone explain why this is? It runs complicated queries.... >> > > Ok, > > now I've read it a couple of times, I understand it more or less. > > A (long) query is constructed to determine the inode, with the help of a > LEFT JOIN. > > This query does grow bigger as the path is longer (deeper). > > I'm looking into the mysql syntax now. Any hints about it welcome. > Yes, the query grows bigger, and it is executed every time a lstat() or lstat64() syscall happens - which makes performance really bad on ndbcluster with distributed clients on different locations. We should cache it somewhere (either in mysqlfs_getattr, or as cache somewhere in query_inode_full. the cache must be cleared for the inode, if write/rename or chown/chmod is executed so that query_inode_full always returns proper values. i thought about that inode-cache - it shouldn't be more than an array containing values which has to be checked before the query is run. as for the locking-problem, what about creating a table metadata to store information on which inode is currently locked? metadata could also be stored with ndbcluster-engine, and thus we would easily be able to implement file locking. -- Mit freundlichen Grüßen, Florian Wiessner Smart Weblications GmbH Martinsberger Str. 1 D-95119 Naila fon.: +49 700 762 789 32 - 0,12 EUR/Min* fax.: +49 700 762 789 32 - 0,12 EUR/Min* 24/7: +49 900 311 886 00 - 1,99 EUR/Min* http://www.smart-weblications.de -- Sitz der Gesellschaft: Naila Geschäftsführer: Florian Wiessner HRB-Nr.: HRB 3840 Amtsgericht Hof *aus dem dt. Festnetz, ggf. abweichende Preise aus dem Mobilfunknetz |
From: Stef B. <st...@bo...> - 2008-03-30 12:54:29
|
Michal Ludvig wrote: > I think what you really want is something like "rdiff-backup" or > "duplicity". Both tools let you maintain backups with history, remove > old revisions, etc. You can indeed store the archives in mysqlfs if you > wish. > > Have a look at: > - http://www.nongnu.org/duplicity/ > - http://rdiff-backup.stanford.edu/ > - http://www.debian-administration.org/articles/209 (not really > debian-specific, don't worry) > Thanks for the links. I'm using rdiff-backup already on some machines now. The reason for trying to do this with mysqlfs is that it has special potential on maintaining versions, cause it's fuse and it's in a database. Then you do not need a complicated script as rdiff-backup, but backups are made on the fly. I'll check the links anyway. Stef Bon |
From: Michal L. <mi...@lo...> - 2008-03-30 12:34:03
|
Stef Bon wrote: > My idea is to only do a copy or a sync of a directory of my choice, for > example my homedirectory, once a day (4.30 at night for example), to the > mounted mysqlfs. > This way not every change will get his way to the database. > > Of course there should be a policy on how long a version will stay in > the database, and there should be some utilities to find previous > versions. But that's not the most difficult step I think. I think what you really want is something like "rdiff-backup" or "duplicity". Both tools let you maintain backups with history, remove old revisions, etc. You can indeed store the archives in mysqlfs if you wish. Have a look at: - http://www.nongnu.org/duplicity/ - http://rdiff-backup.stanford.edu/ - http://www.debian-administration.org/articles/209 (not really debian-specific, don't worry) HTH. Michal |
From: Stef B. <st...@bo...> - 2008-03-30 12:08:50
|
Michal Ludvig wrote: > Binlog was designed for a totally different purpose and as such is > hardly usable for a convenient revision tracking on mysqlfs. > > For instance you'll have to keep all binlogs from when the FS was > created to be able to "replay" the filesystem to some point in time (and > honestly who keeps binlogs longer then a few days after they get their > way to slaves). > Well, I did not know this way there are backups already available in the MySQL server. I've just read the MySQL reference on the Internet, and it is possible to do binary logging for at least the mysqlfs database. > It would work on the whole-filesystem level, not for particular files > (which may or may not be what Stef wanted). > Yes, indeed. With mysqlbinlog it's possible to go back to a previous version of a database, and copy that one to a temporary database. But this is step one. The next step would be to find the particular file (for a given time). Ok, it's possible this way, but very complicated if you would like to check only one file. But when reacting on your first remark. In my idea I woul like to use the mysql filesystem as a backup, not as a filesystem where users have direct access to. I do understand that when users (or processes) have direct access, there will a lot of changes, which all get maintained... My idea is to only do a copy or a sync of a directory of my choice, for example my homedirectory, once a day (4.30 at night for example), to the mounted mysqlfs. This way not every change will get his way to the database. Of course there should be a policy on how long a version will stay in the database, and there should be some utilities to find previous versions. But that's not the most difficult step I think. Stef Bon |
From: Michal L. <mi...@lo...> - 2008-03-30 11:14:06
|
Smart Weblications GmbH, Florian Wiessner wrote: > Hi, > > Michal Ludvig schrieb: >> But that all depends on your exact requirements. > > Yes, you are right. And for that purpose there is mysql-binlog. Binlog was designed for a totally different purpose and as such is hardly usable for a convenient revision tracking on mysqlfs. For instance you'll have to keep all binlogs from when the FS was created to be able to "replay" the filesystem to some point in time (and honestly who keeps binlogs longer then a few days after they get their way to slaves). It would work on the whole-filesystem level, not for particular files (which may or may not be what Stef wanted). It isn't user-accessible. Etc. Michal |
From: Smart W. G. F. W. <f.w...@sm...> - 2008-03-30 10:54:20
|
Hi, Michal Ludvig schrieb: > Hi, > > - I personally think you'll run out of space pretty quickly ;-) > > - There need to be some way to remove old revisions. > > - You probably only want to keep revisions of files in some defined > state, i.e. when they are closed. Keeping each and every revision after > each write() is useless as in most cases the file won't be consistent > anyway. > > But that all depends on your exact requirements. Yes, you are right. And for that purpose there is mysql-binlog. -- Mit freundlichen Grüßen, Florian Wiessner Smart Weblications GmbH Martinsberger Str. 1 D-95119 Naila fon.: +49 700 762 789 32 - 0,12 EUR/Min* fax.: +49 700 762 789 32 - 0,12 EUR/Min* 24/7: +49 900 311 886 00 - 1,99 EUR/Min* http://www.smart-weblications.de -- Sitz der Gesellschaft: Naila Geschäftsführer: Florian Wiessner HRB-Nr.: HRB 3840 Amtsgericht Hof *aus dem dt. Festnetz, ggf. abweichende Preise aus dem Mobilfunknetz |
From: Michal L. <mi...@lo...> - 2008-03-30 10:19:55
|
Hi, - I personally think you'll run out of space pretty quickly ;-) - There need to be some way to remove old revisions. - You probably only want to keep revisions of files in some defined state, i.e. when they are closed. Keeping each and every revision after each write() is useless as in most cases the file won't be consistent anyway. But that all depends on your exact requirements. Michal Stef Bon wrote: > Hello, > > I would like to try to add the ability to mysqlfs to keep track of all > previous versions > of files and directories (and..). Imho mysqlfs is ver suitable for doing > so, because it's already in a database. > > First (before getting technical) I would like to know how you (reader) > feel about this. Is there anyone having thoughts > about this? > > Stef Bon > > ------------------------------------------------------------------------- > Check out the new SourceForge.net Marketplace. > It's the best place to buy or sell services for > just about anything Open Source. > http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace > _______________________________________________ > Mysqlfs-general mailing list > Mys...@li... > https://lists.sourceforge.net/lists/listinfo/mysqlfs-general |
From: Stef B. <st...@bo...> - 2008-03-30 10:08:02
|
Hello, I would like to try to add the ability to mysqlfs to keep track of all previous versions of files and directories (and..). Imho mysqlfs is ver suitable for doing so, because it's already in a database. First (before getting technical) I would like to know how you (reader) feel about this. Is there anyone having thoughts about this? Stef Bon |
From: Stef B. <st...@bo...> - 2008-03-16 09:47:51
|
Stef Bon wrote: > Hello, > > now I can check the flow of what happens (by activating debugging). It's > much more clear now. > > But when I look at query_inode_full, I do not understand what happens > there at all. > The purpose of this function is to translate a path into a inode, right? > But it does much more things. > It "returns" a table nlinks, the inode, the parent. (and more?) > > Can someone explain why this is? It runs complicated queries.... > Ok, now I've read it a couple of times, I understand it more or less. A (long) query is constructed to determine the inode, with the help of a LEFT JOIN. This query does grow bigger as the path is longer (deeper). I'm looking into the mysql syntax now. Any hints about it welcome. Stef Bon |
From: Stef B. <st...@bo...> - 2008-03-09 11:45:11
|
Hello, now I can check the flow of what happens (by activating debugging). It's much more clear now. But when I look at query_inode_full, I do not understand what happens there at all. The purpose of this function is to translate a path into a inode, right? But it does much more things. It "returns" a table nlinks, the inode, the parent. (and more?) Can someone explain why this is? It runs complicated queries.... Thanks in advance, Stef Bon |
From: Michal L. <mi...@lo...> - 2008-03-06 10:34:06
|
Stef Bon wrote: > Michal Ludvig wrote: >> have a look at mysqlfs_truncate() in mysqlfs.c and query_truncate() in >> query.c. These two ensure that the non-used blocks are removed. > > Ok I've found these functions. Are they called by Fuse, cause I did not > find where this truncate > will be done. Yes, mysqlfs_truncate is called by fuse, or more precisely, by the kernel. Recompile with debugging on and you'll get a log message for each function called. Then you will get a picture of the workflow. Michal |
From: Stef B. <st...@bo...> - 2008-03-06 10:19:51
|
Michal Ludvig wrote: > Stef Bon wrote: > > >> First the overwriting of a file by another smaller one. This makes >> (if both files differ much in size and both are big enough) that >> datablocks will be deleted. I've been searching the >> sourcecode, but cannot find out how this works exactly. So before a file >> is written, there should be a procedure which >> checks the file already exits, and removes the datablocks which will not >> be used. Is this right? If so, where can I find this >> in the sourcecode? >> > > Hi, > have a look at mysqlfs_truncate() in mysqlfs.c and query_truncate() in > query.c. These two ensure that the non-used blocks are removed. > > Thanks for the fast reply. Ok I've found these functions. Are they called by Fuse, cause I did not find where this truncate will be done. Stef Bon > Michal > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Mysqlfs-general mailing list > Mys...@li... > https://lists.sourceforge.net/lists/listinfo/mysqlfs-general > > |
From: Michal L. <mi...@lo...> - 2008-03-05 20:35:17
|
Stef Bon wrote: > First the overwriting of a file by another smaller one. This makes > (if both files differ much in size and both are big enough) that > datablocks will be deleted. I've been searching the > sourcecode, but cannot find out how this works exactly. So before a file > is written, there should be a procedure which > checks the file already exits, and removes the datablocks which will not > be used. Is this right? If so, where can I find this > in the sourcecode? Hi, have a look at mysqlfs_truncate() in mysqlfs.c and query_truncate() in query.c. These two ensure that the non-used blocks are removed. Michal |
From: Stef B. <st...@bo...> - 2008-03-05 16:58:48
|
Hello, earlier I've posted a message in this list. I've mentioned the "multithreading (im)possibilities" of mysqlfs. I thought mysqlfs does not support multithreading. Just recent I've discovered that it does. At least, it's possible to make use of the same mountpoint by more users the same time. I would like to investigate it's possible to add support for "maintaining versions of files and directories". I think this is very possible because it's in a database. But first I'm trying to understand how mysqlfs works. So I've got some questions. If anyone would be so kind to answer them. First the overwriting of a file by another smaller one. This makes (if both files differ much in size and both are big enough) that datablocks will be deleted. I've been searching the sourcecode, but cannot find out how this works exactly. So before a file is written, there should be a procedure which checks the file already exits, and removes the datablocks which will not be used. Is this right? If so, where can I find this in the sourcecode? Stef Bon |