Menu
▾
▴
moosefs-users
|
From: JJ <jj...@ci...> - 2012-02-13 18:20:53
|
Now that our moosefs install is functional, We want to provide access to our Clients that are bound to a Window-based OS. I would like to know if mounting (using mfsmount, running ./configure per the client installl) on the mfsmaster is a good idea? If it suggested ok to do, then I can ask our Windows users to install WinSCP and then I'd alter the mfs user's $home directory to be /mnt/mfs -- JJ Support Engineer Cirrhus9.com |
|
From: Travis H. <tra...@tr...> - 2012-02-13 18:38:23
|
You would need to consider your network topology (e.g. in our environment we have the moosefs master and chunk servers and mfs mount clients in their own back end network segment, and a virtual machine that runs mfsmount and samba is the access point or "gateway" into the file system for the windows users. We tried to follow the design as one would do an iSCSI san, where dedicated "SAN" network segments are used for the back end storage to the application servers that is different from the front end facing network segments the end users and clients invoke the applications with. You might also consider the amount of network IO operations that would be done between the windows machines and the (SCP operations?) mount point that exposes the mfsmount-ed file system. Where by depending on the speed of the network (100MB, GB ethernet) and the amount of concurent requests, you might find contention for the network link and this could reduce the performance of the file system as a whole. but this is entirely subjective . But functionally there is no technical consideration why you could not run mfsmount on the mfsmaster node. On 12-02-13 1:17 PM, JJ wrote: > Now that our moosefs install is functional, > We want to provide access to our Clients that are bound > to a Window-based OS. > > I would like to know if mounting (using mfsmount, running ./configure > per the client installl) on the mfsmaster is a good idea? > > If it suggested ok to do, then I can ask our Windows users to install > WinSCP and then I'd alter the mfs user's $home directory to be /mnt/mfs > |
|
From: JJ <jj...@ci...> - 2012-02-13 18:42:04
|
Perfect! Thank you. JJ Support Engineer Cirrhus9.com On 02/13/2012 01:38 PM, Travis Hein wrote: > ... > But functionally there is no technical consideration why you could not > run mfsmount on the mfsmaster node. > ... |
|
From: JJ <jj...@ci...> - 2012-02-13 22:01:19
|
What would be the correct ./configure options|command to install mfsmount on a working mfsmaster? I tried ./configure --prefix=/usr --sysconfdir=/etc/mfs --localstatedir=/var/lib --with-default-user=mfs --with-default-group=mfs --disable-mfschunkserver && make with a find looking for it. Nothing. Tried fewer options (--disable*), no joy. Thank you for your time. JJ Support Engineer Cirrhus9.com On 02/13/2012 01:38 PM, Travis Hein wrote: > You would need to consider your network topology (e.g. in our > environment we have the moosefs master and chunk servers and mfs mount > clients in their own back end network segment, and a virtual machine > that runs mfsmount and samba is the access point or "gateway" into the > file system for the windows users. We tried to follow the design as one > would do an iSCSI san, where dedicated "SAN" network segments are used > for the back end storage to the application servers that is different > from the front end facing network segments the end users and clients > invoke the applications with. > > You might also consider the amount of network IO operations that would > be done between the windows machines and the (SCP operations?) mount > point that exposes the mfsmount-ed file system. Where by depending on > the speed of the network (100MB, GB ethernet) and the amount of > concurent requests, you might find contention for the network link and > this could reduce the performance of the file system as a whole. but > this is entirely subjective . > > But functionally there is no technical consideration why you could not > run mfsmount on the mfsmaster node. > > On 12-02-13 1:17 PM, JJ wrote: >> Now that our moosefs install is functional, >> We want to provide access to our Clients that are bound >> to a Window-based OS. >> >> I would like to know if mounting (using mfsmount, running ./configure >> per the client installl) on the mfsmaster is a good idea? >> >> If it suggested ok to do, then I can ask our Windows users to install >> WinSCP and then I'd alter the mfs user's $home directory to be /mnt/mfs >> > > > ------------------------------------------------------------------------------ > Try before you buy = See our experts in action! > The most comprehensive online learning library for Microsoft developers > is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, > Metro Style Apps, more. Free future releases when you subscribe now! > http://p.sf.net/sfu/learndevnow-dev2 > _______________________________________________ > moosefs-users mailing list > moo...@li... > https://lists.sourceforge.net/lists/listinfo/moosefs-users |
|
From: Steve <st...@bo...> - 2012-02-14 17:15:11
|
Hi, Is async enabled by default ? What do I look for ? I mainly use one share but windows desktop drops out periodically seems more often with > 4 torrent downloads at unthrottled adsl bandwidths. I did think this was ipcop as a reboot fixes but now I think back unlikely because ive moved from ipcop1.4 to ipcop2 on different hardware and same problem exists. ipcop is linking desktop to mfs hardware subnet. Steve -------Original Message------- From: Robert Sandilands Date: 14/02/2012 00:13:42 To: moo...@li... Subject: Re: [Moosefs-users] Question re: win32 client/access I have to disagree. mfsmaster is very sensitive to load. You really want to run a dedicated master server. Setting up a (virtual) machine or even multiple (virtual) machines to mount the file system is a better idea. Ensure that you disable all asynchronous file access in Samba. FUSE and sendfile() is not a great combination. I would also recommend having a mount per SMB share. For example if you plan to have 3 shares called "tools", "docs" and "tmp" then I would mount MooseFS on three folders: /mnt/mfs_tools, /mnt/mfs_docs and /mnt/mfs_tmp and share those points using SMB. This implies running 3 instances of mfsmount. It works around some bottlenecks in mfsmount. Robert On 2/13/12 1:38 PM, JJ wrote: > Perfect! > > Thank you. > > > JJ > Support Engineer > Cirrhus9.com > > On 02/13/2012 01:38 PM, Travis Hein wrote: >> ... >> But functionally there is no technical consideration why you could not >> run mfsmount on the mfsmaster node. >> ... > ----------------------------------------------------------------------------- > Try before you buy = See our experts in action! > The most comprehensive online learning library for Microsoft developers > is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, > Metro Style Apps, more. Free future releases when you subscribe now! > http://p.sf.net/sfu/learndevnow-dev2 > _______________________________________________ > moosefs-users mailing list > moo...@li... > https://lists.sourceforge.net/lists/listinfo/moosefs-users ----------------------------------------------------------------------------- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d _______________________________________________ moosefs-users mailing list moo...@li... https://lists.sourceforge.net/lists/listinfo/moosefs-users |
|
From: Robert S. <rsa...@ne...> - 2012-02-15 00:29:32
|
man smb.conf Search for sendfile. Robert On 2/14/12 12:14 PM, Steve wrote: > Hi, > > > > Is async enabled by default ? What do I look for ? > > > > I mainly use one share but windows desktop drops out periodically seems more > often with> 4 torrent downloads at unthrottled adsl bandwidths. I did think > this was ipcop as a reboot fixes but now I think back unlikely because ive > moved from ipcop1.4 to ipcop2 on different hardware and same problem exists. > ipcop is linking desktop to mfs hardware subnet. > > > > Steve > > > > > > > > > > -------Original Message------- > > > > From: Robert Sandilands > > Date: 14/02/2012 00:13:42 > > To: moo...@li... > > Subject: Re: [Moosefs-users] Question re: win32 client/access > > > > I have to disagree. > > > > mfsmaster is very sensitive to load. You really want to run a dedicated > > master server. Setting up a (virtual) machine or even multiple (virtual) > > machines to mount the file system is a better idea. > > > > Ensure that you disable all asynchronous file access in Samba. FUSE and > > sendfile() is not a great combination. > > > > I would also recommend having a mount per SMB share. > > > > For example if you plan to have 3 shares called "tools", "docs" and > > "tmp" then I would mount MooseFS on three folders: /mnt/mfs_tools, > > /mnt/mfs_docs and /mnt/mfs_tmp and share those points using SMB. This > > implies running 3 instances of mfsmount. It works around some > > bottlenecks in mfsmount. > > > > Robert > > > > On 2/13/12 1:38 PM, JJ wrote: > >> Perfect! >> Thank you. >> JJ >> Support Engineer >> Cirrhus9.com >> On 02/13/2012 01:38 PM, Travis Hein wrote: >>> ... >>> But functionally there is no technical consideration why you could not >>> run mfsmount on the mfsmaster node. >>> ... > ----------------------------------------------------------------------------- > > >> Try before you buy = See our experts in action! >> The most comprehensive online learning library for Microsoft developers >> is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, >> Metro Style Apps, more. Free future releases when you subscribe now! >> http://p.sf.net/sfu/learndevnow-dev2 >> _______________________________________________ >> moosefs-users mailing list >> moo...@li... >> https://lists.sourceforge.net/lists/listinfo/moosefs-users > > > > > ----------------------------------------------------------------------------- > > > Keep Your Developer Skills Current with LearnDevNow! > > The most comprehensive online learning library for Microsoft developers > > is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, > > Metro Style Apps, more. Free future releases when you subscribe now! > > http://p.sf.net/sfu/learndevnow-d2d > > _______________________________________________ > > moosefs-users mailing list > > moo...@li... > > https://lists.sourceforge.net/lists/listinfo/moosefs-users > > |
|
From: Travis H. <tra...@tr...> - 2012-02-13 22:22:14
|
try adding the configure parameter "--enable-mfsmount" this will force the system prerequisite libraries to be present, otherwise the configure script will fail with the message "require ..... for mfsmount" . The mfsmount uses the FUSE libraries and the zlibrary, and the system requires (in addition to the usual build tools like make, gcc), the developer packages of these libraries to be installed, so the headers are available. By default the configure script will just decide to not bother trying to build the mfsmount client if one or more of these libraries / headers are not available, and continue to build the rest of it. e.g. usually on my CentOS machines, I install these fuse-devel packages with yum install fuse-devel zlib-devel and then (in your case) ./configure --prefix=/usr --sysconfdir=/etc/mfs --localstatedir=/var/lib --with-default-user=mfs --with-default-group=mfs --disable-mfschunkserver --enable-mfsmount On 12-02-13 4:57 PM, JJ wrote: > What would be the correct ./configure options|command to install > mfsmount on a working mfsmaster? > > I tried ./configure --prefix=/usr --sysconfdir=/etc/mfs > --localstatedir=/var/lib --with-default-user=mfs > --with-default-group=mfs --disable-mfschunkserver&& make > > with a find looking for it. Nothing. > > Tried fewer options (--disable*), no joy. > > Thank you for your time. > > > JJ > Support Engineer > Cirrhus9.com > |
|
From: JJ <jj...@ci...> - 2012-02-13 22:44:30
|
Perfect. Thanks! JJ Support Engineer Cirrhus9.com On 02/13/2012 05:22 PM, Travis Hein wrote: > try adding the configure parameter "--enable-mfsmount" |
|
From: Robert S. <rsa...@ne...> - 2012-02-14 00:11:55
|
I have to disagree. mfsmaster is very sensitive to load. You really want to run a dedicated master server. Setting up a (virtual) machine or even multiple (virtual) machines to mount the file system is a better idea. Ensure that you disable all asynchronous file access in Samba. FUSE and sendfile() is not a great combination. I would also recommend having a mount per SMB share. For example if you plan to have 3 shares called "tools", "docs" and "tmp" then I would mount MooseFS on three folders: /mnt/mfs_tools, /mnt/mfs_docs and /mnt/mfs_tmp and share those points using SMB. This implies running 3 instances of mfsmount. It works around some bottlenecks in mfsmount. Robert On 2/13/12 1:38 PM, JJ wrote: > Perfect! > > Thank you. > > > JJ > Support Engineer > Cirrhus9.com > > On 02/13/2012 01:38 PM, Travis Hein wrote: >> ... >> But functionally there is no technical consideration why you could not >> run mfsmount on the mfsmaster node. >> ... > ------------------------------------------------------------------------------ > Try before you buy = See our experts in action! > The most comprehensive online learning library for Microsoft developers > is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, > Metro Style Apps, more. Free future releases when you subscribe now! > http://p.sf.net/sfu/learndevnow-dev2 > _______________________________________________ > moosefs-users mailing list > moo...@li... > https://lists.sourceforge.net/lists/listinfo/moosefs-users |
|
From: Palak, G. (N. - PL/Wroclaw) <grz...@ns...> - 2012-02-17 09:44:09
|
@Robert What mfsmount bottlenecks you have on mind? Have you noticed any performance related bottlenecks? Greg -----Original Message----- From: ext Robert Sandilands [mailto:rsa...@ne...] Sent: Tuesday, February 14, 2012 12:56 AM To: moo...@li... Subject: Re: [Moosefs-users] Question re: win32 client/access I have to disagree. mfsmaster is very sensitive to load. You really want to run a dedicated master server. Setting up a (virtual) machine or even multiple (virtual) machines to mount the file system is a better idea. Ensure that you disable all asynchronous file access in Samba. FUSE and sendfile() is not a great combination. I would also recommend having a mount per SMB share. For example if you plan to have 3 shares called "tools", "docs" and "tmp" then I would mount MooseFS on three folders: /mnt/mfs_tools, /mnt/mfs_docs and /mnt/mfs_tmp and share those points using SMB. This implies running 3 instances of mfsmount. It works around some bottlenecks in mfsmount. Robert On 2/13/12 1:38 PM, JJ wrote: > Perfect! > > Thank you. > > > JJ > Support Engineer > Cirrhus9.com > > On 02/13/2012 01:38 PM, Travis Hein wrote: >> ... >> But functionally there is no technical consideration why you could not >> run mfsmount on the mfsmaster node. >> ... > ------------------------------------------------------------------------ ------ > Try before you buy = See our experts in action! > The most comprehensive online learning library for Microsoft developers > is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, > Metro Style Apps, more. Free future releases when you subscribe now! > http://p.sf.net/sfu/learndevnow-dev2 > _______________________________________________ > moosefs-users mailing list > moo...@li... > https://lists.sourceforge.net/lists/listinfo/moosefs-users ------------------------------------------------------------------------ ------ Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d _______________________________________________ moosefs-users mailing list moo...@li... https://lists.sourceforge.net/lists/listinfo/moosefs-users |
|
From: Robert S. <rsa...@ne...> - 2012-02-17 14:16:30
|
Hi Greg, I have isolated several issues: 1. mfsmaster is single threaded. This implies that if it has to share time with any other processes on the same machine then performance can suffer significantly. 2. mfsmount has a single TCP connection to mfsmaster. This implies that most accesses to all the files on a mount has to share a single TCP connection to the master. This is a significant bottleneck. 3. mfschunkserver has 10 threads doing work for all the mfsmount's and 10 threads doing work for mfsmaster. This easily gets overwhelmed on a busy system. Running mfsmaster on a dedicated machine helps with the first issue. Running multiple copies of mfsmount helps with the second issue. Limiting the number of disks managed by a specific mfschunkserver instance helps with the third issue. My guess is that you should have one instance of mfsmount for about every 20 simultaneous accesses. You also should have 1 instance of mfschunkserver for every 10 disks (spindles). A dedicated mfsmaster is also a very good idea. Robert On 2/17/12 4:43 AM, Palak, Grzegorz (NSN - PL/Wroclaw) wrote: > @Robert > What mfsmount bottlenecks you have on mind? > Have you noticed any performance related bottlenecks? > > Greg > > -----Original Message----- > From: ext Robert Sandilands [mailto:rsa...@ne...] > Sent: Tuesday, February 14, 2012 12:56 AM > To: moo...@li... > Subject: Re: [Moosefs-users] Question re: win32 client/access > > I have to disagree. > > mfsmaster is very sensitive to load. You really want to run a dedicated > master server. Setting up a (virtual) machine or even multiple (virtual) > > machines to mount the file system is a better idea. > > Ensure that you disable all asynchronous file access in Samba. FUSE and > sendfile() is not a great combination. > > I would also recommend having a mount per SMB share. > > For example if you plan to have 3 shares called "tools", "docs" and > "tmp" then I would mount MooseFS on three folders: /mnt/mfs_tools, > /mnt/mfs_docs and /mnt/mfs_tmp and share those points using SMB. This > implies running 3 instances of mfsmount. It works around some > bottlenecks in mfsmount. > > Robert > > On 2/13/12 1:38 PM, JJ wrote: >> Perfect! >> >> Thank you. >> >> >> JJ >> Support Engineer >> Cirrhus9.com >> >> On 02/13/2012 01:38 PM, Travis Hein wrote: >>> ... >>> But functionally there is no technical consideration why you could > not >>> run mfsmount on the mfsmaster node. >>> ... > ------------------------------------------------------------------------ > ------ >> Try before you buy = See our experts in action! >> The most comprehensive online learning library for Microsoft > developers >> is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, > MVC3, >> Metro Style Apps, more. Free future releases when you subscribe now! >> http://p.sf.net/sfu/learndevnow-dev2 >> _______________________________________________ >> moosefs-users mailing list >> moo...@li... >> https://lists.sourceforge.net/lists/listinfo/moosefs-users > > ------------------------------------------------------------------------ > ------ > Keep Your Developer Skills Current with LearnDevNow! > The most comprehensive online learning library for Microsoft developers > is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, > Metro Style Apps, more. Free future releases when you subscribe now! > http://p.sf.net/sfu/learndevnow-d2d > _______________________________________________ > moosefs-users mailing list > moo...@li... > https://lists.sourceforge.net/lists/listinfo/moosefs-users |
|
From: Atom P. <ap...@di...> - 2012-02-17 22:20:40
|
Robert, your advice helped me design my MFS system. One more thing I would add: chunk replication and client caching had a large effect on the performance of my clients. I have a dedicated mfsmaster with a fast CPU and plenty of RAM, an mfsmount for each service that uses mfs (1-8 per host, Ubuntu), and each chunkservers has one large disk (FreeBSD,zfs). I had originally increased my chunk replication from write-1,read-5 to write-5,read-15 while I was adding chunkservers to the system. Once replication was done and I had added many clients (~12) I noticed that listing files on mfs mounts was slow, 10-20 seconds for a simple 'ls'. The master charts showed no change in CPU or anything else before and after the slowness started. Reducing the replication back to 1,5 restored performance. I still don't know why; all chunkservers were already balanced. Also, enabling 'mfscachefiles' on the clients made a huge improvement on CPU use on the mfsmaster. On 02/17/2012 06:16 AM, Robert Sandilands wrote: > Hi Greg, > > I have isolated several issues: > > 1. mfsmaster is single threaded. This implies that if it has to share > time with any other processes on the same machine then performance can > suffer significantly. > 2. mfsmount has a single TCP connection to mfsmaster. This implies that > most accesses to all the files on a mount has to share a single TCP > connection to the master. This is a significant bottleneck. > 3. mfschunkserver has 10 threads doing work for all the mfsmount's and > 10 threads doing work for mfsmaster. This easily gets overwhelmed on a > busy system. > > Running mfsmaster on a dedicated machine helps with the first issue. > > Running multiple copies of mfsmount helps with the second issue. > > Limiting the number of disks managed by a specific mfschunkserver > instance helps with the third issue. > > My guess is that you should have one instance of mfsmount for about > every 20 simultaneous accesses. You also should have 1 instance of > mfschunkserver for every 10 disks (spindles). A dedicated mfsmaster is > also a very good idea. > -- -- Perfection is just a word I use occasionally with mustard. --Atom Powers-- Director of IT DigiPen Institute of Technology +1 (425) 895-4443 |
×
Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.
Thanks for helping keep SourceForge clean.
X