Menu
▾
▴
moosefs-users
|
From: <pen...@ic...> - 2015-09-18 07:22:36
|
hi all:
How many clients does Moosefs max support?
Thanks for your help.
best regards.
pen...@ic...
|
|
From: Aleksander W. <ale...@mo...> - 2015-09-18 09:31:00
|
Hi All depends. Question is how many clients you want to have and how many operations they will perform on one master? We have instances with about 300 client in production environment. In our software we don't have any client connection limits, but number of parallel connections close to 10 000 creates different level of problem. Best regards Aleksander Wieliczko Technical Support Engineer MooseFS.com <moosefs.com> On 18.09.2015 09:20, pen...@ic... wrote: > hi all: > > How many clients does Moosefs max support? > > Thanks for your help. > > best regards. > ------------------------------------------------------------------------ > pen...@ic... > > > ------------------------------------------------------------------------------ > > > _________________________________________ > moosefs-users mailing list > moo...@li... > https://lists.sourceforge.net/lists/listinfo/moosefs-users |
|
From: Davies L. <dav...@gm...> - 2015-09-18 17:21:29
|
I think MFS can't scale well if there are more than 2k clients. We saw the CPU usage of master went to almost 100% with about 1k clients and not high traffic. In order to scale to thousands of clients, MFS should use epoll/kqueue. I have a branch that use epoll instead of poll: https://github.com/davies/moosefs/commits/douban On Fri, Sep 18, 2015 at 2:31 AM, Aleksander Wieliczko <ale...@mo...> wrote: > Hi > All depends. > Question is how many clients you want to have and how many operations they > will perform on one master? > > We have instances with about 300 client in production environment. > In our software we don't have any client connection limits, but number of > parallel connections close to 10 000 creates different level of problem. > > Best regards > Aleksander Wieliczko > Technical Support Engineer > MooseFS.com > > On 18.09.2015 09:20, pen...@ic... wrote: > > hi all: > > How many clients does Moosefs max support? > > Thanks for your help. > > best regards. > ________________________________ > pen...@ic... > > > ------------------------------------------------------------------------------ > > > > _________________________________________ > moosefs-users mailing list > moo...@li... > https://lists.sourceforge.net/lists/listinfo/moosefs-users > > > > ------------------------------------------------------------------------------ > > _________________________________________ > moosefs-users mailing list > moo...@li... > https://lists.sourceforge.net/lists/listinfo/moosefs-users > -- - Davies |
|
From: <pen...@ic...> - 2015-09-19 01:04:03
|
That's great. But master is single thread, I think this is one of a limitation to master. pen...@ic... From: Davies Liu Date: 2015-09-19 01:21 To: Aleksander Wieliczko CC: pen...@ic...; moosefs-users Subject: Re: [MooseFS-Users] How many clients dose MFS max support? I think MFS can't scale well if there are more than 2k clients. We saw the CPU usage of master went to almost 100% with about 1k clients and not high traffic. In order to scale to thousands of clients, MFS should use epoll/kqueue. I have a branch that use epoll instead of poll: https://github.com/davies/moosefs/commits/douban On Fri, Sep 18, 2015 at 2:31 AM, Aleksander Wieliczko <ale...@mo...> wrote: > Hi > All depends. > Question is how many clients you want to have and how many operations they > will perform on one master? > > We have instances with about 300 client in production environment. > In our software we don't have any client connection limits, but number of > parallel connections close to 10 000 creates different level of problem. > > Best regards > Aleksander Wieliczko > Technical Support Engineer > MooseFS.com > > On 18.09.2015 09:20, pen...@ic... wrote: > > hi all: > > How many clients does Moosefs max support? > > Thanks for your help. > > best regards. > ________________________________ > pen...@ic... > > > ------------------------------------------------------------------------------ > > > > _________________________________________ > moosefs-users mailing list > moo...@li... > https://lists.sourceforge.net/lists/listinfo/moosefs-users > > > > ------------------------------------------------------------------------------ > > _________________________________________ > moosefs-users mailing list > moo...@li... > https://lists.sourceforge.net/lists/listinfo/moosefs-users > -- - Davies |
|
From: Jakub Kruszona-Z. <jak...@ge...> - 2015-09-21 05:21:09
|
On 19 Sep, 2015, at 3:02, pen...@ic... wrote: > That's great. > But master is single thread, I think this is one of a limitation to master. Not exactly. Very often single threaded apps are as fast as multithreaded (sometimes even faster - and always much safer and much less buggy). In multithreaded apps CPU spends a lot of time on synchronization. Also whole architecture of application is usually much worse. Davies is right - to handle thousands of clients the most important improvement is to change poll to epoll/kqueue and it is on our roadmap. This should speed up master significantly in case of huge number of clients/chunkservers. > > pen...@ic... > > From: Davies Liu > Date: 2015-09-19 01:21 > To: Aleksander Wieliczko > CC: pen...@ic...; moosefs-users > Subject: Re: [MooseFS-Users] How many clients dose MFS max support? > I think MFS can't scale well if there are more than 2k clients. We saw > the CPU usage of master went to almost 100% with about 1k clients and > not high traffic. > > In order to scale to thousands of clients, MFS should use > epoll/kqueue. I have a branch that use epoll instead of poll: > https://github.com/davies/moosefs/commits/douban > > On Fri, Sep 18, 2015 at 2:31 AM, Aleksander Wieliczko > <ale...@mo...> wrote: > > Hi > > All depends. > > Question is how many clients you want to have and how many operations they > > will perform on one master? > > > > We have instances with about 300 client in production environment. > > In our software we don't have any client connection limits, but number of > > parallel connections close to 10 000 creates different level of problem. > > > > Best regards > > Aleksander Wieliczko > > Technical Support Engineer > > MooseFS.com > > > > On 18.09.2015 09:20, pen...@ic... wrote: > > > > hi all: > > > > How many clients does Moosefs max support? > > > > Thanks for your help. > > > > best regards. > > ________________________________ > > pen...@ic... > > > > > > ------------------------------------------------------------------------------ > > > > > > > > _________________________________________ > > moosefs-users mailing list > > moo...@li... > > https://lists.sourceforge.net/lists/listinfo/moosefs-users > > > > > > > > ------------------------------------------------------------------------------ > > > > _________________________________________ > > moosefs-users mailing list > > moo...@li... > > https://lists.sourceforge.net/lists/listinfo/moosefs-users > > > > > > -- > - Davies > ------------------------------------------------------------------------------ > _________________________________________ > moosefs-users mailing list > moo...@li... > https://lists.sourceforge.net/lists/listinfo/moosefs-users -- Pozdrawiam, Jakub Kruszona-Zawadzki - - - - - - - - - - - - - - - - Segmentation fault (core dumped) Tel. +48 602 212 039 |
|
From: <pen...@ic...> - 2015-09-18 09:57:11
|
Hi Aleksander Thanks for your answer. In our production environment, clients close to 100 * 1000. we have no idea to test the performance of parallel connections before online. Best regards. pen...@ic... From: Aleksander Wieliczko Date: 2015-09-18 17:31 To: pen...@ic...; moosefs-users Subject: Re: [MooseFS-Users] How many clients dose MFS max support? Hi All depends. Question is how many clients you want to have and how many operations they will perform on one master? We have instances with about 300 client in production environment. In our software we don't have any client connection limits, but number of parallel connections close to 10 000 creates different level of problem. Best regards Aleksander Wieliczko Technical Support Engineer MooseFS.com On 18.09.2015 09:20, pen...@ic... wrote: hi all: How many clients does Moosefs max support? Thanks for your help. best regards. pen...@ic... ------------------------------------------------------------------------------ _________________________________________ moosefs-users mailing list moo...@li... https://lists.sourceforge.net/lists/listinfo/moosefs-users |
|
From: Warren M. <wa...@an...> - 2015-09-18 16:16:29
|
There are a number of mitigation strategies to reduce the number of direct connections to the MFS Master:have MFS power a file server (ownCloud, CIFS, NFS, etc)have MFS power a web/ftp serveretc Warren Myers http://antipaucity.com https://www.digitalocean.com/?refcode=d197a961987a Date: Fri, 18 Sep 2015 17:55:27 +0800 From: pen...@ic... To: ale...@mo...; moo...@li... Subject: Re: [MooseFS-Users] How many clients dose MFS max support? Hi Aleksander Thanks for your answer.In our production environment, clients close to 100 * 1000. we have no idea to test the performance of parallel connections before online. Best regards. pen...@ic... From: Aleksander WieliczkoDate: 2015-09-18 17:31To: pen...@ic...; moosefs-usersSubject: Re: [MooseFS-Users] How many clients dose MFS max support? Hi All depends. Question is how many clients you want to have and how many operations they will perform on one master? We have instances with about 300 client in production environment. In our software we don't have any client connection limits, but number of parallel connections close to 10 000 creates different level of problem. Best regards Aleksander Wieliczko Technical Support Engineer MooseFS.com On 18.09.2015 09:20, pen...@ic... wrote: hi all: How many clients does Moosefs max support? Thanks for your help. best regards. pen...@ic... ------------------------------------------------------------------------------ _________________________________________ moosefs-users mailing list moo...@li... https://lists.sourceforge.net/lists/listinfo/moosefs-users ------------------------------------------------------------------------------ _________________________________________ moosefs-users mailing list moo...@li... https://lists.sourceforge.net/lists/listinfo/moosefs-users |
×
Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.
Thanks for helping keep SourceForge clean.
X