Menu
▾
▴
moosefs-users — Mailing list for MooseFS Users
You can subscribe to this list here.
| 2009 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
(4) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2010 |
Jan
(20) |
Feb
(11) |
Mar
(11) |
Apr
(9) |
May
(22) |
Jun
(85) |
Jul
(94) |
Aug
(80) |
Sep
(72) |
Oct
(64) |
Nov
(69) |
Dec
(89) |
| 2011 |
Jan
(72) |
Feb
(109) |
Mar
(116) |
Apr
(117) |
May
(117) |
Jun
(102) |
Jul
(91) |
Aug
(72) |
Sep
(51) |
Oct
(41) |
Nov
(55) |
Dec
(74) |
| 2012 |
Jan
(45) |
Feb
(77) |
Mar
(99) |
Apr
(113) |
May
(132) |
Jun
(75) |
Jul
(70) |
Aug
(58) |
Sep
(58) |
Oct
(37) |
Nov
(51) |
Dec
(15) |
| 2013 |
Jan
(28) |
Feb
(16) |
Mar
(25) |
Apr
(38) |
May
(23) |
Jun
(39) |
Jul
(42) |
Aug
(19) |
Sep
(41) |
Oct
(31) |
Nov
(18) |
Dec
(18) |
| 2014 |
Jan
(17) |
Feb
(19) |
Mar
(39) |
Apr
(16) |
May
(10) |
Jun
(13) |
Jul
(17) |
Aug
(13) |
Sep
(8) |
Oct
(53) |
Nov
(23) |
Dec
(7) |
| 2015 |
Jan
(35) |
Feb
(13) |
Mar
(14) |
Apr
(56) |
May
(8) |
Jun
(18) |
Jul
(26) |
Aug
(33) |
Sep
(40) |
Oct
(37) |
Nov
(24) |
Dec
(20) |
| 2016 |
Jan
(38) |
Feb
(20) |
Mar
(25) |
Apr
(14) |
May
(6) |
Jun
(36) |
Jul
(27) |
Aug
(19) |
Sep
(36) |
Oct
(24) |
Nov
(15) |
Dec
(16) |
| 2017 |
Jan
(8) |
Feb
(13) |
Mar
(17) |
Apr
(20) |
May
(28) |
Jun
(10) |
Jul
(20) |
Aug
(3) |
Sep
(18) |
Oct
(8) |
Nov
|
Dec
(5) |
| 2018 |
Jan
(15) |
Feb
(9) |
Mar
(12) |
Apr
(7) |
May
(123) |
Jun
(41) |
Jul
|
Aug
(14) |
Sep
|
Oct
(15) |
Nov
|
Dec
(7) |
| 2019 |
Jan
(2) |
Feb
(9) |
Mar
(2) |
Apr
(9) |
May
|
Jun
|
Jul
(2) |
Aug
|
Sep
(6) |
Oct
(1) |
Nov
(12) |
Dec
(2) |
| 2020 |
Jan
(2) |
Feb
|
Mar
|
Apr
(3) |
May
|
Jun
(4) |
Jul
(4) |
Aug
(1) |
Sep
(18) |
Oct
(2) |
Nov
|
Dec
|
| 2021 |
Jan
|
Feb
(3) |
Mar
|
Apr
|
May
|
Jun
|
Jul
(6) |
Aug
|
Sep
(5) |
Oct
(5) |
Nov
(3) |
Dec
|
| 2022 |
Jan
|
Feb
|
Mar
(3) |
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
|
From: Bán M. <ba...@vo...> - 2011-02-09 15:21:57
|
On Wed, 09 Feb 2011 18:09:29 +0300 Boli <bo...@le...> wrote: > just turn it on! > Its easy, thanks. Is it also works if the chunkserver's filesystem was modified and seemingly there is no differences among the chunk copies but there are some vicious chunks? Is the masterserver able to recognize the wrong copies and renew them? Thanks, Miklós > On Wed, 2011-02-09 at 16:06 +0100, Bán Miklós wrote: > > Hi, > > > > I have a chunkserver which have crashed. After two days its ready > > for work again. While it has been stopped the data changed on the > > mfs filesystem, so the chunks are not up-to date on this server. My > > question is, can I turn on this server without any worrying about > > the data integrity or is there a recommended way for this kind of > > situation? It is not really clear for me, is there any data > > integrity check among the chunkservers, or it is my responsibility > > to handle them, e.g. unmount all the clients, clean the affected > > chunserver and turn it on like a new chunkserver and finally mount > > the clients again? > > > > Thanks, Miklós > > > > ------------------------------------------------------------------------------ > > The ultimate all-in-one performance toolkit: Intel(R) Parallel > > Studio XE: Pinpoint memory and threading errors before they happen. > > Find and fix more than 250 security defects in the development > > cycle. Locate bottlenecks in serial and parallel code that limit > > performance. http://p.sf.net/sfu/intel-dev2devfeb > > _______________________________________________ > > moosefs-users mailing list > > moo...@li... > > https://lists.sourceforge.net/lists/listinfo/moosefs-users > |
|
From: Bán M. <ba...@vo...> - 2011-02-09 15:06:31
|
Hi, I have a chunkserver which have crashed. After two days its ready for work again. While it has been stopped the data changed on the mfs filesystem, so the chunks are not up-to date on this server. My question is, can I turn on this server without any worrying about the data integrity or is there a recommended way for this kind of situation? It is not really clear for me, is there any data integrity check among the chunkservers, or it is my responsibility to handle them, e.g. unmount all the clients, clean the affected chunserver and turn it on like a new chunkserver and finally mount the clients again? Thanks, Miklós |
|
From: Anh K. H. <ky...@vi...> - 2011-02-09 10:37:23
|
Hi, On Wed, 9 Feb 2011 08:58:17 +0100 "Michal Borychowski" <mic...@ge...> wrote: > We just tried to install postgresql on the machine: > MFS: 1.6.20 > Ubuntu 10.10 / 64bit > > with the "dpkg-buildpackage -us -nc -rfakeroot" command and > everything went smoothly. And we are not sure where to look further > for any error and how to debug it. Thank you for your information. I've tried as you said, and the results are weird: #1. dpkg-buildpackage -us # failed #2. dpkg-buildpackage -us -rfakeroot # passed #3. dpkg-buildpackage -us -nc -rfakeroot # failed That's really strange. It's seem that the problem happened randomly... I will test and give you more results. Regards, > > Here is a piece of log: > > ./pg_regress --inputdir=. --dlpath=. --multibyte=SQL_ASCII > --temp-install=./tmp_check --top-builddir=../../.. > --schedule=./parallel_schedule numeric_big > ============== creating temporary installation ============== > ============== initializing database system ============== > ============== starting postmaster ============== > running on port 57235 with pid 32628 > ============== creating database "regression" ============== > CREATE DATABASE > ALTER DATABASE > ============== running regression test queries ============== > test tablespace ... ok > parallel group (17 tests): int2 float4 name oid money text varchar > char int4 int8 boolean txid float8 bit uuid enum numeric > boolean ... ok > char ... ok > name ... ok > varchar ... ok > text ... ok > int2 ... ok > int4 ... ok > int8 ... ok > oid ... ok > float4 ... ok > float8 ... ok > bit ... ok > numeric ... ok > txid ... ok > uuid ... ok > enum ... ok > money ... ok > test strings ... ok > test numerology ... ok > parallel group (19 tests): point circle comments tstypes box lseg > abstime reltime tinterval timetz date polygon macaddr time > timestamp path interval inet timestamptz > point ... ok > lseg ... ok > box ... ok > path ... ok > polygon ... ok > circle ... ok > date ... ok > time ... ok > timetz ... ok > timestamp ... ok > timestamptz ... ok > interval ... ok > abstime ... ok > reltime ... ok > tinterval ... ok > inet ... ok > macaddr ... ok > tstypes ... ok > comments ... ok > parallel group (5 tests): geometry horology type_sanity oidjoins > opr_sanity geometry ... ok > horology ... ok > oidjoins ... ok > type_sanity ... ok > opr_sanity ... ok > test insert ... ok > test create_function_1 ... ok > test create_type ... ok > test create_table ... ok > test create_function_2 ... ok > parallel group (2 tests): copyselect copy > copy ... ok > copyselect ... ok > parallel group (10 tests): create_aggregate create_operator > create_cast drop_if_exists typed_table create_misc triggers vacuum > constraints inherit constraints ... ok > triggers ... ok > create_misc ... ok > create_aggregate ... ok > create_operator ... ok > inherit ... ok > typed_table ... ok > vacuum ... ok > drop_if_exists ... ok > create_cast ... ok > parallel group (2 tests): create_view create_index > create_index ... ok > create_view ... ok > test sanity_check ... ok > test errors ... ok > test select ... ok > parallel group (20 tests): select_distinct select_distinct_on > select_into btree_index hash_index random union update > select_having delete namespace case aggregates select_implicit > prepared_xacts subselect transactions portals arrays join > select_into ... ok > select_distinct ... ok > select_distinct_on ... ok > select_implicit ... ok > select_having ... ok > subselect ... ok > union ... ok > case ... ok > join ... ok > aggregates ... ok > transactions ... ok > random ... ok > portals ... ok > arrays ... ok > btree_index ... ok > hash_index ... ok > update ... ok > namespace ... ok > prepared_xacts ... ok > delete ... ok > test privileges ... ok > test misc ... ok > test rules ... ok > parallel group (13 tests): combocid select_views portals_p2 > tsdicts window xmlmap foreign_data guc dependency tsearch bitmapops > cluster foreign_key select_views ... ok > portals_p2 ... ok > foreign_key ... ok > cluster ... ok > dependency ... ok > guc ... ok > bitmapops ... ok > combocid ... ok > tsearch ... ok > tsdicts ... ok > foreign_data ... ok > window ... ok > xmlmap ... ok > parallel group (19 tests): limit conversion xml prepare > polymorphism largeobject without_oid with copy2 returning rowtypes > plancache sequence temp rangefuncs domain plpgsql truncate > alter_table plancache ... ok > limit ... ok > plpgsql ... ok > copy2 ... ok > temp ... ok > domain ... ok > rangefuncs ... ok > prepare ... ok > without_oid ... ok > conversion ... ok > truncate ... ok > alter_table ... ok > sequence ... ok > polymorphism ... ok > rowtypes ... ok > returning ... ok > largeobject ... ok > with ... ok > xml ... ok > test stats ... ok > test numeric_big ... ok > ============== shutting down postmaster ============== > > ======================= > All 123 tests passed. > ======================= > > > > > Regards > Michal > > > -----Original Message----- > From: Anh K. Huynh [mailto:ky...@vi...] > Sent: Tuesday, February 08, 2011 6:57 AM > To: moosefs-users > Subject: [Moosefs-users] (FYI) fail to build postgresql on MFS disk > > Hi, > > I am trying to build `postgresql-9.0.3` on a Lenny system. The > source is located on a MFS disk. The build process is good > (binaries are created), but the tests fails (for example, test > "numeric_big" as below); and many sql statements cannot be > executed. Please note that during the test, Debian system will run > a temporary database server to feed SQL statements. > > I move source to a NFS disk, and things go well. So this post may > help anyone who intends to build `postgresql` on MFS disk. If you > need any information, just let me know. > > Regards, > > Logs: > > *** > /home/debian/psql.source/postgresql-9.0-9.0.3/src/test/regress/expected/nume > ric_big.out 2011-01-27 18:21:31.000000000 -0800 > --- > /home/debian/psql.source/postgresql-9.0-9.0.3/src/test/regress/results/numer > ic_big.out 2011-02-07 20:17:12.000000000 -0800 > *************** > *** 500,505 **** > --- 500,506 ---- |
|
From: Sollix - A. C. <acu...@so...> - 2011-02-09 08:54:38
|
Hi, I'm my only solution. In my project, all chunkserver is on Internet. I really like MooseFS, more than GlusterFS etc ... I really want to use it in this project. All my connection Internet have 1mbits upload. To optimize the traffic between chunkserver, i set a goal of 1 to all new file. When the mfs client has finished to upload each file, i do a script to set the goal to 3. If i don't do this, my mfs client is limited to 1mbits because the chunkserver coype the file to the another chunkserver in the same time to be synchronizes , i think ... Cordialement, Alexandre CUVELIER ----- Mail original ----- De: "Michal Borychowski" <mic...@ge...> À: "Sollix - Alexandre CUVELIER" <acu...@so...> Cc: moo...@li... Envoyé: Mercredi 9 Février 2011 07:23:18 Objet: RE: [Moosefs-users] My Project with MooseFS Hi! As far as I understand you try to make MooseFS work over WAN connection? Unfortunately unless you have a really quick connection it’d rather not work optimally. You may also have a look at this reply: http://sourceforge.net/mailarchive/message.php?msg_id=26792119 If you need any further assistance please let us know. Kind regards Michał Borychowski MooseFS Support Manager _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Gemius S.A. ul. Wołoska 7, 02-672 Warszawa Budynek MARS, klatka D Tel.: +4822 874-41-00 Fax : +4822 874-41-01 From: Sollix - Alexandre CUVELIER [mailto:acu...@so...] Sent: Monday, February 07, 2011 9:55 PM To: moo...@li... Subject: [Moosefs-users] My Project with MooseFS Hello everybody, I'm a new french user (sorry for my bad english) of the Moose FS. I have installed Moose on a development infrastructure to test it. I will try to explain you what i want to do : I'm a reseller IT and i sell Linux server to my customer. The aim : do backup of all server. They are all connected to my VPN (server 100Mbits in datacenter). The Master are setup on it. I want to install chunkserver on all server. Near 300To will be available. Each server will copy data on this FS with a goal = X (for example 4). All server are connect to internet thanks ADSL. After some test, i have a little problem with network performance. Example : If i copy file1.txt on the MooseFS with a goal of 5, the node will sent the file to the other node to have the replication. So the server upload 5x the size of the file. It's not very optimized. I search a solution to upload the file to the VPN Server (i can setup a chunkserver on it) and only the VPN Server will deploy the file on the another chunkserver to use the big bandwidth of this server -> 100Mbits I hope i was clear. Thanks you Cordialement, Alexandre CUVELIER |
|
From: Michal B. <mic...@ge...> - 2011-02-09 07:58:33
|
Hi!
We just tried to install postgresql on the machine:
MFS: 1.6.20
Ubuntu 10.10 / 64bit
with the "dpkg-buildpackage -us -nc -rfakeroot" command and everything went
smoothly. And we are not sure where to look further for any error and how to
debug it.
Here is a piece of log:
./pg_regress --inputdir=. --dlpath=. --multibyte=SQL_ASCII
--temp-install=./tmp_check --top-builddir=../../..
--schedule=./parallel_schedule numeric_big
============== creating temporary installation ==============
============== initializing database system ==============
============== starting postmaster ==============
running on port 57235 with pid 32628
============== creating database "regression" ==============
CREATE DATABASE
ALTER DATABASE
============== running regression test queries ==============
test tablespace ... ok
parallel group (17 tests): int2 float4 name oid money text varchar char
int4 int8 boolean txid float8 bit uuid enum numeric
boolean ... ok
char ... ok
name ... ok
varchar ... ok
text ... ok
int2 ... ok
int4 ... ok
int8 ... ok
oid ... ok
float4 ... ok
float8 ... ok
bit ... ok
numeric ... ok
txid ... ok
uuid ... ok
enum ... ok
money ... ok
test strings ... ok
test numerology ... ok
parallel group (19 tests): point circle comments tstypes box lseg abstime
reltime tinterval timetz date polygon macaddr time timestamp path interval
inet timestamptz
point ... ok
lseg ... ok
box ... ok
path ... ok
polygon ... ok
circle ... ok
date ... ok
time ... ok
timetz ... ok
timestamp ... ok
timestamptz ... ok
interval ... ok
abstime ... ok
reltime ... ok
tinterval ... ok
inet ... ok
macaddr ... ok
tstypes ... ok
comments ... ok
parallel group (5 tests): geometry horology type_sanity oidjoins opr_sanity
geometry ... ok
horology ... ok
oidjoins ... ok
type_sanity ... ok
opr_sanity ... ok
test insert ... ok
test create_function_1 ... ok
test create_type ... ok
test create_table ... ok
test create_function_2 ... ok
parallel group (2 tests): copyselect copy
copy ... ok
copyselect ... ok
parallel group (10 tests): create_aggregate create_operator create_cast
drop_if_exists typed_table create_misc triggers vacuum constraints inherit
constraints ... ok
triggers ... ok
create_misc ... ok
create_aggregate ... ok
create_operator ... ok
inherit ... ok
typed_table ... ok
vacuum ... ok
drop_if_exists ... ok
create_cast ... ok
parallel group (2 tests): create_view create_index
create_index ... ok
create_view ... ok
test sanity_check ... ok
test errors ... ok
test select ... ok
parallel group (20 tests): select_distinct select_distinct_on select_into
btree_index hash_index random union update select_having delete namespace
case aggregates select_implicit prepared_xacts subselect transactions
portals arrays join
select_into ... ok
select_distinct ... ok
select_distinct_on ... ok
select_implicit ... ok
select_having ... ok
subselect ... ok
union ... ok
case ... ok
join ... ok
aggregates ... ok
transactions ... ok
random ... ok
portals ... ok
arrays ... ok
btree_index ... ok
hash_index ... ok
update ... ok
namespace ... ok
prepared_xacts ... ok
delete ... ok
test privileges ... ok
test misc ... ok
test rules ... ok
parallel group (13 tests): combocid select_views portals_p2 tsdicts window
xmlmap foreign_data guc dependency tsearch bitmapops cluster foreign_key
select_views ... ok
portals_p2 ... ok
foreign_key ... ok
cluster ... ok
dependency ... ok
guc ... ok
bitmapops ... ok
combocid ... ok
tsearch ... ok
tsdicts ... ok
foreign_data ... ok
window ... ok
xmlmap ... ok
parallel group (19 tests): limit conversion xml prepare polymorphism
largeobject without_oid with copy2 returning rowtypes plancache sequence
temp rangefuncs domain plpgsql truncate alter_table
plancache ... ok
limit ... ok
plpgsql ... ok
copy2 ... ok
temp ... ok
domain ... ok
rangefuncs ... ok
prepare ... ok
without_oid ... ok
conversion ... ok
truncate ... ok
alter_table ... ok
sequence ... ok
polymorphism ... ok
rowtypes ... ok
returning ... ok
largeobject ... ok
with ... ok
xml ... ok
test stats ... ok
test numeric_big ... ok
============== shutting down postmaster ==============
=======================
All 123 tests passed.
=======================
Regards
Michal
-----Original Message-----
From: Anh K. Huynh [mailto:ky...@vi...]
Sent: Tuesday, February 08, 2011 6:57 AM
To: moosefs-users
Subject: [Moosefs-users] (FYI) fail to build postgresql on MFS disk
Hi,
I am trying to build `postgresql-9.0.3` on a Lenny system. The source is
located on a MFS disk. The build process is good (binaries are created), but
the tests fails (for example, test "numeric_big" as below); and many sql
statements cannot be executed. Please note that during the test, Debian
system will run a temporary database server to feed SQL statements.
I move source to a NFS disk, and things go well. So this post may help
anyone who intends to build `postgresql` on MFS disk. If you need any
information, just let me know.
Regards,
Logs:
***
/home/debian/psql.source/postgresql-9.0-9.0.3/src/test/regress/expected/nume
ric_big.out 2011-01-27 18:21:31.000000000 -0800
---
/home/debian/psql.source/postgresql-9.0-9.0.3/src/test/regress/results/numer
ic_big.out 2011-02-07 20:17:12.000000000 -0800
***************
*** 500,505 ****
--- 500,506 ----
CREATE UNIQUE INDEX num_exp_log10_idx ON num_exp_log10 (id);
CREATE UNIQUE INDEX num_exp_power_10_ln_idx ON num_exp_power_10_ln (id);
VACUUM ANALYZE num_exp_add;
+ WARNING: could not open statistics file "pg_stat_tmp/pgstat.stat":
Operation not permitted
VACUUM ANALYZE num_exp_sub;
VACUUM ANALYZE num_exp_div;
VACUUM ANALYZE num_exp_mul;
--
Anh Ky Huynh @ UTC+7
----------------------------------------------------------------------------
--
The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
Pinpoint memory and threading errors before they happen.
Find and fix more than 250 security defects in the development cycle.
Locate bottlenecks in serial and parallel code that limit performance.
http://p.sf.net/sfu/intel-dev2devfeb
_______________________________________________
moosefs-users mailing list
moo...@li...
https://lists.sourceforge.net/lists/listinfo/moosefs-users
|
|
From: Michal B. <mic...@ge...> - 2011-02-09 06:23:37
|
Hi! As far as I understand you try to make MooseFS work over WAN connection? Unfortunately unless you have a really quick connection it’d rather not work optimally. You may also have a look at this reply: http://sourceforge.net/mailarchive/message.php?msg_id=26792119 If you need any further assistance please let us know. Kind regards Michał Borychowski MooseFS Support Manager _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Gemius S.A. ul. Wołoska 7, 02-672 Warszawa Budynek MARS, klatka D Tel.: +4822 874-41-00 Fax : +4822 874-41-01 From: Sollix - Alexandre CUVELIER [mailto:acu...@so...] Sent: Monday, February 07, 2011 9:55 PM To: moo...@li... Subject: [Moosefs-users] My Project with MooseFS Hello everybody, I'm a new french user (sorry for my bad english) of the Moose FS. I have installed Moose on a development infrastructure to test it. I will try to explain you what i want to do : I'm a reseller IT and i sell Linux server to my customer. The aim : do backup of all server. They are all connected to my VPN (server 100Mbits in datacenter). The Master are setup on it. I want to install chunkserver on all server. Near 300To will be available. Each server will copy data on this FS with a goal = X (for example 4). All server are connect to internet thanks ADSL. After some test, i have a little problem with network performance. Example : If i copy file1.txt on the MooseFS with a goal of 5, the node will sent the file to the other node to have the replication. So the server upload 5x the size of the file. It's not very optimized. I search a solution to upload the file to the VPN Server (i can setup a chunkserver on it) and only the VPN Server will deploy the file on the another chunkserver to use the big bandwidth of this server -> 100Mbits I hope i was clear. Thanks you Cordialement, Alexandre CUVELIER |
|
From: Michal B. <mic...@ge...> - 2011-02-09 06:14:50
|
Hi Alexandre! Are your computers located in the same LAN? It looks like some hardware problem. You can run “tcpdump” to see what packets are transmitted in the network. Especially with this chunkserver, that is: tcpdump -n host 10.142.0.11 and port 9422 Kind regards Michał Borychowski MooseFS Support Manager _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Gemius S.A. ul. Wołoska 7, 02-672 Warszawa Budynek MARS, klatka D Tel.: +4822 874-41-00 Fax : +4822 874-41-01 From: Sollix - Alexandre CUVELIER [mailto:acu...@so...] Sent: Tuesday, February 08, 2011 11:11 PM To: moosefs-users Subject: Re: [Moosefs-users] Operation timed out Feb 8 23:15:45 srv1 mfsmount[5518]: file: 787, index: 0 - can't connect to proper chunkserver (try counter: 1) Feb 8 23:15:53 srv1 mfsmount[5518]: can't connect to (0A8E000B:9422): ETIMEDOUT (Operation timed out) Feb 8 23:15:53 srv1 mfsmount[5518]: file: 788, index: 0 - can't connect to proper chunkserver (try counter: 1) Feb 8 23:15:54 srv1 mfsmount[5518]: can't connect to (0A8E000B:9422): ETIMEDOUT (Operation timed out) Feb 8 23:15:54 srv1 mfsmount[5518]: file: 788, index: 0 - can't connect to proper chunkserver (try counter: 2) Feb 8 23:15:55 srv1 mfsmount[5518]: can't connect to (0A8E000B:9422): ETIMEDOUT (Operation timed out) Feb 8 23:15:55 srv1 mfsmount[5518]: file: 788, index: 0 - can't connect to proper chunkserver (try counter: 3) Feb 8 23:16:09 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:16:09 srv1 mfsmount[5518]: file: 1274, index: 0 - can't connect to proper chunkserver (try counter: 1) Feb 8 23:16:10 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:16:10 srv1 mfsmount[5518]: file: 1274, index: 0 - can't connect to proper chunkserver (try counter: 2) Feb 8 23:16:11 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:16:11 srv1 mfsmount[5518]: file: 1274, index: 0 - can't connect to proper chunkserver (try counter: 3) Feb 8 23:16:13 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:16:13 srv1 mfsmount[5518]: file: 1274, index: 0 - can't connect to proper chunkserver (try counter: 4) Feb 8 23:16:15 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:16:15 srv1 mfsmount[5518]: file: 1274, index: 0 - can't connect to proper chunkserver (try counter: 5) Feb 8 23:16:17 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:16:17 srv1 mfsmount[5518]: file: 1274, index: 0 - can't connect to proper chunkserver (try counter: 6) Feb 8 23:16:20 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:16:20 srv1 mfsmount[5518]: file: 1274, index: 0 - can't connect to proper chunkserver (try counter: 7) Feb 8 23:16:23 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:16:23 srv1 mfsmount[5518]: file: 1274, index: 0 - can't connect to proper chunkserver (try counter: 8) Feb 8 23:16:26 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:16:26 srv1 mfsmount[5518]: file: 1274, index: 0 - can't connect to proper chunkserver (try counter: 9) Feb 8 23:16:30 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:16:30 srv1 mfsmount[5518]: file: 1274, index: 0 - can't connect to proper chunkserver (try counter: 10) Feb 8 23:16:34 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:16:34 srv1 mfsmount[5518]: file: 1274, index: 0 - can't connect to proper chunkserver (try counter: 11) Feb 8 23:16:38 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:16:38 srv1 mfsmount[5518]: file: 1274, index: 0 - can't connect to proper chunkserver (try counter: 12) Feb 8 23:16:43 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:16:43 srv1 mfsmount[5518]: file: 1274, index: 0 - can't connect to proper chunkserver (try counter: 13) Feb 8 23:16:48 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:16:48 srv1 mfsmount[5518]: file: 1274, index: 0 - can't connect to proper chunkserver (try counter: 14) Feb 8 23:16:53 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:16:53 srv1 mfsmount[5518]: file: 1274, index: 0 - can't connect to proper chunkserver (try counter: 15) Feb 8 23:16:59 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:16:59 srv1 mfsmount[5518]: file: 1274, index: 0 - can't connect to proper chunkserver (try counter: 16) Feb 8 23:17:05 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:17:05 srv1 mfsmount[5518]: file: 1274, index: 0 - can't connect to proper chunkserver (try counter: 17) Feb 8 23:17:11 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:17:11 srv1 mfsmount[5518]: file: 1274, index: 0 - can't connect to proper chunkserver (try counter: 18) Feb 8 23:17:18 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:17:18 srv1 mfsmount[5518]: file: 1274, index: 0 - can't connect to proper chunkserver (try counter: 19) Feb 8 23:17:25 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:17:25 srv1 mfsmount[5518]: file: 1274, index: 0 - can't connect to proper chunkserver (try counter: 20) Feb 8 23:17:33 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:17:33 srv1 mfsmount[5518]: file: 1274, index: 0 - can't connect to proper chunkserver (try counter: 21) Feb 8 23:17:41 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:17:41 srv1 mfsmount[5518]: file: 1274, index: 0 - can't connect to proper chunkserver (try counter: 22) Feb 8 23:17:49 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:17:49 srv1 mfsmount[5518]: file: 1274, index: 0 - can't connect to proper chunkserver (try counter: 23) Feb 8 23:17:57 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:17:57 srv1 mfsmount[5518]: file: 1274, index: 0 - can't connect to proper chunkserver (try counter: 24) Feb 8 23:18:06 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:18:06 srv1 mfsmount[5518]: file: 1274, index: 0 - can't connect to proper chunkserver (try counter: 25) Feb 8 23:18:15 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:18:15 srv1 mfsmount[5518]: file: 1274, index: 0 - can't connect to proper chunkserver (try counter: 26) Feb 8 23:18:24 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Cordialement, Alexandre CUVELIER _____ De: "Sollix - Alexandre CUVELIER" <acu...@so...> À: "moosefs-users" <moo...@li...> Envoyé: Mardi 8 Février 2011 23:08:30 Objet: [Moosefs-users] Operation timed out Hi, I have a problem on the Linux Client with mfsmount : Feb 8 23:14:18 srv1 mfsmount[5518]: file: 1360, index: 0, chunk: 5905, version: 1 - writeworker: connection with (0A8E0008:9422) was timed out (unfinished writes: 5; try counter: 11) Feb 8 23:14:19 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:14:20 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:14:21 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:14:23 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:14:25 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:14:27 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:14:30 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:14:33 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:14:36 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:14:40 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) etc ... I don't understand because the chunkserver is available and i show it on mfscgi. And why mfsmount doesn't try an another chuckserver ? For information, the infrastructure using Wan to work. I need to increase the timeout ? If yes, where ? I doesn't use option on mfsmount, i need it ? Cordialement, Alexandre CUVELIER ------------------------------------------------------------------------------ The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE: Pinpoint memory and threading errors before they happen. Find and fix more than 250 security defects in the development cycle. Locate bottlenecks in serial and parallel code that limit performance. http://p.sf.net/sfu/intel-dev2devfeb _______________________________________________ moosefs-users mailing list moo...@li... https://lists.sourceforge.net/lists/listinfo/moosefs-users |
|
From: Sollix - A. C. <acu...@so...> - 2011-02-08 22:19:44
|
Feb 8 23:15:45 srv1 mfsmount[5518]: file: 787, index: 0 - can't connect to proper chunkserver (try counter: 1) Feb 8 23:15:53 srv1 mfsmount[5518]: can't connect to (0A8E000B:9422): ETIMEDOUT (Operation timed out) Feb 8 23:15:53 srv1 mfsmount[5518]: file: 788, index: 0 - can't connect to proper chunkserver (try counter: 1) Feb 8 23:15:54 srv1 mfsmount[5518]: can't connect to (0A8E000B:9422): ETIMEDOUT (Operation timed out) Feb 8 23:15:54 srv1 mfsmount[5518]: file: 788, index: 0 - can't connect to proper chunkserver (try counter: 2) Feb 8 23:15:55 srv1 mfsmount[5518]: can't connect to (0A8E000B:9422): ETIMEDOUT (Operation timed out) Feb 8 23:15:55 srv1 mfsmount[5518]: file: 788, index: 0 - can't connect to proper chunkserver (try counter: 3) Feb 8 23:16:09 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:16:09 srv1 mfsmount[5518]: file: 1274, index: 0 - can't connect to proper chunkserver (try counter: 1) Feb 8 23:16:10 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:16:10 srv1 mfsmount[5518]: file: 1274, index: 0 - can't connect to proper chunkserver (try counter: 2) Feb 8 23:16:11 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:16:11 srv1 mfsmount[5518]: file: 1274, index: 0 - can't connect to proper chunkserver (try counter: 3) Feb 8 23:16:13 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:16:13 srv1 mfsmount[5518]: file: 1274, index: 0 - can't connect to proper chunkserver (try counter: 4) Feb 8 23:16:15 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:16:15 srv1 mfsmount[5518]: file: 1274, index: 0 - can't connect to proper chunkserver (try counter: 5) Feb 8 23:16:17 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:16:17 srv1 mfsmount[5518]: file: 1274, index: 0 - can't connect to proper chunkserver (try counter: 6) Feb 8 23:16:20 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:16:20 srv1 mfsmount[5518]: file: 1274, index: 0 - can't connect to proper chunkserver (try counter: 7) Feb 8 23:16:23 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:16:23 srv1 mfsmount[5518]: file: 1274, index: 0 - can't connect to proper chunkserver (try counter: 8) Feb 8 23:16:26 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:16:26 srv1 mfsmount[5518]: file: 1274, index: 0 - can't connect to proper chunkserver (try counter: 9) Feb 8 23:16:30 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:16:30 srv1 mfsmount[5518]: file: 1274, index: 0 - can't connect to proper chunkserver (try counter: 10) Feb 8 23:16:34 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:16:34 srv1 mfsmount[5518]: file: 1274, index: 0 - can't connect to proper chunkserver (try counter: 11) Feb 8 23:16:38 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:16:38 srv1 mfsmount[5518]: file: 1274, index: 0 - can't connect to proper chunkserver (try counter: 12) Feb 8 23:16:43 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:16:43 srv1 mfsmount[5518]: file: 1274, index: 0 - can't connect to proper chunkserver (try counter: 13) Feb 8 23:16:48 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:16:48 srv1 mfsmount[5518]: file: 1274, index: 0 - can't connect to proper chunkserver (try counter: 14) Feb 8 23:16:53 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:16:53 srv1 mfsmount[5518]: file: 1274, index: 0 - can't connect to proper chunkserver (try counter: 15) Feb 8 23:16:59 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:16:59 srv1 mfsmount[5518]: file: 1274, index: 0 - can't connect to proper chunkserver (try counter: 16) Feb 8 23:17:05 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:17:05 srv1 mfsmount[5518]: file: 1274, index: 0 - can't connect to proper chunkserver (try counter: 17) Feb 8 23:17:11 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:17:11 srv1 mfsmount[5518]: file: 1274, index: 0 - can't connect to proper chunkserver (try counter: 18) Feb 8 23:17:18 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:17:18 srv1 mfsmount[5518]: file: 1274, index: 0 - can't connect to proper chunkserver (try counter: 19) Feb 8 23:17:25 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:17:25 srv1 mfsmount[5518]: file: 1274, index: 0 - can't connect to proper chunkserver (try counter: 20) Feb 8 23:17:33 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:17:33 srv1 mfsmount[5518]: file: 1274, index: 0 - can't connect to proper chunkserver (try counter: 21) Feb 8 23:17:41 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:17:41 srv1 mfsmount[5518]: file: 1274, index: 0 - can't connect to proper chunkserver (try counter: 22) Feb 8 23:17:49 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:17:49 srv1 mfsmount[5518]: file: 1274, index: 0 - can't connect to proper chunkserver (try counter: 23) Feb 8 23:17:57 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:17:57 srv1 mfsmount[5518]: file: 1274, index: 0 - can't connect to proper chunkserver (try counter: 24) Feb 8 23:18:06 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:18:06 srv1 mfsmount[5518]: file: 1274, index: 0 - can't connect to proper chunkserver (try counter: 25) Feb 8 23:18:15 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:18:15 srv1 mfsmount[5518]: file: 1274, index: 0 - can't connect to proper chunkserver (try counter: 26) Feb 8 23:18:24 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Cordialement, Alexandre CUVELIER De: "Sollix - Alexandre CUVELIER" <acu...@so...> À: "moosefs-users" <moo...@li...> Envoyé: Mardi 8 Février 2011 23:08:30 Objet: [Moosefs-users] Operation timed out Hi, I have a problem on the Linux Client with mfsmount : Feb 8 23:14:18 srv1 mfsmount[5518]: file: 1360, index: 0, chunk: 5905, version: 1 - writeworker: connection with (0A8E0008:9422) was timed out (unfinished writes: 5; try counter: 11) Feb 8 23:14:19 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:14:20 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:14:21 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:14:23 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:14:25 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:14:27 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:14:30 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:14:33 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:14:36 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:14:40 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) etc ... I don't understand because the chunkserver is available and i show it on mfscgi. And why mfsmount doesn't try an another chuckserver ? For information, the infrastructure using Wan to work. I need to increase the timeout ? If yes, where ? I doesn't use option on mfsmount, i need it ? Cordialement, Alexandre CUVELIER ------------------------------------------------------------------------------ The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE: Pinpoint memory and threading errors before they happen. Find and fix more than 250 security defects in the development cycle. Locate bottlenecks in serial and parallel code that limit performance. http://p.sf.net/sfu/intel-dev2devfeb _______________________________________________ moosefs-users mailing list moo...@li... https://lists.sourceforge.net/lists/listinfo/moosefs-users |
|
From: Sollix - A. C. <acu...@so...> - 2011-02-08 22:17:13
|
Hi, I have a problem on the Linux Client with mfsmount : Feb 8 23:14:18 srv1 mfsmount[5518]: file: 1360, index: 0, chunk: 5905, version: 1 - writeworker: connection with (0A8E0008:9422) was timed out (unfinished writes: 5; try counter: 11) Feb 8 23:14:19 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:14:20 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:14:21 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:14:23 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:14:25 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:14:27 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:14:30 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:14:33 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:14:36 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) Feb 8 23:14:40 srv1 mfsmount[5518]: can't connect to (0A8E0008:9422): ETIMEDOUT (Operation timed out) etc ... I don't understand because the chunkserver is available and i show it on mfscgi. And why mfsmount doesn't try an another chuckserver ? For information, the infrastructure using Wan to work. I need to increase the timeout ? If yes, where ? I doesn't use option on mfsmount, i need it ? Cordialement, Alexandre CUVELIER |
|
From: Thomas S H. <tha...@gm...> - 2011-02-08 16:42:23
|
This helps a lot! Thanks Michal 2011/2/8 Michal Borychowski <mic...@ge...> > Hi! > > > > This is very interesting question. Our experience shows that when you have > a big file to write (eg. 100 GB) it is good to logically divide it into > blocks of one chunk size (64MB) and write these blocks to different files > and later merge these files into one by using *mfsappendchunks* tool. This > allows us to optimally use the throughput of the network. Mind that the last > block may be filled with “zeros” up to 64MB. > > > > Another option is to also divide a big file into chunks and write them to > the target file simultaneously do different positions of the file (using > seek). So each “writer” saves a portion of data of [ chunksize*i, > chunksize*(i+1)] range to different position. > > > > Unfortunately I am not sure if this could be implemented for optimal > writing of vm images… > > > > > > Regards > > Michal > > > > > > *From:* Thomas S Hatch [mailto:tha...@gm...] > *Sent:* Thursday, January 27, 2011 9:41 PM > *To:* moosefs-users > *Subject:* [Moosefs-users] Optimal writes for MooseFS > > > > We already know that vm image writes are not the optimal way to write to > MooseFS, and randomIO is obviously not the best, so let me ask, what it the > optimal way to write to a MooseFS filesystem? just sequentially? Is an > append to an existing file faster? > > > > Thanks! > > > > -Thomas S Hatch > |
|
From: Jun C. P. <jun...@gm...> - 2011-02-08 16:36:27
|
> If you’d be eager to experiment, let us know, we will prepare a special
> version with this option for tests.
We are very keen to use MFS in our real production because it seems to
fit well to most of our needs. However, the fdatasync problem is a
critical issue that I have to clarify before making our final
decision. It would be greatly appreciated if you provide the special
version with the option of by-passing fdatasync() as soon as possible.
Thanks,
-Jun
2011/2/8 Michal Borychowski <mic...@ge...>:
> Hi Jun and Thomas!
>
>
>
> We are aware of "non-persistent connections" and soon we’ll improve this
> behavior.
>
>
>
> But unfortunately "persistent connections" would not help much in this
> scenario. “fdatasync” causes immediate dispatch of all data from cache to
> external disks – this is a “costly” operation (even if the connection would
> be sustained) and generally speaking eliminates all profits from having the
> cache.
>
>
>
> We can add to mfsmount an option like “mfsignorefsync” which would cause
> that “fdatasync” does nothing and data would be sent at “its own speed” and
> would use single connection for the whole group of data.
>
>
>
> If you’d be eager to experiment, let us know, we will prepare a special
> version with this option for tests.
>
>
>
>
>
> Regards
>
> -Michal
>
>
>
>
>
>
>
> From: Thomas S Hatch [mailto:tha...@gm...]
> Sent: Tuesday, February 08, 2011 12:32 AM
> To: Jun Cheol Park
> Cc: moosefs-users
> Subject: Re: [Moosefs-users] Why too slow in calling fdatasync()? (or
> futex())
>
>
>
>
>
> On Mon, Feb 7, 2011 at 4:25 PM, Jun Cheol Park <jun...@gm...>
> wrote:
>
> Hi,
>
> I found a more specific case about the performance slowness issue that
> I experienced before.
>
> One of important commands in using KVM (kernel-based Virtual Machine)
> is qemu-img that generates a special form of file (i.e., qcow2) as
> follows.
>
> # strace qemu-img convert -f qcow2 -O qcow2 ........
>
> read(4,
> "\f\0\0\0\0\0\0\0\0\0\0\0\260r\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"...,
> 128) = 128
> pwrite(5,
> "\0\1\0\1\0\1\0\1\0\1\0\1\0\1\0\1\0\1\0\1\0\1\0\1\0\1\0\1\0\1\0\1"...,
> 512, 196608) = 512
> fdatasync(5) = 0
> futex(0x63eec4, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x63eec0, {FUTEX_OP_SET,
> 0, FUTEX_OP_CMP_GT, 1}) = 1
> select(5, [4], [], NULL, NULL) = 1 (in [4])
> read(4,
> "\f\0\0\0\0\0\0\0\0\0\0\0\260r\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"...,
> 128) = 128
> pwrite(5,
> "\200\0\0\0\0\27\0\0\200\0\0\0\0\30\0\0\200\0\0\0\0\31\0\0\200\0\0\0\0\32\0\0"...,
> 512, 263168) = 512
> fdatasync(5)
> futex(0x63eec4, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x63eec0, {FUTEX_OP_SET,
> 0, FUTEX_OP_CMP_GT, 1}) = 1
>
> The problem of this command (unlike 'cp' where no calling of
> fdatasync) is that, for every pwrite operation, fdatasync() and
> futex() are called. Then, the MFS write performance is significantly
> reduced from 60 M/s (with 'cp') to 1M/s (with qemu-img). I noticed
> via netstat with TIME_WAIT that, while the qemu-img command is
> running, there are a lot of non-persistent TCP connections.
>
> Is there any way to improve this situation?
>
> Thanks,
>
> -Jun
>
> ------------------------------------------------------------------------------
> The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
> Pinpoint memory and threading errors before they happen.
> Find and fix more than 250 security defects in the development cycle.
> Locate bottlenecks in serial and parallel code that limit performance.
> http://p.sf.net/sfu/intel-dev2devfeb
> _______________________________________________
> moosefs-users mailing list
> moo...@li...
> https://lists.sourceforge.net/lists/listinfo/moosefs-users
>
>
>
>
>
> Thanks Jun, that explains a lot, I usually prepare my qcow images on a build
> machine with local disks and then my datacenters running moosefs pull from
> the build machine over the internet. But I have never been able to get
> qemu-img convert to create a qcow image on a moosefs mount, this answers a
> lot of questions here, and it raises my curiosity about the connection
> handling.
>
>
>
> As I said before, I trust the MooseFS devs, but I wonder if I can ask, from
> an academic perspective, why has it been designed this way? and could this
> be an opportunity for performance improvement?
>
>
>
> -Thomas S Hatch
|
|
From: Michal B. <mic...@ge...> - 2011-02-08 08:59:31
|
Hi Jun and Thomas!
We are aware of "non-persistent connections" and soon we'll improve this
behavior.
But unfortunately "persistent connections" would not help much in this
scenario. "fdatasync" causes immediate dispatch of all data from cache to
external disks - this is a "costly" operation (even if the connection would
be sustained) and generally speaking eliminates all profits from having the
cache.
We can add to mfsmount an option like "mfsignorefsync" which would cause
that "fdatasync" does nothing and data would be sent at "its own speed" and
would use single connection for the whole group of data.
If you'd be eager to experiment, let us know, we will prepare a special
version with this option for tests.
Regards
-Michal
From: Thomas S Hatch [mailto:tha...@gm...]
Sent: Tuesday, February 08, 2011 12:32 AM
To: Jun Cheol Park
Cc: moosefs-users
Subject: Re: [Moosefs-users] Why too slow in calling fdatasync()? (or
futex())
On Mon, Feb 7, 2011 at 4:25 PM, Jun Cheol Park <jun...@gm...>
wrote:
Hi,
I found a more specific case about the performance slowness issue that
I experienced before.
One of important commands in using KVM (kernel-based Virtual Machine)
is qemu-img that generates a special form of file (i.e., qcow2) as
follows.
# strace qemu-img convert -f qcow2 -O qcow2 ........
read(4,
"\f\0\0\0\0\0\0\0\0\0\0\0\260r\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"...,
128) = 128
pwrite(5,
"\0\1\0\1\0\1\0\1\0\1\0\1\0\1\0\1\0\1\0\1\0\1\0\1\0\1\0\1\0\1\0\1"...,
512, 196608) = 512
fdatasync(5) = 0
futex(0x63eec4, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x63eec0, {FUTEX_OP_SET,
0, FUTEX_OP_CMP_GT, 1}) = 1
select(5, [4], [], NULL, NULL) = 1 (in [4])
read(4,
"\f\0\0\0\0\0\0\0\0\0\0\0\260r\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"...,
128) = 128
pwrite(5,
"\200\0\0\0\0\27\0\0\200\0\0\0\0\30\0\0\200\0\0\0\0\31\0\0\200\0\0\0\0\32\0\
0"...,
512, 263168) = 512
fdatasync(5)
futex(0x63eec4, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x63eec0, {FUTEX_OP_SET,
0, FUTEX_OP_CMP_GT, 1}) = 1
The problem of this command (unlike 'cp' where no calling of
fdatasync) is that, for every pwrite operation, fdatasync() and
futex() are called. Then, the MFS write performance is significantly
reduced from 60 M/s (with 'cp') to 1M/s (with qemu-img). I noticed
via netstat with TIME_WAIT that, while the qemu-img command is
running, there are a lot of non-persistent TCP connections.
Is there any way to improve this situation?
Thanks,
-Jun
----------------------------------------------------------------------------
--
The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
Pinpoint memory and threading errors before they happen.
Find and fix more than 250 security defects in the development cycle.
Locate bottlenecks in serial and parallel code that limit performance.
http://p.sf.net/sfu/intel-dev2devfeb
_______________________________________________
moosefs-users mailing list
moo...@li...
https://lists.sourceforge.net/lists/listinfo/moosefs-users
Thanks Jun, that explains a lot, I usually prepare my qcow images on a build
machine with local disks and then my datacenters running moosefs pull from
the build machine over the internet. But I have never been able to get
qemu-img convert to create a qcow image on a moosefs mount, this answers a
lot of questions here, and it raises my curiosity about the connection
handling.
As I said before, I trust the MooseFS devs, but I wonder if I can ask, from
an academic perspective, why has it been designed this way? and could this
be an opportunity for performance improvement?
-Thomas S Hatch
|
|
From: Michal B. <mic...@ge...> - 2011-02-08 08:26:22
|
Hi! Thanks for the remarks, I'll try to make the answers more clear. Regards Michal -----Original Message----- From: Anh K. Huynh [mailto:ky...@vi...] Sent: Tuesday, February 08, 2011 9:23 AM To: Michal Borychowski Cc: moo...@li... Subject: Re: [Moosefs-users] Some new FAQ entries on the website Hi, On Mon, 7 Feb 2011 14:13:38 +0100 "Michal Borychowski" <mic...@ge...> wrote: > I've just added 11 new Frequently Asked Questions (with > answers! :)) which were coming from you during several last months. > > The new part starts here: > http://www.moosefs.org/moosefs-faq.html#file-locking. I hope they > would clarify some points with MooseFS! Thank you very much for the great FAQ entries. That would help me so much :) I've read the section "http://www.moosefs.org/moosefs-faq.html#master-online", and there's (item 2.) > For example if we have MooseFS installation in /mnt/mfs then > stat /mnt/mfs command (in Linux) will show [...] I think /mnt/mfs would be a MFS mount point (not a "MooseFS installation" which indicates the location we install binaries, and which is often /usr/local/). > Additionaly mfsmount creates a virtual hidden file .stats in the > root mounted folder. For example, to get the statistics of mfsmount > when MooseFS is mounted we can cat this .stats file, eg.: > > $ cat /mnt/mfs/.stats The author mentioned "virtual hidden file .stats", which make me confused :) After trying with "ls -la" I gave up; fortunately, "ls -la .stats" works: $ ls -la |grep stat $ ls -la .stats -rw-r--r-- 1 root root 0 1969-12-31 16:00 .stats The file ".stats" isn't really hidden IMHO :P Regards, -- Anh Ky Huynh @ UTC+7 ---------------------------------------------------------------------------- -- The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE: Pinpoint memory and threading errors before they happen. Find and fix more than 250 security defects in the development cycle. Locate bottlenecks in serial and parallel code that limit performance. http://p.sf.net/sfu/intel-dev2devfeb _______________________________________________ moosefs-users mailing list moo...@li... https://lists.sourceforge.net/lists/listinfo/moosefs-users |
|
From: Anh K. H. <ky...@vi...> - 2011-02-08 08:23:02
|
Hi, On Mon, 7 Feb 2011 14:13:38 +0100 "Michal Borychowski" <mic...@ge...> wrote: > I've just added 11 new Frequently Asked Questions (with > answers! :)) which were coming from you during several last months. > > The new part starts here: > http://www.moosefs.org/moosefs-faq.html#file-locking. I hope they > would clarify some points with MooseFS! Thank you very much for the great FAQ entries. That would help me so much :) I've read the section "http://www.moosefs.org/moosefs-faq.html#master-online", and there's (item 2.) > For example if we have MooseFS installation in /mnt/mfs then > stat /mnt/mfs command (in Linux) will show [...] I think /mnt/mfs would be a MFS mount point (not a "MooseFS installation" which indicates the location we install binaries, and which is often /usr/local/). > Additionaly mfsmount creates a virtual hidden file .stats in the > root mounted folder. For example, to get the statistics of mfsmount > when MooseFS is mounted we can cat this .stats file, eg.: > > $ cat /mnt/mfs/.stats The author mentioned "virtual hidden file .stats", which make me confused :) After trying with "ls -la" I gave up; fortunately, "ls -la .stats" works: $ ls -la |grep stat $ ls -la .stats -rw-r--r-- 1 root root 0 1969-12-31 16:00 .stats The file ".stats" isn't really hidden IMHO :P Regards, -- Anh Ky Huynh @ UTC+7 |
|
From: youngcow <you...@gm...> - 2011-02-08 08:15:37
|
Thanks, anyone has the script or a example script and I can change it for my requirement. > Hi! > > To be honest there is no optimal method to measure it. There is "mfsdirinfo" > which returns occupied space but it gives wrong results in case of snapshots > (and hard links to be precise). So if one i-node in a subtree occurs twice > (as in hard link) it would be also counted twice. Similarly if a chunk (in > case of snapshot) occurs twice it would be also counted twice. > > One could probably try to write a special script which would calculate it > precisely using information in metadata. > > > Kind regards > Michał Borychowski > MooseFS Support Manager > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > Gemius S.A. > ul. Wołoska 7, 02-672 Warszawa > Budynek MARS, klatka D > Tel.: +4822 874-41-00 > Fax : +4822 874-41-01 > > > > -----Original Message----- > From: youngcow [mailto:you...@gm...] > Sent: Tuesday, February 08, 2011 7:43 AM > To: moosefs-users > Subject: [Moosefs-users] how to calculate the space used by snapshot > > Hello! > > I'm testing MooseFs. I have a question about snapshot. > When I make a snapshot for a directory such as dir1, the snapshot don't > use disk space.But when I change the source dir1's content(such as > change the content of a file), the snapshot will use the disk space. > Now I want to calcuate the disk space usage for snapshot, I can't find > the method. > > Thanks. > > ---------------------------------------------------------------------------- > -- > The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE: > Pinpoint memory and threading errors before they happen. > Find and fix more than 250 security defects in the development cycle. > Locate bottlenecks in serial and parallel code that limit performance. > http://p.sf.net/sfu/intel-dev2devfeb > _______________________________________________ > moosefs-users mailing list > moo...@li... > https://lists.sourceforge.net/lists/listinfo/moosefs-users > |
|
From: Anh K. H. <ky...@vi...> - 2011-02-08 08:12:20
|
On Tue, 8 Feb 2011 07:32:53 +0100 "Michal Borychowski" <mic...@ge...> wrote: > Hi! > > This > > + WARNING: could not open statistics file > > "pg_stat_tmp/pgstat.stat": Operation not permitted > > looks more like a problem with access rights than with the MooseFS > itself. Would you check it? This file is deleted after the error is captured, so I have no idea what it is :( I asked on #debian, but I didn't receive any reply. Will try on the list debian-users instead. I've also tried to build the package under root account, and the problem is still the same. The full log can be found at http://pastebin.com/x6YdPuvE (> 5400 lines) Regards, > -----Original Message----- > From: Anh K. Huynh [mailto:ky...@vi...] > Sent: Tuesday, February 08, 2011 6:56 AM > To: moosefs-users > Subject: [Moosefs-users] (FYI) fail to build postgresql on MFS disk > > Hi, > > I am trying to build `postgresql-9.0.3` on a Lenny system. The > source is located on a MFS disk. The build process is good > (binaries are created), but the tests fails (for example, test > "numeric_big" as below); and many sql statements cannot be > executed. Please note that during the test, Debian system will run > a temporary database server to feed SQL statements. > > I move source to a NFS disk, and things go well. So this post may > help anyone who intends to build `postgresql` on MFS disk. If you > need any information, just let me know. > > Regards, > > Logs: > > *** > /home/debian/psql.source/postgresql-9.0-9.0.3/src/test/regress/expected/nume > ric_big.out 2011-01-27 18:21:31.000000000 -0800 > --- > /home/debian/psql.source/postgresql-9.0-9.0.3/src/test/regress/results/numer > ic_big.out 2011-02-07 20:17:12.000000000 -0800 > *************** > *** 500,505 **** > --- 500,506 ---- > CREATE UNIQUE INDEX num_exp_log10_idx ON num_exp_log10 (id); > CREATE UNIQUE INDEX num_exp_power_10_ln_idx ON > num_exp_power_10_ln (id); VACUUM ANALYZE num_exp_add; > + WARNING: could not open statistics file > "pg_stat_tmp/pgstat.stat": Operation not permitted > VACUUM ANALYZE num_exp_sub; > VACUUM ANALYZE num_exp_div; > VACUUM ANALYZE num_exp_mul; > -- Anh Ky Huynh @ UTC+7 |
|
From: Michal B. <mic...@ge...> - 2011-02-08 08:01:57
|
Hi! To be honest there is no optimal method to measure it. There is "mfsdirinfo" which returns occupied space but it gives wrong results in case of snapshots (and hard links to be precise). So if one i-node in a subtree occurs twice (as in hard link) it would be also counted twice. Similarly if a chunk (in case of snapshot) occurs twice it would be also counted twice. One could probably try to write a special script which would calculate it precisely using information in metadata. Kind regards Michał Borychowski MooseFS Support Manager _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Gemius S.A. ul. Wołoska 7, 02-672 Warszawa Budynek MARS, klatka D Tel.: +4822 874-41-00 Fax : +4822 874-41-01 -----Original Message----- From: youngcow [mailto:you...@gm...] Sent: Tuesday, February 08, 2011 7:43 AM To: moosefs-users Subject: [Moosefs-users] how to calculate the space used by snapshot Hello! I'm testing MooseFs. I have a question about snapshot. When I make a snapshot for a directory such as dir1, the snapshot don't use disk space.But when I change the source dir1's content(such as change the content of a file), the snapshot will use the disk space. Now I want to calcuate the disk space usage for snapshot, I can't find the method. Thanks. ---------------------------------------------------------------------------- -- The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE: Pinpoint memory and threading errors before they happen. Find and fix more than 250 security defects in the development cycle. Locate bottlenecks in serial and parallel code that limit performance. http://p.sf.net/sfu/intel-dev2devfeb _______________________________________________ moosefs-users mailing list moo...@li... https://lists.sourceforge.net/lists/listinfo/moosefs-users |
|
From: Michal B. <mic...@ge...> - 2011-02-08 07:53:15
|
Hi! This is nothing serious. These are files which were opened by a user and were deleted. POSIX demands to have the file opened even if it was deleted as long as it is opened. So that's why these files appear. You can find this file in a "reserved" folder when you mount "meta" resource (option -o mfsmeta for mfsmount). It would stay there as long as it is used by some client(s). If none of the clients uses it it would be automatically deleted (by default after 2 hours as far as we remember). So again - this is nothing to worry about. Kind regards Michał Borychowski MooseFS Support Manager _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Gemius S.A. ul. Wołoska 7, 02-672 Warszawa Budynek MARS, klatka D Tel.: +4822 874-41-00 Fax : +4822 874-41-01 -----Original Message----- From: Giovanni Toraldo [mailto:gt...@li...] Sent: Thursday, January 27, 2011 10:24 AM To: moo...@li... Subject: [Moosefs-users] currently unavailable chunk and reserved files Hi, I was testing master recovery after failure: after switching master, 127 missing chunks appeared. currently unavailable chunk 0000000000001244 (inode: 193 ; index: 0) currently unavailable chunk 0000000000001245 (inode: 193 ; index: 1) [..] currently unavailable chunk 00000000000012C1 (inode: 193 ; index: 125) currently unavailable chunk 00000000000012C2 (inode: 193 ; index: 126) + currently unavailable reserved file 193: images/60a81e97504d92ce5238400baba858954bce2ead unavailable chunks: 127 unavailable reserved files: 1 I didn't found any data corruption on my services (I use mfs as OpenNebula storage backend). What to do when unavailable chunk appears? There is a way to tell Moose to forget them after sometime? And what about unavailable reserved file? I have no problems accessing that file (I did a cat file > /dev/null). Thanks for any reply. -- Giovanni Toraldo http://www.libersoft.it/ |
|
From: Michal B. <mic...@ge...> - 2011-02-08 07:47:04
|
Hi! This is very interesting question. Our experience shows that when you have a big file to write (eg. 100 GB) it is good to logically divide it into blocks of one chunk size (64MB) and write these blocks to different files and later merge these files into one by using mfsappendchunks tool. This allows us to optimally use the throughput of the network. Mind that the last block may be filled with "zeros" up to 64MB. Another option is to also divide a big file into chunks and write them to the target file simultaneously do different positions of the file (using seek). So each "writer" saves a portion of data of [ chunksize*i, chunksize*(i+1)] range to different position. Unfortunately I am not sure if this could be implemented for optimal writing of vm images. Regards Michal From: Thomas S Hatch [mailto:tha...@gm...] Sent: Thursday, January 27, 2011 9:41 PM To: moosefs-users Subject: [Moosefs-users] Optimal writes for MooseFS We already know that vm image writes are not the optimal way to write to MooseFS, and randomIO is obviously not the best, so let me ask, what it the optimal way to write to a MooseFS filesystem? just sequentially? Is an append to an existing file faster? Thanks! -Thomas S Hatch |
|
From: Michal B. <mic...@ge...> - 2011-02-08 07:04:41
|
Hi! We should publish a new release of MooseFS capable of caching symlinks soon. Kind regards Michal Borychowski MooseFS Support Manager _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Gemius S.A. ul. Wołoska 7, 02-672 Warszawa Budynek MARS, klatka D Tel.: +4822 874-41-00 Fax : +4822 874-41-01 -----Original Message----- From: Flow Jiang [mailto:fl...@gm...] Sent: Wednesday, January 26, 2011 3:49 PM To: moo...@li... Subject: [Moosefs-users] Performance issue with symlinks Hi, I setup 2 Dell PowerEdge 1950 servers with moosefs-1.6.17 recently. Everything looks better than our original NFS mounted FS but when compiling source codes with gcc, the performance is bad. It takes 2~3 times longer than NFS to generate dependency files. I did many tests to find out the reason, including changing the cache parameters for mfsmount, but 1Gbps mounted MFS is still not as good as 100Mbps mounted NFS. Today I happened to see the report generated by mfscgiserv and one interesting finding is the "readlink" count to the source code folder is huge. And our source code is linked with symlinks for compiling so I think it's caused by the activity of reading source files. I googled a while and found there's an old thread talked about caching for symlinks with fuse: http://sourceforge.net/mailarchive/forum.php?thread_name=E1K8yNn-00017E-PD%4 0pomaz-ex.szeredi.hu&forum_name=fuse-devel. So would that be the root cause of my problem? Or any other suggestions? Many Thanks Flow ---------------------------------------------------------------------------- -- Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)! Finally, a world-class log management solution at an even better price-free! Download using promo code Free_Logger_4_Dev2Dev. Offer expires February 28th, so secure your free ArcSight Logger TODAY! http://p.sf.net/sfu/arcsight-sfd2d _______________________________________________ moosefs-users mailing list moo...@li... https://lists.sourceforge.net/lists/listinfo/moosefs-users |
|
From: youngcow <you...@gm...> - 2011-02-08 06:43:08
|
Hello! I'm testing MooseFs. I have a question about snapshot. When I make a snapshot for a directory such as dir1, the snapshot don't use disk space.But when I change the source dir1's content(such as change the content of a file), the snapshot will use the disk space. Now I want to calcuate the disk space usage for snapshot, I can't find the method. Thanks. |
|
From: Michal B. <mic...@ge...> - 2011-02-08 06:33:08
|
Hi! This > + WARNING: could not open statistics file "pg_stat_tmp/pgstat.stat": > Operation not permitted looks more like a problem with access rights than with the MooseFS itself. Would you check it? Regards Michal -----Original Message----- From: Anh K. Huynh [mailto:ky...@vi...] Sent: Tuesday, February 08, 2011 6:56 AM To: moosefs-users Subject: [Moosefs-users] (FYI) fail to build postgresql on MFS disk Hi, I am trying to build `postgresql-9.0.3` on a Lenny system. The source is located on a MFS disk. The build process is good (binaries are created), but the tests fails (for example, test "numeric_big" as below); and many sql statements cannot be executed. Please note that during the test, Debian system will run a temporary database server to feed SQL statements. I move source to a NFS disk, and things go well. So this post may help anyone who intends to build `postgresql` on MFS disk. If you need any information, just let me know. Regards, Logs: *** /home/debian/psql.source/postgresql-9.0-9.0.3/src/test/regress/expected/nume ric_big.out 2011-01-27 18:21:31.000000000 -0800 --- /home/debian/psql.source/postgresql-9.0-9.0.3/src/test/regress/results/numer ic_big.out 2011-02-07 20:17:12.000000000 -0800 *************** *** 500,505 **** --- 500,506 ---- CREATE UNIQUE INDEX num_exp_log10_idx ON num_exp_log10 (id); CREATE UNIQUE INDEX num_exp_power_10_ln_idx ON num_exp_power_10_ln (id); VACUUM ANALYZE num_exp_add; + WARNING: could not open statistics file "pg_stat_tmp/pgstat.stat": Operation not permitted VACUUM ANALYZE num_exp_sub; VACUUM ANALYZE num_exp_div; VACUUM ANALYZE num_exp_mul; -- Anh Ky Huynh @ UTC+7 ---------------------------------------------------------------------------- -- The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE: Pinpoint memory and threading errors before they happen. Find and fix more than 250 security defects in the development cycle. Locate bottlenecks in serial and parallel code that limit performance. http://p.sf.net/sfu/intel-dev2devfeb _______________________________________________ moosefs-users mailing list moo...@li... https://lists.sourceforge.net/lists/listinfo/moosefs-users |
|
From: Anh K. H. <ky...@vi...> - 2011-02-08 05:56:42
|
Hi, I am trying to build `postgresql-9.0.3` on a Lenny system. The source is located on a MFS disk. The build process is good (binaries are created), but the tests fails (for example, test "numeric_big" as below); and many sql statements cannot be executed. Please note that during the test, Debian system will run a temporary database server to feed SQL statements. I move source to a NFS disk, and things go well. So this post may help anyone who intends to build `postgresql` on MFS disk. If you need any information, just let me know. Regards, Logs: *** /home/debian/psql.source/postgresql-9.0-9.0.3/src/test/regress/expected/numeric_big.out 2011-01-27 18:21:31.000000000 -0800 --- /home/debian/psql.source/postgresql-9.0-9.0.3/src/test/regress/results/numeric_big.out 2011-02-07 20:17:12.000000000 -0800 *************** *** 500,505 **** --- 500,506 ---- CREATE UNIQUE INDEX num_exp_log10_idx ON num_exp_log10 (id); CREATE UNIQUE INDEX num_exp_power_10_ln_idx ON num_exp_power_10_ln (id); VACUUM ANALYZE num_exp_add; + WARNING: could not open statistics file "pg_stat_tmp/pgstat.stat": Operation not permitted VACUUM ANALYZE num_exp_sub; VACUUM ANALYZE num_exp_div; VACUUM ANALYZE num_exp_mul; -- Anh Ky Huynh @ UTC+7 |
|
From: Thomas S H. <tha...@gm...> - 2011-02-07 23:32:03
|
On Mon, Feb 7, 2011 at 4:25 PM, Jun Cheol Park <jun...@gm...>wrote:
> Hi,
>
> I found a more specific case about the performance slowness issue that
> I experienced before.
>
> One of important commands in using KVM (kernel-based Virtual Machine)
> is qemu-img that generates a special form of file (i.e., qcow2) as
> follows.
>
> # strace qemu-img convert -f qcow2 -O qcow2 ........
>
> read(4,
> "\f\0\0\0\0\0\0\0\0\0\0\0\260r\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"...,
> 128) = 128
> pwrite(5,
> "\0\1\0\1\0\1\0\1\0\1\0\1\0\1\0\1\0\1\0\1\0\1\0\1\0\1\0\1\0\1\0\1"...,
> 512, 196608) = 512
> fdatasync(5) = 0
> futex(0x63eec4, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x63eec0, {FUTEX_OP_SET,
> 0, FUTEX_OP_CMP_GT, 1}) = 1
> select(5, [4], [], NULL, NULL) = 1 (in [4])
> read(4,
> "\f\0\0\0\0\0\0\0\0\0\0\0\260r\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"...,
> 128) = 128
> pwrite(5,
> "\200\0\0\0\0\27\0\0\200\0\0\0\0\30\0\0\200\0\0\0\0\31\0\0\200\0\0\0\0\32\0\0"...,
> 512, 263168) = 512
> fdatasync(5)
> futex(0x63eec4, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x63eec0, {FUTEX_OP_SET,
> 0, FUTEX_OP_CMP_GT, 1}) = 1
>
> The problem of this command (unlike 'cp' where no calling of
> fdatasync) is that, for every pwrite operation, fdatasync() and
> futex() are called. Then, the MFS write performance is significantly
> reduced from 60 M/s (with 'cp') to 1M/s (with qemu-img). I noticed
> via netstat with TIME_WAIT that, while the qemu-img command is
> running, there are a lot of non-persistent TCP connections.
>
> Is there any way to improve this situation?
>
> Thanks,
>
> -Jun
>
>
> ------------------------------------------------------------------------------
> The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
> Pinpoint memory and threading errors before they happen.
> Find and fix more than 250 security defects in the development cycle.
> Locate bottlenecks in serial and parallel code that limit performance.
> http://p.sf.net/sfu/intel-dev2devfeb
> _______________________________________________
> moosefs-users mailing list
> moo...@li...
> https://lists.sourceforge.net/lists/listinfo/moosefs-users
>
Thanks Jun, that explains a lot, I usually prepare my qcow images on a build
machine with local disks and then my datacenters running moosefs pull from
the build machine over the internet. But I have never been able to get
qemu-img convert to create a qcow image on a moosefs mount, this answers a
lot of questions here, and it raises my curiosity about the connection
handling.
As I said before, I trust the MooseFS devs, but I wonder if I can ask, from
an academic perspective, why has it been designed this way? and could this
be an opportunity for performance improvement?
-Thomas S Hatch
|
|
From: Jun C. P. <jun...@gm...> - 2011-02-07 23:25:18
|
Hi,
I found a more specific case about the performance slowness issue that
I experienced before.
One of important commands in using KVM (kernel-based Virtual Machine)
is qemu-img that generates a special form of file (i.e., qcow2) as
follows.
# strace qemu-img convert -f qcow2 -O qcow2 ........
read(4, "\f\0\0\0\0\0\0\0\0\0\0\0\260r\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"...,
128) = 128
pwrite(5, "\0\1\0\1\0\1\0\1\0\1\0\1\0\1\0\1\0\1\0\1\0\1\0\1\0\1\0\1\0\1\0\1"...,
512, 196608) = 512
fdatasync(5) = 0
futex(0x63eec4, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x63eec0, {FUTEX_OP_SET,
0, FUTEX_OP_CMP_GT, 1}) = 1
select(5, [4], [], NULL, NULL) = 1 (in [4])
read(4, "\f\0\0\0\0\0\0\0\0\0\0\0\260r\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"...,
128) = 128
pwrite(5, "\200\0\0\0\0\27\0\0\200\0\0\0\0\30\0\0\200\0\0\0\0\31\0\0\200\0\0\0\0\32\0\0"...,
512, 263168) = 512
fdatasync(5)
futex(0x63eec4, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x63eec0, {FUTEX_OP_SET,
0, FUTEX_OP_CMP_GT, 1}) = 1
The problem of this command (unlike 'cp' where no calling of
fdatasync) is that, for every pwrite operation, fdatasync() and
futex() are called. Then, the MFS write performance is significantly
reduced from 60 M/s (with 'cp') to 1M/s (with qemu-img). I noticed
via netstat with TIME_WAIT that, while the qemu-img command is
running, there are a lot of non-persistent TCP connections.
Is there any way to improve this situation?
Thanks,
-Jun
|
84 messages has been excluded from this view by a project administrator.
