vfer-users — Mailing list for those interested in using\following and developing for VFER

Re: [vfer-users] vfer for Internet transfers

[Ivan B. <iv...@cs...>]

Yang,

You might also consider tuning TCP for better performance. If used
out of the box on any of today's operating systems, TCP is typically
configured with abysmal defaults. For example, Stas wrote a
prescriptive troubleshooting and performance tuning guide
for TCP which you might find of interest:

http://shlang.com/writing/tcp-perf.html

ivan.

On Nov 5, 2007, at 6:35 PM, Yang wrote:

> Not too surprisingly, when I try to FTP large files between two
> (Internet-connected) hosts on opposite coasts of the US, I can achieve
> only low throughput, presumably due in large part to high
> bandwidth-delay (and TCP's behavior under those conditions). This
> triggered my curiosity in alternative bulk transfer protocols.
>
> Could vfer be such a protocol (i.e., likely to improve my throughput)?
> If not, any pointers to such a protocol (available for use, not just a
> publication)? If not, is there any reason why I probably shouldn't
> expect to ever find such a protocol?
>
> (I also looked into http://udt.sf.net/, but found its throughput to be
> lower than TCP's.)
>
> Thanks for any answers.
>
> ---------------------------------------------------------------------- 
> ---
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems?  Stop.
> Now Search log events and configuration files using AJAX and a  
> browser.
> Download your FREE copy of Splunk now >> http://get.splunk.com/
> _______________________________________________
> http://vfer.sf.net
> vfer-users mailing list
> vfe...@li...
> https://lists.sourceforge.net/lists/listinfo/vfer-users

Re: [vfer-users] Can't use vfer_rcp

[Ivan B. <iv...@cs...>]

Hi Yang,

Thanks for trying VFER. Unfortunately, as you might have noticed from  
the lack of
project activity, VFER is in a dormant state. I haven't found the  
time to hack on it
since I've started graduate school and new projects have been taking  
up my all
of my time.

 From the debug output you've included it looks like you might be  
running VFER on an
OS that does not support the way VFER calls the local socket layer  
connect function.
The code should work between two Linux hosts. OS X has also been tested.
That's my guess from the output. If you dig deeper and find another  
diagnosis or perhaps
resolve the problem, let the list know.

thanks,
ivan.

On Nov 5, 2007, at 7:40 PM, Yang wrote:

> I've been trying to use vfer_rcp to see how fast files transfer
> between two Internet-connected hosts.
>
> [root@harvard vfer-0.98]# bin/vfer_rcpd -vv -P 9876 start
> VFER_RCPD     [ main                 ] starting vferd
> DEBUG_API     [ vfer_socket          ] Entered
> DEBUG_PMTUD   [ Control_Initialize_PMTUD ] Unable to create ICMP
> socket. Continuing anyways
> DEBUG_API     [ vfer_bind            ] Entered
> DEBUG_PMTUD   [ vfer_bind            ] Unable to bind ICMP socket.
> Continuing without ICMP.
> DEBUG_API     [ vfer_bind            ] Trying to set SO_RCVBUF to  
> 3150000
> DEBUG_API     [ vfer_bind            ] Trying to set SO_SNDBUF to  
> 3150000
> DEBUG_API     [ vfer_listen          ] Entered
> DEBUG_CTL     [ Control_Register_Socket ] Starting ControlT thread,
> num_active = 1
> VFER_RCPD     [ main                 ] started with authenticating
> username: root, home_dir: /root
> DEBUG_API     [ vfer_accept          ] Entered
> DEBUG_CTL     [ Control_Accept       ] Entered with listening sock- 
> >fd[3]
> DEBUG_CTL     [ ControlT             ] Entered with num_active[1]
> DEBUG_PMTUD   [ Control_Initialize_PMTUD ] Unable to create ICMP
> socket. Continuing anyways
> DEBUG_CTL     [ Control_Accept       ] New port socket: 0.0.0.0.47174
>
> However, on the other host:
>
> $ sudo ./vfer_rcp -vv -P 9876 ya...@ha...:/tmp/seq .
> [sudo] password for yang:
> VFER_RCP      [ main                 ] parsed: from_remote[0] lpath[.]
> uname[yang] host[harvard.csail.mit.edu] port[9876] rpath[/tmp/seq]
> VFER_RCP      [ rcp_connect          ] remote (harvard.csail.mit.edu :
> 128.30.76.131)
> VFER_RCP      [ rcp_connect          ] local  (yang-xps410 :  
> 127.0.1.1)
> DEBUG_API     [ vfer_socket          ] Entered
> DEBUG_PMTUD   [ Control_Initialize_PMTUD ] Unable to create ICMP
> socket. Continuing anyways
> DEBUG_API     [ vfer_bind            ] Entered
> DEBUG_PMTUD   [ vfer_bind            ] Unable to bind ICMP socket.
> Continuing without ICMP.
> DEBUG_API     [ vfer_bind            ] Trying to set SO_RCVBUF to  
> 3150000
> DEBUG_API     [ vfer_bind            ] Trying to set SO_SNDBUF to  
> 3150000
> DEBUG_API     [ vfer_connect         ] Entered
> DEBUG_CTL     [ Control_Connect      ] Entered sock->fd[3]
> Control_Connect :: connect: Invalid argument
> DEBUG_CTL     [ Control_Connect      ] connect failed
> VFER_RCP      [ rcp_connect          ] DEBUG_API     [ vfer_errortext
>      ] Entered
> vfer_connect failed [not able to connect on a socket]
>
> I wasn't able to find documentation on these two apps. Thanks in
> advance for any hints.
>
> ---------------------------------------------------------------------- 
> ---
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems?  Stop.
> Now Search log events and configuration files using AJAX and a  
> browser.
> Download your FREE copy of Splunk now >> http://get.splunk.com/
> _______________________________________________
> http://vfer.sf.net
> vfer-users mailing list
> vfe...@li...
> https://lists.sourceforge.net/lists/listinfo/vfer-users

[vfer-users] Can't use vfer_rcp

[Yang <u01...@sn...>]

I've been trying to use vfer_rcp to see how fast files transfer
between two Internet-connected hosts.

[root@harvard vfer-0.98]# bin/vfer_rcpd -vv -P 9876 start
VFER_RCPD     [ main                 ] starting vferd
DEBUG_API     [ vfer_socket          ] Entered
DEBUG_PMTUD   [ Control_Initialize_PMTUD ] Unable to create ICMP
socket. Continuing anyways
DEBUG_API     [ vfer_bind            ] Entered
DEBUG_PMTUD   [ vfer_bind            ] Unable to bind ICMP socket.
Continuing without ICMP.
DEBUG_API     [ vfer_bind            ] Trying to set SO_RCVBUF to 3150000
DEBUG_API     [ vfer_bind            ] Trying to set SO_SNDBUF to 3150000
DEBUG_API     [ vfer_listen          ] Entered
DEBUG_CTL     [ Control_Register_Socket ] Starting ControlT thread,
num_active = 1
VFER_RCPD     [ main                 ] started with authenticating
username: root, home_dir: /root
DEBUG_API     [ vfer_accept          ] Entered
DEBUG_CTL     [ Control_Accept       ] Entered with listening sock->fd[3]
DEBUG_CTL     [ ControlT             ] Entered with num_active[1]
DEBUG_PMTUD   [ Control_Initialize_PMTUD ] Unable to create ICMP
socket. Continuing anyways
DEBUG_CTL     [ Control_Accept       ] New port socket: 0.0.0.0.47174

However, on the other host:

$ sudo ./vfer_rcp -vv -P 9876 ya...@ha...:/tmp/seq .
[sudo] password for yang:
VFER_RCP      [ main                 ] parsed: from_remote[0] lpath[.]
uname[yang] host[harvard.csail.mit.edu] port[9876] rpath[/tmp/seq]
VFER_RCP      [ rcp_connect          ] remote (harvard.csail.mit.edu :
128.30.76.131)
VFER_RCP      [ rcp_connect          ] local  (yang-xps410 : 127.0.1.1)
DEBUG_API     [ vfer_socket          ] Entered
DEBUG_PMTUD   [ Control_Initialize_PMTUD ] Unable to create ICMP
socket. Continuing anyways
DEBUG_API     [ vfer_bind            ] Entered
DEBUG_PMTUD   [ vfer_bind            ] Unable to bind ICMP socket.
Continuing without ICMP.
DEBUG_API     [ vfer_bind            ] Trying to set SO_RCVBUF to 3150000
DEBUG_API     [ vfer_bind            ] Trying to set SO_SNDBUF to 3150000
DEBUG_API     [ vfer_connect         ] Entered
DEBUG_CTL     [ Control_Connect      ] Entered sock->fd[3]
Control_Connect :: connect: Invalid argument
DEBUG_CTL     [ Control_Connect      ] connect failed
VFER_RCP      [ rcp_connect          ] DEBUG_API     [ vfer_errortext
     ] Entered
vfer_connect failed [not able to connect on a socket]

I wasn't able to find documentation on these two apps. Thanks in
advance for any hints.

[vfer-users] vfer for Internet transfers

[Yang <u01...@sn...>]

Not too surprisingly, when I try to FTP large files between two
(Internet-connected) hosts on opposite coasts of the US, I can achieve
only low throughput, presumably due in large part to high
bandwidth-delay (and TCP's behavior under those conditions). This
triggered my curiosity in alternative bulk transfer protocols.

Could vfer be such a protocol (i.e., likely to improve my throughput)?
If not, any pointers to such a protocol (available for use, not just a
publication)? If not, is there any reason why I probably shouldn't
expect to ever find such a protocol?

(I also looked into http://udt.sf.net/, but found its throughput to be
lower than TCP's.)

Thanks for any answers.

[vfer-users] VFER 0.98 Released

[Ivan B. <iv...@cs...>]

Hello everybody,

VFER 0.98 was released last week. The new release focuses on integrating 
code from students as part of the Google's summer of code program. It 
includes code from Nikoluas Rath who has worked on connection 
authentication using existing SSH keys and encryption, the new VSL layer 
(VFER sucre layer) specifications may be found online under the doxygen 
docs repository ( http://vfer.sourceforge.net/files/doxygen/html/vsl.html 
and http://vfer.sourceforge.net/files/doxygen/html/vsl__api_8h.html ). 
Also included in this release is code by Andrew Lake 
implementing the new path MTU discovery algorithm (RFC draft- 
http://www.ietf.org/internet-drafts/draft-ietf-pmtud-method-06.txt ). The 
path mtu discovery algorithm necessitated that the vector ack packet be changed 
to a packet carrying missing data ranges rather than a nack of received 
fragments. The new information is now payloaded on the CC ACK packets or 
just ACK packets. Also in this release are new tools that use the VSL 
layer, particularly a file transfer client which establishes an 
authetnication connection using existing SSH keys. This release also has a 
slightly changed delay based congestion control which we found to be more 
responsive to queueing delay than the algorithm which used base min - 
current min relative queueing delay. The release is still somewhat buggy 
and bug reports and general testing would be much appreciated.

thanks,
ivan.

Re: [vfer-users] [vfer-devel] VFER-0.97 Released

[Nikolaus R. <Nik...@ra...>]

Ivan Beschastnikh <iv...@cs...> writes:
> Thanks for the note. However i'm able to fully compile the released=20=20
> code on every type of OS. The Makefile now specifies std=3Dgnu99=20=20
> argument for gcc to use, perhaps you're using a different compiler or=20=
=20

Btw, the code doesn't depend on any GNU features, it is just C99.
However, GCC's C99 support works apparently better in gnu99 mode than
in c99 mode. For any other compiler, C99 mode should do fine.

   --Nikolaus

--=20
Disziplin ist die F=C3=A4higkeit, d=C3=BCmmer zu erscheinen als der Chef.

Re: [vfer-users] [vfer-devel] VFER-0.97 Released

[Gudmundson M. A <gud...@st...>]

Yes I'm sure it's a compiler difference.  It's really not a big problem, =
but I will try compiling on a different machine on Monday when I'm in.


-----Original Message-----
From: vfe...@li... on behalf of Ivan =
Beschastnikh
Sent: Fri 7/21/2006 11:11 AM
To: vfe...@li...; vfe...@li...
Subject: Re: [vfer-devel] VFER-0.97 Released
=20
This didn't seem to go through so I'm sending it again to the lists.

-------- Original Message --------
Subject: 	Re: [vfer-users] [vfer-devel] VFER-0.97 Released
Date: 	Thu, 20 Jul 2006 19:56:31 -0700
From: 	Ivan Beschastnikh <iv...@cs...>
To: 	Gudmundson Mitchell A <gud...@st...>
CC: 	<vfe...@li...>, =
<vfe...@li...>

=09


Hi Mitchell,

Thanks for the note. However i'm able to fully compile the released =20
code on every type of OS. The Makefile now specifies std=3Dgnu99 =20
argument for gcc to use, perhaps you're using a different compiler or =20
an older version of gcc (3.3.4 works for me here) ? Under the C99 =20
standard you can declare variables in middle of the code blocks so =20
vfer_getsockname() should compile ok. Nikolaus added this function =20
and that's the programming style he uses in the VSL lib codes but it =20
shouldn't impact compilation as long as the std flag is honored I =20
believe. Can you try on another machine and\or with a different gcc =20
version ?

thanks,

ivan.

On Jul 18, 2006, at 1:37 PM, Gudmundson Mitchell A wrote:

> You might want to change the function vfer_getsockname to at least =20
> declare or declare and initialize the vfer_sock * sock before the =20
> first if test is done.  I just downloaded the new release and did a =20
> make lib and got a complie error.  Just thought I'd let you know.
>
> -Mitchell
>
> -----Original Message-----
> From: vfe...@li... on behalf of Ivan =20
> Beschastnikh
> Sent: Tue 7/18/2006 12:56 AM
> To: vfe...@li...; vfe...@li...
> Subject: [vfer-devel] VFER-0.97 Released
>
> Hello,
>
> We've finally got around to releasing VFER-0.97. This timely release
> includes the long awaited delay based congestion control. The default
> parameters are a GAIN of 8 packets/RTT and a TARGET queueing delay of
> 8ms. The code has been run over LAN and WAN links and performs
> relatively well for a user space implementation with experimental CC.
>
> The release also includes quite a number of changes to the VFER
> protocol including data range vector nacks instead of fragment vector
> nacks, an AVL binary tree delay history data structure, and one of
> this summer of code's student's work implementing authentication and
> encryption within VFER. That code still needs work but it has already
> been merged into the main branch and testing is under way for the
> next release which will include authentication support over existing
> user SSH keys. Many bugs have been squashed and features added. There
> are links to the sources .tar.gz file from the VFER website- http://
> vfer.sf.net
>
>
> thanks all,
>
> ivan.



-------------------------------------------------------------------------=

Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share =
your
opinions on IT & business topics through brief surveys -- and earn cash
http://www.techsay.com/default.php?page=3Djoin.php&p=3Dsourceforge&CID=3D=
DEVDEV
_______________________________________________
vfer-devel mailing list
vfe...@li...
https://lists.sourceforge.net/lists/listinfo/vfer-devel

Re: [vfer-users] [vfer-devel] VFER-0.97 Released

[Ivan B. <iv...@cs...>]

This didn't seem to go through so I'm sending it again to the lists.

-------- Original Message --------
Subject: 	Re: [vfer-users] [vfer-devel] VFER-0.97 Released
Date: 	Thu, 20 Jul 2006 19:56:31 -0700
From: 	Ivan Beschastnikh <iv...@cs...>
To: 	Gudmundson Mitchell A <gud...@st...>
CC: 	<vfe...@li...>, <vfe...@li...>

	


Hi Mitchell,

Thanks for the note. However i'm able to fully compile the released  
code on every type of OS. The Makefile now specifies std=gnu99  
argument for gcc to use, perhaps you're using a different compiler or  
an older version of gcc (3.3.4 works for me here) ? Under the C99  
standard you can declare variables in middle of the code blocks so  
vfer_getsockname() should compile ok. Nikolaus added this function  
and that's the programming style he uses in the VSL lib codes but it  
shouldn't impact compilation as long as the std flag is honored I  
believe. Can you try on another machine and\or with a different gcc  
version ?

thanks,

ivan.

On Jul 18, 2006, at 1:37 PM, Gudmundson Mitchell A wrote:

> You might want to change the function vfer_getsockname to at least  
> declare or declare and initialize the vfer_sock * sock before the  
> first if test is done.  I just downloaded the new release and did a  
> make lib and got a complie error.  Just thought I'd let you know.
>
> -Mitchell
>
> -----Original Message-----
> From: vfe...@li... on behalf of Ivan  
> Beschastnikh
> Sent: Tue 7/18/2006 12:56 AM
> To: vfe...@li...; vfe...@li...
> Subject: [vfer-devel] VFER-0.97 Released
>
> Hello,
>
> We've finally got around to releasing VFER-0.97. This timely release
> includes the long awaited delay based congestion control. The default
> parameters are a GAIN of 8 packets/RTT and a TARGET queueing delay of
> 8ms. The code has been run over LAN and WAN links and performs
> relatively well for a user space implementation with experimental CC.
>
> The release also includes quite a number of changes to the VFER
> protocol including data range vector nacks instead of fragment vector
> nacks, an AVL binary tree delay history data structure, and one of
> this summer of code's student's work implementing authentication and
> encryption within VFER. That code still needs work but it has already
> been merged into the main branch and testing is under way for the
> next release which will include authentication support over existing
> user SSH keys. Many bugs have been squashed and features added. There
> are links to the sources .tar.gz file from the VFER website- http://
> vfer.sf.net
>
>
> thanks all,
>
> ivan.

Re: [vfer-users] [vfer-devel] VFER-0.97 Released

[Ivan B. <iv...@cs...>]

Hi Mitchell,

Thanks for the note. However i'm able to fully compile the released  
code on every type of OS. The Makefile now specifies std=gnu99  
argument for gcc to use, perhaps you're using a different compiler or  
an older version of gcc (3.3.4 works for me here) ? Under the C99  
standard you can declare variables in middle of the code blocks so  
vfer_getsockname() should compile ok. Nikolaus added this function  
and that's the programming style he uses in the VSL lib codes but it  
shouldn't impact compilation as long as the std flag is honored I  
believe. Can you try on another machine and\or with a different gcc  
version ?

thanks,

ivan.

On Jul 18, 2006, at 1:37 PM, Gudmundson Mitchell A wrote:

> You might want to change the function vfer_getsockname to at least  
> declare or declare and initialize the vfer_sock * sock before the  
> first if test is done.  I just downloaded the new release and did a  
> make lib and got a complie error.  Just thought I'd let you know.
>
> -Mitchell
>
> -----Original Message-----
> From: vfe...@li... on behalf of Ivan  
> Beschastnikh
> Sent: Tue 7/18/2006 12:56 AM
> To: vfe...@li...; vfe...@li...
> Subject: [vfer-devel] VFER-0.97 Released
>
> Hello,
>
> We've finally got around to releasing VFER-0.97. This timely release
> includes the long awaited delay based congestion control. The default
> parameters are a GAIN of 8 packets/RTT and a TARGET queueing delay of
> 8ms. The code has been run over LAN and WAN links and performs
> relatively well for a user space implementation with experimental CC.
>
> The release also includes quite a number of changes to the VFER
> protocol including data range vector nacks instead of fragment vector
> nacks, an AVL binary tree delay history data structure, and one of
> this summer of code's student's work implementing authentication and
> encryption within VFER. That code still needs work but it has already
> been merged into the main branch and testing is under way for the
> next release which will include authentication support over existing
> user SSH keys. Many bugs have been squashed and features added. There
> are links to the sources .tar.gz file from the VFER website- http://
> vfer.sf.net
>
>
> thanks all,
>
> ivan.
>
> ---------------------------------------------------------------------- 
> ---
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to  
> share your
> opinions on IT & business topics through brief surveys -- and earn  
> cash
> http://www.techsay.com/default.php? 
> page=join.php&p=sourceforge&CID=DEVDEV
> _______________________________________________
> vfer-devel mailing list
> vfe...@li...
> https://lists.sourceforge.net/lists/listinfo/vfer-devel
>
>
> ---------------------------------------------------------------------- 
> ---
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to  
> share your
> opinions on IT & business topics through brief surveys -- and earn  
> cash
> http://www.techsay.com/default.php? 
> page=join.php&p=sourceforge&CID=DEVDEV
> _______________________________________________
> http://vfer.sf.net
> vfer-users mailing list
> vfe...@li...
> https://lists.sourceforge.net/lists/listinfo/vfer-users

Re: [vfer-users] [vfer-devel] VFER-0.97 Released

[Gudmundson M. A <gud...@st...>]

You might want to change the function vfer_getsockname to at least =
declare or declare and initialize the vfer_sock * sock before the first =
if test is done.  I just downloaded the new release and did a make lib =
and got a complie error.  Just thought I'd let you know.

-Mitchell

-----Original Message-----
From: vfe...@li... on behalf of Ivan =
Beschastnikh
Sent: Tue 7/18/2006 12:56 AM
To: vfe...@li...; vfe...@li...
Subject: [vfer-devel] VFER-0.97 Released
=20
Hello,

We've finally got around to releasing VFER-0.97. This timely release =20
includes the long awaited delay based congestion control. The default =20
parameters are a GAIN of 8 packets/RTT and a TARGET queueing delay of =20
8ms. The code has been run over LAN and WAN links and performs =20
relatively well for a user space implementation with experimental CC.

The release also includes quite a number of changes to the VFER =20
protocol including data range vector nacks instead of fragment vector =20
nacks, an AVL binary tree delay history data structure, and one of =20
this summer of code's student's work implementing authentication and =20
encryption within VFER. That code still needs work but it has already =20
been merged into the main branch and testing is under way for the =20
next release which will include authentication support over existing =20
user SSH keys. Many bugs have been squashed and features added. There =20
are links to the sources .tar.gz file from the VFER website- http://=20
vfer.sf.net


thanks all,

ivan.

-------------------------------------------------------------------------=

Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share =
your
opinions on IT & business topics through brief surveys -- and earn cash
http://www.techsay.com/default.php?page=3Djoin.php&p=3Dsourceforge&CID=3D=
DEVDEV
_______________________________________________
vfer-devel mailing list
vfe...@li...
https://lists.sourceforge.net/lists/listinfo/vfer-devel

[vfer-users] VFER-0.97 Released

[Ivan B. <iv...@cs...>]

Hello,

We've finally got around to releasing VFER-0.97. This timely release  
includes the long awaited delay based congestion control. The default  
parameters are a GAIN of 8 packets/RTT and a TARGET queueing delay of  
8ms. The code has been run over LAN and WAN links and performs  
relatively well for a user space implementation with experimental CC.

The release also includes quite a number of changes to the VFER  
protocol including data range vector nacks instead of fragment vector  
nacks, an AVL binary tree delay history data structure, and one of  
this summer of code's student's work implementing authentication and  
encryption within VFER. That code still needs work but it has already  
been merged into the main branch and testing is under way for the  
next release which will include authentication support over existing  
user SSH keys. Many bugs have been squashed and features added. There  
are links to the sources .tar.gz file from the VFER website- http:// 
vfer.sf.net


thanks all,

ivan.

[vfer-users] Announcing VFER-0.96

[Ivan B. <iv...@cs...>]

VFER 0.96 Released.

This release finalizes many simplifications to the architecture of the
implementation. Of note: there is one library thread managing all of
the protocol logic, vfer fds (typedef int) are now used in all vfer
library calls by the client to refer to vfer sockets, and lastly the
closing handshake has been simplified a great deal while the initial
handshake has been made to be more failsafe. These changes make future
development easier and speed up the implementation. The internal
congestion control and the protocol haven't been changed in this
release. You can download this release here. The doxygen documentation
has received some extra effort and is now up to date. The protocol
specs on the other hand need to be revised and are unreliable.


You can find the .tar.gz release file here:
http://sourceforge.net/project/showfiles.php?group_id=3D151673


thanks,

ivan.

Re: [vfer-users] bulk transport api question

[Ivan B. <iv...@cs...>]

On 08 May 2006 15:49:21 -0400, stanislav shalunov
<sha...@in...> wrote:
> Is there anything that prevents VFER_SOCKET from being typedef'ed as an i=
nt?

That's what I would prefer in the implementation as it would simplify
many cases. I'm not sure if this is okay with the drafted API however.

ivan.

Re: [vfer-users] bulk transport api question

[stanislav s. <sha...@in...>]

Oh, I see now.  It has ``X_SOCKET *'' in all the prototypes.  How
would you get access to the stats of a given fd given a fd?  Your
reason (3), indeed, would be a good reason to just use integer
descriptors in general, in VFER or anywhere else.

This is what implementation experience is supposed to be about.

Can you send Steven textual changes?

-- 
Stanislav Shalunov		http://www.internet2.edu/~shalunov/

Heather has two mommies.  That's nothing.  The Internet has at least
two dozen daddies.

[vfer-users] Announcing VFER-0.95

[Ivan B. <iv...@cs...>]

VFer 0.95 Released.

This is a long awaited release that includes many improvements to all
parts of the code. The release contains a fully functional file
transfer client and server that report transfer speeds. The protocol
uses a TCP-like AIMD congestion control and performs fairly well on
large files (approx 60Mb/s data rates over 100Mb link in LAN
environments and up to 80Mb/s over 100Mb link in WAN environments,
although results may vary :-) The internal threading arch has also
been greatly improved.

You can find the .tar.gz release file here:
http://prdownloads.sourceforge.net/vfer/vfer-0.95.tar.gz?use_mirror=3Dumn

Please download the release let us know what you think.



thanks,

ivan.

Re: [vfer-users] bulk transport api question

[Ivan B. <iv...@cs...>]

On 08 May 2006 16:07:01 -0400, stanislav shalunov
<sha...@in...> wrote:
> How would you get access to the stats of a given fd given a fd?

The X_STATS * x_sockstats(X_SOCKET *skt)
which is the way current draft specifies stats retrieval, would
instead look like this:

X_STATS x_sockstats(X_FD)

It can return a pointer to an allocated structure but (as I said in
the first email), to me it would seem like a better idea to return the
full data structure rather than a ptr to it.

> Can you send Steven textual changes?

Do you mean the proposed textual changes to the API draft? Should I
forward to him my first email with the reasons for the change[s] to
see what he thinks, as he might not be actively monitoring this list.

Re: [vfer-users] bulk transport api question

[stanislav s. <sha...@in...>]

"Ivan Beschastnikh" <iv...@cs...> writes:

> On 08 May 2006 16:07:01 -0400, stanislav shalunov
> <sha...@in...> wrote:
> > How would you get access to the stats of a given fd given a fd?
> 
> The X_STATS * x_sockstats(X_SOCKET *skt)
> which is the way current draft specifies stats retrieval, would
> instead look like this:
> 
> X_STATS x_sockstats(X_FD)

Why not just replace ``X_STATS *'' with ``X_STATS'' and ``X_SOCKET *''
with ``X_SOCKET'' throughout?  The implementation should then be left
free to define X_SOCKET as an integer, no?

> Do you mean the proposed textual changes to the API draft? Should I
> forward to him my first email with the reasons for the change[s] to
> see what he thinks, as he might not be actively monitoring this list.

Well, Steven's on vfer-users.

-- 
Stanislav Shalunov		http://www.internet2.edu/~shalunov/

I never let school stand in the way of my education.  -- Mark Twain

Re: [vfer-users] bulk transport api question

[Steven S. <se...@cs...>]

I think this would be fine.

- steve

On May 8, 2006, at 2:49 PM, stanislav shalunov wrote:

> Is there anything that prevents VFER_SOCKET from being typedef'ed  
> as an int?
>
> -- 
> Stanislav Shalunov		http://www.internet2.edu/~shalunov/
>
> This message is designed to be viewed at sea level.
>
>
> -------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services,  
> security?
> Get stuff done quickly with pre-integrated technology to make your  
> job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache  
> Geronimo
> http://sel.as-us.falkag.net/sel? 
> cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> http://vfer.sf.net
> vfer-users mailing list
> vfe...@li...
> https://lists.sourceforge.net/lists/listinfo/vfer-users

Re: [vfer-users] bulk transport api question

[stanislav s. <sha...@in...>]

Is there anything that prevents VFER_SOCKET from being typedef'ed as an int?

-- 
Stanislav Shalunov		http://www.internet2.edu/~shalunov/

This message is designed to be viewed at sea level.

[vfer-users] bulk transport api question

[Ivan B. <iv...@cs...>]

Hi, I have a few questions concerning the bulk trasport api.

My main question concerns the use of X_SOCKET rather than an integer
fd like identifier for the socket. My problems with it are the
following:

1. Possible unsafe\incorrect pointer passed to a vfer library function
means we can't always bail gracefully and be very careful about
locking anything for example before accessing the pointer.

2. User has to worry about deallocation of memory for this structure
rather than let the lib handle the details of this.

3. Because the structure can't contain more than a stats structure and
error status (wouldn't want the user to access/tweak internally
maintained vars that might require a mutex lock before access), means
that internally there will be another bigger structure to which
X_SOCKET must have a reference, probably an fd-like equivalent. It
would simplify things if we dealt with an fd.

4. Berkeley sockets api deals with fds, so it might be more
transparent for users to use fds with VFER, even less work for them to
translate existing code to use VFER.

To handle errors, I can return a negative integer which will carry the
error code that wouldn't have to be looked up with x_sockerror() and
can be translated with errortext() as defined.

My second question concerns the semantics of x_sockstats(). If the
socket structure contains a stats structure, is that the outdated
structure? Does this call update the socket's stats structure, and if
so, why not return void and have the user access the stats structure
using the X_SOCKET ptr? Also why is the return type a pointer, does
this mean that the lib will allocate the memory for the stats
structure and its up to the user to release it, perhaps returning a
stats structure itself rather than a pointer would be better, as the
user wouldn't have to deal with mem deallocation?

thanks,

ivan.

Re: [vfer-users] vfer current cvs thread deadlocks on files larger than 1mb

[stanislav s. <sha...@in...>]

"Ivan Beschastnikh" <iv...@cs...> writes:

> Its actually not very difficult to impose a nonthreading model.

The key is probably getting vfer_select() right.  Since both blocking
and non-blocking modes are necessary, it will probably be easiest to
implement blocking mode as vfer_select() with a single descriptor
underneath.

-- 
Stanislav Shalunov		http://www.internet2.edu/~shalunov/

Religion is the opium of them asses.		--Karlm Arx

Re: [vfer-users] vfer current cvs thread deadlocks on files larger than 1mb

[Ivan B. <iv...@cs...>]

On 5/2/06, Markus Koetter <koe...@gm...> wrote:
>
> It is a threading deadlock, maybe the protocol bugs too, but what I saw
> was a definitly a threading deadlock.
>
>   PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
>  7249 me        16   0 56716  22m  692 S 90.1  2.2   3:16.98 vfer
>
> select(0, NULL, NULL, NULL, {1, 0})     =3D 0 (Timeout)
> futex(0x8062988, FUTEX_WAIT, 2, NULL
>
> and then the cpu peaks at 99% for the process.
>
> =3D=3D8071=3D=3D    at 0x1B949199: __lll_mutex_lock_wait (in
> /lib/tls/libpthread-2.3.6.so)
> =3D=3D8071=3D=3D    by 0x804F105: Control_Recv (control.c:600)
> =3D=3D8071=3D=3D    by 0x804B1ED: vfer_recv (api.c:930)
> =3D=3D8071=3D=3D    by 0x804AF44: vfer_recvfile (api.c:872)
> =3D=3D8071=3D=3D    by 0x805D3C8: main (vfer_rcp.c:407)

I haven't seen deadlocks such as this in my testing. Send me more info
so I can try to reproduce what you're seeing.

> Murphys law says "Interchangeable parts-won't", and the socket io used
> in vfer was not even planned as interchangeable, replacing it will be
> the hell. One has to get a _real_ close view on all parts of the code,
> draft a new layout, and restructure the existing code, while rewriting
> the socket io.

Its actually not very difficult to impose a nonthreading model.
Reading and writing is relegated to reader.c and writer.c which are
impl as two threads. Meshing them together is not very difficult.
Instead of building up a private packet buffer, the packets can be
handled as they come in and that removes almost all mutexes.
Appropriate functions from control.c such as the main thread loop
(ControlT) can be modified to proccess packets as they come in,
instead of blocking until they appear in the packet buffer. If each
socket is handled by a different thread, the modifications you have to
make are not very difficult. I'll be working on this project over the
summer as well, and will provide whatever help and code guidance that
will be necessary to make new modifications and future development
easy. The code is very well documented so getting to know it isn't
very difficult either. These are my personal opinions, you are
welcomed to disagree :-)


ivan.

Re: [vfer-users] vfer current cvs thread deadlocks on files larger than 1mb

[Markus K. <koe...@gm...>]

Hi,

I guess some people might be offended by this email, but I don't intend
to start a flamewar, and as the list has about 4 users, there is only a
slight chance somebody will take it that way.

Try to take it as constructive critics, I spend two days on evaluating
the code, more than half of this was burned on the last released
version, which is far beyond my vocabular.

I expected the codebase from last years endofsoc to work, maybe with
little problems, but the rare opposite is the fact, even after a whole
year, the current cvs code still suffers from its very early design
decisions.



>> I already profiled the code, and checked where to find the deadlocks,
>> (not how to defeat them), so if these problems are not known, drop me a
>> line and I'll followup with a more complete bugreport.
> 
> 
> I'm working on getting rid of these before the next release which will
> come out this week. Its a known issue but it has less to do with
> deadlocks and more to do with the logic of the protocol. I've observed
> deadlock scenarios in which RTT estimation fails to approximate a
> reasonable number for example and the two endpoints fall out of sync
> and take an effectively infinite time to complete. Packets are
> streaming (if you use the -v option) but data packets aren't being
> sent. If you turn off the congestion control, transfers complete but
> with intermediate congestion collapse.

It is a threading deadlock, maybe the protocol bugs too, but what I saw
was a definitly a threading deadlock.

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 7249 me        16   0 56716  22m  692 S 90.1  2.2   3:16.98 vfer

select(0, NULL, NULL, NULL, {1, 0})     = 0 (Timeout)
futex(0x8062988, FUTEX_WAIT, 2, NULL

and then the cpu peaks at 99% for the process.

==8071==    at 0x1B949199: __lll_mutex_lock_wait (in
/lib/tls/libpthread-2.3.6.so)
==8071==    by 0x804F105: Control_Recv (control.c:600)
==8071==    by 0x804B1ED: vfer_recv (api.c:930)
==8071==    by 0x804AF44: vfer_recvfile (api.c:872)
==8071==    by 0x805D3C8: main (vfer_rcp.c:407)

>> From my point of view the whole threading used for the socketio is far
>> to complex for the task itself, incomplete and therefore pretty error
>> prone, for example
....
> 
> 
> Yes, this is definitely a handicap of the implementation, but it also
> has some benefits which I won't go into. We're hoping to find a
> student for Google's summer of code 2006 to experiment with non
> threaded alternatives and perform performance measurements.

I'd be pleased to hear about these benefits.
I like sockets, and was about to apply for this project, but ... 3 month
is far too less to fix a 12 month history of visions, even with code reuse.

I already solved the first tasks, measure performance and check if its
threading problems.
Compiling with -pg and running gprof on the code, valgrind, gdb, and I
still can't understand how you could start implementing the protocol
without a reliable working socket base.

With threads this will *never* run stable and offer the protocols
possible performance, therefore protocol measurements and improvements
are wasted time, to go without threads, somebody has to rewrite it.


Murphys law says "Interchangeable parts-won't", and the socket io used
in vfer was not even planned as interchangeable, replacing it will be
the hell. One has to get a _real_ close view on all parts of the code,
draft a new layout, and restructure the existing code, while rewriting
the socket io.


> The impl only uses blocking mutex acquisitions. The short story is
> that there are two sets of mutexes, those that control bins that
> receive packets and those that control access to the sockets array.
> Receiving packets into and reading from the same bin makes more sense
> with a blocking mutex, and socket array access likewise doesn't
> benefit from a non blocking trylock call. The sockets array mutexes
> haven't been tested in full since one socket suffices to find the
> important bugs right now. The bin mutexes have been thoroughly tested.
> 
...
> 
> None of these are actually being used. These are there for the
> vfer_select() call which is not fully implemented right now. Condition
> vars are not being used anywhere in the active code.

Thats the problem, not the solution.

Don't take it bad, but the used threading lacks any design idea or
knowledge about thread synchronisation, so how can you claim benefits
using threads here?

There has to be no no blocking code, everything that has to be done can
be done nonblocking, sendmsg, recvmsg, writev, readv,
Just pure memory operations, and some computations, nothing that could
b/lock the process over the time to make threads a usefull feature.


As said before, I was about to apply, but after getting a deeper view, I
won't, this won't last 3 month, it will take at least (!) one year.
And the application simply lies, it talks about improvements, it should
state 'fix somebodys visions' and get the skill ranking 'hardco(r|d)e'.

My respect for the choosen one who applies to this task and solves it,
even if he makes me look like an arrogant liar, I'll owe him a beer.


For all those reading the ml as they want to assign for the project, if
you want your summer of code to be fun, don't apply here,
this is a summer of pain project.


MfG
Markus Koetter

Re: [vfer-users] vfer current cvs thread deadlocks on files larger than 1mb

[Ivan B. <iv...@cs...>]

Hi Markus,

Thanks for trying the protocol and for the good discussion. My replies
are interleaved below.

On 5/1/06, Markus Koetter <koe...@gm...> wrote:
> Hi,
>
> I compiled vfer current cvs on a debian & gentoo host in my lan, and hit
> some strange borders.
>
> both boxes are single cpu boxes, so smp/ht here.
>
> I was able to transfer a 1mb file at something like 22mbit, larger files
> like 100mb or 300mb would either deadlock the server, the client,
> or interrupt with an 'error', the server claims the client is guilty,
> the client disconnects with an error.
>
> I already profiled the code, and checked where to find the deadlocks,
> (not how to defeat them), so if these problems are not known, drop me a
> line and I'll followup with a more complete bugreport.

I'm working on getting rid of these before the next release which will
come out this week. Its a known issue but it has less to do with
deadlocks and more to do with the logic of the protocol. I've observed
deadlock scenarios in which RTT estimation fails to approximate a
reasonable number for example and the two endpoints fall out of sync
and take an effectively infinite time to complete. Packets are
streaming (if you use the -v option) but data packets aren't being
sent. If you turn off the congestion control, transfers complete but
with intermediate congestion collapse.

> From my point of view the whole threading used for the socketio is far
> to complex for the task itself, incomplete and therefore pretty error
> prone, for example

Yes, this is definitely a handicap of the implementation, but it also
has some benefits which I won't go into. We're hoping to find a
student for Google's summer of code 2006 to experiment with non
threaded alternatives and perform performance measurements.

>
> grep pthread_mutex_trylock * -R | wc
>       0       0       0

The impl only uses blocking mutex acquisitions. The short story is
that there are two sets of mutexes, those that control bins that
receive packets and those that control access to the sockets array.
Receiving packets into and reading from the same bin makes more sense
with a blocking mutex, and socket array access likewise doesn't
benefit from a non blocking trylock call. The sockets array mutexes
haven't been tested in full since one socket suffices to find the
important bugs right now. The bin mutexes have been thoroughly tested.

>
> no checking for *possible* deadlocks
>
> grep pthread_cond_ * -R
> src/api.c:      pthread_cond_init(&skt->cond_readable, NULL);
> src/globals.h:  pthread_cond_t cond_readable;
> src/globals.h:  pthread_cond_t cond_writeable;
> src/globals.h:  pthread_cond_t cond_exception;

None of these are actually being used. These are there for the
vfer_select() call which is not fully implemented right now. Condition
vars are not being used anywhere in the active code.

> src/packet.h:// pthread_cond_t cond;

This one is commented out and is part of an older 'vision'.

> my approach was to rewrite the socket io threadless and nonblocking, as
> the whole protocol implementation is useless (as it can neither be
> measured nor improved), if the socketio is unreliable due to threading
> problems.

As I said above, hopefully we can experiment this summer.


Thanks very much!

ivan.

[vfer-users] vfer current cvs thread deadlocks on files larger than 1mb

[Markus K. <koe...@gm...>]

Hi,

I compiled vfer current cvs on a debian & gentoo host in my lan, and hit
some strange borders.

both boxes are single cpu boxes, so smp/ht here.

I was able to transfer a 1mb file at something like 22mbit, larger files
like 100mb or 300mb would either deadlock the server, the client,
or interrupt with an 'error', the server claims the client is guilty,
the client disconnects with an error.

I already profiled the code, and checked where to find the deadlocks,
(not how to defeat them), so if these problems are not known, drop me a
line and I'll followup with a more complete bugreport.


From my point of view the whole threading used for the socketio is far
to complex for the task itself, incomplete and therefore pretty error
prone, for example

grep pthread_mutex_trylock * -R | wc
      0       0       0

no checking for *possible* deadlocks

grep pthread_cond_ * -R
src/api.c:      pthread_cond_init(&skt->cond_readable, NULL);
src/globals.h:  pthread_cond_t cond_readable;
src/globals.h:  pthread_cond_t cond_writeable;
src/globals.h:  pthread_cond_t cond_exception;
src/packet.h:// pthread_cond_t cond;


no signaling to tell a thread a mutex got unlocked, and a new ressource
is availible.



my approach was to rewrite the socket io threadless and nonblocking, as
the whole protocol implementation is useless (as it can neither be
measured nor improved), if the socketio is unreliable due to threading
problems.



MfG
Markus Koetter