aoetools-discuss — General discussion list for AoE and aoetools

Re: [Aoetools-discuss] Query on AoE initiator accessing target started locally over lo

[Ed C. <ed....@ac...>]

On 08/17/2017 10:39 PM, Nivin Lawrence wrote:
...
> __1)__Is all the code for AoE initiator/target located @ 
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/block/aoe 
> <http://%2522/>?____

No, that's the initiator for the Linux kernel, the most popular initiator.

The most popular open source target is the vblade.  You can find it in 
the OpenAoE project on github.

> __2)__Is it possible to cleanly stop the AoE target, ensuring the cache 
> is written to disk and any cleanup needed is done?____

Yes, the steps would be:

   * On the initiator safely shut down each layer using
     the storage, from highest to lowest level.

     * E.g., umount, vgchange -a n ...

   * Run "sync" on the initiator

     There are some kernels (old, probably) that do not
     automatically sync on unmount.

   * Interrupt the networking that supports the AoE command stream

     This is just a paranoia step.  No AoE commands should be going
     out after you stop using the AoE target.

   * Run sync on the host running the target.

   * Kill vblade (assuming that's what it is) with SIGTERM.

   * Run sync on the target host again, for paranoia.

Now if the target is exporting a file on some underlying storage that 
needs to be quiesced or something, you'd have to quiesce it, but that 
seems unlikely.

More comments follow quotes below.

> __3)__I have a use case where I need to use AoE over network initially, 
> but later stop the remote target, restart target locally on loopback 
> interface. The storage that is accessed in both cases is the same, so no 
> issues with data being in sync. I have a few clarification requests in 
> this regard:____

That's confusing me.  Can you be really concrete, like with hostnames, 
network interface names, and network switch names and ports, etc.?

> __a.__Wanted to understand if the AoE model has specific optimizations 
> done to make this as close as possible to a regular local disk access.____

The biggest optimization you can take advantage of is fast networking 
with jumbo (e.g., 9000 MTU at all network endpoints) packets.

...
> __c.__Do you think further optimizations are theoretically possible to 
> take care of this specific scenario to make it just like a regular local 
> disk access?____

If you search the archives you can find repeated but infrequent posts 
about a performance problem that can occur when the alignment of I/O 
requests on the initiator doesn't match the backing storage.  Then you 
wind up doing extra I/O.  You could look into that problem.  It would be 
easier to use 40 GbE, though.

>     4)__Is there a way to pause the initiator from trying to communicate
>     with the target once it is decided that the target has to be shut
>     and restarted locally?

You can pause the initiator by interrupting networking, e.g., by setting 
the AoE network interface(s) to a "down" state.  The aoe initiator will 
retransmit commands until it gets a response.  If aoe_deadsecs seconds 
elapse, though, the aoe driver will mark the device as down and fail all 
outstanding I/O requests, so you can set that to a large enough number 
of seconds to support the interruption.

...
> __5)__For use of AoE in production, I was wondering if there would be 
> any kind of metric which would help with details on the stability and 
> availability expectations and any feedback from current usage in 
> production.____

The code itself is stable and mature.  Storage over networking has its 
pitfalls, whether it's enterprise stuff or something you put together 
with open source components.  E.g., if your network switch has buggy 
firmware, you could have problems.  The best way to gain confidence is 
to use it in systems where you can make mistakes and learn a use pattern 
that works for you.

-- 
   Ed

[Aoetools-discuss] Query on AoE initiator accessing target started locally over lo

[Nivin L. <niv...@gm...>]

Hi Experts,

      Could you please help with your comments for the below.



1)      Is all the code for AoE initiator/target located @
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/
linux.git/tree/drivers/block/aoe <http://%2522/>?


2)      Is it possible to cleanly stop the AoE target, ensuring the cache
is written to disk and any cleanup needed is done?


3)      I have a use case where I need to use AoE over network initially,
but later stop the remote target, restart target locally on loopback
interface. The storage that is accessed in both cases is the same, so no
issues with data being in sync. I have a few clarification requests in this
regard:

a.       Wanted to understand if the AoE model has specific optimizations
done to make this as close as possible to a regular local disk access.

b.      I see the Linux storage stack @ https://upload.wikimedia.org/
wikipedia/commons/3/30/IO_stack_of_the_Linux_kernel.svg <http://%2522/>, I
was wondering about the overhead involved in accessing storage from
application POV once I restart the AoE target locally.

                                                              i.      Overhead
from the perspective of additional LOC involved as compared to a local disk
access

                                                            ii.      Overhead
from the perspective of throughput possible as compared to a local disk
access

c.       Do you think further optimizations are theoretically possible to
take care of this specific scenario to make it just like a regular local
disk access?

4)      Is there a way to pause the initiator from trying to communicate
with the target once it is decided that the target has to be shut and
restarted locally?

a.       I would like to avoid having to drop and retry in this specific
case as it’s a controlled operation. I understand this might not be a use
case generally.

b.      Would be ideal if we can flush any outstanding requests, stop
further requests from being sent and queue it up, receive all the responses
so that there are no outstanding requests from initiator POV. This would be
a good point to shut down the remote AoE target and restart target locally.


5)      For use of AoE in production, I was wondering if there would be any
kind of metric which would help with details on the stability and
availability expectations and any feedback from current usage in production.



Thanks,

Nivin

Re: [Aoetools-discuss] Tuning

[ray k. <jul...@ya...>]

Thanks for this common sense reply from the "Of course if you put it that way" department. Would it even make sense to cut back on the number of ports per initiator to reduce congestion? or would a 10GB switch upgrade make more sense?  

    On Wednesday, 12 April 2017, 17:39, Ed Cashin <ed...@ca...> wrote:
 

 You can echo into the module parameter files under /sys to change the settings on the fly.
If you have more total bandwidth on the sum of the initiator ports (16 Gbps) than for the total bandwidth of the target ports (8 Gbps), heavy write workloads will cause congestion on the target side of the data path.
There's only so much the initiator can do to compensate. 
"Kickme" means that the driver wanted to send more commands to the target but found out there were already the max number of commands waiting for a response. So later it will try again after being kicked by the periodic timer handler routine.

On Mar 31, 2017, at 1:24 PM, ray klassen <jul...@ya...> wrote:


Several years of use on an aoe installation and we seem to be running into performance issues. My theory is that we've hit some kind of ceiling that my rudimentary tuning at install time is not sufficient to deal with
description: dedicated 1Gb switch with target connected with 8 ports and 4 initiators (proxmox nodes) connected with 4 ports each. 
Target is running ggaoed. The targets are debian stock installs of aoe-tools. (i may reduce that to 7 ports on the target)

on the target, ifconfig shows a slowly growing number of dropped packets and overruns. about .01% of Rx Packets, which doesn't look too big when put in terms of percentage but probably means an iowait on the initiator because some percentage of writes have to be repeated.
Recently I increased the ring buffer on ggaoed. which seems to have improved things somewhat. Earlier on this mailing list someone referenced the aoe_maxout parameter which I have never set but the /sys/module/aoe/parameters/aoe_maxout records as 128 on all initiators https://www.spinics.net/lists/aoetools/msg00058.html



the recommendation in the earlier message on this mailing list was to set it at 8 instead of 16. 128 is much more than either of those I was wondering if it's auto chosen by the module when it loads. 

questions: in /sys/block/etherd\!e0.0/debug, what does the kicked value mean?                Can aoe kernel module parameters such as aoe_maxout be altered on the fly using sysctl or something? 

                (It's tough doing tuning on a live system)


Thank you for any response



------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

_______________________________________________
Aoetools-discuss mailing list
Aoe...@li...
https://lists.sourceforge.net/lists/listinfo/aoetools-discuss



   

Re: [Aoetools-discuss] Tuning

[Ed C. <ed...@ca...>]

You can echo into the module parameter files under /sys to change the settings on the fly.

If you have more total bandwidth on the sum of the initiator ports (16 Gbps) than for the total bandwidth of the target ports (8 Gbps), heavy write workloads will cause congestion on the target side of the data path.

There's only so much the initiator can do to compensate. 

"Kickme" means that the driver wanted to send more commands to the target but found out there were already the max number of commands waiting for a response. So later it will try again after being kicked by the periodic timer handler routine.

> On Mar 31, 2017, at 1:24 PM, ray klassen <jul...@ya...> wrote:
> 
> Several years of use on an aoe installation and we seem to be running into performance issues. My theory is that we've hit some kind of ceiling that my rudimentary tuning at install time is not sufficient to deal with
> 
> description: dedicated 1Gb switch with target connected with 8 ports and 4 initiators (proxmox nodes) connected with 4 ports each. 
> Target is running ggaoed. The targets are debian stock installs of aoe-tools. (i may reduce that to 7 ports on the target)
> 
> on the target, ifconfig shows a slowly growing number of dropped packets and overruns. about .01% of Rx Packets, which doesn't look too big when put in terms of percentage but probably means an iowait on the initiator because some percentage of writes have to be repeated.
> 
> Recently I increased the ring buffer on ggaoed. which seems to have improved things somewhat. Earlier on this mailing list someone referenced the aoe_maxout parameter which I have never set but the /sys/module/aoe/parameters/aoe_maxout records as 128 on all initiators https://www.spinics.net/lists/aoetools/msg00058.html
> 
> 
> 
> the recommendation in the earlier message on this mailing list was to set it at 8 instead of 16. 128 is much more than either of those I was wondering if it's auto chosen by the module when it loads. 
> 
> questions: in /sys/block/etherd\!e0.0/debug, what does the kicked value mean?
>                 Can aoe kernel module parameters such as aoe_maxout be altered on the fly using sysctl or something? 
> 
>                 (It's tough doing tuning on a live system)
> 
> 
> Thank you for any response
> 
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Aoetools-discuss mailing list
> Aoe...@li...
> https://lists.sourceforge.net/lists/listinfo/aoetools-discuss

[Aoetools-discuss] Tuning

[ray k. <jul...@ya...>]

Several years of use on an aoe installation and we seem to be running into performance issues. My theory is that we've hit some kind of ceiling that my rudimentary tuning at install time is not sufficient to deal with
description: dedicated 1Gb switch with target connected with 8 ports and 4 initiators (proxmox nodes) connected with 4 ports each. 
Target is running ggaoed. The targets are debian stock installs of aoe-tools. (i may reduce that to 7 ports on the target)

on the target, ifconfig shows a slowly growing number of dropped packets and overruns. about .01% of Rx Packets, which doesn't look too big when put in terms of percentage but probably means an iowait on the initiator because some percentage of writes have to be repeated.
Recently I increased the ring buffer on ggaoed. which seems to have improved things somewhat. Earlier on this mailing list someone referenced the aoe_maxout parameter which I have never set but the /sys/module/aoe/parameters/aoe_maxout records as 128 on all initiators https://www.spinics.net/lists/aoetools/msg00058.html



the recommendation in the earlier message on this mailing list was to set it at 8 instead of 16. 128 is much more than either of those I was wondering if it's auto chosen by the module when it loads. 

questions: in /sys/block/etherd\!e0.0/debug, what does the kicked value mean?                Can aoe kernel module parameters such as aoe_maxout be altered on the fly using sysctl or something? 

                (It's tough doing tuning on a live system)


Thank you for any response

Re: [Aoetools-discuss] AOE client behavior on server restart

[Ed C. <ed....@ac...>]

Hi, Nivin Lawrence.  Answers appear between selected quotes below.

On 03/27/2017 11:12 AM, Nivin Lawrence wrote:
> Hi experts,
>     I am trying at understanding the implications of any network drops
> and AOE/vblade server restart.
>
> 1) Is it expected to not see any loss in the Ethernet network where the
> client/server resides or is the protocol capable of handling drops in
> the network?

For storage, the "target" is the place where the data is persisted, and 
the "initiator" is the software that wants to read and write the data. 
I think you mean initiator by client, but I'll keep using the storage 
oriented words, because it matches the documentation.

No, Ethernet is expected to drop packets, and the initiator is expected 
to retransmit commands.  The tag field in the AoE header identifies a 
command, and the initiator gets to pick it.  The target copies that tag 
into the response to the command, so that the initiator can match the 
response to the waiting request.

> 2) If the server restarts and remains unavailable for say 5 seconds,
> what happens to the clients?

That's up to the initiator.  For example, the aoe driver in Linux has a 
module parameter, aoe_deadsecs, that controls this behavior.  If it's 
set to 4, then it will fail all the AoE commands (reads and writes) that 
have been waiting for a response for 4 seconds.

But the default is higher: three minutes worth of seconds.

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/block/aoe/aoecmd.c#n29

...
>      - Would be good to understand if there is an option where client
> wont need to do a mount again once the server becomes available and if
> the transactions could continue just as if there was a network
> connectivity issue for 5 seconds.

 From the initiator's perspective, that's already the case: Some people 
plug the initiator and target into a new networks switch without even 
unmounting.  That's not recommended, but it has happened a lot.

Now there's also the issue of the target.  Nothing I've said addresses 
the possibility of write caching.  Without write caching there is no 
problem with the target power cycling or something, you mentioned the 
server restarting.  If the server is running a vblade using regular I/O, 
you can't just restart that system.  You'd have to make sure that 
incoming AoE commands stop, all write operations finish, buffers are 
synced from RAM to disk, and *then* the server restarts.

When AoE resumes, some of the commands can be performed again, the ones 
that were performed without the response reaching the initiator. 
Usually that will mean the same data gets written to the same place as 
before.

There are ways to avoid write caching on the storage target, but they 
often entail a significant performance hit.  There are ways to make the 
whole write cache path keep the data safe in case of power interruption, 
but the ones I'm aware of use special hardware.

-- 
   Ed

[Aoetools-discuss] AOE client behavior on server restart

[Nivin L. <niv...@gm...>]

Hi experts,
    I am trying at understanding the implications of any network drops and
AOE/vblade server restart.

1) Is it expected to not see any loss in the Ethernet network where the
client/server resides or is the protocol capable of handling drops in the
network?

2) If the server restarts and remains unavailable for say 5 seconds, what
happens to the clients?
     - Does the clients keep retrying and ensure no transaction is lost
once the server comes backup?
     - Would be good to understand if there is an option where client wont
need to do a mount again once the server becomes available and if the
transactions could continue just as if there was a network connectivity
issue for 5 seconds.

thanks,
nivin

Re: [Aoetools-discuss] Maintaining aoetools and vblade in Debian

[Ed C. <ed....@ac...>]

OK, the master branches of vblade and aoetools repositories
on the openaoe project have been updated with the patches
from the debian maintainers.

Thank you very much, Christoph!

-- 
   Ed

Re: [Aoetools-discuss] [SPAM] Maintaining aoetools and vblade in Debian

[Christoph B. <sou...@ma...>]

Ed Cashin wrote...

> What's the best way to get a plain-text version of those patches,
> please?

Ups, the pages I mentioned do not provide a download/raw view link,
I'll ask for an update. Anyway, click the version number in the
"breadcrumb" line " Patches / Patch / aoetools / 36-2" on the top of
the page to get a list of all patches. Or in short:


> > - https://sources.debian.net/patches/aoetools/36-2/02_no_hyphen_in_manpages.patch/

https://sources.debian.net/data/main/a/aoetools/36-2/debian/patches/02_no_hyphen_in_manpages.patch

> > - https://sources.debian.net/patches/aoetools/36-2/fix-typos.patch/

https://sources.debian.net/data/main/a/aoetools/36-2/debian/patches/fix-typos.patch

> > - https://sources.debian.net/patches/vblade/23-1/manpage-escape-dash.patch/

https://sources.debian.net/data/main/v/vblade/23-1/debian/patches/manpage-escape-dash.patch

    Christoph

Re: [Aoetools-discuss] [SPAM] Maintaining aoetools and vblade in Debian

[Ed C. <ed....@ac...>]

What's the best way to get a plain-text version of those patches,
please?

On 12/27/2016 04:21 PM, Christoph Biedl wrote:
> Hello,
>
> in the Debian distribution, both the aoetools and vblade packages were
> without an active maintainer ("orphaned"). So I took the opportunity
> and these two are mine now. Activity on them has been pretty low over
> time, so I except a rather smooth ride. I might however bring bugs
> upstream (i.e. here) when applicable.
>
> For the time being, some nitpicking. Please consider
>
> * for aoetools
>
> - https://sources.debian.net/patches/aoetools/36-2/02_no_hyphen_in_manpages.patch/
>
> This patch existed before, so I'll have to guess as I'm not quite fluent
> in *roff. Appearently "-" is translated into a hyphen while a dash is
> intended, the achieve the latter it has to be escaped.
>
> There's also a typo fix.
>
> - https://sources.debian.net/patches/aoetools/36-2/fix-typos.patch/
>
> More typos fixed.
>
> * for vblade
>
> - https://sources.debian.net/patches/vblade/23-1/manpage-escape-dash.patch/
>
> As above.
>
> Cheers,
>
>     Christoph
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> _______________________________________________
> Aoetools-discuss mailing list
> Aoe...@li...
> https://lists.sourceforge.net/lists/listinfo/aoetools-discuss
>

Re: [Aoetools-discuss] Extracting in real time outgoing/incoming file content

[Ed C. <ed....@ac...>]

The aoetools are user-space programs that are supposed to make it more
convenient to use ATA over Ethernet.  They include aoeping and
aoe-discover.

The aoe driver is a kernel module that makes remote AoE targets
appear as local block devices like /dev/etherd/e0.1.

The terms "client" and "server" are not used much---data is written
by the initiator to the storage target, usually that means it's
written by the aoe driver to the SRX or vblade, via Ethernet.

On the initiator host, if the userland process using the block device
/dev/etherd/e0.1 opens e0.1 using the O_DIRECT flag, then the kernel
doesn't do the usual page caching.  The page cache normally would
store up all the writes in memory as dirty pages to be written out
to the storage target in a batch later.  But O_DIRECT tells the kernel
to write directly to the storage target.

On 12/31/2016 06:30 PM, NavAil wrote:
> I understand. Does that mean that once the incoming data reaches the
> AOETools layer, it is send immediately, with no latency, to the server
> and no cache mechanism is applied at that layer (and nothing can be done
> at the aoetools level)?
>
>
> ---
> PS
> I am sorry about the unintentional duplicates. I've tried to post via
> Yahoo 10 or more hours ago and it didn't work then, that is why I've
> tried later via another email service. But it seems Yahoo is actually
> having some server problems and sends the emails hours later causing
> this mess.
>
> 31. Dec 2016 21:48 by :
>
>     For item two, you're probably best off bypassing the page cache by
>     opening the device with O_DIRECT.
>
>     On Dec 31, 2016 7:22 AM, som...@tu...
>     <mailto:som...@tu...> wrote:
>
>         Hello All,
>
>         I'd appreciate some help on the following  case:
>
>         1. A big file (2-3GB or more) is saved/written at the aoe-client
>         side (to the e0.1 device)
>
>         2. The data written needs to be continuously send over the
>         network to the aoe-server immediately, with no latency
>
>         3. At the server side I need to extract in real time the
>         available incoming raw content fragments (ordered correctly)
>         immediately after they are received and not wait for the whole
>         file to be received, reconstructed and made available via the fs
>
>         What would be the easiest way to do that?
>         I assume some coding would be necessary. What could be done and
>         which are the right driver source files/functions to modify in
>         order to achieve all that?
>
>         Any help would be much appreciated!
>         Thanks
>
>

Re: [Aoetools-discuss] Extracting in real time outgoing/incoming file content

[NavAil <som...@tu...>]

I understand. Does that mean that once the incoming data reaches the AOETools layer, it is send immediately, with no latency, to the server and no cache mechanism is applied at that layer (and nothing can be done at the aoetools level)?


---
PS   
I am sorry about the unintentional duplicates. I've tried to post via Yahoo 10 or more hours ago and it didn't work then, that is why I've tried later via another email service. But it seems Yahoo is actually having some server problems and sends the emails hours later causing this mess. 

31. Dec 2016 21:48 by :


> For item two, you're probably best off bypassing the page cache by opening the device with O_DIRECT. 
> On Dec 31, 2016 7:22 AM, > som...@tu...>  wrote:
>
>>           >> Hello All,
>>
>> I'd appreciate some help on the following  case:
>>
>> 1. A big file (2-3GB or more) is saved/written at the aoe-client side (to the e0.1 device)
>>
>> 2. The data written needs to be continuously send over the network to the aoe-server immediately, with no latency
>>
>> 3. At the server side I need to extract in real time the available incoming raw content fragments (ordered correctly) immediately after they are received and not wait for the whole file to be received, reconstructed and made available via the fs
>>
>> What would be the easiest way to do that?
>> I assume some coding would be necessary. What could be done and which are the right driver source files/functions to modify in order to achieve all that?
>>
>> Any help would be much appreciated!
>> Thanks
>>
>>   
>
>

[Aoetools-discuss] Extracting in real time outgoing/incoming file content

[NavAil <som...@ya...>]

Hello All,

I'd appreciate some help on the following  case:

1. A big file (2-3GB or more) is saved/written at the aoe-client side 
(to the e0.1 device)

2. The data written needs to be continuously send over the network to 
the aoe-server immediately, with no latency

3. At the server side I need to extract in real time the available 
incoming raw content fragments (ordered correctly) immediately after 
they are received and not wait for the whole file to be received, 
reconstructed and made available via the fs

What would be the easiest way to do that?
I assume some coding would be necessary. What could be done and which 
are the right driver source files/functions to modify in order to 
achieve all that?

Any help would be much appreciated!
Thanks

[Aoetools-discuss] Extracting in real time outgoing/incoming file content

[NavAil <som...@ya...>]

Hello All,

I'd appreciate some help on the following  case:

1. A big file (2-3GB or more) is saved/written at the aoe-client side 
(to the e0.1 device)

2. The data written needs to be continuously send over the network to 
the aoe-server /immediately, with no latency/

3. At the server side I need to extract in real timethe available 
incoming raw content fragments (ordered correctly) immediately after 
they are received and not wait for the whole file to be received, 
reconstructed and made available via the fs

What would be the easiest way to do that?
I assume some coding would be necessary. What could be done and which 
are the right driver source files/functions to modify in order to 
achieve all that?

Any help would be much appreciated!
Thanks

[Aoetools-discuss] Maintaining aoetools and vblade in Debian

[Christoph B. <sou...@ma...>]

Hello,

in the Debian distribution, both the aoetools and vblade packages were
without an active maintainer ("orphaned"). So I took the opportunity
and these two are mine now. Activity on them has been pretty low over
time, so I except a rather smooth ride. I might however bring bugs
upstream (i.e. here) when applicable.

For the time being, some nitpicking. Please consider

* for aoetools

- https://sources.debian.net/patches/aoetools/36-2/02_no_hyphen_in_manpages.patch/

This patch existed before, so I'll have to guess as I'm not quite fluent
in *roff. Appearently "-" is translated into a hyphen while a dash is
intended, the achieve the latter it has to be escaped.

There's also a typo fix.

- https://sources.debian.net/patches/aoetools/36-2/fix-typos.patch/

More typos fixed.

* for vblade

- https://sources.debian.net/patches/vblade/23-1/manpage-escape-dash.patch/

As above.

Cheers,

    Christoph

[Aoetools-discuss] Extracting in real time outgoing/incoming file content

[<som...@tu...>]

Hello All,

I'd appreciate some help on the following  case:

1. A big file (2-3GB or more) is saved/written at the aoe-client side (to the e0.1 device)

2. The data written needs to be continuously send over the network to the aoe-server immediately, with no latency

3. At the server side I need to extract in real time the available incoming raw content fragments (ordered correctly) immediately after they are received and not wait for the whole file to be received, reconstructed and made available via the fs

What would be the easiest way to do that?
I assume some coding would be necessary. What could be done and which are the right driver source files/functions to modify in order to achieve all that?

Any help would be much appreciated!
Thanks

Re: [Aoetools-discuss] Looking for AoE specification

[Lars T. <ta...@bb...>]

The official protocol is still available here:
http://brantleycoilecompany.com/AoEr11.pdf

This company seems to be the successor of the former coraid corporation in some way.

Lars

Fri, 26 Aug 2016 12:11:14 +0300
Sergey Matveev <sta...@st...> ==> aoe...@li... :
> Greetings!
> 
> I am looking for AoE specification and still can not find any. All links
> lead to http://support.coraid.com/documents/AoEr11.txt , but this link
> is dead (404). Where can I find that file?
> 


-- 
                            Informationstechnologie
Berlin-Brandenburgische Akademie der Wissenschaften
Jägerstraße 22-23                      10117 Berlin
Tel.: +49 30 20370-352           http://www.bbaw.de

Re: [Aoetools-discuss] Looking for AoE specification

[Sergey M. <sta...@st...>]

*** Marco De Benedetto <de...@ga...> [2016-08-26 13:05]:
>I have found the file here: http://archive.is/9GCnK

Perfect! Thank you very much!

-- 
Happy hacking

Re: [Aoetools-discuss] Looking for AoE specification

[Marco De B. <de...@ga...>]

On Fri 26 Aug, Sergey Matveev wrote:
> Greetings!
> 
> I am looking for AoE specification and still can not find any. All links
> lead to http://support.coraid.com/documents/AoEr11.txt , but this link
> is dead (404). Where can I find that file?

Hi,

I have found the file here: http://archive.is/9GCnK

-- 
Marco

[Aoetools-discuss] Looking for AoE specification

[Sergey M. <sta...@st...>]

Greetings!

I am looking for AoE specification and still can not find any. All links
lead to http://support.coraid.com/documents/AoEr11.txt , but this link
is dead (404). Where can I find that file?

-- 
Happy hacking

Re: [Aoetools-discuss] [Aoetools] aoe driver on Ubuntu 14.04

[Daofeng Li <li...@gm...>]

Oh my god.....my stupid....after modprobe aoe these errors are
gone.....thanks :)

Ok, got new problem, I could not see any disks. I'll dig more.

Thanks again!

Daofeng

On Wed, Dec 16, 2015 at 4:08 PM, Christoph Biedl <
sou...@ma...> wrote:

> Daofeng Li wrote...
>
> > # aoe-discover
> > aoe-discover: /dev/etherd/discover does not exist or is not writeable.
>
> Has the aoe module been loaded, in other words: Do things improve
> after an "sudo modprobe aoe"?
>
>     Christoph
>
>
> ------------------------------------------------------------------------------
> _______________________________________________
> Aoetools-discuss mailing list
> Aoe...@li...
> https://lists.sourceforge.net/lists/listinfo/aoetools-discuss
>

Re: [Aoetools-discuss] [Aoetools] aoe driver on Ubuntu 14.04

[Christoph B. <sou...@ma...>]

Daofeng Li wrote...

> # aoe-discover
> aoe-discover: /dev/etherd/discover does not exist or is not writeable.

Has the aoe module been loaded, in other words: Do things improve
after an "sudo modprobe aoe"?

    Christoph

[Aoetools-discuss] [Aoetools] aoe driver on Ubuntu 14.04

[Daofeng Li <li...@gm...>]

Dear list,

I would like to attach some aoe disks to my system, I assume I don't need
to install aoe drivers as Ubuntu 14.04 already have the driver. I connect
the disks to the server, but aoe-stat gives me nothing, aoe-discover get
error.

# aoe-discover
aoe-discover: /dev/etherd/discover does not exist or is not writeable.

I did some google search seems related to udev rules, but not quite sure
how to fix it.
Could anyone please suggest how to fix it? Thanks a lot.

Daofeng

Re: [Aoetools-discuss] Slow transfer speed of AoE

[Ed C. <ed....@ac...>]

Firstly, I don't know.

Secondly, one thing to always check are caching effects.

So you cannot be sure that you're getting 55 or 35 MBps unless you know 
that you're not just measuring reads and writes from RAM.  The kernel, 
if left alone, will aggressively attempt to make sure that's what happens.

Some good tools are the drop_caches file under /proc/sys/vm (with 
documentation in the kernel source, unfortunately), the boot 
parameter("mem=") that allows you to use less than the installed amount 
of RAM, and workloads for testing that include more data than can fit 
into RAM.

On 10/08/2015 02:59 PM, TuxOholic wrote:
> Hi experts
>
> I use Debian Jessie on two Banana-PI with kernel 3.19 and have 
> installed aoe.ko, vblade and aoetools. Both systems are connected by 
> Gbit LAN via a Gbit switch. The MTU is set at 1500 and can't be 
> changed because the Ethernet driver does not support it for that hardware.
>
> Without AoE, when I read/write dataof 1GB size on the other system via 
> nfs, I get speeds of 55 / 35 MB/s respectively. Writing is probably so 
> much slower due to a bug in the Allwinner A20 silicone or somewhere in 
> software.
>
> However, if I map the drive of one system to the other via AoE and do 
> reading/writing to this mapped raw device (/dev/etherd/e0.1), the 
> speeds get very low, typically 21/7 MB/s respectively. Only when 
> rather small amount of data like 150 MB files are transferred, the 
> speed is higher like 35/25 MB/s.
>
> I wonder why AoE is so slow and hope you can tell me what must be 
> changed to get better transfer rates.
>
> Many thanks!
>
> TuxOholic
>
>
>
> ------------------------------------------------------------------------------
>
>
> _______________________________________________
> Aoetools-discuss mailing list
> Aoe...@li...
> https://lists.sourceforge.net/lists/listinfo/aoetools-discuss

[Aoetools-discuss] Slow transfer speed of AoE

[TuxOholic <tux...@ho...>]

Hi experts

I use Debian Jessie on two Banana-PI with kernel 3.19 and have installed
aoe.ko, vblade and aoetools. Both systems are connected by Gbit LAN via
a Gbit switch. The MTU is set at 1500 and can't be changed because the
Ethernet driver does not support it for that hardware.

Without AoE, when I read/write dataof 1GB size on the other system via
nfs, I get speeds of 55 / 35 MB/s respectively. Writing is probably so
much slower due to a bug in the Allwinner A20 silicone or somewhere in
software.

However, if I map the drive of one system to the other via AoE and do
reading/writing to this mapped raw device (/dev/etherd/e0.1), the speeds
get very low, typically 21/7 MB/s respectively. Only when rather small
amount of data like 150 MB files are transferred, the speed is higher
like 35/25 MB/s.

I wonder why AoE is so slow and hope you can tell me what must be
changed to get better transfer rates.

Many thanks!

TuxOholic