curl-loader-devel

re: how can i use random mac addresses for curl-loader tests? Current test all use a single MAC address and I need to simulate more.

[<Mal...@sw...>]

Hi curl-loader devs,
First of all, thank you for your work in creating a Smartbits Avalanche "replacement" tool. I have found it very helpful and much more affordable than the Smartbits.
I am currently trying to run some load tests with your tool (to replace our aging Avalanche), however I am missing one element to make my tests a success. The gateway that I am testing requires that each client has a distinct MAC address, and I cannot find any way of adding this to curl-loader. Any ideas or pointers that you have will be much appreciated. I have looked at multiple load test tools and curl-loader is the best in the FOSS space for this particular task that I have, however, I need the MAC addresses to be different per request. I would really appreciate if you can confirm whether this is possible, how much coding work would be needed, or If anyone else has had a similar requirement. Thanks in advance.
Malick SY

Re: how can i use random mac addresses for curl-loader tests? Current test all use a single MAC address and I need to simulate more.

[Robert I. <cor...@gm...>]

Dear Malick,
Thank you for your appreciation.

There's no option to generate random Macs in curl-loader.
Sorry.

Regards,
Robert

On Thu, Mar 19, 2015 at 1:20 PM, <Mal...@sw...> wrote:

>  Hi curl-loader devs,
> First of all, thank you for your work in creating a Smartbits Avalanche
> “replacement” tool. I have found it very helpful and much more affordable
> than the Smartbits.
> I am currently trying to run some load tests with your tool (to replace
> our aging Avalanche), however I am missing one element to make my tests a
> success. The gateway that I am testing requires that each client has a
> distinct MAC address, and I cannot find any way of adding this to
> curl-loader. Any ideas or pointers that you have will be much appreciated.
> I have looked at multiple load test tools and curl-loader is the best in
> the FOSS space for this particular task that I have, however, I need the
> MAC addresses to be different per request. I would really appreciate if you
> can confirm whether this is possible, how much coding work would be needed,
> or If anyone else has had a similar requirement. Thanks in advance.
> Malick SY
>
>
> ------------------------------------------------------------------------------
> Dive into the World of Parallel Programming The Go Parallel Website,
> sponsored
> by Intel and developed in partnership with Slashdot Media, is your hub for
> all
> things parallel software development, from weekly thought leadership blogs
> to
> news, videos, case studies, tutorials and more. Take a look and join the
> conversation now. http://goparallel.sourceforge.net/
> _______________________________________________
> curl-loader-devel mailing list
> cur...@li...
> https://lists.sourceforge.net/lists/listinfo/curl-loader-devel
>
>

Re: how can i use random mac addresses for curl-loader tests? Current test all use a single MAC address and I need to simulate more.

[<Mal...@sw...>]

Hi Robert,
Thanks very much for your speedy response. Another question, is it possible to specify more than one interface for curl-loader to run on?
Thanks
Malick

From: Robert Iakobashvili <cor...@gm...<mailto:cor...@gm...>>
Reply-To: curl-loader-devel <cur...@li...<mailto:cur...@li...>>
Date: Thursday, 19 March 2015 12:48
To: curl-loader-devel <cur...@li...<mailto:cur...@li...>>
Subject: Re: how can i use random mac addresses for curl-loader tests? Current test all use a single MAC address and I need to simulate more.

Dear Malick,
Thank you for your appreciation.

There's no option to generate random Macs in curl-loader.
Sorry.

Regards,
Robert

On Thu, Mar 19, 2015 at 1:20 PM, <Mal...@sw...<mailto:Mal...@sw...>> wrote:
Hi curl-loader devs,
First of all, thank you for your work in creating a Smartbits Avalanche "replacement" tool. I have found it very helpful and much more affordable than the Smartbits.
I am currently trying to run some load tests with your tool (to replace our aging Avalanche), however I am missing one element to make my tests a success. The gateway that I am testing requires that each client has a distinct MAC address, and I cannot find any way of adding this to curl-loader. Any ideas or pointers that you have will be much appreciated. I have looked at multiple load test tools and curl-loader is the best in the FOSS space for this particular task that I have, however, I need the MAC addresses to be different per request. I would really appreciate if you can confirm whether this is possible, how much coding work would be needed, or If anyone else has had a similar requirement. Thanks in advance.
Malick SY

------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the
conversation now. http://goparallel.sourceforge.net/
_______________________________________________
curl-loader-devel mailing list
cur...@li...<mailto:cur...@li...>
https://lists.sourceforge.net/lists/listinfo/curl-loader-devel

Re: how can i use random mac addresses for curl-loader tests? Current test all use a single MAC address and I need to simulate more.

[Vincent Li <vin...@gm...>]

Hi Malick,

I am glad for you to abandon Avalanche :)

the your request is unusual. I think the closest thing to achieve what
you need is DPDK http://www.dpdk.org/   and  MoonGen
https://github.com/emmericp/MoonGen . it appears you are doing
http/https type of test. you could randomize MAC address and ip
address in MoonGen, then "hardcode " http payload in tcp data section.

i worked on a project to   use DPDK and MoonGen to generate ~7 million
packet per second DNS  request to test most powerful F5 hardware
platform, this is enterprise level of load testing using open source
technology

don't get me wrong, I used curl-loader and love it :) but for million
connections, packet per second level of test, the bottleneck is in the
Linux kernel tcp/ip stack that curl-loader runs on.

see

The Secret to 10 Million Concurrent Connections -The Kernel is the
Problem, Not the Solution

 http://highscalability.com/blog/2013/5/13/the-secret-to-10-million-concurrent-connections-the-kernel-i.html


I bought a used $300 dell poweredge R210 and i can achieve 2.5 million
http concurrent connections.

http://ickernel.blogspot.com/2015/10/25-million-tcphttp-connection-with-mtcp.html


Vincent


On Thu, Mar 19, 2015 at 4:20 AM,  <Mal...@sw...> wrote:
> Hi curl-loader devs,
> First of all, thank you for your work in creating a Smartbits Avalanche
> “replacement” tool. I have found it very helpful and much more affordable
> than the Smartbits.
> I am currently trying to run some load tests with your tool (to replace our
> aging Avalanche), however I am missing one element to make my tests a
> success. The gateway that I am testing requires that each client has a
> distinct MAC address, and I cannot find any way of adding this to
> curl-loader. Any ideas or pointers that you have will be much appreciated. I
> have looked at multiple load test tools and curl-loader is the best in the
> FOSS space for this particular task that I have, however, I need the MAC
> addresses to be different per request. I would really appreciate if you can
> confirm whether this is possible, how much coding work would be needed, or
> If anyone else has had a similar requirement. Thanks in advance.
> Malick SY
>
> ------------------------------------------------------------------------------
> Dive into the World of Parallel Programming The Go Parallel Website,
> sponsored
> by Intel and developed in partnership with Slashdot Media, is your hub for
> all
> things parallel software development, from weekly thought leadership blogs
> to
> news, videos, case studies, tutorials and more. Take a look and join the
> conversation now. http://goparallel.sourceforge.net/
> _______________________________________________
> curl-loader-devel mailing list
> cur...@li...
> https://lists.sourceforge.net/lists/listinfo/curl-loader-devel
>

Re: how can i use random mac addresses for curl-loader tests? Current test all use a single MAC address and I need to simulate more.

[Vincent Li <vin...@gm...>]

it turned out pretty easy to use mTCP + DPDK to achieve to
randomization of source MAC, then you can send http request with
random source MAC.

patch mTCP/src/eth_out.c as below:

diff --git a/mtcp/src/eth_out.c b/mtcp/src/eth_out.c
index 7fd1097..b064e9c 100644
--- a/mtcp/src/eth_out.c
+++ b/mtcp/src/eth_out.c
@@ -181,6 +181,44 @@ FlushSendChunkBuf(mtcp_manager_t mtcp, int nif)
 }
 #endif /* E_PSIO */
 /*----------------------------------------------------------------------------*/
+
+/**
+ * Get a pseudo-random value.
+ *
+ * This function generates pseudo-random numbers using the linear
+ * congruential algorithm and 48-bit integer arithmetic, called twice
+ * to generate a 64-bit value.
+ *
+ * @return
+ *   A pseudo-random value between 0 and (1<<64)-1.
+ */
+static inline uint64_t
+rte_rand(void)
+{
+        uint64_t val;
+        val = lrand48();
+        val <<= 32;
+        val += lrand48();
+        return val;
+}
+
+
+static void
+generate_random_mac_addr(struct ethhdr *mac_addr)
+{
+        uint64_t random;
+
+        /* Set Organizationally Unique Identifier (OUI) prefix. */
+        mac_addr->h_source[0] = 0xa0;
+        mac_addr->h_source[1] = 0x36;
+        mac_addr->h_source[2] = 0x9f;
+        /* Force indication of locally assigned MAC address. */
+//        mac_addr->h_source[0] |= ETHER_LOCAL_ADMIN_ADDR;
+        /* Generate the last 3 bytes of the MAC address with a random
number. */
+        random = rte_rand();
+        memcpy(&mac_addr->h_source[3], &random, 3);
+}
+
 uint8_t *
 EthernetOutput(struct mtcp_manager *mtcp, uint16_t h_proto,
                int nif, unsigned char* dst_haddr, uint16_t iplen)
@@ -213,8 +251,9 @@ EthernetOutput(struct mtcp_manager *mtcp, uint16_t h_proto,
 #endif

        ethh = (struct ethhdr *)buf;
+       generate_random_mac_addr(ethh);
        for (i = 0; i < ETH_ALEN; i++) {
-               ethh->h_source[i] = CONFIG.eths[nif].haddr[i];
+               //ethh->h_source[i] = CONFIG.eths[nif].haddr[i];
                ethh->h_dest[i] = dst_haddr[i];
        }
        ethh->h_proto = htons(h_proto);

On Fri, Nov 13, 2015 at 4:09 PM, Vincent Li <vin...@gm...> wrote:
> Hi Malick,
>
> I am glad for you to abandon Avalanche :)
>
> the your request is unusual. I think the closest thing to achieve what
> you need is DPDK http://www.dpdk.org/   and  MoonGen
> https://github.com/emmericp/MoonGen . it appears you are doing
> http/https type of test. you could randomize MAC address and ip
> address in MoonGen, then "hardcode " http payload in tcp data section.
>
> i worked on a project to   use DPDK and MoonGen to generate ~7 million
> packet per second DNS  request to test most powerful F5 hardware
> platform, this is enterprise level of load testing using open source
> technology
>
> don't get me wrong, I used curl-loader and love it :) but for million
> connections, packet per second level of test, the bottleneck is in the
> Linux kernel tcp/ip stack that curl-loader runs on.
>
> see
>
> The Secret to 10 Million Concurrent Connections -The Kernel is the
> Problem, Not the Solution
>
>  http://highscalability.com/blog/2013/5/13/the-secret-to-10-million-concurrent-connections-the-kernel-i.html
>
>
> I bought a used $300 dell poweredge R210 and i can achieve 2.5 million
> http concurrent connections.
>
> http://ickernel.blogspot.com/2015/10/25-million-tcphttp-connection-with-mtcp.html
>
>
> Vincent
>
>
> On Thu, Mar 19, 2015 at 4:20 AM,  <Mal...@sw...> wrote:
>> Hi curl-loader devs,
>> First of all, thank you for your work in creating a Smartbits Avalanche
>> “replacement” tool. I have found it very helpful and much more affordable
>> than the Smartbits.
>> I am currently trying to run some load tests with your tool (to replace our
>> aging Avalanche), however I am missing one element to make my tests a
>> success. The gateway that I am testing requires that each client has a
>> distinct MAC address, and I cannot find any way of adding this to
>> curl-loader. Any ideas or pointers that you have will be much appreciated. I
>> have looked at multiple load test tools and curl-loader is the best in the
>> FOSS space for this particular task that I have, however, I need the MAC
>> addresses to be different per request. I would really appreciate if you can
>> confirm whether this is possible, how much coding work would be needed, or
>> If anyone else has had a similar requirement. Thanks in advance.
>> Malick SY
>>
>> ------------------------------------------------------------------------------
>> Dive into the World of Parallel Programming The Go Parallel Website,
>> sponsored
>> by Intel and developed in partnership with Slashdot Media, is your hub for
>> all
>> things parallel software development, from weekly thought leadership blogs
>> to
>> news, videos, case studies, tutorials and more. Take a look and join the
>> conversation now. http://goparallel.sourceforge.net/
>> _______________________________________________
>> curl-loader-devel mailing list
>> cur...@li...
>> https://lists.sourceforge.net/lists/listinfo/curl-loader-devel
>>