Re: curl-loader https: low performance due to DNS query?

[Robert I. <cor...@gm...>]

I see.

On Wed, Mar 7, 2012 at 3:49 PM, Fred Huang <di...@gm...> wrote:
> I tried -t 16 or -t 32, the performance drops to approx. 50 CAPS...
>
>
> 2012/3/7 Robert Iakobashvili <cor...@gm...>
>>
>> Fred,
>>
>> On more suggestion, which is a general one.
>>
>> If N is the number of your logical CPUs,
>> usually, I'd recommend to run 2*N + B threads.
>>
>> Thus, you can try:
>>
>> -t 16
>>
>> -t 32
>>
>> and it MAY increased a bit CAPS.
>>
>> Thanks,
>> Robert
>>
>> On Wed, Mar 7, 2012 at 2:43 AM, Fred Huang <di...@gm...> wrote:
>> > yes, I use -t 8 option because the test box has 8 logic CPUs (Dell 990,
>> > Intel Core i7 2600, 16GB memory).
>> > if I do not use -t option, the CPU used by curl-loader will be 100% on
>> > one
>> > of 8 logic CPU (12% of total CPU), and the CAPS drops to 100:
>> > # gprof /usr/bin/curl-loader gmon.out -p | head -20
>> > Flat profile:
>> > Each sample counts as 0.01 seconds.
>> >   %   cumulative   self              self     total
>> >  time   seconds   seconds    calls   s/call   s/call  name
>> >  34.12      2.74     2.74    17385     0.00     0.00  create_conn
>> >  11.21      3.64     0.90 30998583     0.00     0.00
>> >  hostcache_timestamp_remove
>> >   9.84      4.43     0.79    13433     0.00     0.00
>> >  Curl_hash_clean_with_criterium
>> >   7.97      5.07     0.64   381818     0.00     0.00  Curl_hash_pick
>> >   5.85      5.54     0.47    17385     0.00     0.00  ConnectionStore
>> >   5.11      5.95     0.41    16385     0.00     0.00
>> >  curl_multi_remove_handle
>> >   4.11      6.28     0.33 16509889     0.00     0.00
>> >  Curl_str_key_compare
>> >   3.36      6.55     0.27 23959914     0.00     0.00  Curl_socket_check
>> >   3.24      6.81     0.26    56703     0.00     0.00  Curl_hash_add
>> >   2.62      7.02     0.21   635750     0.00     0.00  dprintf_formatf
>> >   1.25      7.12     0.10 92421190     0.00     0.00
>> >  ares__is_list_empty
>> >   0.75      7.18     0.06    97957     0.00     0.00  Curl_readwrite
>> >   0.62      7.23     0.05 23422260     0.00     0.00  addbyter
>> >   0.62      7.28     0.05  1136775     0.00     0.00
>> >  client_tracing_function
>> >   0.50      7.32     0.04  3091375     0.00     0.00
>> >  ares__init_list_head
>> >
>> > I also tried to use
>> >  curl_easy_setopt (handle, CURLOPT_DNS_CACHE_TIMEOUT, 20);
>> > but get no improvement on CAPS.
>> >
>> >
>> > Regards,
>> > Fred
>> >
>> > 2012/3/7 Robert Iakobashvili <cor...@gm...>
>> >>
>> >> Dear Fred,
>> >>
>> >> If I understand correctly your observations, the issue is not in:
>> >>
>> >> 1. DNS resolving;
>> >> 2. Version of cares or libcurl;
>> >>
>> >> but rather in
>> >>
>> >> Less effective handling of HTTPS versus HTTP in libcurl.
>> >>
>> >> Please, confirm that my understanding is correct.
>> >>
>> >>
>> >> HTTPS URLs are using an extra locking in openssl with -t option
>> >> that could be making the issue.
>> >> If you are using -t option, try to load without it and let's say with
>> >> 300  clients.
>> >>
>> >> If you are not using -t, thus the only way to proceed is to share
>> >> your observations with libcurl people at
>> >>
>> >> libcurl development <cur...@co...>
>> >>
>> >> Thank you.
>> >>
>> >> Best wishes.
>> >> Robert
>> >>
>> >> On Tue, Mar 6, 2012 at 6:17 PM, Fred Huang <di...@gm...> wrote:
>> >> > 1. the major CPU consumer is curl-loader. here is the top:
>> >> > top - 07:36:36 up 23:31,  5 users,  load average: 4.32, 2.26, 0.91
>> >> > Tasks: 237 total,   2 running, 235 sleeping,   0 stopped,   0 zombie
>> >> > Cpu0  : 76.6%us,  3.6%sy,  0.0%ni, 19.4%id,  0.0%wa,  0.0%hi,
>> >> >  0.3%si,
>> >> >  0.0%
>> >> > Cpu1  : 72.4%us,  4.0%sy,  0.0%ni, 22.9%id,  0.0%wa,  0.0%hi,
>> >> >  0.7%si,
>> >> >  0.0%
>> >> > Cpu2  : 65.8%us,  3.3%sy,  0.0%ni, 30.6%id,  0.0%wa,  0.0%hi,
>> >> >  0.3%si,
>> >> >  0.0%
>> >> > Cpu3  : 75.2%us,  4.3%sy,  0.0%ni, 19.5%id,  0.0%wa,  0.0%hi,
>> >> >  1.0%si,
>> >> >  0.0%
>> >> > Cpu4  : 79.1%us,  4.0%sy,  0.0%ni, 16.8%id,  0.0%wa,  0.0%hi,
>> >> >  0.0%si,
>> >> >  0.0%
>> >> > Cpu5  : 75.8%us,  4.3%sy,  0.0%ni, 19.2%id,  0.0%wa,  0.0%hi,
>> >> >  0.7%si,
>> >> >  0.0%
>> >> > Cpu6  : 74.1%us,  5.0%sy,  0.0%ni, 20.3%id,  0.0%wa,  0.0%hi,
>> >> >  0.7%si,
>> >> >  0.0%
>> >> > Cpu7  : 78.7%us,  4.0%sy,  0.0%ni, 16.6%id,  0.0%wa,  0.0%hi,
>> >> >  0.7%si,
>> >> >  0.0%
>> >> > Mem:  16209932k total,  3829500k used, 12380432k free,    99112k
>> >> > buffers
>> >> > Swap: 18448376k total,        0k used, 18448376k free,   320284k
>> >> > cached
>> >> >
>> >> >   PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
>> >> > 30795 root      20   0 3200m 2.8g 2296 S 79.2 18.4  21:26.84
>> >> > curl-loader
>> >> >  1832 nobody    20   0 13944 1736  580 S  0.1  0.0   0:32.65 dnsmasq
>> >> >     9 root      20   0     0    0    0 S  0.0  0.0   0:00.53
>> >> > ksoftirqd/1
>> >> > 30825 root      20   0 15220 1360  960 R  0.0  0.0   0:00.42 top
>> >> >     1 root      20   0 19400 1568 1256 S  0.0  0.0   0:00.75 init
>> >> >     2 root      20   0     0    0    0 S  0.0  0.0   0:00.00 kthreadd
>> >> >     3 root      RT   0     0    0    0 S  0.0  0.0   0:00.00
>> >> > migration/0
>> >> >     4 root      20   0     0    0    0 S  0.0  0.0   0:00.06
>> >> > ksoftirqd/0
>> >> >     5 root      RT   0     0    0    0 S  0.0  0.0   0:00.00
>> >> > migration/0
>> >> >
>> >> >
>> >> > 2. 1000 loading clients
>> >> >
>> >> > 3. yes, tried 0.52 but no lucky -- CAPS 380, throughput 200Mbps, CPU
>> >> > 90%
>> >> > top - 08:17:10 up 1 day, 12 min,  8 users,  load average: 4.66, 3.39,
>> >> > 1.88
>> >> > Tasks: 245 total,   2 running, 243 sleeping,   0 stopped,   0 zombie
>> >> > Cpu0  : 82.2%us,  4.0%sy,  0.0%ni, 13.9%id,  0.0%wa,  0.0%hi,
>> >> >  0.0%si,
>> >> >  0.0%
>> >> > Cpu1  : 86.1%us,  4.3%sy,  0.0%ni,  7.9%id,  0.0%wa,  0.0%hi,
>> >> >  1.7%si,
>> >> >  0.0%
>> >> > Cpu2  : 85.3%us,  4.0%sy,  0.0%ni, 10.0%id,  0.0%wa,  0.0%hi,
>> >> >  0.7%si,
>> >> >  0.0%
>> >> > Cpu3  : 81.8%us,  3.0%sy,  0.0%ni, 14.2%id,  0.0%wa,  0.0%hi,
>> >> >  1.0%si,
>> >> >  0.0%
>> >> > Cpu4  : 83.7%us,  4.0%sy,  0.0%ni, 12.3%id,  0.0%wa,  0.0%hi,
>> >> >  0.0%si,
>> >> >  0.0%
>> >> > Cpu5  : 92.7%us,  3.3%sy,  0.0%ni,  3.3%id,  0.0%wa,  0.0%hi,
>> >> >  0.7%si,
>> >> >  0.0%
>> >> > Cpu6  : 89.7%us,  2.0%sy,  0.0%ni,  8.0%id,  0.0%wa,  0.0%hi,
>> >> >  0.3%si,
>> >> >  0.0%
>> >> > Cpu7  : 88.1%us,  3.6%sy,  0.0%ni,  6.6%id,  0.0%wa,  0.0%hi,
>> >> >  1.7%si,
>> >> >  0.0%
>> >> > Mem:  16209932k total,  1795748k used, 14414184k free,   102824k
>> >> > buffers
>> >> > Swap: 18448376k total,        0k used, 18448376k free,   382988k
>> >> > cached
>> >> >
>> >> >   PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
>> >> > 12236 root      20   0 1150m 894m 2572 S 90.7  5.7  51:11.91
>> >> > curl-loader
>> >> >  1832 nobody    20   0 13944 1736  580 S  0.1  0.0   0:47.04 dnsmasq
>> >> >
>> >> > ]# gprof /usr/bin/curl-loader gmon.out -p | head -20
>> >> > Flat profile:
>> >> > Each sample counts as 0.01 seconds.
>> >> >   %   cumulative   self              self     total
>> >> >  time   seconds   seconds    calls   s/call   s/call  name
>> >> >  43.92    151.95   151.95   157295     0.00     0.00
>> >> >  Curl_hash_clean_with_criterium
>> >> >  22.90    231.17    79.22 1712190971     0.00     0.00
>> >> >  hostcache_timestamp_remove
>> >> >  13.15    276.68    45.51  3260321     0.00     0.00  Curl_hash_pick
>> >> >   7.51    302.65    25.97   394171     0.00     0.00  Curl_hash_add
>> >> >   5.98    323.34    20.68 567437518     0.00     0.00
>> >> >  Curl_str_key_compare
>> >> >   1.00    326.80     3.46  5380220     0.00     0.00  dprintf_formatf
>> >> >   0.41    328.21     1.41
>> >> > locking_function
>> >> >   0.32    329.32     1.11 469222138     0.00     0.00
>> >> >  ares__is_list_empty
>> >> >   0.31    330.40     1.08  3284508     0.00     0.00  multi_runsingle
>> >> >   0.28    331.38     0.98 10006806     0.00     0.00
>> >> >  client_tracing_function
>> >> >   0.24    332.21     0.83   151951     0.00     0.00
>> >> >  Curl_num_addresses
>> >> >   0.22    332.97     0.77 188075672     0.00     0.00  addbyter
>> >> >   0.19    333.64     0.67   841840     0.00     0.00  Curl_readwrite
>> >> >   0.19    334.29     0.65                             id_function
>> >> >   0.18    334.90     0.61    39843     0.00     0.01
>> >> >  curl_multi_socket_all
>> >> >
>> >> >
>> >> >
>> >> > 4. yes, it is correct for 0.56. If use same URL list by replacing
>> >> > https
>> >> > to
>> >> > http, I can get CAPS 1500 and throughput 870Mbps with CPU% < 60%
>> >> >
>> >> >
>> >> > 2012/3/6 Robert Iakobashvili <cor...@gm...>
>> >> >>
>> >> >> Several questions:
>> >> >>
>> >> >> 1. Could you please run top and see if there are other processes
>> >> >> burning
>> >> >> CPU?
>> >> >> 2. What is the number of loading clients that you use?
>> >> >> 3. Could you try curl-loader-0.52 if it compiles for you?
>> >> >> If not, please try to place curl-loader-7.19.7.tar.gz and
>> >> >> c-ares-16.0.tar.gz,
>> >> >> update their versions in Makefile and rebuild by make cleanall; make
>> >> >>
>> >> >> 4. Is it correct to say that for HTTP URLs (not HTTPS) you are
>> >> >> getting
>> >> >> a good throughput
>> >> >>    and CAPS with curl-loader-0.56?
>> >> >>
>> >> >> Thanks,
>> >> >> Robert
>> >> >>
>> >> >> On Tue, Mar 6, 2012 at 3:52 AM, Fred Huang <di...@gm...> wrote:
>> >> >> > TIMEOUT -1 -> 60 works a little better but no performance
>> >> >> > improvement
>> >> >> > on
>> >> >> > CAPS or throughput.
>> >> >> >
>> >> >> > test 5:
>> >> >> > enable c-ares;
>> >> >> > curl_easy_setopt (handle, CURLOPT_DNS_CACHE_TIMEOUT, 60);
>> >> >> >
>> >> >> > dns server: dnsmasq@127.0.0.1, 2,000,000 dns entry cache, resolve
>> >> >> > *.com
>> >> >> > to
>> >> >> > one IP address
>> >> >> > number of domain names in URLs: 780,000
>> >> >> > number of client: 1000
>> >> >> > CPU usage: 74%
>> >> >> >     cpu%  irq% sirq%  sys% iowt%  mem_used  buf&cached
>> >> >> >     74.0   0.0   0.7   4.2   0.0  3530.6Mb     250.8Mb
>> >> >> > SSL TPS: 360
>> >> >> > SSL throughput: 220Mbps
>> >> >> >
>> >> >> > # gprof /usr/bin/curl-loader gmon.out -p | head -50
>> >> >> > Flat profile:
>> >> >> > Each sample counts as 0.01 seconds.
>> >> >> >   %   cumulative   self              self     total
>> >> >> >  time   seconds   seconds    calls   s/call   s/call  name
>> >> >> >  26.63     23.13    23.13   119226     0.00     0.00
>> >> >> >  Curl_hash_clean_with_criterium
>> >> >> >  20.34     40.80    17.67 317792146     0.00     0.00
>> >> >> >  hostcache_timestamp_remove
>> >> >> >  11.27     50.59     9.79  2579903     0.00     0.00
>> >> >> >  Curl_hash_pick
>> >> >> >   7.84     57.40     6.81   120606     0.00     0.00  create_conn
>> >> >> >   5.65     62.31     4.91 119117401     0.00     0.00
>> >> >> >  Curl_str_key_compare
>> >> >> >   4.98     66.64     4.33   329829     0.00     0.00
>> >> >> >  Curl_hash_add
>> >> >> >   2.80     69.08     2.44  4476606     0.00     0.00
>> >> >> >  dprintf_formatf
>> >> >> >   2.57     71.31     2.23   124019     0.00     0.00
>> >> >> >  ConnectionStore
>> >> >> >   1.50     72.61     1.30 740997609     0.00     0.00
>> >> >> >  ares__is_list_empty
>> >> >> >   1.16     73.61     1.01
>> >> >> > locking_function
>> >> >> >   0.76     74.27     0.66   238744     0.00     0.00  ares_cancel
>> >> >> >   0.70     74.88     0.61   457564     0.00     0.00
>> >> >> >  curl_multi_socket_action
>> >> >> >   0.67     75.46     0.58   118616     0.00     0.00
>> >> >> >  curl_multi_remove_handle
>> >> >> >   0.58     75.97     0.51  8211502     0.00     0.00
>> >> >> >  client_tracing_function
>> >> >> >   0.53     76.43     0.46   733561     0.00     0.00
>> >> >> >  Curl_readwrite
>> >> >> >   0.51     76.87     0.44  2183179     0.00     0.00
>> >> >> >  multi_runsingle
>> >> >> >   0.51     77.31     0.44 166585567     0.00     0.00  addbyter
>> >> >> >   0.44     77.69     0.38  2175805     0.00     0.00
>> >> >> >  Curl_raw_equal
>> >> >> >   0.43     78.06     0.38                             id_function
>> >> >> >   0.40     78.41     0.35 29789346     0.00     0.00
>> >> >> >  Curl_socket_check
>> >> >> >   0.40     78.76     0.35   479671     0.00     0.00
>> >> >> >  ossl_connect_common
>> >> >> >   0.40     79.11     0.35   490639     0.00     0.00
>> >> >> >  ares__init_list_node
>> >> >> >   0.33     79.40     0.29 94197670     0.00     0.00
>> >> >> >  Curl_raw_toupper
>> >> >> >   0.33     79.69     0.29  3268031     0.00     0.00
>> >> >> >  Curl_pgrsUpdate
>> >> >> >   0.28     79.93     0.24     8797     0.00     0.00
>> >> >> >  curl_multi_perform
>> >> >> >   0.27     80.17     0.24  2233641     0.00     0.00  Curl_infof
>> >> >> >   0.26     80.40     0.23  1839837     0.00     0.00  Curl_splay
>> >> >> >   0.24     80.61     0.21 121713028     0.00     0.00
>> >> >> >  curl_strequal
>> >> >> >   0.21     80.79     0.18   243333     0.00     0.00
>> >> >> >  Curl_ssl_getsessionid
>> >> >> >   0.18     80.95     0.16   122180     0.00     0.00
>> >> >> >  Curl_num_addresses
>> >> >> >   0.17     81.09     0.15  8271666     0.00     0.00  Curl_debug
>> >> >> >   0.16     81.23     0.14 18902452     0.00     0.00
>> >> >> >  alloc_addbyter
>> >> >> >   0.16     81.37     0.14     8851     0.00     0.00
>> >> >> >  curl_multi_socket_all
>> >> >> >   0.14     81.50     0.13  2236769     0.00     0.00  Curl_read
>> >> >> >   0.14     81.62     0.12  2271700     0.00     0.00
>> >> >> >  fd_key_compare
>> >> >> >   0.14     81.74     0.12   323167     0.00     0.00  event_del
>> >> >> >   0.14     81.86     0.12   205436     0.00     0.00
>> >> >> >  read_udp_packets
>> >> >> >   0.14     81.98     0.12   200979     0.00     0.00  processfds
>> >> >> >   0.13     82.09     0.11   123886     0.00     0.00
>> >> >> >  Curl_if_is_interface_name
>> >> >> >   0.13     82.20     0.11   120350     0.00     0.00  Curl_http
>> >> >> >   0.13     82.31     0.11
>> >> >> > ssl_tls_trace
>> >> >> >   0.12     82.41     0.10   119229     0.00     0.00
>> >> >> >  Curl_splayremovebyaddr
>> >> >> >   0.12     82.51     0.10  7798717     0.00     0.00
>> >> >> >  Curl_raw_nequal
>> >> >> >   0.12     82.61     0.10   784485     0.00     0.00  event_add
>> >> >> >   0.10     82.70     0.09   643482     0.00     0.00  Curl_expire
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > test 6:
>> >> >> > enable c-ares;
>> >> >> > curl_easy_setopt (handle, CURLOPT_DNS_CACHE_TIMEOUT, 60);
>> >> >> > CURL_VER:=7.24.0-20120109 ->  CURL_VER:=7.23.0
>> >> >> >
>> >> >> > dns server: dnsmasq@127.0.0.1, 2,000,000 dns entry cache, resolve
>> >> >> > *.com
>> >> >> > to
>> >> >> > one IP address
>> >> >> > number of domain names in URLs: 780,000
>> >> >> > number of client: 1000
>> >> >> > CPU usage: 86%
>> >> >> >     cpu%  irq% sirq%  sys% iowt%  mem_used  buf&cached
>> >> >> >      85.9   0.0   0.2   2.1   0.0  3537.2Mb     261.4Mb
>> >> >> > SSL TPS: 360
>> >> >> > SSL throughput: 200Mbps
>> >> >> >
>> >> >> > # gprof /usr/bin/curl-loader gmon.out -p | head -20
>> >> >> > Flat profile:
>> >> >> > Each sample counts as 0.01 seconds.
>> >> >> >   %   cumulative   self              self     total
>> >> >> >  time   seconds   seconds    calls   s/call   s/call  name
>> >> >> >  27.19     19.96    19.96   103472     0.00     0.00
>> >> >> >  Curl_hash_clean_with_criterium
>> >> >> >  21.73     35.91    15.95 274168981     0.00     0.00
>> >> >> >  hostcache_timestamp_remove
>> >> >> >  10.33     43.49     7.58  2256441     0.00     0.00
>> >> >> >  Curl_hash_pick
>> >> >> >   7.21     48.78     5.29   104540     0.00     0.00  create_conn
>> >> >> >   6.20     53.33     4.55 103413405     0.00     0.00
>> >> >> >  Curl_str_key_compare
>> >> >> >   4.88     56.91     3.58   287109     0.00     0.00
>> >> >> >  Curl_hash_add
>> >> >> >   3.02     59.13     2.22  3797855     0.00     0.00
>> >> >> >  dprintf_formatf
>> >> >> >   2.10     60.67     1.54   107711     0.00     0.00
>> >> >> >  ConnectionStore
>> >> >> >   1.65     61.88     1.21 642725327     0.00     0.00
>> >> >> >  ares__is_list_empty
>> >> >> >   1.19     62.76     0.88
>> >> >> > locking_function
>> >> >> >   0.87     63.40     0.64   207817     0.00     0.00  ares_cancel
>> >> >> >   0.56     63.81     0.41   404108     0.00     0.00
>> >> >> >  curl_multi_socket_action
>> >> >> >   0.54     64.21     0.40   425603     0.00     0.00
>> >> >> >  ares__init_list_node
>> >> >> >   0.53     64.60     0.39  7068112     0.00     0.00
>> >> >> >  client_tracing_function
>> >> >> >   0.49     64.96     0.36  1908587     0.00     0.00
>> >> >> >  multi_runsingle
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > test 7:
>> >> >> > enable c-ares;
>> >> >> > curl_easy_setopt (handle, CURLOPT_DNS_CACHE_TIMEOUT, -1);
>> >> >> > CURL_VER:=7.24.0-20120109 ->  CURL_VER:=7.23.0
>> >> >> >
>> >> >> > dns server: dnsmasq@127.0.0.1, 2,000,000 dns entry cache, resolve
>> >> >> > *.com
>> >> >> > to
>> >> >> > one IP address
>> >> >> > number of domain names in URLs: 780,000
>> >> >> > number of client: 1000
>> >> >> > CPU usage: 86%
>> >> >> >     cpu%  irq% sirq%  sys% iowt%  mem_used  buf&cached
>> >> >> >     86.6   0.0   0.6   3.3   0.0  3624.8Mb     263.0Mb
>> >> >> > SSL TPS: 300
>> >> >> > SSL throughput: 200Mbps
>> >> >> >
>> >> >> > # gprof /usr/bin/curl-loader gmon.out -p | head -20
>> >> >> > Flat profile:
>> >> >> > Each sample counts as 0.01 seconds.
>> >> >> >   %   cumulative   self              self     total
>> >> >> >  time   seconds   seconds    calls   s/call   s/call  name
>> >> >> >  39.83    369.71   369.71   207173     0.00     0.00
>> >> >> >  Curl_hash_clean_with_criterium
>> >> >> >  24.27    594.98   225.27 3198337733     0.00     0.00
>> >> >> >  hostcache_timestamp_remove
>> >> >> >  13.75    722.59   127.61  4312864     0.00     0.00
>> >> >> >  Curl_hash_pick
>> >> >> >   9.19    807.87    85.28   546507     0.00     0.00
>> >> >> >  Curl_hash_add
>> >> >> >   6.14    864.88    57.01 1022419684     0.00     0.00
>> >> >> >  Curl_str_key_compare
>> >> >> >   2.06    884.00    19.12   203749     0.00     0.00  create_conn
>> >> >> >   0.54    888.99     4.99   213306     0.00     0.00
>> >> >> >  ConnectionStore
>> >> >> >   0.49    893.50     4.51  7331251     0.00     0.00
>> >> >> >  dprintf_formatf
>> >> >> >   0.29    896.16     2.66 1287182329     0.00     0.00
>> >> >> >  ares__is_list_empty
>> >> >> >   0.23    898.28     2.12
>> >> >> > locking_function
>> >> >> >   0.20    900.10     1.82   212210     0.00     0.00
>> >> >> >  Curl_num_addresses
>> >> >> >   0.17    901.66     1.56   198385     0.00     0.00
>> >> >> >  curl_multi_remove_handle
>> >> >> >   0.12    902.81     1.15   411114     0.00     0.00  ares_cancel
>> >> >> >   0.12    903.95     1.14  3683448     0.00     0.00
>> >> >> >  multi_runsingle
>> >> >> >   0.11    904.97     1.03 13664120     0.00     0.00
>> >> >> >  client_tracing_function
>> >> >> >
>> >> >> >
>> >> >> > test 8:
>> >> >> > enable c-ares;
>> >> >> > curl_easy_setopt (handle, CURLOPT_DNS_CACHE_TIMEOUT, 60);
>> >> >> > CURL_VER:=7.24.0-20120109 ->  CURL_VER:=7.24.0
>> >> >> >
>> >> >> > dns server: dnsmasq@127.0.0.1, 2,000,000 dns entry cache, resolve
>> >> >> > *.com
>> >> >> > to
>> >> >> > one IP address
>> >> >> > number of domain names in URLs: 780,000
>> >> >> > number of client: 1000
>> >> >> > CPU usage: 75%
>> >> >> >     cpu%  irq% sirq%  sys% iowt%  mem_used  buf&cached
>> >> >> >      75.2   0.0   0.2   2.1   0.0  3537.2Mb     261.4Mb
>> >> >> > SSL TPS: 380
>> >> >> > SSL throughput: 220Mbps
>> >> >> >
>> >> >> > # gprof /usr/bin/curl-loader gmon.out -p | head -20
>> >> >> > Flat profile:
>> >> >> > Each sample counts as 0.01 seconds.
>> >> >> >   %   cumulative   self              self     total
>> >> >> >  time   seconds   seconds    calls   s/call   s/call  name
>> >> >> >  27.58     12.31    12.31    71140     0.00     0.00
>> >> >> >  Curl_hash_clean_with_criterium
>> >> >> >  19.09     20.83     8.52 176729692     0.00     0.00
>> >> >> >  hostcache_timestamp_remove
>> >> >> >  10.44     25.49     4.66  1589309     0.00     0.00
>> >> >> >  Curl_hash_pick
>> >> >> >   6.85     28.55     3.06    73204     0.00     0.00  create_conn
>> >> >> >   5.56     31.03     2.48 67620530     0.00     0.00
>> >> >> >  Curl_str_key_compare
>> >> >> >   4.41     33.00     1.97   200684     0.00     0.00
>> >> >> >  Curl_hash_add
>> >> >> >   3.56     34.59     1.59  2717956     0.00     0.00
>> >> >> >  dprintf_formatf
>> >> >> >   1.72     35.36     0.77    75247     0.00     0.00
>> >> >> >  ConnectionStore
>> >> >> >   1.70     36.12     0.76 444342256     0.00     0.00
>> >> >> >  ares__is_list_empty
>> >> >> >   1.23     36.67     0.55
>> >> >> > locking_function
>> >> >> >   1.16     37.19     0.52   143342     0.00     0.00  ares_cancel
>> >> >> >   0.99     37.63     0.44    72211     0.00     0.00
>> >> >> >  curl_multi_remove_handle
>> >> >> >   0.74     37.96     0.33  1333371     0.00     0.00
>> >> >> >  multi_runsingle
>> >> >> >   0.68     38.27     0.31 101147798     0.00     0.00  addbyter
>> >> >> >   0.58     38.53     0.26   288737     0.00     0.00
>> >> >> >  curl_multi_socket_action
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > test 9:
>> >> >> > enable c-ares;
>> >> >> > curl_easy_setopt (handle, CURLOPT_DNS_CACHE_TIMEOUT, -1);
>> >> >> > CURL_VER:=7.24.0-20120109 ->  CURL_VER:=7.24.0
>> >> >> >
>> >> >> > dns server: dnsmasq@127.0.0.1, 2,000,000 dns entry cache, resolve
>> >> >> > *.com
>> >> >> > to
>> >> >> > one IP address
>> >> >> > number of domain names in URLs: 780,000
>> >> >> > number of client: 1000
>> >> >> > CPU usage: 75%
>> >> >> >     cpu%  irq% sirq%  sys% iowt%  mem_used  buf&cached
>> >> >> >     75.1   0.0   0.6   4.1   0.0  3545.2Mb     260.1Mb
>> >> >> > SSL TPS: 380
>> >> >> > SSL throughput: 200Mbps
>> >> >> >
>> >> >> > # gprof /usr/bin/curl-loader gmon.out -p | head -20
>> >> >> > Flat profile:
>> >> >> > Each sample counts as 0.01 seconds.
>> >> >> >   %   cumulative   self              self     total
>> >> >> >  time   seconds   seconds    calls   s/call   s/call  name
>> >> >> >  34.00     31.13    31.13    72495     0.00     0.00
>> >> >> >  Curl_hash_clean_with_criterium
>> >> >> >  23.64     52.78    21.65 386070412     0.00     0.00
>> >> >> >  hostcache_timestamp_remove
>> >> >> >  13.28     64.94    12.16  1619440     0.00     0.00
>> >> >> >  Curl_hash_pick
>> >> >> >   5.92     70.36     5.42 137653129     0.00     0.00
>> >> >> >  Curl_str_key_compare
>> >> >> >   5.50     75.40     5.04   203018     0.00     0.00
>> >> >> >  Curl_hash_add
>> >> >> >   4.55     79.57     4.17    74618     0.00     0.00  create_conn
>> >> >> >   1.70     81.13     1.56  2757148     0.00     0.00
>> >> >> >  dprintf_formatf
>> >> >> >   0.98     82.03     0.90    76945     0.00     0.00
>> >> >> >  ConnectionStore
>> >> >> >   0.95     82.90     0.87 455775100     0.00     0.00
>> >> >> >  ares__is_list_empty
>> >> >> >   0.57     83.42     0.53
>> >> >> > locking_function
>> >> >> >   0.39     83.78     0.36   146742     0.00     0.00  ares_cancel
>> >> >> >   0.35     84.10     0.32 102968806     0.00     0.00  addbyter
>> >> >> >   0.35     84.42     0.32   456587     0.00     0.00
>> >> >> >  Curl_readwrite
>> >> >> >   0.35     84.74     0.32    73133     0.00     0.00
>> >> >> >  curl_multi_remove_handle
>> >> >> >   0.34     85.05     0.31  1360545     0.00     0.00
>> >> >> >  multi_runsingle
>> >> >> >
>> >> >> >
>> >> >> > 2012/3/6 Robert Iakobashvili <cor...@gm...>
>> >> >> >>
>> >> >> >> If it will not be helpful this means that recent
>> >> >> >> changes in libcurl library have broken DNS interface.
>> >> >> >>
>> >> >> >> You can try a previos version and the latest libcurl, like:
>> >> >> >>
>> >> >> >> http://curl.haxx.se/download/curl-7.23.0.tar.gz
>> >> >> >> and
>> >> >> >> http://curl.haxx.se/downloadcurl-7.24.0.tar.gz
>> >> >> >>
>> >> >> >> by:
>> >> >> >>
>> >> >> >> 1. downloading a libcurl version and placing it to
>> >> >> >> curl-loader-0.56/packages ;
>> >> >> >> 2. correcting the Makefile in curl-loader-0.56
>> >> >> >> CURL_VER:=7.24.0-20120109 ->  CURL_VER:=7.24 or CURL_VER:=7.23
>> >> >> >> 3. make cleanall; make
>> >> >> >>
>> >> >> >> Please, send us your observations.
>> >> >> >> Thanks!
>> >> >> >> Robert
>> >> >> >>
>> >> >> >> On Mon, Mar 5, 2012 at 6:57 PM, Robert Iakobashvili
>> >> >> >> <cor...@gm...>
>> >> >> >> wrote:
>> >> >> >> > If so, please try to change
>> >> >> >> >
>> >> >> >> > loader.c, line 578 which looks like:
>> >> >> >> > curl_easy_setopt (handle, CURLOPT_DNS_CACHE_TIMEOUT, -1);
>> >> >> >> >
>> >> >> >> > 1. instead of -1 try to set some value like 60
>> >> >> >> > 2. make clean; make
>> >> >> >> >
>> >> >> >> > Thanks,
>> >> >> >> > Robert
>> >> >> >> >
>> >> >> >> > On Mon, Mar 5, 2012 at 6:19 PM, Fred Huang <di...@gm...>
>> >> >> >> > wrote:
>> >> >> >> >>
>> >> >> >> >> Looks like the bottleneck comes from libcurl other than
>> >> >> >> >> c-ares...
>> >> >> >> >> test 3 (no ares patch applied):
>> >> >> >> >> dns server: dnsmasq@127.0.0.1, 2,000,000 dns entry cache,
>> >> >> >> >> resolve
>> >> >> >> >> *.com
>> >> >> >> >> to one IP address
>> >> >> >> >> number of domain names in URLs: 780,000
>> >> >> >> >> number of client: 1000
>> >> >> >> >> CPU usage: 85%
>> >> >> >> >>     cpu%  irq% sirq%  sys% iowt%  mem_used  buf&cached
>> >> >> >> >>     85.6   0.0   0.8   5.9   0.0  3334.3Mb     112.0Mb
>> >> >> >> >> SSL TPS: 320
>> >> >> >> >> SSL throughput: 200Mbps
>> >> >> >> >>
>> >> >> >> >> # gprof /usr/bin/curl-loader gmon.out -p | head -50
>> >> >> >> >> Flat profile:
>> >> >> >> >> Each sample counts as 0.01 seconds.
>> >> >> >> >>   %   cumulative   self              self     total
>> >> >> >> >>  time   seconds   seconds    calls   s/call   s/call  name
>> >> >> >> >>  40.80    120.16   120.16   116079     0.00     0.00
>> >> >> >> >>  Curl_hash_clean_with_criterium
>> >> >> >> >>  24.46    192.18    72.02 983166323     0.00     0.00
>> >> >> >> >>  hostcache_timestamp_remove
>> >> >> >> >>  13.30    231.35    39.17  2059341     0.00     0.00
>> >> >> >> >>  Curl_hash_pick
>> >> >> >> >>   6.45    250.35    19.00   240087     0.00     0.00
>> >> >> >> >>  Curl_hash_add
>> >> >> >> >>   4.81    264.51    14.16 334257102     0.00     0.00
>> >> >> >> >>  Curl_str_key_compare
>> >> >> >> >>   3.40    274.52    10.01   119514     0.00     0.00
>> >> >> >> >>  create_conn
>> >> >> >> >>   0.93    277.25     2.73  4478910     0.00     0.00
>> >> >> >> >>  dprintf_formatf
>> >> >> >> >>   0.90    279.91     2.66   122209     0.00     0.00
>> >> >> >> >>  ConnectionStore
>> >> >> >> >>   0.40    281.09     1.18
>> >> >> >> >> locking_function
>> >> >> >> >>   0.27    281.88     0.79   114759     0.00     0.00
>> >> >> >> >>  curl_multi_remove_handle
>> >> >> >> >>   0.21    282.51     0.63  7993664     0.00     0.00
>> >> >> >> >>  client_tracing_function
>> >> >> >> >>   0.21    283.12     0.61   122685     0.00     0.00
>> >> >> >> >>  Curl_num_addresses
>> >> >> >> >>   0.19    283.68     0.56   397028     0.00     0.00
>> >> >> >> >>  curl_multi_socket_action
>> >> >> >> >>   0.19    284.23     0.55  2036116     0.00     0.00
>> >> >> >> >>  multi_runsingle
>> >> >> >> >>   0.17    284.72     0.49   703687     0.00     0.00
>> >> >> >> >>  Curl_readwrite
>> >> >> >> >>   0.16    285.19     0.47
>> >> >> >> >> id_function
>> >> >> >> >>   0.15    285.62     0.43 29692029     0.00     0.00
>> >> >> >> >>  Curl_socket_check
>> >> >> >> >>   0.14    286.02     0.40 91973856     0.00     0.00
>> >> >> >> >>  Curl_raw_toupper
>> >> >> >> >>   0.13    286.40     0.38 164664144     0.00     0.00
>> >> >> >> >>  addbyter
>> >> >> >> >>   0.13    286.77     0.37  2180797     0.00     0.00
>> >> >> >> >>  Curl_infof
>> >> >> >> >>   0.11    287.10     0.33   122675     0.00     0.00
>> >> >> >> >>  Curl_if_is_interface_name
>> >> >> >> >>   0.10    287.39     0.29   471318     0.00     0.00
>> >> >> >> >>  ossl_connect_common
>> >> >> >> >>   0.10    287.67     0.28  2113528     0.00     0.00
>> >> >> >> >>  Curl_raw_equal
>> >> >> >> >>   0.09    287.93     0.26     8676     0.00     0.02
>> >> >> >> >>  curl_multi_socket_all
>> >> >> >> >>   0.08    288.17     0.24  1663262     0.00     0.00
>> >> >> >> >>  Curl_splay
>> >> >> >> >>   0.08    288.41     0.24     8505     0.00     0.01
>> >> >> >> >>  curl_multi_perform
>> >> >> >> >>   0.07    288.63     0.22  3114576     0.00     0.00
>> >> >> >> >>  Curl_pgrsUpdate
>> >> >> >> >>   0.06    288.80     0.17   119637     0.00     0.00
>> >> >> >> >>  singleipconnect
>> >> >> >> >>   0.06    288.97     0.17  7994818     0.00     0.00
>> >> >> >> >>  Curl_debug
>> >> >> >> >>   0.05    289.13     0.16   235820     0.00     0.00
>> >> >> >> >>  event_del
>> >> >> >> >>   0.05    289.29     0.16 119393071     0.00     0.00
>> >> >> >> >>  curl_strequal
>> >> >> >> >>   0.05    289.45     0.16 18535473     0.00     0.00
>> >> >> >> >>  alloc_addbyter
>> >> >> >> >>   0.05    289.61     0.16   356074     0.00     0.00
>> >> >> >> >>  Curl_hash_str
>> >> >> >> >>   0.05    289.76     0.15  1797233     0.00     0.00
>> >> >> >> >>  fd_key_compare
>> >> >> >> >>   0.05    289.91     0.15   234966     0.00     0.00
>> >> >> >> >>  Curl_ssl_getsessionid
>> >> >> >> >>   0.04    290.04     0.13   585853     0.00     0.00
>> >> >> >> >>  Curl_llist_destroy
>> >> >> >> >>   0.04    290.16     0.12   240826     0.00     0.00
>> >> >> >> >>  Curl_resolv
>> >> >> >> >>   0.03    290.26     0.10  7514328     0.00     0.00
>> >> >> >> >>  Curl_raw_nequal
>> >> >> >> >>   0.03    290.36     0.10  2214436     0.00     0.00
>> >> >> >> >>  Curl_read
>> >> >> >> >>   0.03    290.46     0.10   116872     0.00     0.00
>> >> >> >> >>  Curl_http_readwrite_headers
>> >> >> >> >>   0.03    290.56     0.10
>> >> >> >> >> ssl_tls_trace
>> >> >> >> >>   0.03    290.66     0.10  8013097     0.00     0.00
>> >> >> >> >>  scan_response
>> >> >> >> >>   0.03    290.75     0.09 29488555     0.00     0.00
>> >> >> >> >>  curlx_tvdiff
>> >> >> >> >>   0.03    290.84     0.09  2142468     0.00     0.00
>> >> >> >> >>  Curl_setopt
>> >> >> >> >>   0.03    290.93     0.09   476299     0.00     0.00
>> >> >> >> >>  Curl_expire
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >> 2012/3/5 Robert Iakobashvili <cor...@gm...>
>> >> >> >> >>>
>> >> >> >> >>> Dear Fred Huang,
>> >> >> >> >>>
>> >> >> >> >>> Thanks for your posting.
>> >> >> >> >>>
>> >> >> >> >>> Could you please check if this is due to some badness at
>> >> >> >> >>> cares?
>> >> >> >> >>> Thanks.
>> >> >> >> >>>
>> >> >> >> >>> Let's exclude usage of cares based DNS resolving by applying
>> >> >> >> >>> the
>> >> >> >> >>> patch
>> >> >> >> >>> attached
>> >> >> >> >>> and use instead global DNS cache.
>> >> >> >> >>>
>> >> >> >> >>> cd ./curl-loader-0.56
>> >> >> >> >>> patch -p1 < path-to/disable-cares-use-global-dns.patch
>> >> >> >> >>> make cleanall; make
>> >> >> >> >>>
>> >> >> >> >>> See if it helps.
>> >> >> >> >>>
>> >> >> >> >>> Global cache, unfortunately, is not thread safe.
>> >> >> >> >>>
>> >> >> >> >>> If it helps, we'll try to restore the previous version of
>> >> >> >> >>> cares.
>> >> >> >> >>>
>> >> >> >> >>> Thanks,
>> >> >> >> >>> Robert
>> >> >> >> >>>
>> >> >> >> >>> On Mon, Mar 5, 2012 at 4:28 PM, Fred Huang <di...@gm...>
>> >> >> >> >>> wrote:
>> >> >> >> >>>>
>> >> >> >> >>>> test 1:
>> >> >> >> >>>>
>> >> >> >> >>>> dns server: dnsmasq@127.0.0.1, 2,000,000 dns entry cache,
>> >> >> >> >>>> resolve
>> >> >> >> >>>> *.com to one IP address
>> >> >> >> >>>> number of domain names in all https URLs: 780,000
>> >> >> >> >>>> number of client: 1000
>> >> >> >> >>>> CPU usage: 78%
>> >> >> >> >>>>     cpu%  irq% sirq%  sys% iowt%  mem_used  buf&cached
>> >> >> >> >>>>     78.8   0.0   0.6   3.8   0.0  3640.9Mb     282.1Mb
>> >> >> >> >>>> SSL TPS: 300
>> >> >> >> >>>> SSL throughput: 200Mbps
>> >> >> >> >>>>
>> >> >> >> >>>> # gprof /usr/bin/curl-loader gmon.out -p | head -50
>> >> >> >> >>>> Flat profile:
>> >> >> >> >>>> Each sample counts as 0.01 seconds.
>> >> >> >> >>>>   %   cumulative   self              self     total
>> >> >> >> >>>>  time   seconds   seconds    calls   s/call   s/call  name
>> >> >> >> >>>>  38.55    127.10   127.10   130975     0.00     0.00
>> >> >> >> >>>>  Curl_hash_clean_with_criterium
>> >> >> >> >>>>  24.63    208.32    81.22 1247528407     0.00     0.00
>> >> >> >> >>>>  hostcache_timestamp_remove
>> >> >> >> >>>>  13.08    251.44    43.12  2791988     0.00     0.00
>> >> >> >> >>>>  Curl_hash_pick
>> >> >> >> >>>>   7.30    275.51    24.07   352490     0.00     0.00
>> >> >> >> >>>>  Curl_hash_add
>> >> >> >> >>>>   6.01    295.31    19.80 410760770     0.00     0.00
>> >> >> >> >>>>  Curl_str_key_compare
>> >> >> >> >>>>   2.95    305.04     9.73   131015     0.00     0.00
>> >> >> >> >>>>  create_conn
>> >> >> >> >>>>   0.88    307.94     2.90  4841390     0.00     0.00
>> >> >> >> >>>>  dprintf_formatf
>> >> >> >> >>>>   0.74    310.38     2.44   136036     0.00     0.00
>> >> >> >> >>>>  ConnectionStore
>> >> >> >> >>>>   0.48    311.96     1.58 814604911     0.00     0.00
>> >> >> >> >>>>  ares__is_list_empty
>> >> >> >> >>>>   0.36    313.16     1.20
>> >> >> >> >>>> locking_function
>> >> >> >> >>>>   0.27    314.06     0.90   261608     0.00     0.00
>> >> >> >> >>>>  ares_cancel
>> >> >> >> >>>>   0.22    314.79     0.73   127417     0.00     0.00
>> >> >> >> >>>>  curl_multi_remove_handle
>> >> >> >> >>>>   0.19    315.40     0.62  8906301     0.00     0.00
>> >> >> >> >>>>  client_tracing_function
>> >> >> >> >>>>   0.17    315.97     0.57   135305     0.00     0.00
>> >> >> >> >>>>  Curl_num_addresses
>> >> >> >> >>>>   0.17    316.54     0.57 181937907     0.00     0.00
>> >> >> >> >>>>  addbyter
>> >> >> >> >>>>   0.17    317.09     0.55  2374363     0.00     0.00
>> >> >> >> >>>>  multi_runsingle
>> >> >> >> >>>>   0.16    317.62     0.53   541954     0.00     0.00
>> >> >> >> >>>>  ares__init_list_node
>> >> >> >> >>>>   0.16    318.14     0.52   494412     0.00     0.00
>> >> >> >> >>>>  curl_multi_socket_action
>> >> >> >> >>>>   0.14    318.60     0.46 33578897     0.00     0.00
>> >> >> >> >>>>  Curl_socket_check
>> >> >> >> >>>>   0.13    319.04     0.44 102951052     0.00     0.00
>> >> >> >> >>>>  Curl_raw_toupper
>> >> >> >> >>>>   0.13    319.48     0.44   787558     0.00     0.00
>> >> >> >> >>>>  Curl_readwrite
>> >> >> >> >>>>   0.13    319.91     0.44
>> >> >> >> >>>> id_function
>> >> >> >> >>>>   0.12    320.29     0.38   391390     0.00     0.00
>> >> >> >> >>>>  Curl_hash_str
>> >> >> >> >>>>   0.09    320.58     0.29     9216     0.00     0.00
>> >> >> >> >>>>  curl_multi_perform
>> >> >> >> >>>>   0.08    320.86     0.28  2350908     0.00     0.00
>> >> >> >> >>>>  Curl_raw_equal
>> >> >> >> >>>>   0.08    321.14     0.28  2019415     0.00     0.00
>> >> >> >> >>>>  Curl_splay
>> >> >> >> >>>>   0.08    321.40     0.26   520625     0.00     0.00
>> >> >> >> >>>>  ossl_connect_common
>> >> >> >> >>>>   0.08    321.65     0.25  3532992     0.00     0.00
>> >> >> >> >>>>  Curl_pgrsUpdate
>> >> >> >> >>>>   0.07    321.87     0.22 131778537     0.00     0.00
>> >> >> >> >>>>  curl_strequal
>> >> >> >> >>>>   0.05    322.05     0.18  8954118     0.00     0.00
>> >> >> >> >>>>  scan_response
>> >> >> >> >>>>   0.05    322.23     0.18   533644     0.00     0.00
>> >> >> >> >>>>  ares_expand_name
>> >> >> >> >>>>   0.05    322.40     0.17  2477345     0.00     0.00
>> >> >> >> >>>>  fd_key_compare
>> >> >> >> >>>>   0.05    322.57     0.17  2429135     0.00     0.00
>> >> >> >> >>>>  Curl_infof
>> >> >> >> >>>>   0.05    322.74     0.17   130966     0.00     0.00
>> >> >> >> >>>>  singleipconnect
>> >> >> >> >>>>   0.05    322.90     0.16     9292     0.00     0.00
>> >> >> >> >>>>  curl_multi_socket_all
>> >> >> >> >>>>   0.05    323.06     0.16   270137     0.00     0.00
>> >> >> >> >>>>  ares__get_hostent
>> >> >> >> >>>>   0.05    323.21     0.15  8943657     0.00     0.00
>> >> >> >> >>>>  Curl_debug
>> >> >> >> >>>>   0.04    323.35     0.14   262493     0.00     0.00
>> >> >> >> >>>>  Curl_ssl_getsessionid
>> >> >> >> >>>>   0.04    323.48     0.13   566006     0.00     0.00
>> >> >> >> >>>>  socket_callback
>> >> >> >> >>>>   0.04    323.61     0.13 33983864     0.00     0.00
>> >> >> >> >>>>  curlx_tvdiff
>> >> >> >> >>>>   0.04    323.74     0.13   132853     0.00     0.00
>> >> >> >> >>>>  Curl_http_readwrite_headers
>> >> >> >> >>>>   0.04    323.86     0.12   344032     0.00     0.00
>> >> >> >> >>>>  epoll_del
>> >> >> >> >>>>   0.04    323.98     0.12   223477     0.00     0.00
>> >> >> >> >>>>  Curl_poll
>> >> >> >> >>>>   0.04    324.10     0.12  8526280     0.00     0.00
>> >> >> >> >>>>  Curl_raw_nequal
>> >> >> >> >>>>   0.03    324.21     0.11   348839     0.00     0.00
>> >> >> >> >>>>  event_del
>> >> >> >> >>>>
>> >> >> >> >>>>
>> >> >> >> >>>>
>> >> >> >> >>>>
>> >> >> >> >>>> test 2:
>> >> >> >> >>>>
>> >> >> >> >>>> dns server: dnsmasq@127.0.0.1, 2,000,000 dns entry cache,
>> >> >> >> >>>> resolve
>> >> >> >> >>>> *.com to one IP address
>> >> >> >> >>>> number of domain names in all https URLs: 1
>> >> >> >> >>>> number of client: 1000
>> >> >> >> >>>> CPU usage: 75%
>> >> >> >> >>>> SSL TPS: 1300
>> >> >> >> >>>> SSL throughput: 700Mbps
>> >> >> >> >>>>
>> >> >> >> >>>> # gprof /usr/bin/curl-loader gmon.out -p | head -50
>> >> >> >> >>>> Flat profile:
>> >> >> >> >>>> Each sample counts as 0.01 seconds.
>> >> >> >> >>>>   %   cumulative   self              self     total
>> >> >> >> >>>>  time   seconds   seconds    calls   s/call   s/call  name
>> >> >> >> >>>>   8.98      2.45     2.45  5127312     0.00     0.00
>> >> >> >> >>>>  dprintf_formatf
>> >> >> >> >>>>   8.90      4.88     2.43 1292796503     0.00     0.00
>> >> >> >> >>>>  ares__is_list_empty
>> >> >> >> >>>>   6.12      6.55     1.67   322461     0.00     0.00
>> >> >> >> >>>>  create_conn
>> >> >> >> >>>>   4.58      7.80     1.25   419872     0.00     0.00
>> >> >> >> >>>>  ares_cancel
>> >> >> >> >>>>   4.10      8.92     1.12  2100162     0.00     0.00
>> >> >> >> >>>>  Curl_readwrite
>> >> >> >> >>>>   3.92      9.99     1.07  4527388     0.00     0.00
>> >> >> >> >>>>  Curl_hash_pick
>> >> >> >> >>>>   3.43     10.93     0.94
>> >> >> >> >>>> locking_function
>> >> >> >> >>>>   3.04     11.76     0.83   909759     0.00     0.00
>> >> >> >> >>>>  curl_multi_socket_action
>> >> >> >> >>>>   3.00     12.58     0.82    97252     0.00     0.00
>> >> >> >> >>>>  Curl_hash_clean_with_criterium
>> >> >> >> >>>>   2.71     13.32     0.74 179704016     0.00     0.00
>> >> >> >> >>>>  Curl_raw_toupper
>> >> >> >> >>>>   2.68     14.05     0.73  4646524     0.00     0.00
>> >> >> >> >>>>  multi_runsingle
>> >> >> >> >>>>   2.44     14.71     0.67 13862992     0.00     0.00
>> >> >> >> >>>>  client_tracing_function
>> >> >> >> >>>>   2.42     15.37     0.66   319735     0.00     0.00
>> >> >> >> >>>>  curl_multi_remove_handle
>> >> >> >> >>>>   2.42     16.03     0.66       22     0.03     0.03
>> >> >> >> >>>>  ares__init_list_node
>> >> >> >> >>>>   1.94     16.56     0.53  9930244     0.00     0.00
>> >> >> >> >>>>  hostcache_timestamp_remove
>> >> >> >> >>>>   1.94     17.09     0.53 169320088     0.00     0.00
>> >> >> >> >>>>  addbyter
>> >> >> >> >>>>   1.80     17.58     0.49
>> >> >> >> >>>> id_function
>> >> >> >> >>>>   1.36     17.95     0.37  7057146     0.00     0.00
>> >> >> >> >>>>  Curl_pgrsUpdate
>> >> >> >> >>>>   1.36     18.32     0.37    16835     0.00     0.00
>> >> >> >> >>>>  curl_multi_socket_all
>> >> >> >> >>>>   1.25     18.66     0.34    16719     0.00     0.00
>> >> >> >> >>>>  curl_multi_perform
>> >> >> >> >>>>   1.21     18.99     0.33   329403     0.00     0.00
>> >> >> >> >>>>  Curl_http_readwrite_headers
>> >> >> >> >>>>   1.17     19.31     0.32  4036624     0.00     0.00
>> >> >> >> >>>>  Curl_splay
>> >> >> >> >>>>   1.14     19.62     0.31 20838902     0.00     0.00
>> >> >> >> >>>>  Curl_raw_nequal
>> >> >> >> >>>>   0.82     19.85     0.23  2764123     0.00     0.00
>> >> >> >> >>>>  Curl_infof
>> >> >> >> >>>>   0.81     20.07     0.22  5933857     0.00     0.00
>> >> >> >> >>>>  ossl_recv
>> >> >> >> >>>>   0.77     20.28     0.21   323112     0.00     0.00
>> >> >> >> >>>>  Curl_splayremovebyaddr
>> >> >> >> >>>>   0.75     20.48     0.21  5943991     0.00     0.00
>> >> >> >> >>>>  Curl_read
>> >> >> >> >>>>   0.70     20.67     0.19   425174     0.00     0.00
>> >> >> >> >>>>  event_del
>> >> >> >> >>>>   0.70     20.86     0.19    98652     0.00     0.00
>> >> >> >> >>>>  Curl_if_is_interface_name
>> >> >> >> >>>>   0.70     21.05     0.19  8677733     0.00     0.00
>> >> >> >> >>>>  Curl_socket_check
>> >> >> >> >>>>   0.59     21.21     0.16  4129554     0.00     0.00
>> >> >> >> >>>>  fd_key_compare
>> >> >> >> >>>>   0.57     21.37     0.16 14017019     0.00     0.00
>> >> >> >> >>>>  Curl_debug
>> >> >> >> >>>>   0.55     21.52     0.15  9376861     0.00     0.00
>> >> >> >> >>>>  stat_data_in_add
>> >> >> >> >>>>   0.55     21.67     0.15  6840468     0.00     0.00
>> >> >> >> >>>>  Curl_timeleft
>> >> >> >> >>>>   0.55     21.82     0.15  2054602     0.00     0.00
>> >> >> >> >>>>  Curl_raw_equal
>> >> >> >> >>>>   0.55     21.97     0.15  1291611     0.00     0.00
>> >> >> >> >>>>  Curl_expire
>> >> >> >> >>>>   0.51     22.11     0.14 16570063     0.00     0.00
>> >> >> >> >>>>  curlx_tvnow
>> >> >> >> >>>>   0.51     22.25     0.14 13863284     0.00     0.00
>> >> >> >> >>>>  scan_response
>> >> >> >> >>>>   0.51     22.39     0.14  5498864     0.00     0.00
>> >> >> >> >>>>  Curl_setopt
>> >> >> >> >>>>   0.48     22.52     0.13 97490291     0.00     0.00
>> >> >> >> >>>>  curl_strequal
>> >> >> >> >>>>   0.44     22.64     0.12   324253     0.00     0.00
>> >> >> >> >>>>  Curl_http
>> >> >> >> >>>>   0.44     22.76     0.12   394386     0.00     0.00
>> >> >> >> >>>>  ossl_connect_common
>> >> >> >> >>>>   0.40     22.87     0.11  3581335     0.00     0.00
>> >> >> >> >>>>  Curl_getinfo
>> >> >> >> >>>>   0.38     22.97     0.11 26983092     0.00     0.00
>> >> >> >> >>>>  alloc_addbyter
>> >> >> >> >>>>   0.37     23.07     0.10  8729044     0.00     0.00
>> >> >> >> >>>>  Curl_client_write
>> >> >> >> >>>>
>> >> >> >> >>>>
>> >> >> >> >>>>
>> >> >> >> >>>>
>> >> >> >> >>>>
>> >> >> >> >>>>
>> >> >> >> >>>> ------------------------------------------------------------------------------
>> >> >> >> >>>> Try before you buy = See our experts in action!
>> >> >> >> >>>> The most comprehensive online learning library for Microsoft
>> >> >> >> >>>> developers
>> >> >> >> >>>> is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5,
>> >> >> >> >>>> CSS3,
>> >> >> >> >>>> MVC3,
>> >> >> >> >>>> Metro Style Apps, more. Free future releases when you
>> >> >> >> >>>> subscribe
>> >> >> >> >>>> now!
>> >> >> >> >>>> http://p.sf.net/sfu/learndevnow-dev2
>> >> >> >> >>>> _______________________________________________
>> >> >> >> >>>> curl-loader-devel mailing list
>> >> >> >> >>>> cur...@li...
>> >> >> >> >>>>
>> >> >> >> >>>> https://lists.sourceforge.net/lists/listinfo/curl-loader-devel
>> >> >> >> >>>>
>> >> >> >> >>>
>> >> >> >> >>>
>> >> >> >> >>>
>> >> >> >> >>> --
>> >> >> >> >>> Regards,
>> >> >> >> >>> Robert Iakobashvili, Ph.D.
>> >> >> >> >>>
>> >> >> >> >>> Home:       http://www.ghotit.com
>> >> >> >> >>> Blog:         http://www.ghotit.com/category/ghotit-blogs/
>> >> >> >> >>> Twitter:      http://twitter.com/ghotit
>> >> >> >> >>> Facebook: http://facebook.com/ghotit
>> >> >> >> >>>
>> >> >> >> >>>
>> >> >> >> >>>
>> >> >> >> >>> ......................................................................
>> >> >> >> >>>                    Ghotit Dyslexia
>> >> >> >> >>> Assistive technology that understands you
>> >> >> >> >>>
>> >> >> >> >>>
>> >> >> >> >>>
>> >> >> >> >>> ......................................................................
>> >> >> >> >>>
>> >> >> >> >>>
>> >> >> >> >>>
>> >> >> >> >>>
>> >> >> >> >>>
>> >> >> >> >>>
>> >> >> >> >>> ------------------------------------------------------------------------------
>> >> >> >> >>> Try before you buy = See our experts in action!
>> >> >> >> >>> The most comprehensive online learning library for Microsoft
>> >> >> >> >>> developers
>> >> >> >> >>> is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5,
>> >> >> >> >>> CSS3,
>> >> >> >> >>> MVC3,
>> >> >> >> >>> Metro Style Apps, more. Free future releases when you
>> >> >> >> >>> subscribe
>> >> >> >> >>> now!
>> >> >> >> >>> http://p.sf.net/sfu/learndevnow-dev2
>> >> >> >> >>> _______________________________________________
>> >> >> >> >>> curl-loader-devel mailing list
>> >> >> >> >>> cur...@li...
>> >> >> >> >>>
>> >> >> >> >>> https://lists.sourceforge.net/lists/listinfo/curl-loader-devel
>> >> >> >> >>>
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >> ------------------------------------------------------------------------------
>> >> >> >> >> Try before you buy = See our experts in action!
>> >> >> >> >> The most comprehensive online learning library for Microsoft
>> >> >> >> >> developers
>> >> >> >> >> is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5,
>> >> >> >> >> CSS3,
>> >> >> >> >> MVC3,
>> >> >> >> >> Metro Style Apps, more. Free future releases when you
>> >> >> >> >> subscribe
>> >> >> >> >> now!
>> >> >> >> >> http://p.sf.net/sfu/learndevnow-dev2
>> >> >> >> >> _______________________________________________
>> >> >> >> >> curl-loader-devel mailing list
>> >> >> >> >> cur...@li...
>> >> >> >> >> https://lists.sourceforge.net/lists/listinfo/curl-loader-devel
>> >> >> >> >>
>> >> >> >> >
>> >> >> >> >
>> >> >> >> >
>> >> >> >> > --
>> >> >> >> > Regards,
>> >> >> >> > Robert Iakobashvili, Ph.D.
>> >> >> >> >
>> >> >> >> > Home:       http://www.ghotit.com
>> >> >> >> > Blog:         http://www.ghotit.com/category/ghotit-blogs/
>> >> >> >> > Twitter:      http://twitter.com/ghotit
>> >> >> >> > Facebook: http://facebook.com/ghotit
>> >> >> >> >
>> >> >> >> >
>> >> >> >> >
>> >> >> >> > ......................................................................
>> >> >> >> >                    Ghotit Dyslexia
>> >> >> >> > Assistive technology that understands you
>> >> >> >> >
>> >> >> >> >
>> >> >> >> >
>> >> >> >> > ......................................................................
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >> --
>> >> >> >> Regards,
>> >> >> >> Robert Iakobashvili, Ph.D.
>> >> >> >>
>> >> >> >> Home:       http://www.ghotit.com
>> >> >> >> Blog:         http://www.ghotit.com/category/ghotit-blogs/
>> >> >> >> Twitter:      http://twitter.com/ghotit
>> >> >> >> Facebook: http://facebook.com/ghotit
>> >> >> >>
>> >> >> >>
>> >> >> >> ......................................................................
>> >> >> >>                    Ghotit Dyslexia
>> >> >> >> Assistive technology that understands you
>> >> >> >>
>> >> >> >>
>> >> >> >> ......................................................................
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >> ------------------------------------------------------------------------------
>> >> >> >> Try before you buy = See our experts in action!
>> >> >> >> The most comprehensive online learning library for Microsoft
>> >> >> >> developers
>> >> >> >> is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5,
>> >> >> >> CSS3,
>> >> >> >> MVC3,
>> >> >> >> Metro Style Apps, more. Free future releases when you subscribe
>> >> >> >> now!
>> >> >> >> http://p.sf.net/sfu/learndevnow-dev2
>> >> >> >> _______________________________________________
>> >> >> >> curl-loader-devel mailing list
>> >> >> >> cur...@li...
>> >> >> >> https://lists.sourceforge.net/lists/listinfo/curl-loader-devel
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > ------------------------------------------------------------------------------
>> >> >> > Keep Your Developer Skills Current with LearnDevNow!
>> >> >> > The most comprehensive online learning library for Microsoft
>> >> >> > developers
>> >> >> > is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3,
>> >> >> > MVC3,
>> >> >> > Metro Style Apps, more. Free future releases when you subscribe
>> >> >> > now!
>> >> >> > http://p.sf.net/sfu/learndevnow-d2d
>> >> >> > _______________________________________________
>> >> >> > curl-loader-devel mailing list
>> >> >> > cur...@li...
>> >> >> > https://lists.sourceforge.net/lists/listinfo/curl-loader-devel
>> >> >> >
>> >> >>
>> >> >>
>> >> >>
>> >> >> --
>> >> >> Regards,
>> >> >> Robert Iakobashvili, Ph.D.
>> >> >>
>> >> >> Home:       http://www.ghotit.com
>> >> >> Blog:         http://www.ghotit.com/category/ghotit-blogs/
>> >> >> Twitter:      http://twitter.com/ghotit
>> >> >> Facebook: http://facebook.com/ghotit
>> >> >>
>> >> >> ......................................................................
>> >> >>                    Ghotit Dyslexia
>> >> >> Assistive technology that understands you
>> >> >>
>> >> >> ......................................................................
>> >> >>
>> >> >>
>> >> >>
>> >> >>
>> >> >> ------------------------------------------------------------------------------
>> >> >> Keep Your Developer Skills Current with LearnDevNow!
>> >> >> The most comprehensive online learning library for Microsoft
>> >> >> developers
>> >> >> is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3,
>> >> >> MVC3,
>> >> >> Metro Style Apps, more. Free future releases when you subscribe now!
>> >> >> http://p.sf.net/sfu/learndevnow-d2d
>> >> >> _______________________________________________
>> >> >> curl-loader-devel mailing list
>> >> >> cur...@li...
>> >> >> https://lists.sourceforge.net/lists/listinfo/curl-loader-devel
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > ------------------------------------------------------------------------------
>> >> > Keep Your Developer Skills Current with LearnDevNow!
>> >> > The most comprehensive online learning library for Microsoft
>> >> > developers
>> >> > is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3,
>> >> > MVC3,
>> >> > Metro Style Apps, more. Free future releases when you subscribe now!
>> >> > http://p.sf.net/sfu/learndevnow-d2d
>> >> > _______________________________________________
>> >> > curl-loader-devel mailing list
>> >> > cur...@li...
>> >> > https://lists.sourceforge.net/lists/listinfo/curl-loader-devel
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> Regards,
>> >> Robert Iakobashvili, Ph.D.
>> >>
>> >> Home:       http://www.ghotit.com
>> >> Blog:         http://www.ghotit.com/category/ghotit-blogs/
>> >> Twitter:      http://twitter.com/ghotit
>> >> Facebook: http://facebook.com/ghotit
>> >> ......................................................................
>> >>                    Ghotit Dyslexia
>> >> Assistive technology that understands you
>> >> ......................................................................
>> >>
>> >>
>> >>
>> >> ------------------------------------------------------------------------------
>> >> Keep Your Developer Skills Current with LearnDevNow!
>> >> The most comprehensive online learning library for Microsoft developers
>> >> is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3,
>> >> MVC3,
>> >> Metro Style Apps, more. Free future releases when you subscribe now!
>> >> http://p.sf.net/sfu/learndevnow-d2d
>> >> _______________________________________________
>> >> curl-loader-devel mailing list
>> >> cur...@li...
>> >> https://lists.sourceforge.net/lists/listinfo/curl-loader-devel
>> >
>> >
>> >
>> >
>> > ------------------------------------------------------------------------------
>> > Virtualization & Cloud Management Using Capacity Planning
>> > Cloud computing makes use of virtualization - but cloud computing
>> > also focuses on allowing computing to be delivered as a service.
>> > http://www.accelacomm.com/jaw/sfnl/114/51521223/
>> > _______________________________________________
>> > curl-loader-devel mailing list
>> > cur...@li...
>> > https://lists.sourceforge.net/lists/listinfo/curl-loader-devel
>> >
>>
>>
>>
>> --
>> Regards,
>> Robert Iakobashvili, Ph.D.
>>
>> Home:       http://www.ghotit.com
>> Blog:         http://www.ghotit.com/category/ghotit-blogs/
>> Twitter:      http://twitter.com/ghotit
>> Facebook: http://facebook.com/ghotit
>> ......................................................................
>>                    Ghotit Dyslexia
>> Assistive technology that understands you
>> ......................................................................
>>
>>
>> ------------------------------------------------------------------------------
>> Virtualization & Cloud Management Using Capacity Planning
>> Cloud computing makes use of virtualization - but cloud computing
>> also focuses on allowing computing to be delivered as a service.
>> http://www.accelacomm.com/jaw/sfnl/114/51521223/
>> _______________________________________________
>> curl-loader-devel mailing list
>> cur...@li...
>> https://lists.sourceforge.net/lists/listinfo/curl-loader-devel
>
>
>
> ------------------------------------------------------------------------------
> Virtualization & Cloud Management Using Capacity Planning
> Cloud computing makes use of virtualization - but cloud computing
> also focuses on allowing computing to be delivered as a service.
> http://www.accelacomm.com/jaw/sfnl/114/51521223/
> _______________________________________________
> curl-loader-devel mailing list
> cur...@li...
> https://lists.sourceforge.net/lists/listinfo/curl-loader-devel
>



-- 
Regards,
Robert Iakobashvili, Ph.D.

Home:       http://www.ghotit.com
Blog:         http://www.ghotit.com/category/ghotit-blogs/
Twitter:      http://twitter.com/ghotit
Facebook: http://facebook.com/ghotit
......................................................................
                   Ghotit Dyslexia
Assistive technology that understands you
......................................................................