#66 Skip network tests

Platform_support
open
nobody
None
5
2014-05-26
2013-05-02
No

It would be nice to be able to run the tests when building the libevent Fedora package. However, the Fedora builders do not have network access and so we get failures like:

 regress: 
  SKIP regress_util.c:1017: Couldn't resolve www.google.com[Timed out] 
  FAIL regress_dns.c:156: assert(dns_ok == DNS_IPv4_A): 0 vs 1dns/gethostbyname: 
  FAIL regress_main.c:332: Legacy unit test failed
  [gethostbyname FAILED]
[Timed out] dns/gethostbyname6: 
  FAIL regress_main.c:332: Legacy unit test failed
  [gethostbyname6 FAILED]
[Timed out] 
  FAIL regress_dns.c:188: assert(dns_ok == DNS_PTR): 0 vs 2dns/gethostbyaddr: 
  FAIL regress_main.c:332: Legacy unit test failed
  [gethostbyaddr FAILED]
[Timed out] 
  FAIL regress_dns.c:213: assert(dns_ok == DNS_PTR): 0 vs 2dns/resolve_reverse: 
  [resolve_reverse FAILED]
4/179 TESTS FAILED. (0 skipped)
FAILED

It would be nice to be able to skip the network tests somehow.

Related

Feature Requests: #66

Discussion

  • KuldeepGupta

    KuldeepGupta - 2014-01-07

    Dear Poplawski,

    I just looked into this issue, In the latest libevent master branch, i can see that there is a flag "TT_OFF_BY_DEFAULT"- "Flag for a test that's off by default."
    this flag is used specifically for disabling network specific tests and used in

    regress_dns.c
    DNS_LEGACY(gethostbyname, TT_FORK|TT_NEED_BASE|TT_NEED_DNS|TT_OFF_BY_DEFAULT),
    DNS_LEGACY(gethostbyname6, TT_FORK|TT_NEED_BASE|TT_NEED_DNS|TT_OFF_BY_DEFAULT),
    DNS_LEGACY(gethostbyaddr, TT_FORK|TT_NEED_BASE|TT_NEED_DNS|TT_OFF_BY_DEFAULT),
    { "resolve_reverse", dns_resolve_reverse, TT_FORK|TT_OFF_BY_DEFAULT, NULL, NULL },

    regress_http.c

    HTTP(connection_fail),{ "connection_retry", http_connection_retry_test,TT_ISOLATED|TT_OFF_BY_DEFAULT, &basic_setup, NULL },

    regress_thread.c

    TEST(conditions_simple),{ "deferred_cb_skew", thread_deferred_cb_skew, TT_FORK|TT_NEED_THREADS|TT_OFF_BY_DEFAULT, &basic_setup, NULL }

    regress_util.c
    { "getaddrinfo_live", test_evutil_getaddrinfo_live, TT_FORK|TT_OFF_BY_DEFAULT, NULL, NULL },
    { "monotonic_res_precise", test_evutil_monotonic_res, TT_OFF_BY_DEFAULT, &basic_setup, (void)"precise" },
    { "monotonic_res_fallback", test_evutil_monotonic_res, TT_OFF_BY_DEFAULT, &basic_setup, (void
    )"fallback" },

    So whenever user runs ./regress inside test folder
    Test results shows disabled for network related tests as below.

    util/getaddrinfo_live: DISABLED
    util/monotonic_res_precise: DISABLED
    util/monotonic_res_fallback: DISABLED
    dns/gethostbyname: DISABLED
    dns/gethostbyname6: DISABLED
    dns/gethostbyaddr: DISABLED
    dns/resolve_reverse: DISABLED
    thread/deferred_cb_skew: DISABLED
    http/connection_retry: DISABLED
    0/219 TESTS FAILED. (9 skipped)

    so network related tests are skipped (by default disabled) now in the latest code.

    To run/test the disabled test, user can run individual tests
    ./regress +thread/deferred_cb_skew
    ./regress +util/getaddrinfo_live
    ./regress +util/monotonic_res_precise
    ./regress +dns/gethostbyname
    ./regress +dns/gethostbyname6
    ./regress +dns/gethostbyaddr
    ./regress +dns/resolve_reverse
    ./regress +thread/deferred_cb_skew
    ./regress +http/connection_retry

    Thanks & Regards,
    Kuldeep Gupta

     
    • Orion Poplawski

      Orion Poplawski - 2014-01-07

      Kuldeep -

      Thanks. I'm passing this on to the libevent maintainers.

      • Orion

      On 01/06/2014 11:28 PM, KuldeepGupta wrote:

      Dear Poplawski,

      I just looked into this issue, In the latest libevent master branch, i can see that there is a flag "TT_OFF_BY_DEFAULT"- "Flag for a test that's off by default."
      this flag is used specifically for disabling network specific tests and used in

      regress_dns.c
      DNS_LEGACY(gethostbyname, TT_FORK|TT_NEED_BASE|TT_NEED_DNS|TT_OFF_BY_DEFAULT),
      DNS_LEGACY(gethostbyname6, TT_FORK|TT_NEED_BASE|TT_NEED_DNS|TT_OFF_BY_DEFAULT),
      DNS_LEGACY(gethostbyaddr, TT_FORK|TT_NEED_BASE|TT_NEED_DNS|TT_OFF_BY_DEFAULT),
      { "resolve_reverse", dns_resolve_reverse, TT_FORK|TT_OFF_BY_DEFAULT, NULL, NULL },

      regress_http.c

      HTTP(connection_fail),{ "connection_retry", http_connection_retry_test,TT_ISOLATED|TT_OFF_BY_DEFAULT, &basic_setup, NULL },

      regress_thread.c

      TEST(conditions_simple),{ "deferred_cb_skew", thread_deferred_cb_skew, TT_FORK|TT_NEED_THREADS|TT_OFF_BY_DEFAULT, &basic_setup, NULL }

      regress_util.c
      { "getaddrinfo_live", test_evutil_getaddrinfo_live, TT_FORK|TT_OFF_BY_DEFAULT, NULL, NULL },
      { "monotonic_res_precise", test_evutil_monotonic_res, TT_OFF_BY_DEFAULT, &basic_setup, (void)"precise" },
      { "monotonic_res_fallback", test_evutil_monotonic_res, TT_OFF_BY_DEFAULT, &basic_setup, (void
      )"fallback" },

      So whenever user runs ./regress inside test folder
      Test results shows disabled for network related tests as below.

      util/getaddrinfo_live: DISABLED
      util/monotonic_res_precise: DISABLED
      util/monotonic_res_fallback: DISABLED
      dns/gethostbyname: DISABLED
      dns/gethostbyname6: DISABLED
      dns/gethostbyaddr: DISABLED
      dns/resolve_reverse: DISABLED
      thread/deferred_cb_skew: DISABLED
      http/connection_retry: DISABLED
      0/219 TESTS FAILED. (9 skipped)

      so network related tests are skipped (by default disabled) now in the latest code.

      To run/test the disabled test, user can run individual tests
      ./regress +thread/deferred_cb_skew
      ./regress +util/getaddrinfo_live
      ./regress +util/monotonic_res_precise
      ./regress +dns/gethostbyname
      ./regress +dns/gethostbyname6
      ./regress +dns/gethostbyaddr
      ./regress +dns/resolve_reverse
      ./regress +thread/deferred_cb_skew
      ./regress +http/connection_retry

      Thanks & Regards,
      Kuldeep Gupta


      [feature-requests:#66] Skip network tests

      Status: open
      Created: Thu May 02, 2013 03:50 PM UTC by Orion Poplawski
      Last Updated: Thu May 02, 2013 03:50 PM UTC
      Owner: nobody

      It would be nice to be able to run the tests when building the libevent Fedora package. However, the Fedora builders do not have network access and so we get failures like:

      ~~~~
      regress:
      SKIP regress_util.c:1017: Couldn't resolve www.google.com[Timed out]
      FAIL regress_dns.c:156: assert(dns_ok == DNS_IPv4_A): 0 vs 1dns/gethostbyname:
      FAIL regress_main.c:332: Legacy unit test failed
      [gethostbyname FAILED]
      [Timed out] dns/gethostbyname6:
      FAIL regress_main.c:332: Legacy unit test failed
      [gethostbyname6 FAILED]
      [Timed out]
      FAIL regress_dns.c:188: assert(dns_ok == DNS_PTR): 0 vs 2dns/gethostbyaddr:
      FAIL regress_main.c:332: Legacy unit test failed
      [gethostbyaddr FAILED]
      [Timed out]
      FAIL regress_dns.c:213: assert(dns_ok == DNS_PTR): 0 vs 2dns/resolve_reverse:
      [resolve_reverse FAILED]
      4/179 TESTS FAILED. (0 skipped)
      FAILED
      ~~~~

      It would be nice to be able to skip the network tests somehow.


      Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/levent/feature-requests/66/

      To unsubscribe from further messages, please visit https://sourceforge.net/auth/subscriptions/

      --
      Orion Poplawski
      Technical Manager 303-415-9701 x222
      NWRA, Boulder/CoRA Office FAX: 303-415-9702
      3380 Mitchell Lane orion@nwra.com
      Boulder, CO 80301 http://www.nwra.com

       

      Related

      Feature Requests: #66

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:





No, thanks