check-users Mailing List for check (Page 3)

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 422-6466

hey

is it normal that tcase_set_timeout() can't be used on Windows even with 
NO_FORK ?

thanks

Vincent Torri



>
> I’m seeing some encouraging results by commenting out the call to
> `pthread_exit` in Class.c. Based on your knowledge of check, could this
> call explain the unpredictable behavior I was seeing?


It may not be valid to call pthread_exit() from atexit according to this:
http://stackoverflow.com/questions/11953955/using-atexit-to-call-pthread-exit

The only work which has been done in check with respect to atexit is that
if a test fails in fork mode _exit() is invoked. This results in a unit
test process not invoking atexit(), as it may be in a bad state.

- Branden

On Mon, Nov 24, 2014 at 5:28 PM, Jay Dolan <jay...@gm...> wrote:

> Just to answer my own question: I ended up using AM_TESTS_ENVIRONMENT in
> my tests’ Makefile.am to set an environment variable that my code could
> check for so that, if running unit tests via `make check`, I do not call
> pthread_exit.
>
> Someone may want to update the check docs to mention that if you call
> pthread_exit in the code you’re testing, you’re gonna have a bad time :D
>
> Thanks again Branden,
>
> Jay Dolan
> 508-415-1866
>
> On Nov 24, 2014, at 4:02 PM, Jay Dolan <jay...@gm...> wrote:
>
> Branden,
>
> I’m seeing some encouraging results by commenting out the call to
> `pthread_exit` in Class.c. Based on your knowledge of check, could this
> call explain the unpredictable behavior I was seeing?
>
> Next question: since this call is in fact valid for my application, does
> check set an environment variable I can inspect in my teardown routine to
> “elegantly” avoid calling `pthread_exit` when running unit tests?
>
> Cheers,
>
> Jay
>
> Jay Dolan
> 508-415-1866
>
> On Nov 24, 2014, at 9:47 AM, Jay Dolan <jay...@gm...> wrote:
>
> Hi Branden,
>
> That’s really interesting. I wonder if it has to do with my use of atexit
> or pthread_exit. You can see how I perform some cleanup / teardown at the
> top of Class.c. I’ll try setting some breakpoints in there to see if that’s
> influencing check’s behavior. Maybe check is hitting that code path at the
> end of every test case, and things are going south?
>
> Thanks again,
>
> Jay Dolan
> 508-415-1866
>
> On Nov 24, 2014, at 9:24 AM, Branden Archer <b.m...@gm...> wrote:
>
> I was able to reproduce the same behavior. Several of the test programs
> appear to sporadically hang during execution and are eventually aborted due
> to a timeout.
>
> One of them, the Objects program, was taken as an example. Running it in
> both fork and no-fork mode it occasionally hung. Adding some printf
> statements throughout the test showed that the test did reach the end
> before it started to hang. More interesting what the observation that once
> all of the real content of the unit test was commented out, leaving the
> printf statements, no further hangs were observed.
>
> At that point, I become suspicious that some sort of memory corruption was
> occurring during the unit test run and ran valgrind:
>
> $ CK_FORK=no valgrind --leak-check=full Tests/Objectively/Object
>
> However, nothing conclusive was detected. Though, I did have to compile
> the trunk of valgrind to get it to work on OS X 10.10, so maybe it is not
> ready, not sure.
>
> Interesting, though, when run in CK_FORK=no mode, the test program
> eventually does exit, though it may take 10 seconds:
>
> $ for i in `seq 1 10`; do (time CK_FORK=no Tests/Objectively/Object) 2>&1
> | grep real; done
> real *0m6.661s*
> real 0m0.007s
> real 0m0.004s
> real 0m0.003s
> real 0m0.003s
> real 0m0.003s
> real 0m0.003s
> real *0m9.987s*
> real 0m0.023s
> real 0m0.005s
>
> Also interesting is that the hang occurs after the results are printed.
> That leads me to believe that there is another thread somewhere still
> running. During a hang in CK_FORK=no mode I attached lldb and printed out
> the stack trace of all current threads. This is the only thread running:
>
> (lldb) thread backtrace
> * thread #1: tid = 0x4a07f, 0x00007fff8fcfa946
> libsystem_kernel.dylib`__workq_kernreturn + 10, stop reason = signal SIGSTOP
>   * frame #0: 0x00007fff8fcfa946 libsystem_kernel.dylib`__workq_kernreturn
> + 10
>     frame #1: 0x00007fff8f761757 libsystem_pthread.dylib`_pthread_wqthread
> + 869
>     frame #2: 0x00007fff8f75f4a1 libsystem_pthread.dylib`start_wqthread +
> 13
>
> which does not make much sense to me.
>
> As a final move, I commented out everything in the Object test except for
> the initial Object allocation. With that the hang would still happen.
> However, if even that is commented out the hang does not occur.
>
> So, in short, I've never seen behavior before where a unit test program
> will not return after its tests have completed Also I've no idea why it is
> happening in this case.
>
> If you do get to the bottom of it, I would be curious to hear what the
> underlying issue was. (:
>
> - Branden
>
>
> On Sun, Nov 23, 2014 at 11:23 PM, Jay Dolan <jay...@gm...> wrote:
>
>> Hi Branden,
>>
>> Thanks for looking at the issue. That error is certainly valid, thanks
>> for pointing that out. The “Hello” binary that’s failing to compile / link
>> is only a demonstration program. If you’re willing to run `make install`,
>> the quickest way forward would be to adjust Tests/Makefile.am and comment
>> out noinst_PROGRAMS so that Hello is skipped. Then run `make install`. Then
>> `make check`. Afterwards, running `make uninstall` will remove Objectively
>> from your system.
>>
>> Sorry the build is a little ugly right now. It’s in flux. And thanks
>> again!
>>
>> Jay Dolan
>> 508-415-1866
>>
>> On Nov 23, 2014, at 10:56 PM, Branden Archer <b.m...@gm...>
>> wrote:
>>
>> In trying to build the project and reproduce the problems I'm running
>> into some issues.
>>
>> My system is using the same versions of everything (exception that OS X
>> is 10.10). The necessary tools are from brew.sh instead of Macports. The
>> following steps:
>>
>> $ autoreconf -i
>> $ ./configure
>> $ make
>>
>> leads to the following error:
>>
>>
>> Brandens-MacBook-Pro:objectify.git brarcher$ make
>> /Library/Developer/CommandLineTools/usr/bin/make  all-recursive
>> Making all in Sources
>> Making all in Objectively
>>   CC       libObjectively_la-Array.lo
>>   CC       libObjectively_la-Class.lo
>>   CC       libObjectively_la-Condition.lo
>>   CC       libObjectively_la-Date.lo
>>   CC       libObjectively_la-DateFormatter.lo
>>   CC       libObjectively_la-Dictionary.lo
>>   CC       libObjectively_la-Lock.lo
>>   CC       libObjectively_la-Log.lo
>>   CC       libObjectively_la-Object.lo
>>   CC       libObjectively_la-String.lo
>>   CC       libObjectively_la-Thread.lo
>>   CCLD     libObjectively.la
>> make[3]: Nothing to be done for `all-am'.
>> Making all in Tests
>> Making all in Objectively
>>   CC       Hello.o
>> *Hello.c:5:10: **fatal error: **'Objectively.h' file not found*
>> #include <Objectively.h>
>> *         ^*
>> 1 error generated.
>> make[3]: *** [Hello.o] Error 1
>> make[2]: *** [all-recursive] Error 1
>> make[1]: *** [all-recursive] Error 1
>> make: *** [all] Error 2
>>
>>
>> It seems that the include path for the unit tests does not include the
>> header from the source. Attempting to hack that in:
>>
>>
>> *diff --git a/Tests/Objectively/Makefile.am
>> b/Tests/Objectively/Makefile.am*
>> *index b32458d..fb9e2ea 100644*
>> *--- a/Tests/Objectively/Makefile.am*
>> *+++ b/Tests/Objectively/Makefile.am*
>> @@ -10,6 +10,7 @@ TESTS = \
>>
>>  CFLAGS += \
>>         -I$(includedir) \
>> +       -I../../Sources \
>>         @CHECK_CFLAGS@
>>
>>  LDADD = \
>>
>>
>> Leads to a failure when looking for the Objectively library:
>>
>>
>> Brandens-MacBook-Pro:objectify.git brarcher$ make
>> /Library/Developer/CommandLineTools/usr/bin/make  all-recursive
>> Making all in Sources
>> Making all in Objectively
>> make[3]: Nothing to be done for `all'.
>> make[3]: Nothing to be done for `all-am'.
>> Making all in Tests
>> Making all in Objectively
>>   CCLD     Hello
>> ld: library not found for -lObjectively
>> clang: error: linker command failed with exit code 1 (use -v to see
>> invocation)
>> make[3]: *** [Hello] Error 1
>> make[2]: *** [all-recursive] Error 1
>> make[1]: *** [all-recursive] Error 1
>> make: *** [all] Error 2
>>
>>
>> How does one build the project such that the unit tests build against the
>> Objectively from the source tree? Did I miss a few steps somewhere?
>>
>> - Branden
>>
>>
>>
>> On Fri, Nov 21, 2014 at 10:30 AM, Jay Dolan <jay...@gm...>
>> wrote:
>>
>>> I’ve seen some _really_ inconsistent behavior from `make check` and I’m
>>> wondering if I’m alone.
>>>
>>>
>>>    - Sometimes the test runner will indicate that all tests are passing
>>>    despite the individual test logs clearly calling out errors.
>>>    - Sometimes the test runner will hang up for seemingly no reason at
>>>    all, causing the tests to timeout and thus fail.
>>>    - Setting the CK_FORK environment variable to `no` seems to help
>>>    where it really shouldn’t matter.
>>>
>>>
>>> My environment is:
>>>
>>> OS X 10.10.1
>>> Apple LLVM 6.0 / Clang (latest Xcode tool chain)
>>> Automake 1.14.1 via Macports
>>> Check 0.9.14 via Macports.
>>>
>>> The project I’m compiling and seeing these issues with is Objectively:
>>> https://github.com/jdolan/objectively
>>>
>>> All but one of my test cases are single-threaded, and I can easily
>>> reproduce these issues while omitting that particular test case. The test
>>> sources are here:
>>> https://github.com/jdolan/objectively/tree/master/Tests/Objectively
>>>
>>> Am I Doing It Wrong(tm), or is there something more to this?
>>>
>>> Jay
>>>
>>> Jay Dolan
>>> 508-415-1866
>>>
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
>>> from Actuate! Instantly Supercharge Your Business Reports and Dashboards
>>> with Interactivity, Sharing, Native Excel Exports, App Integration & more
>>> Get technology previously reserved for billion-dollar corporations, FREE
>>>
>>> http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
>>> _______________________________________________
>>> Check-users mailing list
>>> Che...@li...
>>> https://lists.sourceforge.net/lists/listinfo/check-users
>>>
>>>
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
>> from Actuate! Instantly Supercharge Your Business Reports and Dashboards
>> with Interactivity, Sharing, Native Excel Exports, App Integration & more
>> Get technology previously reserved for billion-dollar corporations, FREE
>>
>> http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
>> _______________________________________________
>> Check-users mailing list
>> Che...@li...
>> https://lists.sourceforge.net/lists/listinfo/check-users
>>
>>
>
>
>
>
>
> ------------------------------------------------------------------------------
> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
> from Actuate! Instantly Supercharge Your Business Reports and Dashboards
> with Interactivity, Sharing, Native Excel Exports, App Integration & more
> Get technology previously reserved for billion-dollar corporations, FREE
>
> http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
> _______________________________________________
> Check-users mailing list
> Che...@li...
> https://lists.sourceforge.net/lists/listinfo/check-users
>
>

Just to answer my own question: I ended up using AM_TESTS_ENVIRONMENT in my tests’ Makefile.am to set an environment variable that my code could check for so that, if running unit tests via `make check`, I do not call pthread_exit.

Someone may want to update the check docs to mention that if you call pthread_exit in the code you’re testing, you’re gonna have a bad time :D

Thanks again Branden,

Jay Dolan
508-415-1866

> On Nov 24, 2014, at 4:02 PM, Jay Dolan <jay...@gm...> wrote:
> 
> Branden,
> 
> I’m seeing some encouraging results by commenting out the call to `pthread_exit` in Class.c. Based on your knowledge of check, could this call explain the unpredictable behavior I was seeing?
> 
> Next question: since this call is in fact valid for my application, does check set an environment variable I can inspect in my teardown routine to “elegantly” avoid calling `pthread_exit` when running unit tests?
> 
> Cheers,
> 
> Jay
> 
> Jay Dolan
> 508-415-1866
> 
>> On Nov 24, 2014, at 9:47 AM, Jay Dolan <jay...@gm... <mailto:jay...@gm...>> wrote:
>> 
>> Hi Branden,
>> 
>> That’s really interesting. I wonder if it has to do with my use of atexit or pthread_exit. You can see how I perform some cleanup / teardown at the top of Class.c. I’ll try setting some breakpoints in there to see if that’s influencing check’s behavior. Maybe check is hitting that code path at the end of every test case, and things are going south?
>> 
>> Thanks again,
>> 
>> Jay Dolan
>> 508-415-1866
>> 
>>> On Nov 24, 2014, at 9:24 AM, Branden Archer <b.m...@gm... <mailto:b.m...@gm...>> wrote:
>>> 
>>> I was able to reproduce the same behavior. Several of the test programs appear to sporadically hang during execution and are eventually aborted due to a timeout.
>>> 
>>> One of them, the Objects program, was taken as an example. Running it in both fork and no-fork mode it occasionally hung. Adding some printf statements throughout the test showed that the test did reach the end before it started to hang. More interesting what the observation that once all of the real content of the unit test was commented out, leaving the printf statements, no further hangs were observed.
>>> 
>>> At that point, I become suspicious that some sort of memory corruption was occurring during the unit test run and ran valgrind:
>>> 
>>> $ CK_FORK=no valgrind --leak-check=full Tests/Objectively/Object
>>> 
>>> However, nothing conclusive was detected. Though, I did have to compile the trunk of valgrind to get it to work on OS X 10.10, so maybe it is not ready, not sure.
>>> 
>>> Interesting, though, when run in CK_FORK=no mode, the test program eventually does exit, though it may take 10 seconds:
>>> 
>>> $ for i in `seq 1 10`; do (time CK_FORK=no Tests/Objectively/Object) 2>&1 | grep real; done
>>> real	0m6.661s
>>> real	0m0.007s
>>> real	0m0.004s
>>> real	0m0.003s
>>> real	0m0.003s
>>> real	0m0.003s
>>> real	0m0.003s
>>> real	0m9.987s
>>> real	0m0.023s
>>> real	0m0.005s
>>> 
>>> Also interesting is that the hang occurs after the results are printed. That leads me to believe that there is another thread somewhere still running. During a hang in CK_FORK=no mode I attached lldb and printed out the stack trace of all current threads. This is the only thread running:
>>> 
>>> (lldb) thread backtrace
>>> * thread #1: tid = 0x4a07f, 0x00007fff8fcfa946 libsystem_kernel.dylib`__workq_kernreturn + 10, stop reason = signal SIGSTOP
>>>   * frame #0: 0x00007fff8fcfa946 libsystem_kernel.dylib`__workq_kernreturn + 10
>>>     frame #1: 0x00007fff8f761757 libsystem_pthread.dylib`_pthread_wqthread + 869
>>>     frame #2: 0x00007fff8f75f4a1 libsystem_pthread.dylib`start_wqthread + 13
>>> 
>>> which does not make much sense to me.
>>> 
>>> As a final move, I commented out everything in the Object test except for the initial Object allocation. With that the hang would still happen. However, if even that is commented out the hang does not occur.
>>> 
>>> So, in short, I've never seen behavior before where a unit test program will not return after its tests have completed Also I've no idea why it is happening in this case.
>>> 
>>> If you do get to the bottom of it, I would be curious to hear what the underlying issue was. (:
>>> 
>>> - Branden
>>> 
>>> 
>>> On Sun, Nov 23, 2014 at 11:23 PM, Jay Dolan <jay...@gm... <mailto:jay...@gm...>> wrote:
>>> Hi Branden,
>>> 
>>> Thanks for looking at the issue. That error is certainly valid, thanks for pointing that out. The “Hello” binary that’s failing to compile / link is only a demonstration program. If you’re willing to run `make install`, the quickest way forward would be to adjust Tests/Makefile.am and comment out noinst_PROGRAMS so that Hello is skipped. Then run `make install`. Then `make check`. Afterwards, running `make uninstall` will remove Objectively from your system.
>>> 
>>> Sorry the build is a little ugly right now. It’s in flux. And thanks again!
>>>   
>>> Jay Dolan
>>> 508-415-1866 <tel:508-415-1866>
>>> 
>>>> On Nov 23, 2014, at 10:56 PM, Branden Archer <b.m...@gm... <mailto:b.m...@gm...>> wrote:
>>>> 
>>>> In trying to build the project and reproduce the problems I'm running into some issues.
>>>> 
>>>> My system is using the same versions of everything (exception that OS X is 10.10). The necessary tools are from brew.sh instead of Macports. The following steps:
>>>> 
>>>> $ autoreconf -i
>>>> $ ./configure
>>>> $ make
>>>> 
>>>> leads to the following error:
>>>> 
>>>> 
>>>> Brandens-MacBook-Pro:objectify.git brarcher$ make
>>>> /Library/Developer/CommandLineTools/usr/bin/make  all-recursive
>>>> Making all in Sources
>>>> Making all in Objectively
>>>>   CC       libObjectively_la-Array.lo
>>>>   CC       libObjectively_la-Class.lo
>>>>   CC       libObjectively_la-Condition.lo
>>>>   CC       libObjectively_la-Date.lo
>>>>   CC       libObjectively_la-DateFormatter.lo
>>>>   CC       libObjectively_la-Dictionary.lo
>>>>   CC       libObjectively_la-Lock.lo
>>>>   CC       libObjectively_la-Log.lo
>>>>   CC       libObjectively_la-Object.lo
>>>>   CC       libObjectively_la-String.lo
>>>>   CC       libObjectively_la-Thread.lo
>>>>   CCLD     libObjectively.la
>>>> make[3]: Nothing to be done for `all-am'.
>>>> Making all in Tests
>>>> Making all in Objectively
>>>>   CC       Hello.o
>>>> Hello.c:5:10: fatal error: 'Objectively.h' file not found
>>>> #include <Objectively.h>
>>>>          ^
>>>> 1 error generated.
>>>> make[3]: *** [Hello.o] Error 1
>>>> make[2]: *** [all-recursive] Error 1
>>>> make[1]: *** [all-recursive] Error 1
>>>> make: *** [all] Error 2
>>>> 
>>>> 
>>>> It seems that the include path for the unit tests does not include the header from the source. Attempting to hack that in:
>>>> 
>>>> 
>>>> diff --git a/Tests/Objectively/Makefile.am b/Tests/Objectively/Makefile.am
>>>> index b32458d..fb9e2ea 100644
>>>> --- a/Tests/Objectively/Makefile.am
>>>> +++ b/Tests/Objectively/Makefile.am
>>>> @@ -10,6 +10,7 @@ TESTS = \
>>>>  
>>>>  CFLAGS += \
>>>>         -I$(includedir) \
>>>> +       -I../../Sources \
>>>>         @CHECK_CFLAGS@
>>>>  
>>>>  LDADD = \
>>>> 
>>>> 
>>>> Leads to a failure when looking for the Objectively library:
>>>> 
>>>> 
>>>> Brandens-MacBook-Pro:objectify.git brarcher$ make
>>>> /Library/Developer/CommandLineTools/usr/bin/make  all-recursive
>>>> Making all in Sources
>>>> Making all in Objectively
>>>> make[3]: Nothing to be done for `all'.
>>>> make[3]: Nothing to be done for `all-am'.
>>>> Making all in Tests
>>>> Making all in Objectively
>>>>   CCLD     Hello
>>>> ld: library not found for -lObjectively
>>>> clang: error: linker command failed with exit code 1 (use -v to see invocation)
>>>> make[3]: *** [Hello] Error 1
>>>> make[2]: *** [all-recursive] Error 1
>>>> make[1]: *** [all-recursive] Error 1
>>>> make: *** [all] Error 2
>>>> 
>>>> 
>>>> How does one build the project such that the unit tests build against the Objectively from the source tree? Did I miss a few steps somewhere?
>>>> 
>>>> - Branden
>>>> 
>>>> 
>>>> 
>>>> On Fri, Nov 21, 2014 at 10:30 AM, Jay Dolan <jay...@gm... <mailto:jay...@gm...>> wrote:
>>>> I’ve seen some _really_ inconsistent behavior from `make check` and I’m wondering if I’m alone.
>>>> 
>>>> Sometimes the test runner will indicate that all tests are passing despite the individual test logs clearly calling out errors.
>>>> Sometimes the test runner will hang up for seemingly no reason at all, causing the tests to timeout and thus fail.
>>>> Setting the CK_FORK environment variable to `no` seems to help where it really shouldn’t matter.
>>>> 
>>>> My environment is:
>>>> 
>>>> OS X 10.10.1
>>>> Apple LLVM 6.0 / Clang (latest Xcode tool chain)
>>>> Automake 1.14.1 via Macports
>>>> Check 0.9.14 via Macports.
>>>> 
>>>> The project I’m compiling and seeing these issues with is Objectively:
>>>> https://github.com/jdolan/objectively <https://github.com/jdolan/objectively>
>>>> 
>>>> All but one of my test cases are single-threaded, and I can easily reproduce these issues while omitting that particular test case. The test sources are here:
>>>> https://github.com/jdolan/objectively/tree/master/Tests/Objectively <https://github.com/jdolan/objectively/tree/master/Tests/Objectively>
>>>> 
>>>> Am I Doing It Wrong(tm), or is there something more to this?
>>>> 
>>>> Jay
>>>> 
>>>> Jay Dolan
>>>> 508-415-1866 <tel:508-415-1866>
>>>> 
>>>> ------------------------------------------------------------------------------
>>>> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
>>>> from Actuate! Instantly Supercharge Your Business Reports and Dashboards
>>>> with Interactivity, Sharing, Native Excel Exports, App Integration & more
>>>> Get technology previously reserved for billion-dollar corporations, FREE
>>>> http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk <http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk>
>>>> _______________________________________________
>>>> Check-users mailing list
>>>> Che...@li... <mailto:Che...@li...>
>>>> https://lists.sourceforge.net/lists/listinfo/check-users <https://lists.sourceforge.net/lists/listinfo/check-users>
>>>> 
>>>> 
>>> 
>>> 
>>> ------------------------------------------------------------------------------
>>> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
>>> from Actuate! Instantly Supercharge Your Business Reports and Dashboards
>>> with Interactivity, Sharing, Native Excel Exports, App Integration & more
>>> Get technology previously reserved for billion-dollar corporations, FREE
>>> http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk <http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk>
>>> _______________________________________________
>>> Check-users mailing list
>>> Che...@li... <mailto:Che...@li...>
>>> https://lists.sourceforge.net/lists/listinfo/check-users <https://lists.sourceforge.net/lists/listinfo/check-users>
>>> 
>>> 
>> 
> 


Branden,

I’m seeing some encouraging results by commenting out the call to `pthread_exit` in Class.c. Based on your knowledge of check, could this call explain the unpredictable behavior I was seeing?

Next question: since this call is in fact valid for my application, does check set an environment variable I can inspect in my teardown routine to “elegantly” avoid calling `pthread_exit` when running unit tests?

Cheers,

Jay

Jay Dolan
508-415-1866

> On Nov 24, 2014, at 9:47 AM, Jay Dolan <jay...@gm...> wrote:
> 
> Hi Branden,
> 
> That’s really interesting. I wonder if it has to do with my use of atexit or pthread_exit. You can see how I perform some cleanup / teardown at the top of Class.c. I’ll try setting some breakpoints in there to see if that’s influencing check’s behavior. Maybe check is hitting that code path at the end of every test case, and things are going south?
> 
> Thanks again,
> 
> Jay Dolan
> 508-415-1866
> 
>> On Nov 24, 2014, at 9:24 AM, Branden Archer <b.m...@gm... <mailto:b.m...@gm...>> wrote:
>> 
>> I was able to reproduce the same behavior. Several of the test programs appear to sporadically hang during execution and are eventually aborted due to a timeout.
>> 
>> One of them, the Objects program, was taken as an example. Running it in both fork and no-fork mode it occasionally hung. Adding some printf statements throughout the test showed that the test did reach the end before it started to hang. More interesting what the observation that once all of the real content of the unit test was commented out, leaving the printf statements, no further hangs were observed.
>> 
>> At that point, I become suspicious that some sort of memory corruption was occurring during the unit test run and ran valgrind:
>> 
>> $ CK_FORK=no valgrind --leak-check=full Tests/Objectively/Object
>> 
>> However, nothing conclusive was detected. Though, I did have to compile the trunk of valgrind to get it to work on OS X 10.10, so maybe it is not ready, not sure.
>> 
>> Interesting, though, when run in CK_FORK=no mode, the test program eventually does exit, though it may take 10 seconds:
>> 
>> $ for i in `seq 1 10`; do (time CK_FORK=no Tests/Objectively/Object) 2>&1 | grep real; done
>> real	0m6.661s
>> real	0m0.007s
>> real	0m0.004s
>> real	0m0.003s
>> real	0m0.003s
>> real	0m0.003s
>> real	0m0.003s
>> real	0m9.987s
>> real	0m0.023s
>> real	0m0.005s
>> 
>> Also interesting is that the hang occurs after the results are printed. That leads me to believe that there is another thread somewhere still running. During a hang in CK_FORK=no mode I attached lldb and printed out the stack trace of all current threads. This is the only thread running:
>> 
>> (lldb) thread backtrace
>> * thread #1: tid = 0x4a07f, 0x00007fff8fcfa946 libsystem_kernel.dylib`__workq_kernreturn + 10, stop reason = signal SIGSTOP
>>   * frame #0: 0x00007fff8fcfa946 libsystem_kernel.dylib`__workq_kernreturn + 10
>>     frame #1: 0x00007fff8f761757 libsystem_pthread.dylib`_pthread_wqthread + 869
>>     frame #2: 0x00007fff8f75f4a1 libsystem_pthread.dylib`start_wqthread + 13
>> 
>> which does not make much sense to me.
>> 
>> As a final move, I commented out everything in the Object test except for the initial Object allocation. With that the hang would still happen. However, if even that is commented out the hang does not occur.
>> 
>> So, in short, I've never seen behavior before where a unit test program will not return after its tests have completed Also I've no idea why it is happening in this case.
>> 
>> If you do get to the bottom of it, I would be curious to hear what the underlying issue was. (:
>> 
>> - Branden
>> 
>> 
>> On Sun, Nov 23, 2014 at 11:23 PM, Jay Dolan <jay...@gm... <mailto:jay...@gm...>> wrote:
>> Hi Branden,
>> 
>> Thanks for looking at the issue. That error is certainly valid, thanks for pointing that out. The “Hello” binary that’s failing to compile / link is only a demonstration program. If you’re willing to run `make install`, the quickest way forward would be to adjust Tests/Makefile.am and comment out noinst_PROGRAMS so that Hello is skipped. Then run `make install`. Then `make check`. Afterwards, running `make uninstall` will remove Objectively from your system.
>> 
>> Sorry the build is a little ugly right now. It’s in flux. And thanks again!
>>   
>> Jay Dolan
>> 508-415-1866 <tel:508-415-1866>
>> 
>>> On Nov 23, 2014, at 10:56 PM, Branden Archer <b.m...@gm... <mailto:b.m...@gm...>> wrote:
>>> 
>>> In trying to build the project and reproduce the problems I'm running into some issues.
>>> 
>>> My system is using the same versions of everything (exception that OS X is 10.10). The necessary tools are from brew.sh instead of Macports. The following steps:
>>> 
>>> $ autoreconf -i
>>> $ ./configure
>>> $ make
>>> 
>>> leads to the following error:
>>> 
>>> 
>>> Brandens-MacBook-Pro:objectify.git brarcher$ make
>>> /Library/Developer/CommandLineTools/usr/bin/make  all-recursive
>>> Making all in Sources
>>> Making all in Objectively
>>>   CC       libObjectively_la-Array.lo
>>>   CC       libObjectively_la-Class.lo
>>>   CC       libObjectively_la-Condition.lo
>>>   CC       libObjectively_la-Date.lo
>>>   CC       libObjectively_la-DateFormatter.lo
>>>   CC       libObjectively_la-Dictionary.lo
>>>   CC       libObjectively_la-Lock.lo
>>>   CC       libObjectively_la-Log.lo
>>>   CC       libObjectively_la-Object.lo
>>>   CC       libObjectively_la-String.lo
>>>   CC       libObjectively_la-Thread.lo
>>>   CCLD     libObjectively.la
>>> make[3]: Nothing to be done for `all-am'.
>>> Making all in Tests
>>> Making all in Objectively
>>>   CC       Hello.o
>>> Hello.c:5:10: fatal error: 'Objectively.h' file not found
>>> #include <Objectively.h>
>>>          ^
>>> 1 error generated.
>>> make[3]: *** [Hello.o] Error 1
>>> make[2]: *** [all-recursive] Error 1
>>> make[1]: *** [all-recursive] Error 1
>>> make: *** [all] Error 2
>>> 
>>> 
>>> It seems that the include path for the unit tests does not include the header from the source. Attempting to hack that in:
>>> 
>>> 
>>> diff --git a/Tests/Objectively/Makefile.am b/Tests/Objectively/Makefile.am
>>> index b32458d..fb9e2ea 100644
>>> --- a/Tests/Objectively/Makefile.am
>>> +++ b/Tests/Objectively/Makefile.am
>>> @@ -10,6 +10,7 @@ TESTS = \
>>>  
>>>  CFLAGS += \
>>>         -I$(includedir) \
>>> +       -I../../Sources \
>>>         @CHECK_CFLAGS@
>>>  
>>>  LDADD = \
>>> 
>>> 
>>> Leads to a failure when looking for the Objectively library:
>>> 
>>> 
>>> Brandens-MacBook-Pro:objectify.git brarcher$ make
>>> /Library/Developer/CommandLineTools/usr/bin/make  all-recursive
>>> Making all in Sources
>>> Making all in Objectively
>>> make[3]: Nothing to be done for `all'.
>>> make[3]: Nothing to be done for `all-am'.
>>> Making all in Tests
>>> Making all in Objectively
>>>   CCLD     Hello
>>> ld: library not found for -lObjectively
>>> clang: error: linker command failed with exit code 1 (use -v to see invocation)
>>> make[3]: *** [Hello] Error 1
>>> make[2]: *** [all-recursive] Error 1
>>> make[1]: *** [all-recursive] Error 1
>>> make: *** [all] Error 2
>>> 
>>> 
>>> How does one build the project such that the unit tests build against the Objectively from the source tree? Did I miss a few steps somewhere?
>>> 
>>> - Branden
>>> 
>>> 
>>> 
>>> On Fri, Nov 21, 2014 at 10:30 AM, Jay Dolan <jay...@gm... <mailto:jay...@gm...>> wrote:
>>> I’ve seen some _really_ inconsistent behavior from `make check` and I’m wondering if I’m alone.
>>> 
>>> Sometimes the test runner will indicate that all tests are passing despite the individual test logs clearly calling out errors.
>>> Sometimes the test runner will hang up for seemingly no reason at all, causing the tests to timeout and thus fail.
>>> Setting the CK_FORK environment variable to `no` seems to help where it really shouldn’t matter.
>>> 
>>> My environment is:
>>> 
>>> OS X 10.10.1
>>> Apple LLVM 6.0 / Clang (latest Xcode tool chain)
>>> Automake 1.14.1 via Macports
>>> Check 0.9.14 via Macports.
>>> 
>>> The project I’m compiling and seeing these issues with is Objectively:
>>> https://github.com/jdolan/objectively <https://github.com/jdolan/objectively>
>>> 
>>> All but one of my test cases are single-threaded, and I can easily reproduce these issues while omitting that particular test case. The test sources are here:
>>> https://github.com/jdolan/objectively/tree/master/Tests/Objectively <https://github.com/jdolan/objectively/tree/master/Tests/Objectively>
>>> 
>>> Am I Doing It Wrong(tm), or is there something more to this?
>>> 
>>> Jay
>>> 
>>> Jay Dolan
>>> 508-415-1866 <tel:508-415-1866>
>>> 
>>> ------------------------------------------------------------------------------
>>> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
>>> from Actuate! Instantly Supercharge Your Business Reports and Dashboards
>>> with Interactivity, Sharing, Native Excel Exports, App Integration & more
>>> Get technology previously reserved for billion-dollar corporations, FREE
>>> http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk <http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk>
>>> _______________________________________________
>>> Check-users mailing list
>>> Che...@li... <mailto:Che...@li...>
>>> https://lists.sourceforge.net/lists/listinfo/check-users <https://lists.sourceforge.net/lists/listinfo/check-users>
>>> 
>>> 
>> 
>> 
>> ------------------------------------------------------------------------------
>> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
>> from Actuate! Instantly Supercharge Your Business Reports and Dashboards
>> with Interactivity, Sharing, Native Excel Exports, App Integration & more
>> Get technology previously reserved for billion-dollar corporations, FREE
>> http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk <http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk>
>> _______________________________________________
>> Check-users mailing list
>> Che...@li... <mailto:Che...@li...>
>> https://lists.sourceforge.net/lists/listinfo/check-users <https://lists.sourceforge.net/lists/listinfo/check-users>
>> 
>> 
> 


Hi Branden,

That’s really interesting. I wonder if it has to do with my use of atexit or pthread_exit. You can see how I perform some cleanup / teardown at the top of Class.c. I’ll try setting some breakpoints in there to see if that’s influencing check’s behavior. Maybe check is hitting that code path at the end of every test case, and things are going south?

Thanks again,

Jay Dolan
508-415-1866

> On Nov 24, 2014, at 9:24 AM, Branden Archer <b.m...@gm...> wrote:
> 
> I was able to reproduce the same behavior. Several of the test programs appear to sporadically hang during execution and are eventually aborted due to a timeout.
> 
> One of them, the Objects program, was taken as an example. Running it in both fork and no-fork mode it occasionally hung. Adding some printf statements throughout the test showed that the test did reach the end before it started to hang. More interesting what the observation that once all of the real content of the unit test was commented out, leaving the printf statements, no further hangs were observed.
> 
> At that point, I become suspicious that some sort of memory corruption was occurring during the unit test run and ran valgrind:
> 
> $ CK_FORK=no valgrind --leak-check=full Tests/Objectively/Object
> 
> However, nothing conclusive was detected. Though, I did have to compile the trunk of valgrind to get it to work on OS X 10.10, so maybe it is not ready, not sure.
> 
> Interesting, though, when run in CK_FORK=no mode, the test program eventually does exit, though it may take 10 seconds:
> 
> $ for i in `seq 1 10`; do (time CK_FORK=no Tests/Objectively/Object) 2>&1 | grep real; done
> real	0m6.661s
> real	0m0.007s
> real	0m0.004s
> real	0m0.003s
> real	0m0.003s
> real	0m0.003s
> real	0m0.003s
> real	0m9.987s
> real	0m0.023s
> real	0m0.005s
> 
> Also interesting is that the hang occurs after the results are printed. That leads me to believe that there is another thread somewhere still running. During a hang in CK_FORK=no mode I attached lldb and printed out the stack trace of all current threads. This is the only thread running:
> 
> (lldb) thread backtrace
> * thread #1: tid = 0x4a07f, 0x00007fff8fcfa946 libsystem_kernel.dylib`__workq_kernreturn + 10, stop reason = signal SIGSTOP
>   * frame #0: 0x00007fff8fcfa946 libsystem_kernel.dylib`__workq_kernreturn + 10
>     frame #1: 0x00007fff8f761757 libsystem_pthread.dylib`_pthread_wqthread + 869
>     frame #2: 0x00007fff8f75f4a1 libsystem_pthread.dylib`start_wqthread + 13
> 
> which does not make much sense to me.
> 
> As a final move, I commented out everything in the Object test except for the initial Object allocation. With that the hang would still happen. However, if even that is commented out the hang does not occur.
> 
> So, in short, I've never seen behavior before where a unit test program will not return after its tests have completed Also I've no idea why it is happening in this case.
> 
> If you do get to the bottom of it, I would be curious to hear what the underlying issue was. (:
> 
> - Branden
> 
> 
> On Sun, Nov 23, 2014 at 11:23 PM, Jay Dolan <jay...@gm... <mailto:jay...@gm...>> wrote:
> Hi Branden,
> 
> Thanks for looking at the issue. That error is certainly valid, thanks for pointing that out. The “Hello” binary that’s failing to compile / link is only a demonstration program. If you’re willing to run `make install`, the quickest way forward would be to adjust Tests/Makefile.am and comment out noinst_PROGRAMS so that Hello is skipped. Then run `make install`. Then `make check`. Afterwards, running `make uninstall` will remove Objectively from your system.
> 
> Sorry the build is a little ugly right now. It’s in flux. And thanks again!
>   
> Jay Dolan
> 508-415-1866 <tel:508-415-1866>
> 
>> On Nov 23, 2014, at 10:56 PM, Branden Archer <b.m...@gm... <mailto:b.m...@gm...>> wrote:
>> 
>> In trying to build the project and reproduce the problems I'm running into some issues.
>> 
>> My system is using the same versions of everything (exception that OS X is 10.10). The necessary tools are from brew.sh instead of Macports. The following steps:
>> 
>> $ autoreconf -i
>> $ ./configure
>> $ make
>> 
>> leads to the following error:
>> 
>> 
>> Brandens-MacBook-Pro:objectify.git brarcher$ make
>> /Library/Developer/CommandLineTools/usr/bin/make  all-recursive
>> Making all in Sources
>> Making all in Objectively
>>   CC       libObjectively_la-Array.lo
>>   CC       libObjectively_la-Class.lo
>>   CC       libObjectively_la-Condition.lo
>>   CC       libObjectively_la-Date.lo
>>   CC       libObjectively_la-DateFormatter.lo
>>   CC       libObjectively_la-Dictionary.lo
>>   CC       libObjectively_la-Lock.lo
>>   CC       libObjectively_la-Log.lo
>>   CC       libObjectively_la-Object.lo
>>   CC       libObjectively_la-String.lo
>>   CC       libObjectively_la-Thread.lo
>>   CCLD     libObjectively.la
>> make[3]: Nothing to be done for `all-am'.
>> Making all in Tests
>> Making all in Objectively
>>   CC       Hello.o
>> Hello.c:5:10: fatal error: 'Objectively.h' file not found
>> #include <Objectively.h>
>>          ^
>> 1 error generated.
>> make[3]: *** [Hello.o] Error 1
>> make[2]: *** [all-recursive] Error 1
>> make[1]: *** [all-recursive] Error 1
>> make: *** [all] Error 2
>> 
>> 
>> It seems that the include path for the unit tests does not include the header from the source. Attempting to hack that in:
>> 
>> 
>> diff --git a/Tests/Objectively/Makefile.am b/Tests/Objectively/Makefile.am
>> index b32458d..fb9e2ea 100644
>> --- a/Tests/Objectively/Makefile.am
>> +++ b/Tests/Objectively/Makefile.am
>> @@ -10,6 +10,7 @@ TESTS = \
>>  
>>  CFLAGS += \
>>         -I$(includedir) \
>> +       -I../../Sources \
>>         @CHECK_CFLAGS@
>>  
>>  LDADD = \
>> 
>> 
>> Leads to a failure when looking for the Objectively library:
>> 
>> 
>> Brandens-MacBook-Pro:objectify.git brarcher$ make
>> /Library/Developer/CommandLineTools/usr/bin/make  all-recursive
>> Making all in Sources
>> Making all in Objectively
>> make[3]: Nothing to be done for `all'.
>> make[3]: Nothing to be done for `all-am'.
>> Making all in Tests
>> Making all in Objectively
>>   CCLD     Hello
>> ld: library not found for -lObjectively
>> clang: error: linker command failed with exit code 1 (use -v to see invocation)
>> make[3]: *** [Hello] Error 1
>> make[2]: *** [all-recursive] Error 1
>> make[1]: *** [all-recursive] Error 1
>> make: *** [all] Error 2
>> 
>> 
>> How does one build the project such that the unit tests build against the Objectively from the source tree? Did I miss a few steps somewhere?
>> 
>> - Branden
>> 
>> 
>> 
>> On Fri, Nov 21, 2014 at 10:30 AM, Jay Dolan <jay...@gm... <mailto:jay...@gm...>> wrote:
>> I’ve seen some _really_ inconsistent behavior from `make check` and I’m wondering if I’m alone.
>> 
>> Sometimes the test runner will indicate that all tests are passing despite the individual test logs clearly calling out errors.
>> Sometimes the test runner will hang up for seemingly no reason at all, causing the tests to timeout and thus fail.
>> Setting the CK_FORK environment variable to `no` seems to help where it really shouldn’t matter.
>> 
>> My environment is:
>> 
>> OS X 10.10.1
>> Apple LLVM 6.0 / Clang (latest Xcode tool chain)
>> Automake 1.14.1 via Macports
>> Check 0.9.14 via Macports.
>> 
>> The project I’m compiling and seeing these issues with is Objectively:
>> https://github.com/jdolan/objectively <https://github.com/jdolan/objectively>
>> 
>> All but one of my test cases are single-threaded, and I can easily reproduce these issues while omitting that particular test case. The test sources are here:
>> https://github.com/jdolan/objectively/tree/master/Tests/Objectively <https://github.com/jdolan/objectively/tree/master/Tests/Objectively>
>> 
>> Am I Doing It Wrong(tm), or is there something more to this?
>> 
>> Jay
>> 
>> Jay Dolan
>> 508-415-1866 <tel:508-415-1866>
>> 
>> ------------------------------------------------------------------------------
>> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
>> from Actuate! Instantly Supercharge Your Business Reports and Dashboards
>> with Interactivity, Sharing, Native Excel Exports, App Integration & more
>> Get technology previously reserved for billion-dollar corporations, FREE
>> http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk <http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk>
>> _______________________________________________
>> Check-users mailing list
>> Che...@li... <mailto:Che...@li...>
>> https://lists.sourceforge.net/lists/listinfo/check-users <https://lists.sourceforge.net/lists/listinfo/check-users>
>> 
>> 
> 
> 
> ------------------------------------------------------------------------------
> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
> from Actuate! Instantly Supercharge Your Business Reports and Dashboards
> with Interactivity, Sharing, Native Excel Exports, App Integration & more
> Get technology previously reserved for billion-dollar corporations, FREE
> http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk <http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk>
> _______________________________________________
> Check-users mailing list
> Che...@li... <mailto:Che...@li...>
> https://lists.sourceforge.net/lists/listinfo/check-users <https://lists.sourceforge.net/lists/listinfo/check-users>
> 
> 


I was able to reproduce the same behavior. Several of the test programs
appear to sporadically hang during execution and are eventually aborted due
to a timeout.

One of them, the Objects program, was taken as an example. Running it in
both fork and no-fork mode it occasionally hung. Adding some printf
statements throughout the test showed that the test did reach the end
before it started to hang. More interesting what the observation that once
all of the real content of the unit test was commented out, leaving the
printf statements, no further hangs were observed.

At that point, I become suspicious that some sort of memory corruption was
occurring during the unit test run and ran valgrind:

$ CK_FORK=no valgrind --leak-check=full Tests/Objectively/Object

However, nothing conclusive was detected. Though, I did have to compile the
trunk of valgrind to get it to work on OS X 10.10, so maybe it is not
ready, not sure.

Interesting, though, when run in CK_FORK=no mode, the test program
eventually does exit, though it may take 10 seconds:

$ for i in `seq 1 10`; do (time CK_FORK=no Tests/Objectively/Object) 2>&1 |
grep real; done

real *0m6.661s*

real 0m0.007s

real 0m0.004s

real 0m0.003s

real 0m0.003s

real 0m0.003s

real 0m0.003s

real *0m9.987s*

real 0m0.023s

real 0m0.005s

Also interesting is that the hang occurs after the results are printed.
That leads me to believe that there is another thread somewhere still
running. During a hang in CK_FORK=no mode I attached lldb and printed out
the stack trace of all current threads. This is the only thread running:

(lldb) thread backtrace

* thread #1: tid = 0x4a07f, 0x00007fff8fcfa946
libsystem_kernel.dylib`__workq_kernreturn + 10, stop reason = signal SIGSTOP

  * frame #0: 0x00007fff8fcfa946 libsystem_kernel.dylib`__workq_kernreturn
+ 10

    frame #1: 0x00007fff8f761757 libsystem_pthread.dylib`_pthread_wqthread
+ 869

    frame #2: 0x00007fff8f75f4a1 libsystem_pthread.dylib`start_wqthread + 13

which does not make much sense to me.

As a final move, I commented out everything in the Object test except for
the initial Object allocation. With that the hang would still happen.
However, if even that is commented out the hang does not occur.

So, in short, I've never seen behavior before where a unit test program
will not return after its tests have completed Also I've no idea why it is
happening in this case.

If you do get to the bottom of it, I would be curious to hear what the
underlying issue was. (:

- Branden


On Sun, Nov 23, 2014 at 11:23 PM, Jay Dolan <jay...@gm...> wrote:

> Hi Branden,
>
> Thanks for looking at the issue. That error is certainly valid, thanks for
> pointing that out. The “Hello” binary that’s failing to compile / link is
> only a demonstration program. If you’re willing to run `make install`, the
> quickest way forward would be to adjust Tests/Makefile.am and comment out
> noinst_PROGRAMS so that Hello is skipped. Then run `make install`. Then
> `make check`. Afterwards, running `make uninstall` will remove Objectively
> from your system.
>
> Sorry the build is a little ugly right now. It’s in flux. And thanks again!
>
> Jay Dolan
> 508-415-1866
>
> On Nov 23, 2014, at 10:56 PM, Branden Archer <b.m...@gm...>
> wrote:
>
> In trying to build the project and reproduce the problems I'm running into
> some issues.
>
> My system is using the same versions of everything (exception that OS X is
> 10.10). The necessary tools are from brew.sh instead of Macports. The
> following steps:
>
> $ autoreconf -i
> $ ./configure
> $ make
>
> leads to the following error:
>
>
> Brandens-MacBook-Pro:objectify.git brarcher$ make
> /Library/Developer/CommandLineTools/usr/bin/make  all-recursive
> Making all in Sources
> Making all in Objectively
>   CC       libObjectively_la-Array.lo
>   CC       libObjectively_la-Class.lo
>   CC       libObjectively_la-Condition.lo
>   CC       libObjectively_la-Date.lo
>   CC       libObjectively_la-DateFormatter.lo
>   CC       libObjectively_la-Dictionary.lo
>   CC       libObjectively_la-Lock.lo
>   CC       libObjectively_la-Log.lo
>   CC       libObjectively_la-Object.lo
>   CC       libObjectively_la-String.lo
>   CC       libObjectively_la-Thread.lo
>   CCLD     libObjectively.la
> make[3]: Nothing to be done for `all-am'.
> Making all in Tests
> Making all in Objectively
>   CC       Hello.o
> *Hello.c:5:10: **fatal error: **'Objectively.h' file not found*
> #include <Objectively.h>
> *         ^*
> 1 error generated.
> make[3]: *** [Hello.o] Error 1
> make[2]: *** [all-recursive] Error 1
> make[1]: *** [all-recursive] Error 1
> make: *** [all] Error 2
>
>
> It seems that the include path for the unit tests does not include the
> header from the source. Attempting to hack that in:
>
>
> *diff --git a/Tests/Objectively/Makefile.am
> b/Tests/Objectively/Makefile.am*
> *index b32458d..fb9e2ea 100644*
> *--- a/Tests/Objectively/Makefile.am*
> *+++ b/Tests/Objectively/Makefile.am*
> @@ -10,6 +10,7 @@ TESTS = \
>
>
>  CFLAGS += \
>         -I$(includedir) \
> +       -I../../Sources \
>         @CHECK_CFLAGS@
>
>
>  LDADD = \
>
>
> Leads to a failure when looking for the Objectively library:
>
>
> Brandens-MacBook-Pro:objectify.git brarcher$ make
> /Library/Developer/CommandLineTools/usr/bin/make  all-recursive
> Making all in Sources
> Making all in Objectively
> make[3]: Nothing to be done for `all'.
> make[3]: Nothing to be done for `all-am'.
> Making all in Tests
> Making all in Objectively
>   CCLD     Hello
> ld: library not found for -lObjectively
> clang: error: linker command failed with exit code 1 (use -v to see
> invocation)
> make[3]: *** [Hello] Error 1
> make[2]: *** [all-recursive] Error 1
> make[1]: *** [all-recursive] Error 1
> make: *** [all] Error 2
>
>
> How does one build the project such that the unit tests build against the
> Objectively from the source tree? Did I miss a few steps somewhere?
>
> - Branden
>
>
>
> On Fri, Nov 21, 2014 at 10:30 AM, Jay Dolan <jay...@gm...> wrote:
>
>> I’ve seen some _really_ inconsistent behavior from `make check` and I’m
>> wondering if I’m alone.
>>
>>
>>    - Sometimes the test runner will indicate that all tests are passing
>>    despite the individual test logs clearly calling out errors.
>>    - Sometimes the test runner will hang up for seemingly no reason at
>>    all, causing the tests to timeout and thus fail.
>>    - Setting the CK_FORK environment variable to `no` seems to help
>>    where it really shouldn’t matter.
>>
>>
>> My environment is:
>>
>> OS X 10.10.1
>> Apple LLVM 6.0 / Clang (latest Xcode tool chain)
>> Automake 1.14.1 via Macports
>> Check 0.9.14 via Macports.
>>
>> The project I’m compiling and seeing these issues with is Objectively:
>> https://github.com/jdolan/objectively
>>
>> All but one of my test cases are single-threaded, and I can easily
>> reproduce these issues while omitting that particular test case. The test
>> sources are here:
>> https://github.com/jdolan/objectively/tree/master/Tests/Objectively
>>
>> Am I Doing It Wrong(tm), or is there something more to this?
>>
>> Jay
>>
>> Jay Dolan
>> 508-415-1866
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
>> from Actuate! Instantly Supercharge Your Business Reports and Dashboards
>> with Interactivity, Sharing, Native Excel Exports, App Integration & more
>> Get technology previously reserved for billion-dollar corporations, FREE
>>
>> http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
>> _______________________________________________
>> Check-users mailing list
>> Che...@li...
>> https://lists.sourceforge.net/lists/listinfo/check-users
>>
>>
>
>
>
> ------------------------------------------------------------------------------
> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
> from Actuate! Instantly Supercharge Your Business Reports and Dashboards
> with Interactivity, Sharing, Native Excel Exports, App Integration & more
> Get technology previously reserved for billion-dollar corporations, FREE
>
> http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
> _______________________________________________
> Check-users mailing list
> Che...@li...
> https://lists.sourceforge.net/lists/listinfo/check-users
>
>

Hi Branden,

Thanks for looking at the issue. That error is certainly valid, thanks for pointing that out. The “Hello” binary that’s failing to compile / link is only a demonstration program. If you’re willing to run `make install`, the quickest way forward would be to adjust Tests/Makefile.am and comment out noinst_PROGRAMS so that Hello is skipped. Then run `make install`. Then `make check`. Afterwards, running `make uninstall` will remove Objectively from your system.

Sorry the build is a little ugly right now. It’s in flux. And thanks again!
  
Jay Dolan
508-415-1866

> On Nov 23, 2014, at 10:56 PM, Branden Archer <b.m...@gm...> wrote:
> 
> In trying to build the project and reproduce the problems I'm running into some issues.
> 
> My system is using the same versions of everything (exception that OS X is 10.10). The necessary tools are from brew.sh instead of Macports. The following steps:
> 
> $ autoreconf -i
> $ ./configure
> $ make
> 
> leads to the following error:
> 
> 
> Brandens-MacBook-Pro:objectify.git brarcher$ make
> /Library/Developer/CommandLineTools/usr/bin/make  all-recursive
> Making all in Sources
> Making all in Objectively
>   CC       libObjectively_la-Array.lo
>   CC       libObjectively_la-Class.lo
>   CC       libObjectively_la-Condition.lo
>   CC       libObjectively_la-Date.lo
>   CC       libObjectively_la-DateFormatter.lo
>   CC       libObjectively_la-Dictionary.lo
>   CC       libObjectively_la-Lock.lo
>   CC       libObjectively_la-Log.lo
>   CC       libObjectively_la-Object.lo
>   CC       libObjectively_la-String.lo
>   CC       libObjectively_la-Thread.lo
>   CCLD     libObjectively.la
> make[3]: Nothing to be done for `all-am'.
> Making all in Tests
> Making all in Objectively
>   CC       Hello.o
> Hello.c:5:10: fatal error: 'Objectively.h' file not found
> #include <Objectively.h>
>          ^
> 1 error generated.
> make[3]: *** [Hello.o] Error 1
> make[2]: *** [all-recursive] Error 1
> make[1]: *** [all-recursive] Error 1
> make: *** [all] Error 2
> 
> 
> It seems that the include path for the unit tests does not include the header from the source. Attempting to hack that in:
> 
> 
> diff --git a/Tests/Objectively/Makefile.am b/Tests/Objectively/Makefile.am
> index b32458d..fb9e2ea 100644
> --- a/Tests/Objectively/Makefile.am
> +++ b/Tests/Objectively/Makefile.am
> @@ -10,6 +10,7 @@ TESTS = \
>  
>  CFLAGS += \
>         -I$(includedir) \
> +       -I../../Sources \
>         @CHECK_CFLAGS@
>  
>  LDADD = \
> 
> 
> Leads to a failure when looking for the Objectively library:
> 
> 
> Brandens-MacBook-Pro:objectify.git brarcher$ make
> /Library/Developer/CommandLineTools/usr/bin/make  all-recursive
> Making all in Sources
> Making all in Objectively
> make[3]: Nothing to be done for `all'.
> make[3]: Nothing to be done for `all-am'.
> Making all in Tests
> Making all in Objectively
>   CCLD     Hello
> ld: library not found for -lObjectively
> clang: error: linker command failed with exit code 1 (use -v to see invocation)
> make[3]: *** [Hello] Error 1
> make[2]: *** [all-recursive] Error 1
> make[1]: *** [all-recursive] Error 1
> make: *** [all] Error 2
> 
> 
> How does one build the project such that the unit tests build against the Objectively from the source tree? Did I miss a few steps somewhere?
> 
> - Branden
> 
> 
> 
> On Fri, Nov 21, 2014 at 10:30 AM, Jay Dolan <jay...@gm... <mailto:jay...@gm...>> wrote:
> I’ve seen some _really_ inconsistent behavior from `make check` and I’m wondering if I’m alone.
> 
> Sometimes the test runner will indicate that all tests are passing despite the individual test logs clearly calling out errors.
> Sometimes the test runner will hang up for seemingly no reason at all, causing the tests to timeout and thus fail.
> Setting the CK_FORK environment variable to `no` seems to help where it really shouldn’t matter.
> 
> My environment is:
> 
> OS X 10.10.1
> Apple LLVM 6.0 / Clang (latest Xcode tool chain)
> Automake 1.14.1 via Macports
> Check 0.9.14 via Macports.
> 
> The project I’m compiling and seeing these issues with is Objectively:
> https://github.com/jdolan/objectively <https://github.com/jdolan/objectively>
> 
> All but one of my test cases are single-threaded, and I can easily reproduce these issues while omitting that particular test case. The test sources are here:
> https://github.com/jdolan/objectively/tree/master/Tests/Objectively <https://github.com/jdolan/objectively/tree/master/Tests/Objectively>
> 
> Am I Doing It Wrong(tm), or is there something more to this?
> 
> Jay
> 
> Jay Dolan
> 508-415-1866 <tel:508-415-1866>
> 
> ------------------------------------------------------------------------------
> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
> from Actuate! Instantly Supercharge Your Business Reports and Dashboards
> with Interactivity, Sharing, Native Excel Exports, App Integration & more
> Get technology previously reserved for billion-dollar corporations, FREE
> http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk <http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk>
> _______________________________________________
> Check-users mailing list
> Che...@li... <mailto:Che...@li...>
> https://lists.sourceforge.net/lists/listinfo/check-users <https://lists.sourceforge.net/lists/listinfo/check-users>
> 
> 


In trying to build the project and reproduce the problems I'm running into
some issues.

My system is using the same versions of everything (exception that OS X is
10.10). The necessary tools are from brew.sh instead of Macports. The
following steps:

$ autoreconf -i
$ ./configure
$ make

leads to the following error:


Brandens-MacBook-Pro:objectify.git brarcher$ make

/Library/Developer/CommandLineTools/usr/bin/make  all-recursive

Making all in Sources

Making all in Objectively

  CC       libObjectively_la-Array.lo

  CC       libObjectively_la-Class.lo

  CC       libObjectively_la-Condition.lo

  CC       libObjectively_la-Date.lo

  CC       libObjectively_la-DateFormatter.lo

  CC       libObjectively_la-Dictionary.lo

  CC       libObjectively_la-Lock.lo

  CC       libObjectively_la-Log.lo

  CC       libObjectively_la-Object.lo

  CC       libObjectively_la-String.lo

  CC       libObjectively_la-Thread.lo

  CCLD     libObjectively.la

make[3]: Nothing to be done for `all-am'.

Making all in Tests

Making all in Objectively

  CC       Hello.o

*Hello.c:5:10: **fatal error: **'Objectively.h' file not found*

#include <Objectively.h>

*         ^*

1 error generated.

make[3]: *** [Hello.o] Error 1

make[2]: *** [all-recursive] Error 1

make[1]: *** [all-recursive] Error 1

make: *** [all] Error 2


It seems that the include path for the unit tests does not include the
header from the source. Attempting to hack that in:


*diff --git a/Tests/Objectively/Makefile.am b/Tests/Objectively/Makefile.am*

*index b32458d..fb9e2ea 100644*

*--- a/Tests/Objectively/Makefile.am*

*+++ b/Tests/Objectively/Makefile.am*

@@ -10,6 +10,7 @@ TESTS = \



 CFLAGS += \

        -I$(includedir) \

+       -I../../Sources \

        @CHECK_CFLAGS@



 LDADD = \


Leads to a failure when looking for the Objectively library:


Brandens-MacBook-Pro:objectify.git brarcher$ make

/Library/Developer/CommandLineTools/usr/bin/make  all-recursive

Making all in Sources

Making all in Objectively

make[3]: Nothing to be done for `all'.

make[3]: Nothing to be done for `all-am'.

Making all in Tests

Making all in Objectively

  CCLD     Hello

ld: library not found for -lObjectively

clang: error: linker command failed with exit code 1 (use -v to see
invocation)

make[3]: *** [Hello] Error 1

make[2]: *** [all-recursive] Error 1

make[1]: *** [all-recursive] Error 1

make: *** [all] Error 2


How does one build the project such that the unit tests build against the
Objectively from the source tree? Did I miss a few steps somewhere?

- Branden



On Fri, Nov 21, 2014 at 10:30 AM, Jay Dolan <jay...@gm...> wrote:

> I’ve seen some _really_ inconsistent behavior from `make check` and I’m
> wondering if I’m alone.
>
>
>    - Sometimes the test runner will indicate that all tests are passing
>    despite the individual test logs clearly calling out errors.
>    - Sometimes the test runner will hang up for seemingly no reason at
>    all, causing the tests to timeout and thus fail.
>    - Setting the CK_FORK environment variable to `no` seems to help where
>    it really shouldn’t matter.
>
>
> My environment is:
>
> OS X 10.10.1
> Apple LLVM 6.0 / Clang (latest Xcode tool chain)
> Automake 1.14.1 via Macports
> Check 0.9.14 via Macports.
>
> The project I’m compiling and seeing these issues with is Objectively:
> https://github.com/jdolan/objectively
>
> All but one of my test cases are single-threaded, and I can easily
> reproduce these issues while omitting that particular test case. The test
> sources are here:
> https://github.com/jdolan/objectively/tree/master/Tests/Objectively
>
> Am I Doing It Wrong(tm), or is there something more to this?
>
> Jay
>
> Jay Dolan
> 508-415-1866
>
>
>
> ------------------------------------------------------------------------------
> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
> from Actuate! Instantly Supercharge Your Business Reports and Dashboards
> with Interactivity, Sharing, Native Excel Exports, App Integration & more
> Get technology previously reserved for billion-dollar corporations, FREE
>
> http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
> _______________________________________________
> Check-users mailing list
> Che...@li...
> https://lists.sourceforge.net/lists/listinfo/check-users
>
>

I’ve seen some _really_ inconsistent behavior from `make check` and I’m wondering if I’m alone.

Sometimes the test runner will indicate that all tests are passing despite the individual test logs clearly calling out errors.
Sometimes the test runner will hang up for seemingly no reason at all, causing the tests to timeout and thus fail.
Setting the CK_FORK environment variable to `no` seems to help where it really shouldn’t matter.

My environment is:

OS X 10.10.1
Apple LLVM 6.0 / Clang (latest Xcode tool chain)
Automake 1.14.1 via Macports
Check 0.9.14 via Macports.

The project I’m compiling and seeing these issues with is Objectively:
https://github.com/jdolan/objectively <https://github.com/jdolan/objectively>

All but one of my test cases are single-threaded, and I can easily reproduce these issues while omitting that particular test case. The test sources are here:
https://github.com/jdolan/objectively/tree/master/Tests/Objectively <https://github.com/jdolan/objectively/tree/master/Tests/Objectively>

Am I Doing It Wrong(tm), or is there something more to this?

Jay

Jay Dolan
508-415-1866


Hi,

Firstly, I'm sorry for the late response.

On Thu, Oct 02, 2014 at 12:10:02AM -0400, Branden Archer wrote:
> Thanks for your interest in Check! Taking a look at your README file in the
> project, what you propose looks rather promising. I'm not all too familiar with
> checkmk, as I've only ever used Check directly in projects. However, the
> simplicity of writing tests using your proposed format seems easy. Are you
> using this actively in any of your other projects?

Yes, I use it in a pet project but it is not yet public.

>
> How robust is the converting of a .ts file to a .c file? Specifically, is one
> able to add functions, includes, or comments to a file which are not part of a
> unit test?

Really robust, for now, in my pet project, no conversion problem was
found. For functions, includes or comments, user code can simply use #global
directive (not really usefull for comments because resulting .c file is
not aimed to be read, there is many #line sync).

>
> It might be beneficial to have tests which accompany the code. Not only would
> they verify for various sample inputs that the output is as expected, but would
> provide for some good non-trivial examples of how the system works.
>

Absolutely, I plan to do that. Probably during the next step in my pet project.


Beside checkgen, I'm thinking about the possibility to generate a test
runner by using only the C preprocessor. Probably with XMACRO
(http://en.wikipedia.org/wiki/X_Macro). I have nothing to show for now
but I'm prototyping that in my head these days. This can be interresting
to get rid of the AWK dependency. What do you think about ?

--
xakz



I would not expect anything on the system to cause a conflict with finding check using pkg-config.

噬血龙 <798...@qq...> wrote:

>dear：
>
>      I install the “CUnit” on my system。does it confilict with the Check?
>
>
>
>------------------ 原始邮件 ------------------
>
>发件人: "Branden Archer";<b.m...@gm...>;
>
>发送时间: 2014年10月2日(星期四) 中午11:32
>
>收件人: "噬血龙"<798...@qq...>; 
>
>抄送: "che...@li..."<che...@li...>; 
>
>主题: Re: "make check" step fails
>
>
>Kindly also CC the check-users mailing list as well, so if others run into the same issue they can reference the discussion or post possible answers.
>
>
>I'm not too familiar with pkg-config or why the setup is failing in your case. Is pkg-config able to find information on other packages in the directory containing package information on your system?
>
>
>- Branden
>
>
>On Wed, Sep 24, 2014 at 9:58 AM, 噬血龙 <798...@qq...> wrote:
>
>dear:
>
>      [root@embedclub ~]# pkg-config check --modversion
>Package check was not found in the pkg-config search path.
>Perhaps you should add the directory containing `check.pc'
>to the PKG_CONFIG_PATH environment variable
>No package 'check' found the directory containing `check.pc'
>to the PKG_CONFIG_PATH environment variable,I tried faily again ,why?
>
> 
>
>After adding 
>
>
>
>------------------ 原始邮件 ------------------
>
>发件人: "Branden Archer";<b.m...@gm...>;
>
>发送时间: 2014年9月23日(星期二) 上午9:30
>
>收件人: "噬血龙"<798...@qq...>; 
>
>抄送: "che...@li..."<che...@li...>; 
>
>主题: Re: "make check" step fails
>
>
>Are you attempting to compile the example project in Check's doc/example folder in its release, and hitting this error?
>
>
>The following line is interesting: 
>
>
>./configure: line 2180: AM_PROG_AR: command not found
>
>
>Google-ing that, I find a bug report in 2012 that mentions that AM_PROG_AR was introduced in Automake 1.11.2. The check for Automake in Check's own configure script and the configure script in the example seems to be incorrect:
>
>
>AM_INIT_AUTOMAKE([-Wall -Werror foreign 1.9.6])
>
>
>I've updated this in Check, so in the next release it will check for the correct version  Either way, your version of Automake may be too low. What version of Automake are you using, out of curiosity?
>
>
>Additionally, in regards to the failure to find Check on the system, what version of Check is installed on your system?
>
>
>$ pkg-config check --modversion
>
>0.9.12
>
>
>Is the version at least 0.9.6? If not, you may need to install a more up to date version then attempt to compile the example again.
>
>
>- Branden
>
>
>
>On Sat, Sep 20, 2014 at 9:46 AM, 噬血龙 <798...@qq...> wrote:
>
>dear:
>
>       "make check" step fails
>
>     details:
>
> 
>
>
>
pkg-config check --modversion
Package check was not found in the pkg-config search path.
Perhaps you should add the directory containing `check.pc'
to the PKG_CONFIG_PATH environment variable
No package 'check' foun
dear：
      I install the “CUnit” on my system。does it confilict with the Check?




------------------ 原始邮件 ------------------
发件人: "Branden Archer";<b.m...@gm...>;
发送时间: 2014年10月2日(星期四) 中午11:32
收件人: "噬血龙"<798...@qq...>; 
抄送: "che...@li..."<che...@li...>; 
主题: Re: "make check" step fails



Kindly also CC the check-users mailing list as well, so if others run into the same issue they can reference the discussion or post possible answers.

I'm not too familiar with pkg-config or why the setup is failing in your case. Is pkg-config able to find information on other packages in the directory containing package information on your system?


- Branden


On Wed, Sep 24, 2014 at 9:58 AM, 噬血龙 <798...@qq...> wrote:
dear:
       [root@embedclub ~]# pkg-config check --modversion
Package check was not found in the pkg-config search path.
Perhaps you should add the directory containing `check.pc'
to the PKG_CONFIG_PATH environment variable
No package 'check' found the directory containing `check.pc'
to the PKG_CONFIG_PATH environment variable,I tried faily again ,why?
  
 After adding 
  

 

 ------------------ 原始邮件 ------------------
  发件人: "Branden Archer";<b.m...@gm...>;
 发送时间: 2014年9月23日(星期二) 上午9:30
 收件人: "噬血龙"<798...@qq...>; 
 抄送: "che...@li..."<che...@li...>; 
 主题: Re: "make check" step fails

 

  Are you attempting to compile the example project in Check's doc/example folder in its release, and hitting this error?
 

The following line is interesting: 

 ./configure: line 2180: AM_PROG_AR: command not found 

 Google-ing that, I find a bug report in 2012 that mentions that AM_PROG_AR was introduced in Automake 1.11.2. The check for Automake in Check's own configure script and the configure script in the example seems to be incorrect:
 

 AM_INIT_AUTOMAKE([-Wall -Werror foreign 1.9.6])
 

 I've updated this in Check, so in the next release it will check for the correct version  Either way, your version of Automake may be too low. What version of Automake are you using, out of curiosity?
 

 Additionally, in regards to the failure to find Check on the system, what version of Check is installed on your system?
 

 $ pkg-config check --modversion
 0.9.12
 

 Is the version at least 0.9.6? If not, you may need to install a more up to date version then attempt to compile the example again.
 

 - Branden
 


 
 On Sat, Sep 20, 2014 at 9:46 AM, 噬血龙 <798...@qq...> wrote:
    dear:
        "make check" step fails
      details:
Thanks for your interest in Check! Taking a look at your README file in the
project, what you propose looks rather promising. I'm not all too familiar
with checkmk, as I've only ever used Check directly in projects. However,
the simplicity of writing tests using your proposed format seems easy. Are
you using this actively in any of your other projects?

How robust is the converting of a .ts file to a .c file? Specifically, is
one able to add functions, includes, or comments to a file which are not
part of a unit test?

It might be beneficial to have tests which accompany the code. Not only
would they verify for various sample inputs that the output is as expected,
but would provide for some good non-trivial examples of how the system
works.

- Branden

On Sat, Sep 20, 2014 at 5:19 AM, xakz <xa...@gm...> wrote:

> Hi,
>
> I wish to present to you a test runner generator that I just developed.
>
> I needed a better approach to fixtures and test naming. So, firstly, I
> wanted extend checkmk and send a patch but the code is hard to extend
> without rewriting big parts. Thus, I rewritten an almost compatible tool
> from scratch that handles fixtures and test names like I want.
>
> If you want test it, it can be downloaded at:
> https://github.com/xakz/checkgen
>
> The full documentation is included, including compatibility details.
>
> Any return or suggestion are greatly appreciated.
>
> --
> Maxime Chatelle
>
>
> ------------------------------------------------------------------------------
> Slashdot TV.  Video for Nerds.  Stuff that Matters.
>
> http://pubads.g.doubleclick.net/gampad/clk?id=160591471&iu=/4140/ostg.clktrk
> _______________________________________________
> Check-users mailing list
> Che...@li...
> https://lists.sourceforge.net/lists/listinfo/check-users
>

Kindly also CC the check-users mailing list as well, so if others run into
the same issue they can reference the discussion or post possible answers.

I'm not too familiar with pkg-config or why the setup is failing in your
case. Is pkg-config able to find information on other packages in the
directory containing package information on your system?

- Branden

On Wed, Sep 24, 2014 at 9:58 AM, 噬血龙 <798...@qq...> wrote:

> dear:
>       [root@embedclub ~]# pkg-config check --modversion
> Package check was not found in the pkg-config search path.
> Perhaps you should add the directory containing `check.pc'
> to the PKG_CONFIG_PATH environment variable
> No package 'check' found the directory containing `check.pc'
> to the PKG_CONFIG_PATH environment variable,I tried faily again ,why?
>
> After adding
>
>
> ------------------ 原始邮件 ------------------
>  *发件人:* "Branden Archer";<b.m...@gm...>;
> *发送时间:* 2014年9月23日(星期二) 上午9:30
> *收件人:* "噬血龙"<798...@qq...>;
> *抄送:* "che...@li..."<
> che...@li...>;
> *主题:* Re: "make check" step fails
>
>  Are you attempting to compile the example project in Check's doc/example
> folder in its release, and hitting this error?
>
> The following line is interesting:
>
> ./configure: line 2180: AM_PROG_AR: command not found
>
>
> Google-ing that, I find a bug report
> <http://lists.gnu.org/archive/html/bug-guile/2012-12/msg00038.html> in
> 2012 that mentions that AM_PROG_AR was introduced in Automake 1.11.2. The
> check for Automake in Check's own configure script and the configure script
> in the example seems to be incorrect:
>
> AM_INIT_AUTOMAKE([-Wall -Werror foreign 1.9.6])
>
> I've updated this in Check, so in the next release it will check for the
> correct version  Either way, your version of Automake may be too low.
> What version of Automake are you using, out of curiosity?
>
> Additionally, in regards to the failure to find Check on the system,
> what version of Check is installed on your system?
>
> $ pkg-config check --modversion
> 0.9.12
>
> Is the version at least 0.9.6? If not, you may need to install a more up
> to date version then attempt to compile the example again.
>
> - Branden
>
>
> On Sat, Sep 20, 2014 at 9:46 AM, 噬血龙 <798...@qq...> wrote:
>
>>  dear:
>>        "make check" step fails
>>      details:
>>
>>
>
>

Are you attempting to compile the example project in Check's doc/example
folder in its release, and hitting this error?

The following line is interesting:

./configure: line 2180: AM_PROG_AR: command not found


Google-ing that, I find a bug report
<http://lists.gnu.org/archive/html/bug-guile/2012-12/msg00038.html> in 2012
that mentions that AM_PROG_AR was introduced in Automake 1.11.2. The check
for Automake in Check's own configure script and the configure script in
the example seems to be incorrect:

AM_INIT_AUTOMAKE([-Wall -Werror foreign 1.9.6])

I've updated this in Check, so in the next release it will check for the
correct version  Either way, your version of Automake may be too low.
What version of Automake are you using, out of curiosity?

Additionally, in regards to the failure to find Check on the system,
what version of Check is installed on your system?

$ pkg-config check --modversion
0.9.12

Is the version at least 0.9.6? If not, you may need to install a more up to
date version then attempt to compile the example again.

- Branden


On Sat, Sep 20, 2014 at 9:46 AM, 噬血龙 <798...@qq...> wrote:

> dear:
>        "make check" step fails
>      details:
>
>

Hi,

I wish to present to you a test runner generator that I just developed.

I needed a better approach to fixtures and test naming. So, firstly, I
wanted extend checkmk and send a patch but the code is hard to extend
without rewriting big parts. Thus, I rewritten an almost compatible tool
from scratch that handles fixtures and test names like I want.

If you want test it, it can be downloaded at:
https://github.com/xakz/checkgen

The full documentation is included, including compatibility details.

Any return or suggestion are greatly appreciated.

--
Maxime Chatelle



Justin,

Thanks for digging up the information. I've added to the online
documentation mention of the Automake version and how it changes the
expected output from v1.13.

- Branden


On Tue, Aug 19, 2014 at 12:11 AM, Justin Charette <cha...@gm...>
wrote:

> And here's the page from the GNU Automake manual describing the
> parallel test framework.
>
>
> http://www.gnu.org/software/automake/manual/html_node/Parallel-Test-Harness.html
>
> On Mon, Aug 18, 2014 at 11:55 PM, Justin Charette <cha...@gm...>
> wrote:
> > Thanks, Branden.  I wasn't sure if I was making a mistake.  It appears
> > I haven't, and your verification against 1.11.3 is appreciated.
> >
> > I started looking for the change in Automake that resulted in my
> > dilemma, and it appears that with v1.13, Automake now runs tests in
> > parallel by default and redirects to the test-suite.log.  I haven't
> > dug too deep, but I found an indirect reference[1] in a bug report.
> > I'll see what else I can find.
> >
> > Chris,  I tried out CK_VERBOSE, but it still wrote to the log.  Lucky
> > me that I started this endeavor with _the_ version of Automake that
> > deviates from the documentation.  Thanks for your reply.
> >
> > [1]  https://bind10.isc.org/ticket/3243
> >
> > On Mon, Aug 18, 2014 at 9:36 PM, Branden Archer <b.m...@gm...>
> wrote:
> >> Justin,
> >>
> >> I've seen others mention output similar to what you see, though have
> never
> >> seen it myself. The version of Automake on my system is 1.11.3, and
> when I
> >> run the example the output is that mentioned in the documentation.
> There is
> >> also no test-suite.log file produced. Maybe a more recent version of
> >> Automake such as yours captures the output of unit test programs and
> only
> >> displays a summary, not sure.
> >>
> >> I've I happen to come upon information as to why the difference in
> output,
> >> I'll try to add it to the documentation.
> >>
> >> - Branden
> >>
> >>
> >> On Mon, Aug 18, 2014 at 12:11 AM, Chris Pickett
> >> <chr...@ma...> wrote:
> >>>
> >>> CK_VERBOSE should work too.
> >>>
> >>> Justin Charette wrote:
> >>> > I just noticed that Automake redirects the output of Check to
> >>> > ./test-suite.log.
> >>> >
> >>> > At the very least I have access to the Check output and it only
> requires
> >>> > a
> >>> > slightly longer command:
> >>> >
> >>> >     $ make check || cat ./test-suite.log
> >>> >
> >>> > On Sun, Aug 17, 2014 at 1:54 AM, Justin Charette <cha...@gm...
> >
> >>> > wrote:
> >>> >> Hi,
> >>> >>
> >>> >> I'm an amateur C programming teaching myself autotools and check
> >>> >> testing with a few personal projects.  I'm following the basic unit
> >>> >> testing tutorial[1].  Section 3.5 indicates that when I run `make
> >>> >> check`, the output from the test runner will appear in addition to
> >>> >> Automake's test output.
> >>> >>
> >>> >>> With the CK_NORMAL flag specified in our main(), let’s rerun make
> >>> >>> check now. As before, we get the following satisfying output:
> >>> >>>
> >>> >>>
> >>> >>>
> >>> >>> Running suite(s): Money
> >>> >>> 0%: Checks: 1, Failures: 1, Errors: 0
> >>> >>> check_money.c:9:F:Core:test_money_create:0: Assertion 'money_amount
> >>> >>> (m)==5' failed:
> >>> >>> money_amount (m)==0, 5==5
> >>> >>> FAIL: check_money
> >>> >>> =====================================================
> >>> >>> 1 of 1 test failed
> >>> >>> Please report to check-devel AT lists.sourceforge.net
> >>> >>> =====================================================
> >>> >>>
> >>> >>> [...]
> >>> >>>
> >>> >>>
> >>> >>>
> >>> >>> After that we have some higher level output generated by Automake:
> the
> >>> >>> check_money program failed, and the bug-report address given in
> >>> >>> ‘configure.ac’ is printed.
> >>> >>
> >>> >> However, I only see the higher level output generated by Automake
> when
> >>> >> I run `make check`.
> >>> >>
> >>> >>     make  check-TESTS
> >>> >>     make[1]: Entering directory
> >>> >> 'files/development/allrgb/simple-color-iterator/repo/build'
> >>> >>     make[2]: Entering directory
> >>> >> 'files/development/allrgb/simple-color-iterator/repo/build'
> >>> >>     FAIL: test/unit/color/rgb12/get
> >>> >>     PASS: test/unit/color/rgb12/new
> >>> >>     make[3]: Entering directory
> >>> >> 'files/development/allrgb/simple-color-iterator/repo/build'
> >>> >>     make  all-am
> >>> >>     make[4]: Entering directory
> >>> >> 'files/development/allrgb/simple-color-iterator/repo/build'
> >>> >>     make[4]: Leaving directory
> >>> >> 'files/development/allrgb/simple-color-iterator/repo/build'
> >>> >>     make[3]: Leaving directory
> >>> >> 'files/development/allrgb/simple-color-iterator/repo/build'
> >>> >>
> >>> >>
> ============================================================================
> >>> >>     Testsuite summary for simple-color-generator 0.0.0
> >>> >>
> >>> >>
> ============================================================================
> >>> >>     # TOTAL: 2
> >>> >>     # PASS:  1
> >>> >>     # SKIP:  0
> >>> >>     # XFAIL: 0
> >>> >>     # FAIL:  1
> >>> >>     # XPASS: 0
> >>> >>     # ERROR: 0
> >>> >>
> >>> >>
> ============================================================================
> >>> >>     See ./test-suite.log
> >>> >>     Please report to xy...@xy...
> >>> >>
> >>> >>
> ============================================================================
> >>> >>
> >>> >> Is there some configuration I've missed?  I want the check output to
> >>> >> appear when I run `make check` as it's more informative than
> Automake.
> >>> >>
> >>> >> I do get the desired output when I run the test running manually.
> e.g.
> >>> >>
> >>> >>     $ cd files/development/allrgb/simple-color-iterator/repo/build
> >>> >>     $ ./test/unit/color/rgb12/get
> >>> >>     Running suite(s): rgb12_suite
> >>> >>     66%: Checks: 3, Failures: 1, Errors: 0
> >>> >>
> >>> >>
> ../test/unit/color/rgb12/get.c:7:F:get:test__rgb12_blue_get__always__return_blue_value:0:
> >>> >> Assertion '1==rgb12_blue_get(o)' failed: 1==1, rgb12_blue_get(o)==2
> >>> >>
> >>> >> Here's my build environment:
> >>> >>
> >>> >>     check 0.9.13
> >>> >>     autoconf 2.69
> >>> >>     automake 1.14.1
> >>> >>     clang 3.4.2
> >>> >>
> >>> >> Thanks,
> >>> >> Justin C.
> >>> >>
> >>> >> [1] http://check.sourceforge.net/doc/check_html/check_3.html
> >>> >
> >>> >
> >>> >
> ------------------------------------------------------------------------------
> >>> > _______________________________________________
> >>> > Check-users mailing list
> >>> > Che...@li...
> >>> > https://lists.sourceforge.net/lists/listinfo/check-users
> >>>
> >>>
> >>>
> ------------------------------------------------------------------------------
> >>> _______________________________________________
> >>> Check-users mailing list
> >>> Che...@li...
> >>> https://lists.sourceforge.net/lists/listinfo/check-users
> >>
> >>
>

On 08/19/2014 02:47 AM, Branden Archer wrote:
> The documentation for using Autotools with Check has been update on our
> website, see Chapter 5: Supported Build Systems
> <http://check.sourceforge.net/doc/check_html/check_5.html#Supported-Build-Systems>.
> It mentions the expected way to use Check using pkg-config (mentioned in
> my previous email), but also continues to mention AM_PATH_CHECK for
> backwards compatibility.


Thanks for the update, Branden.  Kudos!


Zé



And here's the page from the GNU Automake manual describing the
parallel test framework.

http://www.gnu.org/software/automake/manual/html_node/Parallel-Test-Harness.html

On Mon, Aug 18, 2014 at 11:55 PM, Justin Charette <cha...@gm...> wrote:
> Thanks, Branden.  I wasn't sure if I was making a mistake.  It appears
> I haven't, and your verification against 1.11.3 is appreciated.
>
> I started looking for the change in Automake that resulted in my
> dilemma, and it appears that with v1.13, Automake now runs tests in
> parallel by default and redirects to the test-suite.log.  I haven't
> dug too deep, but I found an indirect reference[1] in a bug report.
> I'll see what else I can find.
>
> Chris,  I tried out CK_VERBOSE, but it still wrote to the log.  Lucky
> me that I started this endeavor with _the_ version of Automake that
> deviates from the documentation.  Thanks for your reply.
>
> [1]  https://bind10.isc.org/ticket/3243
>
> On Mon, Aug 18, 2014 at 9:36 PM, Branden Archer <b.m...@gm...> wrote:
>> Justin,
>>
>> I've seen others mention output similar to what you see, though have never
>> seen it myself. The version of Automake on my system is 1.11.3, and when I
>> run the example the output is that mentioned in the documentation. There is
>> also no test-suite.log file produced. Maybe a more recent version of
>> Automake such as yours captures the output of unit test programs and only
>> displays a summary, not sure.
>>
>> I've I happen to come upon information as to why the difference in output,
>> I'll try to add it to the documentation.
>>
>> - Branden
>>
>>
>> On Mon, Aug 18, 2014 at 12:11 AM, Chris Pickett
>> <chr...@ma...> wrote:
>>>
>>> CK_VERBOSE should work too.
>>>
>>> Justin Charette wrote:
>>> > I just noticed that Automake redirects the output of Check to
>>> > ./test-suite.log.
>>> >
>>> > At the very least I have access to the Check output and it only requires
>>> > a
>>> > slightly longer command:
>>> >
>>> >     $ make check || cat ./test-suite.log
>>> >
>>> > On Sun, Aug 17, 2014 at 1:54 AM, Justin Charette <cha...@gm...>
>>> > wrote:
>>> >> Hi,
>>> >>
>>> >> I'm an amateur C programming teaching myself autotools and check
>>> >> testing with a few personal projects.  I'm following the basic unit
>>> >> testing tutorial[1].  Section 3.5 indicates that when I run `make
>>> >> check`, the output from the test runner will appear in addition to
>>> >> Automake's test output.
>>> >>
>>> >>> With the CK_NORMAL flag specified in our main(), let’s rerun make
>>> >>> check now. As before, we get the following satisfying output:
>>> >>>
>>> >>>
>>> >>>
>>> >>> Running suite(s): Money
>>> >>> 0%: Checks: 1, Failures: 1, Errors: 0
>>> >>> check_money.c:9:F:Core:test_money_create:0: Assertion 'money_amount
>>> >>> (m)==5' failed:
>>> >>> money_amount (m)==0, 5==5
>>> >>> FAIL: check_money
>>> >>> =====================================================
>>> >>> 1 of 1 test failed
>>> >>> Please report to check-devel AT lists.sourceforge.net
>>> >>> =====================================================
>>> >>>
>>> >>> [...]
>>> >>>
>>> >>>
>>> >>>
>>> >>> After that we have some higher level output generated by Automake: the
>>> >>> check_money program failed, and the bug-report address given in
>>> >>> ‘configure.ac’ is printed.
>>> >>
>>> >> However, I only see the higher level output generated by Automake when
>>> >> I run `make check`.
>>> >>
>>> >>     make  check-TESTS
>>> >>     make[1]: Entering directory
>>> >> 'files/development/allrgb/simple-color-iterator/repo/build'
>>> >>     make[2]: Entering directory
>>> >> 'files/development/allrgb/simple-color-iterator/repo/build'
>>> >>     FAIL: test/unit/color/rgb12/get
>>> >>     PASS: test/unit/color/rgb12/new
>>> >>     make[3]: Entering directory
>>> >> 'files/development/allrgb/simple-color-iterator/repo/build'
>>> >>     make  all-am
>>> >>     make[4]: Entering directory
>>> >> 'files/development/allrgb/simple-color-iterator/repo/build'
>>> >>     make[4]: Leaving directory
>>> >> 'files/development/allrgb/simple-color-iterator/repo/build'
>>> >>     make[3]: Leaving directory
>>> >> 'files/development/allrgb/simple-color-iterator/repo/build'
>>> >>
>>> >> ============================================================================
>>> >>     Testsuite summary for simple-color-generator 0.0.0
>>> >>
>>> >> ============================================================================
>>> >>     # TOTAL: 2
>>> >>     # PASS:  1
>>> >>     # SKIP:  0
>>> >>     # XFAIL: 0
>>> >>     # FAIL:  1
>>> >>     # XPASS: 0
>>> >>     # ERROR: 0
>>> >>
>>> >> ============================================================================
>>> >>     See ./test-suite.log
>>> >>     Please report to xy...@xy...
>>> >>
>>> >> ============================================================================
>>> >>
>>> >> Is there some configuration I've missed?  I want the check output to
>>> >> appear when I run `make check` as it's more informative than Automake.
>>> >>
>>> >> I do get the desired output when I run the test running manually.  e.g.
>>> >>
>>> >>     $ cd files/development/allrgb/simple-color-iterator/repo/build
>>> >>     $ ./test/unit/color/rgb12/get
>>> >>     Running suite(s): rgb12_suite
>>> >>     66%: Checks: 3, Failures: 1, Errors: 0
>>> >>
>>> >> ../test/unit/color/rgb12/get.c:7:F:get:test__rgb12_blue_get__always__return_blue_value:0:
>>> >> Assertion '1==rgb12_blue_get(o)' failed: 1==1, rgb12_blue_get(o)==2
>>> >>
>>> >> Here's my build environment:
>>> >>
>>> >>     check 0.9.13
>>> >>     autoconf 2.69
>>> >>     automake 1.14.1
>>> >>     clang 3.4.2
>>> >>
>>> >> Thanks,
>>> >> Justin C.
>>> >>
>>> >> [1] http://check.sourceforge.net/doc/check_html/check_3.html
>>> >
>>> >
>>> > ------------------------------------------------------------------------------
>>> > _______________________________________________
>>> > Check-users mailing list
>>> > Che...@li...
>>> > https://lists.sourceforge.net/lists/listinfo/check-users
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> _______________________________________________
>>> Check-users mailing list
>>> Che...@li...
>>> https://lists.sourceforge.net/lists/listinfo/check-users
>>
>>



Thanks, Branden.  I wasn't sure if I was making a mistake.  It appears
I haven't, and your verification against 1.11.3 is appreciated.

I started looking for the change in Automake that resulted in my
dilemma, and it appears that with v1.13, Automake now runs tests in
parallel by default and redirects to the test-suite.log.  I haven't
dug too deep, but I found an indirect reference[1] in a bug report.
I'll see what else I can find.

Chris,  I tried out CK_VERBOSE, but it still wrote to the log.  Lucky
me that I started this endeavor with _the_ version of Automake that
deviates from the documentation.  Thanks for your reply.

[1]  https://bind10.isc.org/ticket/3243

On Mon, Aug 18, 2014 at 9:36 PM, Branden Archer <b.m...@gm...> wrote:
> Justin,
>
> I've seen others mention output similar to what you see, though have never
> seen it myself. The version of Automake on my system is 1.11.3, and when I
> run the example the output is that mentioned in the documentation. There is
> also no test-suite.log file produced. Maybe a more recent version of
> Automake such as yours captures the output of unit test programs and only
> displays a summary, not sure.
>
> I've I happen to come upon information as to why the difference in output,
> I'll try to add it to the documentation.
>
> - Branden
>
>
> On Mon, Aug 18, 2014 at 12:11 AM, Chris Pickett
> <chr...@ma...> wrote:
>>
>> CK_VERBOSE should work too.
>>
>> Justin Charette wrote:
>> > I just noticed that Automake redirects the output of Check to
>> > ./test-suite.log.
>> >
>> > At the very least I have access to the Check output and it only requires
>> > a
>> > slightly longer command:
>> >
>> >     $ make check || cat ./test-suite.log
>> >
>> > On Sun, Aug 17, 2014 at 1:54 AM, Justin Charette <cha...@gm...>
>> > wrote:
>> >> Hi,
>> >>
>> >> I'm an amateur C programming teaching myself autotools and check
>> >> testing with a few personal projects.  I'm following the basic unit
>> >> testing tutorial[1].  Section 3.5 indicates that when I run `make
>> >> check`, the output from the test runner will appear in addition to
>> >> Automake's test output.
>> >>
>> >>> With the CK_NORMAL flag specified in our main(), let’s rerun make
>> >>> check now. As before, we get the following satisfying output:
>> >>>
>> >>>
>> >>>
>> >>> Running suite(s): Money
>> >>> 0%: Checks: 1, Failures: 1, Errors: 0
>> >>> check_money.c:9:F:Core:test_money_create:0: Assertion 'money_amount
>> >>> (m)==5' failed:
>> >>> money_amount (m)==0, 5==5
>> >>> FAIL: check_money
>> >>> =====================================================
>> >>> 1 of 1 test failed
>> >>> Please report to check-devel AT lists.sourceforge.net
>> >>> =====================================================
>> >>>
>> >>> [...]
>> >>>
>> >>>
>> >>>
>> >>> After that we have some higher level output generated by Automake: the
>> >>> check_money program failed, and the bug-report address given in
>> >>> ‘configure.ac’ is printed.
>> >>
>> >> However, I only see the higher level output generated by Automake when
>> >> I run `make check`.
>> >>
>> >>     make  check-TESTS
>> >>     make[1]: Entering directory
>> >> 'files/development/allrgb/simple-color-iterator/repo/build'
>> >>     make[2]: Entering directory
>> >> 'files/development/allrgb/simple-color-iterator/repo/build'
>> >>     FAIL: test/unit/color/rgb12/get
>> >>     PASS: test/unit/color/rgb12/new
>> >>     make[3]: Entering directory
>> >> 'files/development/allrgb/simple-color-iterator/repo/build'
>> >>     make  all-am
>> >>     make[4]: Entering directory
>> >> 'files/development/allrgb/simple-color-iterator/repo/build'
>> >>     make[4]: Leaving directory
>> >> 'files/development/allrgb/simple-color-iterator/repo/build'
>> >>     make[3]: Leaving directory
>> >> 'files/development/allrgb/simple-color-iterator/repo/build'
>> >>
>> >> ============================================================================
>> >>     Testsuite summary for simple-color-generator 0.0.0
>> >>
>> >> ============================================================================
>> >>     # TOTAL: 2
>> >>     # PASS:  1
>> >>     # SKIP:  0
>> >>     # XFAIL: 0
>> >>     # FAIL:  1
>> >>     # XPASS: 0
>> >>     # ERROR: 0
>> >>
>> >> ============================================================================
>> >>     See ./test-suite.log
>> >>     Please report to xy...@xy...
>> >>
>> >> ============================================================================
>> >>
>> >> Is there some configuration I've missed?  I want the check output to
>> >> appear when I run `make check` as it's more informative than Automake.
>> >>
>> >> I do get the desired output when I run the test running manually.  e.g.
>> >>
>> >>     $ cd files/development/allrgb/simple-color-iterator/repo/build
>> >>     $ ./test/unit/color/rgb12/get
>> >>     Running suite(s): rgb12_suite
>> >>     66%: Checks: 3, Failures: 1, Errors: 0
>> >>
>> >> ../test/unit/color/rgb12/get.c:7:F:get:test__rgb12_blue_get__always__return_blue_value:0:
>> >> Assertion '1==rgb12_blue_get(o)' failed: 1==1, rgb12_blue_get(o)==2
>> >>
>> >> Here's my build environment:
>> >>
>> >>     check 0.9.13
>> >>     autoconf 2.69
>> >>     automake 1.14.1
>> >>     clang 3.4.2
>> >>
>> >> Thanks,
>> >> Justin C.
>> >>
>> >> [1] http://check.sourceforge.net/doc/check_html/check_3.html
>> >
>> >
>> > ------------------------------------------------------------------------------
>> > _______________________________________________
>> > Check-users mailing list
>> > Che...@li...
>> > https://lists.sourceforge.net/lists/listinfo/check-users
>>
>>
>> ------------------------------------------------------------------------------
>> _______________________________________________
>> Check-users mailing list
>> Che...@li...
>> https://lists.sourceforge.net/lists/listinfo/check-users
>
>



The documentation for using Autotools with Check has been update on our
website, see Chapter 5: Supported Build Systems
<http://check.sourceforge.net/doc/check_html/check_5.html#Supported-Build-Systems>.
It mentions the expected way to use Check using pkg-config (mentioned in my
previous email), but also continues to mention AM_PATH_CHECK for backwards
compatibility.


On Mon, Aug 18, 2014 at 8:34 PM, Branden Archer <b.m...@gm...>
wrote:

> There is an example of using pkg-config for finding Check in the example
> configure script located in Check's doc/example directory (link here
> <http://sourceforge.net/p/check/code/HEAD/tree/trunk/doc/example/configure.ac>
> ):
>
> PKG_CHECK_MODULES([CHECK], [check >= 0.9.6])
>
> Useful information on the PKG_CHECK_MODULES macro can be found here
> <https://www.flameeyes.eu/autotools-mythbuster/pkgconfig/pkg_check_modules.html>.
> If including that in your configure.ac script does not work out, let us
> know.
>
> Deprecating AM_PATH_CHECK in the documentation was overlooked, thanks for
> mentioning it. Probably chapter 3 should go over how to include Check into
> a project for the build system types currently supported (autotools and
> cmake). I'll mark that as a TODO and see if I can update it before too long.
>
> Thanks for your interest in Check!
>
> - Branden
>
>
>
>
> On Sun, Aug 17, 2014 at 2:51 PM, Zé <jos...@gm...> wrote:
>
>> Check's online manual[¹] mentions the use of autoconf's AM_PATH_CHECK
>> macro.  When that macro is included in configure.ac, the config script
>> outputs a message informing that the script is deprecated and, instead,
>> users should replace it with a call to a pkg-config macro.
>>
>> Are there any plans to update the online docs?
>>
>> Additionally, could anyone add to the tutorial a clear example on where
>> and how to add the macro to the configure.ac script?  The current
>> tutorial doesn't mention it anywhere and the user is left to guess how
>> to do that or is forced to look elsewhere for that information.
>>
>>
>> Best regards
>> Zé
>>
>> [¹] http://check.sourceforge.net/doc/check_html/
>> [²] http://check.sourceforge.net/doc/check_html/check_3.html#Tutorial
>>
>>
>> ------------------------------------------------------------------------------
>> _______________________________________________
>> Check-users mailing list
>> Che...@li...
>> https://lists.sourceforge.net/lists/listinfo/check-users
>>
>
>

Justin,

I've seen others mention output similar to what you see, though have never
seen it myself. The version of Automake on my system is 1.11.3, and when I
run the example the output is that mentioned in the documentation. There is
also no test-suite.log file produced. Maybe a more recent version of
Automake such as yours captures the output of unit test programs and only
displays a summary, not sure.

I've I happen to come upon information as to why the difference in output,
I'll try to add it to the documentation.

- Branden


On Mon, Aug 18, 2014 at 12:11 AM, Chris Pickett <
chr...@ma...> wrote:

> CK_VERBOSE should work too.
>
> Justin Charette wrote:
> > I just noticed that Automake redirects the output of Check to
> ./test-suite.log.
> >
> > At the very least I have access to the Check output and it only requires
> a
> > slightly longer command:
> >
> >     $ make check || cat ./test-suite.log
> >
> > On Sun, Aug 17, 2014 at 1:54 AM, Justin Charette <cha...@gm...>
> wrote:
> >> Hi,
> >>
> >> I'm an amateur C programming teaching myself autotools and check
> >> testing with a few personal projects.  I'm following the basic unit
> >> testing tutorial[1].  Section 3.5 indicates that when I run `make
> >> check`, the output from the test runner will appear in addition to
> >> Automake's test output.
> >>
> >>> With the CK_NORMAL flag specified in our main(), let’s rerun make
> check now. As before, we get the following satisfying output:
> >>>
> >>>
> >>>
> >>> Running suite(s): Money
> >>> 0%: Checks: 1, Failures: 1, Errors: 0
> >>> check_money.c:9:F:Core:test_money_create:0: Assertion 'money_amount
> (m)==5' failed:
> >>> money_amount (m)==0, 5==5
> >>> FAIL: check_money
> >>> =====================================================
> >>> 1 of 1 test failed
> >>> Please report to check-devel AT lists.sourceforge.net
> >>> =====================================================
> >>>
> >>> [...]
> >>>
> >>>
> >>>
> >>> After that we have some higher level output generated by Automake: the
> check_money program failed, and the bug-report address given in ‘
> configure.ac’ is printed.
> >>
> >> However, I only see the higher level output generated by Automake when
> >> I run `make check`.
> >>
> >>     make  check-TESTS
> >>     make[1]: Entering directory
> >> 'files/development/allrgb/simple-color-iterator/repo/build'
> >>     make[2]: Entering directory
> >> 'files/development/allrgb/simple-color-iterator/repo/build'
> >>     FAIL: test/unit/color/rgb12/get
> >>     PASS: test/unit/color/rgb12/new
> >>     make[3]: Entering directory
> >> 'files/development/allrgb/simple-color-iterator/repo/build'
> >>     make  all-am
> >>     make[4]: Entering directory
> >> 'files/development/allrgb/simple-color-iterator/repo/build'
> >>     make[4]: Leaving directory
> >> 'files/development/allrgb/simple-color-iterator/repo/build'
> >>     make[3]: Leaving directory
> >> 'files/development/allrgb/simple-color-iterator/repo/build'
> >>
>  ============================================================================
> >>     Testsuite summary for simple-color-generator 0.0.0
> >>
>  ============================================================================
> >>     # TOTAL: 2
> >>     # PASS:  1
> >>     # SKIP:  0
> >>     # XFAIL: 0
> >>     # FAIL:  1
> >>     # XPASS: 0
> >>     # ERROR: 0
> >>
>  ============================================================================
> >>     See ./test-suite.log
> >>     Please report to xy...@xy...
> >>
>  ============================================================================
> >>
> >> Is there some configuration I've missed?  I want the check output to
> >> appear when I run `make check` as it's more informative than Automake.
> >>
> >> I do get the desired output when I run the test running manually.  e.g.
> >>
> >>     $ cd files/development/allrgb/simple-color-iterator/repo/build
> >>     $ ./test/unit/color/rgb12/get
> >>     Running suite(s): rgb12_suite
> >>     66%: Checks: 3, Failures: 1, Errors: 0
> >>
>  ../test/unit/color/rgb12/get.c:7:F:get:test__rgb12_blue_get__always__return_blue_value:0:
> >> Assertion '1==rgb12_blue_get(o)' failed: 1==1, rgb12_blue_get(o)==2
> >>
> >> Here's my build environment:
> >>
> >>     check 0.9.13
> >>     autoconf 2.69
> >>     automake 1.14.1
> >>     clang 3.4.2
> >>
> >> Thanks,
> >> Justin C.
> >>
> >> [1] http://check.sourceforge.net/doc/check_html/check_3.html
> >
> >
> ------------------------------------------------------------------------------
> > _______________________________________________
> > Check-users mailing list
> > Che...@li...
> > https://lists.sourceforge.net/lists/listinfo/check-users
>
>
> ------------------------------------------------------------------------------
> _______________________________________________
> Check-users mailing list
> Che...@li...
> https://lists.sourceforge.net/lists/listinfo/check-users
>

There is an example of using pkg-config for finding Check in the example
configure script located in Check's doc/example directory (link here
<http://sourceforge.net/p/check/code/HEAD/tree/trunk/doc/example/configure.ac>
):

PKG_CHECK_MODULES([CHECK], [check >= 0.9.6])

Useful information on the PKG_CHECK_MODULES macro can be found here
<https://www.flameeyes.eu/autotools-mythbuster/pkgconfig/pkg_check_modules.html>.
If including that in your configure.ac script does not work out, let us
know.

Deprecating AM_PATH_CHECK in the documentation was overlooked, thanks for
mentioning it. Probably chapter 3 should go over how to include Check into
a project for the build system types currently supported (autotools and
cmake). I'll mark that as a TODO and see if I can update it before too long.

Thanks for your interest in Check!

- Branden




On Sun, Aug 17, 2014 at 2:51 PM, Zé <jos...@gm...> wrote:

> Check's online manual[¹] mentions the use of autoconf's AM_PATH_CHECK
> macro.  When that macro is included in configure.ac, the config script
> outputs a message informing that the script is deprecated and, instead,
> users should replace it with a call to a pkg-config macro.
>
> Are there any plans to update the online docs?
>
> Additionally, could anyone add to the tutorial a clear example on where
> and how to add the macro to the configure.ac script?  The current
> tutorial doesn't mention it anywhere and the user is left to guess how
> to do that or is forced to look elsewhere for that information.
>
>
> Best regards
> Zé
>
> [¹] http://check.sourceforge.net/doc/check_html/
> [²] http://check.sourceforge.net/doc/check_html/check_3.html#Tutorial
>
>
> ------------------------------------------------------------------------------
> _______________________________________________
> Check-users mailing list
> Che...@li...
> https://lists.sourceforge.net/lists/listinfo/check-users
>

I wrote a fair chunk of the documentation in 2007, including the bits
about the Autotools and the tutorial.  I was a student at the time and
writing better documentation for Check was a good way for me to figure out
how it worked.  It was also better than waiting around for somebody else
to fix it.  Since I don't have a lot of time to maintain Check anymore, I
generally ask for contributions as soon as it sounds like somebody has
something to contribute.  So far, this strategy has worked very well.

The correct call to pkg-config is in check/trunk/configure.ac.  We will
commit any patch that corrects the manual.  If nobody sends a patch, we
will add this to our TODO list, eventually, when somebody remembers.  If
someone files a bug report on SourceForge, that practically guarantees
that the issue will be fixed, although a timeframe is harder to provide.

It's frustrating that smaller open source software projects frequently
suffer from poor management, poor documentation, and poor coding
practices.  These are all things that money will buy.  Then again, there
are no ads in Check, you don't have to pay for it, you are free to use it
in the development of commercial projects, and you can fork it too.

As for civility, to each his own, I suppose.

Chris

On Mon, August 18, 2014 6:00 pm, andrew cooke wrote:
>

> For some reason I'm subscribed to this list.
>
>
> I just read back and I don't see anything in the least uncivil about the
> messages from Ze.  If it had been me I would be a lot more emphatic.
>
> What they said made a lot of sense.  Maybe you should go back, read what
> the emails again, and then apologise for trying to find a non-existent
> issue in tone raher than helping?
>
> Andrew
>
>
>
> On Mon, Aug 18, 2014 at 05:21:56PM -0400, Chris Pickett wrote:
>
>> Hi Zé,
>>
>>
>> My apologies.  I thought you had figured out what to do, since your
>> original question was about updating the docs, and not asking what to
>> do. If you don't know what to do, you could look at the configure.ac
>> file for Check itself, since it uses Check for its own tests, and I'm
>> sure there's something about pkg-config in there.  Once you've got it
>> working, if you feel like updating the documentation, a patch would be
>> highly appreciated. I don't have time to update the documentation,
>> that's why I asked you to do it.  Check is a volunteer project, you're
>> not going to get first class support here, although personally I still
>> feel it's rather good.
>>
>> Also, I need you to be more civil in your messages.  Nobody needs the
>> stress of an argument like this over documentation and a macro call.
>>
>> Chris
>>
>>
>> On Mon, August 18, 2014 4:00 pm, Zé wrote:
>>
>>> On 08/18/2014 05:48 PM, Chris Nehren wrote:
>>>
>>>
>>>> No, I don't.  The idea is that the person who's new asks
>>>> questions and the experts help them understand--then they put that
>>>> explanation into their own words, which tends to be most helpful
>>>> for the new people.  Docs written from the perspective of someone
>>>> who is trying to figure out what's going on and trying to learn how
>>>> things work is probably the best kind of documentation for someone
>>>> who is trying to figure out what's going on and trying to learn how
>>>> things work.  Does that make sense?
>>>
>>> Sorry, your opinion doesn't make any sense.  You've proposed a
>>> convoluted approach that places the responsibility of educating others
>>> on those who don't know what they are doing or how to use the tool,
>>> and therefore are the least capable and even entirely incompetent.
>>> Meanwhile,
>>> you're arguing that those who actually know something (and therefore
>>> are competent and can actually help) should keep wasting their own
>>> time and that of others with an unjustifiable bureaucratic dance that
>>> accomplishes nothing.
>>>
>>> We're not talking about minor style issues or rhetoric approaches to
>>> present a specific piece of information.  We're talking about actually
>>>  supplying that information.  Information can only be provided by
>>> those who actually have them, not by those who don't.
>>>
>>>
>>> Zé
>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> ----
>>> -----
>>> _______________________________________________
>>> Check-users mailing list
>>> Che...@li...
>>> https://lists.sourceforge.net/lists/listinfo/check-users
>>>
>>>
>>>
>>
>>
>>
>>
>>
>> -----------------------------------------------------------------------
>> -------
>> _______________________________________________
>> Check-users mailing list
>> Che...@li...
>> https://lists.sourceforge.net/lists/listinfo/check-users
>>
>>
>








2001	Jan	Feb	Mar	Apr	May	Jun	Jul	Aug (11)	Sep (5)	Oct (3)	Nov (3)	Dec (1)
2002	Jan (3)	Feb	Mar (1)	Apr (2)	May (1)	Jun	Jul	Aug	Sep (1)	Oct (1)	Nov	Dec (1)
2003	Jan	Feb (1)	Mar (3)	Apr	May (2)	Jun (1)	Jul	Aug (5)	Sep	Oct (2)	Nov (1)	Dec (6)
2004	Jan (2)	Feb (1)	Mar (2)	Apr	May (1)	Jun	Jul (1)	Aug	Sep	Oct	Nov (1)	Dec
2005	Jan (2)	Feb (1)	Mar	Apr (2)	May	Jun (3)	Jul (2)	Aug (6)	Sep (3)	Oct	Nov (3)	Dec (2)
2006	Jan (4)	Feb (5)	Mar (19)	Apr (4)	May	Jun (5)	Jul (6)	Aug (3)	Sep (7)	Oct (37)	Nov (36)	Dec (36)
2007	Jan (34)	Feb (25)	Mar (39)	Apr (28)	May (17)	Jun (19)	Jul (48)	Aug (56)	Sep (19)	Oct (24)	Nov (21)	Dec
2008	Jan (3)	Feb (3)	Mar (2)	Apr	May	Jun	Jul (18)	Aug (8)	Sep	Oct	Nov (1)	Dec (1)
2009	Jan (27)	Feb (5)	Mar (22)	Apr	May	Jun	Jul	Aug	Sep (2)	Oct	Nov	Dec
2010	Jan (9)	Feb (9)	Mar (2)	Apr (6)	May	Jun	Jul	Aug	Sep	Oct	Nov (16)	Dec (6)
2011	Jan (2)	Feb (3)	Mar (3)	Apr (1)	May	Jun (5)	Jul	Aug	Sep	Oct	Nov (2)	Dec
2012	Jan (3)	Feb	Mar	Apr	May (10)	Jun	Jul (2)	Aug	Sep	Oct (7)	Nov (1)	Dec (4)
2013	Jan (6)	Feb	Mar	Apr (1)	May	Jun	Jul (27)	Aug (5)	Sep	Oct (22)	Nov (3)	Dec (4)
2014	Jan (3)	Feb (12)	Mar (5)	Apr (19)	May (13)	Jun (17)	Jul (6)	Aug (20)	Sep (2)	Oct (6)	Nov (8)	Dec
2015	Jan (3)	Feb (5)	Mar	Apr (4)	May (4)	Jun	Jul	Aug (1)	Sep	Oct	Nov (5)	Dec (1)
2016	Jan	Feb	Mar (4)	Apr	May	Jun	Jul	Aug (4)	Sep	Oct (2)	Nov	Dec
2017	Jan	Feb	Mar (2)	Apr	May	Jun	Jul	Aug	Sep (3)	Oct	Nov (2)	Dec
2018	Jan	Feb	Mar (5)	Apr	May	Jun	Jul	Aug	Sep	Oct	Nov	Dec
2019	Jan	Feb	Mar	Apr	May	Jun	Jul	Aug (3)	Sep	Oct	Nov	Dec
2021	Jan	Feb	Mar	Apr (3)	May	Jun	Jul	Aug	Sep	Oct	Nov	Dec

check-users Mailing List for check (Page 3)

check-users — Mailing list for users of Check