From: Subrata M. <su...@li...> - 2008-05-27 15:16:49
|
Hi Crackerjack Users/Developers, I was happening to browse through your home page: https://sourceforge.net/projects/crackerjack, and also went through your OLS 2007 paper on Regression Test Framework and Kernel Execution Coverage. (http://ols.108.redhat.com/2007/Reprints/yoshioka-Reprint.pdf) Your paper mentions that you can work with LTP to bring Regression Testing to more greater heights. I would also be interested for it. Let me know how we can start and move forward on this. I would also be interested to see if we can leverage your tests for LTP. Regards-- Subrata |
From: Subrata M. <su...@li...> - 2008-06-09 06:01:33
|
I find out that most of the test cases in Crackerjack has been written by you. I have earlier sent a mail regarding leveraging Cracker Syscall tests for LTP. I would like to bridge the gap in no. of test cases available in LTP, by importing the same from Crackerjack project. Since, most of these tests in Crackerjack are licensed under GPLv2, i hope you will not have much problem in donating those useful tests for LTP. Regards-- Subrata On Tue, 2008-05-27 at 20:46 +0530, Subrata Modak wrote: > Hi Crackerjack Users/Developers, > > I was happening to browse through your home page: > > https://sourceforge.net/projects/crackerjack, > > and also went through your OLS 2007 paper on > > Regression Test Framework and Kernel Execution Coverage. > (http://ols.108.redhat.com/2007/Reprints/yoshioka-Reprint.pdf) > > Your paper mentions that you can work with LTP to bring Regression > Testing to more greater heights. I would also be interested for it. Let > me know how we can start and move forward on this. I would also be > interested to see if we can leverage your tests for LTP. > > Regards-- > Subrata > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Ltp-list mailing list > Ltp...@li... > https://lists.sourceforge.net/lists/listinfo/ltp-list |
From: Subrata M. <su...@li...> - 2008-06-14 07:43:29
|
On Fri, 2008-06-13 at 18:35 +0900, Masatake YAMATO wrote: > Subrata, I'll take my spare time. Well, you have absolutely every right to do that. > (But please don't expect too much, I got a baby:-) Congrats. That´s a very good news. > > > I find out that most of the test cases in Crackerjack has been written > > by you. I have earlier sent a mail regarding leveraging Cracker Syscall > > tests for LTP. I would like to bridge the gap in no. of test cases > > available in LTP, by importing the same from Crackerjack project. Since, > > most of these tests in Crackerjack are licensed under GPLv2, i hope you > > will not have much problem in donating those useful tests for LTP. > > > > Regards-- > > Subrata > > About fadvice64 and sendfile64, the testcases in Crackerjack and > that in LTP are the same. I'll check more other test cases whether > we can port them from Crackerjack to LTP. I can see a lot of them. I will try to send out a list to you as well after doing some comparative study. > > I'll do it incrementally. Do you ready to accept the many small patches? Absolutely yes. I want the things to be done in bits and pieces, and in many patches, rather than keeping the development long and finally submitting a huge patch some very day. Nope, that may not be the case. Submit something as soon as you have finished, and then go back and do something more. > I subscribe this mailing list. However I had no time to read them well. > So if I do something wrong, please let me know. > > About this porting effort I'll use ltp...@li... only > because the posts are just noise for the people who are not interested > in porting. So guys, if you are interested in this effort, subscribe > the list. That indeed will be a huge effort and great contribution. I am expecting that it will definitely improve LTP code coverage as syscall tests touch lots of part of kernel. I do not know exactly when can this porting effort be completed by you. But it would be great if we have successfully ported and made the tests stable in LTP by October/Nov 2008 end. What do you say ? Regards-- Subrata > > Masatake YAMATO > |
From: Masatake Y. <ya...@re...> - 2008-06-16 07:45:57
|
> > About fadvice64 and sendfile64, the testcases in Crackerjack and > > that in LTP are the same. I'll check more other test cases whether > > we can port them from Crackerjack to LTP. Sorry. What I wrote was wrong. The test cases for fadvice64 in Crackerjack and that in ltp are completely different implementations. |
From: Masatake Y. <ya...@re...> - 2008-06-13 09:35:55
|
Subrata, I'll take my spare time. (But please don't expect too much, I got a baby:-) > I find out that most of the test cases in Crackerjack has been written > by you. I have earlier sent a mail regarding leveraging Cracker Syscall > tests for LTP. I would like to bridge the gap in no. of test cases > available in LTP, by importing the same from Crackerjack project. Since, > most of these tests in Crackerjack are licensed under GPLv2, i hope you > will not have much problem in donating those useful tests for LTP. > > Regards-- > Subrata About fadvice64 and sendfile64, the testcases in Crackerjack and that in LTP are the same. I'll check more other test cases whether we can port them from Crackerjack to LTP. I'll do it incrementally. Do you ready to accept the many small patches? I subscribe this mailing list. However I had no time to read them well. So if I do something wrong, please let me know. About this porting effort I'll use ltp...@li... only because the posts are just noise for the people who are not interested in porting. So guys, if you are interested in this effort, subscribe the list. Masatake YAMATO |
From: Subrata M. <su...@li...> - 2008-06-14 09:53:40
|
On Sat, 2008-06-14 at 13:12 +0530, Subrata Modak wrote: > On Fri, 2008-06-13 at 18:35 +0900, Masatake YAMATO wrote: > > Subrata, I'll take my spare time. > > Well, you have absolutely every right to do that. > > > (But please don't expect too much, I got a baby:-) > > Congrats. That´s a very good news. > > > > > > I find out that most of the test cases in Crackerjack has been written > > > by you. I have earlier sent a mail regarding leveraging Cracker Syscall > > > tests for LTP. I would like to bridge the gap in no. of test cases > > > available in LTP, by importing the same from Crackerjack project. Since, > > > most of these tests in Crackerjack are licensed under GPLv2, i hope you > > > will not have much problem in donating those useful tests for LTP. > > > > > > Regards-- > > > Subrata > > > > About fadvice64 and sendfile64, the testcases in Crackerjack and > > that in LTP are the same. I'll check more other test cases whether > > we can port them from Crackerjack to LTP. > > I can see a lot of them. I will try to send out a list to you as well > after doing some comparative study. And here goes the list of syscall test cases that are additionally in Crackerjack and can be included in LTP. Please verify it once again. ftruncate64 setgid16 shmget mbind removexattr exit_group io_cancel migrate_pages setgroups16 timer_create getdents64 timer_delete add_key getegid16 io_destroy newfstat set_mempolicy timer_getoverrun fadvise64_64 io_getevents newlstat rt_sigaction sigreturn timer_gettime geteuid16 newuname rt_sigprocmask timer_settime bdflush getgid16 rt_sigqueueinfo ioprio_get rt_sigsuspend setregid16 tkill getgroups16 ioprio_set sched_getaffinity ssetmask fchown16 io_setup mmap2 setresgid16 truncate64 get_mempolicy io_submit stat64 keyctl setresuid16 chown16 fcntl64 move_pages statfs64 clock_getres lchown16 setreuid16 clock_gettime fgetxattr lgetxattr mq_getsetattr unshare clock_nanosleep flistxattr mq_notify ppoll semctl useclock_settime mq_open semget set_thread_area listxattr mq_timedreceive pread64 semop set_tid_address fremovexattr llistxattr mq_timedsend pselect6 utimes fsetxattr mq_unlink gettid lremovexattr quotactl setuid16 fstat64 msgctl setxattr fstatat64 lsetxattr msgget readahead setfsgid16 sgetmask waitid fstatfs64 getuid16 msgrcv shmat getxattr lstat64 msgsnd setfsuid16 shmctl shmdt Regards-- Subrata > > > > > I'll do it incrementally. Do you ready to accept the many small patches? > > Absolutely yes. I want the things to be done in bits and pieces, and in > many patches, rather than keeping the development long and finally > submitting a huge patch some very day. Nope, that may not be the case. > Submit something as soon as you have finished, and then go back and do > something more. > > > I subscribe this mailing list. However I had no time to read them well. > > So if I do something wrong, please let me know. > > > > About this porting effort I'll use ltp...@li... only > > because the posts are just noise for the people who are not interested > > in porting. So guys, if you are interested in this effort, subscribe > > the list. > > That indeed will be a huge effort and great contribution. I am expecting > that it will definitely improve LTP code coverage as syscall tests touch > lots of part of kernel. > > I do not know exactly when can this porting effort be completed by you. > But it would be great if we have successfully ported and made the tests > stable in LTP by October/Nov 2008 end. What do you say ? > > Regards-- > Subrata > > > > > Masatake YAMATO > > > > > ------------------------------------------------------------------------- > Check out the new SourceForge.net Marketplace. > It's the best place to buy or sell services for > just about anything Open Source. > http://sourceforge.net/services/buy/index.php > _______________________________________________ > Ltp-list mailing list > Ltp...@li... > https://lists.sourceforge.net/lists/listinfo/ltp-list |
From: Masatake Y. <ya...@re...> - 2008-06-16 07:56:15
|
> And here goes the list of syscall test cases that are additionally in > Crackerjack and can be included in LTP. Please verify it once again. > ftruncate64 > setgid16 > shmget > mbind ... I've also studied codes. Here is my memorandum. I've used lisp style representation. Here car part of the list stands for the name of system call. (I don't studied *16 and *64 system calls yet.) About :status, not-in-ltp is important. System call taged as not-in-ltp stands for the test case for the system call is not in ltp and is in crackerjack. If :status is a string, the test case for the system call is in ltp already even if it is not complete. I'd like to concentrate on system calls taged as not-in-ltp. ( ("add_key" :status ???) ("bdflush" :status (not-in-ltp no-copyright-notice)) "chown16" ("clock_getres" :status "open_posix_testsuite") ("clock_gettime" :status "open_posix_testsuite") ("clock_nanosleep" :status "open_posix_testsuite") ("exit_group" :status (not-in-ltp too-simple-in-cjk)) "fadvise64_64" "fchown16" "fcntl64" ("fgetxattr" :status ("testcases/kernel/fs/acls/acl_file_test.c" too-simple-in-ltp)) ("flistxattr" :status ("testcases/kernel/fs/acls/acl_file_test.c" too-simple-in-ltp)) ("fremovexattr" :status ("testcases/kernel/fs/acls/acl_file_test.c" too-simple-in-ltp)) ("fsetxattr" :status ("testcases/kernel/fs/acls/acl_file_test.c" too-simple-in-ltp)) "fstat64" "fstatat64" "fstatfs64" "ftruncate64" ("get_mempolicy" :status ("testcases/kernel/hotplug/memory_hotplug/")) "getdents64" "getegid16" "geteuid16" "getgid16" "getgroups16" ("gettid" :status not-in-ltp) "getuid16" ("getxattr" :status "testcases/kernel/fs/acls/acl_link_test.c") ("io_cancel" :status "open_posix_testsuite") ("io_destroy" :status "open_posix_testsuite") ("io_getevents" :status "open_posix_testsuite") ("io_setup" :status "open_posix_testsuite") ("io_submit" :status "open_posix_testsuite") ("ioprio_get" :status (not-in-ltp asked-on-ltp-list) ("ioprio_set" :status (not-in-ltp asked-on-ltp-list)) ("keyctl" :status ???) "lchown16" ("lgetxattr" :status not-in-ltp) ("listxattr" :status "testcases/kernel/fs/acls/acl_link_test.c") ("llistxattr" :status not-in-ltp) ("lremovexattr" :status "testcases/kernel/fs/acls/acl_link_test.c") ("lsetxattr" :status "testcases/kernel/fs/acls/acl_link_test.c") "lstat64" ("mbind" :status exists-in-ltp) ("migrate_pages" :status "/testcases/kernel/hotplug/memory_hotplug") ("mmap2" :status not-in-ltp) ("move_pages" :status not-in-ltp) ("mq_getsetattr" :status (??? "testcases/open_posix_testsuite/conformance/interfaces/mq_getattr")) ("mq_notify" :status "testcases/open_posix_testsuite/conformance/interfaces/") ("mq_open" :status "testcases/open_posix_testsuite/conformance/interfaces/") ("mq_timedreceive" :status "testcases/open_posix_testsuite/conformance/interfaces/") ("mq_timedsend" :status "testcases/open_posix_testsuite/conformance/interfaces/") ("mq_unlink" :status "testcases/open_posix_testsuite/conformance/interfaces/")) ("msgctl" :status "testcases/kernel/ipc/ipc_stress") ("msgget" :status "testcases/kernel/ipc/ipc_stress") ("msgrcv" :status "testcases/kernel/ipc/ipc_stress") ("msgsnd" :status "testcases/kernel/ipc/ipc_stress") ("newfstat" :status (same-as lstat)) ("newlstat" :status (same-as lstat)) ("newuname" :status (same-as uname)) ("ppoll" :status not-in-ltp) "pread64" ("pselect6" :status not-in-ltp) ("quotactl" :status not-in-ltp) ("readahead" :status not-in-ltp) ("removexattr" :status not-in-ltp) ("rt_sigaction" :status not-in-ltp) ("rt_sigprocmask" :status not-in-ltp) ("rt_sigqueueinfo" :status not-in-ltp) ("rt_sigsuspend" :status not-in-ltp) ("sched_getaffinity" :status "testcases/realtime") ("semctl" :status "testcases/kernel/ipc/ipc_stress") ("semget" :status "testcases/kernel/ipc/ipc_stress") ("semop" :status "testcases/kernel/ipc/ipc_stress") ("set_mempolicy" :status (not-in-ltp "memory_hotplug")) ("set_thread_area" :status not-in-ltp) ("set_tid_address" :status not-in-ltp) "setfsgid16" "setfsuid16" "setgid16" "setgroups16" "setregid16" "setresgid16" "setresuid16" "setreuid16" "setuid16" ("setxattr" :status not-in-ltp) ("sgetmask" :status not-in-ltp) ("shmat" :status "testcases/kernel/ipc/ipc_stress") ("shmctl" :status "testcases/kernel/ipc/ipc_stress") ("shmdt" :status "testcases/kernel/ipc/ipc_stress") ("shmget" :status "testcases/kernel/ipc/ipc_stress") ("sigreturn" :status not-in-ltp) ("ssetmask" :status not-in-ltp) "stat64" "statfs64" ("timer_create" :status "open_posix_testsuite/conformance/interfaces") ("timer_delete" :status "open_posix_testsuite/conformance/interfaces") ("timer_getoverrun" :status "open_posix_testsuite/conformance/interfaces") ("timer_gettime" :status "open_posix_testsuite/conformance/interfaces") ("timer_settime" :status "open_posix_testsuite/conformance/interfaces") ("tkill" :status not-in-ltp) "truncate64" ("unshare" :status "testcases/kernel/containers/libclone/libclone.c") ("useclock_settime" :status not-in-ltp) ("utimes" :status ???) ("waitid" :status not-in-ltp) ) > I do not know exactly when can this porting effort be completed by you. > But it would be great if we have successfully ported and made the tests > stable in LTP by October/Nov 2008 end. What do you say ? Sorry. I cannot say nothing abot the schedule. Masatake |
From: Subrata M. <tos...@gm...> - 2008-06-16 09:50:34
|
On Mon, Jun 16, 2008 at 11:20 AM, Masatake YAMATO <ya...@re...> wrote: > (More people are added to Cc by Subrata, so I'll keep the field > in spite of my last mail.) > > > And here goes the list of syscall test cases that are additionally in > > Crackerjack and can be included in LTP. Please verify it once again. > > > > ftruncate64 > > ... > > shmat > > msgsnd > > ... > > shmctl > > shmdt > > ... > > Could you tell me more about "can be included"? > > e.g. shmat, msgsnd, shmctl, and shmdt are tested in > ltp/testcases/kernel/ipc/ipc_stress, aren't they? (Sometimes i need to reply through this id as my IMAP account @ linux.vnet.ibm.com becomes dead slow in delivering mails) Oh Yes. You are correct. I did not look in to that aspect at all. I just did a comparison of whatever we have at *testcases/kernel/syscalls* and what they have in *Crackerjack*. I am listing down my comments here: 1) ftruncate64 Comments: We have ftruncate at *testcases/kernel/syscalls/ftruncate/*. But this may not be doing the ftruncate64() test. You may need to include/port ftruncate64 on similar lines to sendfile64 and fadvise64. 2) setgid16 Comments: We have setgid at *testcases/kernel/syscalls/setgid. *You may need to include/port setgid16 on similar lines to sendfile64 and fadvise64. 3) shmget: Comments: We have this at testcases/kernel/syscalls/ipc/shmget. Need not look into this right now. May be after the porting is over, we can see whether there is some additional functionality that Crackerjack shmget is covering over LTP-shmget. We may then need to add 1/2 test cases in this regard. But, not to be prioritized immediately. 4) mbind Comments: We need this from Crackerjack as we do not have this in LTP anywhere. 5) removexattr Comments: We need this from Crackerjack as we do not have this in LTP anywhere. 6) exit_group Comments: We need this from Crackerjack as we do not have this in LTP anywhere. 7) io_cancel Comments: We need this from Crackerjack as we do not have this in LTP anywhere. 8) migrate_pages Comments: We have a different form of this at recently added Memory Hotplug testcases at testcases/kernel/hotplug/memory_hotplug/. But i would like to see this a tested inside testcases/kernel/syscalls, as hotplug testing presently is optional. So, please include/port this as well. 9) setgroups16 Comments: We have setgroups at *testcases/kernel/syscalls/setgroups. *You may need to include/port setgroups16 on similar lines to sendfile64 and fadvise64. 10) getdents64 Comments: We have getdents at *testcases/kernel/syscalls/getdents. *You may need to include/port getdents64 on similar lines to sendfile64 and fadvise64. 11) add_key Comments: We need this from Crackerjack as we do not have this in LTP anywhere. 12) getegid16 Comments: We have getegid at *testcases/kernel/syscalls/gettegid. *You may need to include/port getegid16 on similar lines to sendfile64 and fadvise64. 13) io_destroy Comments: We need this from Crackerjack as we do not have this in LTP anywhere. 14) newfstat Comments: We need this from Crackerjack as we do not have this in LTP anywhere. 15) set_mempolicy Comments: We need this from Crackerjack as we do not have this in LTP anywhere. 16) timer_getoverrun Comments: We need this from Crackerjack as we do not have this in LTP anywhere. 17) fadvise64_64 Comments: We have fadvise64 support added by you. But i am not sure what this fadvise64_64 is. May need to include/port this as well. 18) io_getevents Comments: We need this from Crackerjack as we do not have this in LTP anywhere. 19) newlstat Comments: We need this from Crackerjack as we do not have this in LTP anywhere. 20) rt_sigaction Comments: We need this from Crackerjack as we do not have this in LTP anywhere. 21) sigreturn Comments: We need this from Crackerjack as we do not have this in LTP anywhere. 22) timer_gettime Comments: We need this from Crackerjack as we do not have this in LTP anywhere. 23) geteuid16 Comments: We have geteuid at *testcases/kernel/syscalls/geteuid. *You may need to include/port geteuid16 on similar lines to sendfile64 and fadvise64. 24) newuname Comments: We need this from Crackerjack as we do not have this in LTP anywhere. 25) rt_sigprocmask Comments: We need this from Crackerjack as we do not have this in LTP anywhere. 26) bdflush Comments: We need this from Crackerjack as we do not have this in LTP anywhere. 27) getgid16 Comments: We have getgid at *testcases/kernel/syscalls/getgid. *You may need to include/port getgid16 on similar lines to sendfile64 and fadvise64. 28) rt_sigqueueinfo Comments: We need this from Crackerjack as we do not have this in LTP anywhere. 29) ioprio_get Comments: We need this from Crackerjack as we do not have this in LTP anywhere. 30) rt_sigsuspend Comments: We need this from Crackerjack as we do not have this in LTP anywhere. 31) setregid16 Comments: We have setregid at *testcases/kernel/syscalls/setregid. *You may need to include/port setregid16 on similar lines to sendfile64 and fadvise64. 32) tkill Comments: We need this from Crackerjack as we do not have this in LTP anywhere. 33) getgroups16 Comments: We have getgroups at *testcases/kernel/syscalls/getgroups. *You may need to include/port getgroups16 on similar lines to sendfile64 and fadvise64. 34) ioprio_set Comments: We need this from Crackerjack as we do not have this in LTP anywhere. 35) sched_getaffinity Comments: We need this from Crackerjack as we do not have this in LTP anywhere. 36) ssetmask Comments: We need this from Crackerjack as we do not have this in LTP anywhere. 37) fchown16 Comments: We have fchown at *testcases/kernel/syscalls/fchown. *You may need to include/port fchown16 on similar lines to sendfile64 and fadvise64. 38) io_setup Comments: We need this from Crackerjack as we do not have this in LTP anywhere. 39) mmap2 Comments: We have mmap2 in testcases/kernel/mem/mtest06/. Need not look into this right now. May be after the porting is over, we can see whether there is some additional functionality that Crackerjack mmap2 is covering over LTP-mmap2. We may then need to add 1/2 test cases in this regard. But, not to be prioritized immediately. 40) setresgid16 Comments: We have setresgid at *testcases/kernel/syscalls/setresgid. *You may need to include/port setresgid16 on similar lines to sendfile64 and fadvise64. 41) truncate64 Comments: We have truncate at *testcases/kernel/syscalls/truncate. *You may need to include/port truncate64 on similar lines to sendfile64 and fadvise64. 42) get_mempolicy Comments: We need this from Crackerjack as we do not have this in LTP anywhere. 43) io_submit Comments: We need this from Crackerjack as we do not have this in LTP anywhere. 44) stat64 Comments: We have stat at *testcases/kernel/syscalls/stat. *You may need to include/port stat64 on similar lines to sendfile64 and fadvise64. 45) keyctl Comments: We need this from Crackerjack as we do not have this in LTP anywhere. 46) setresuid16 Comments: We have setresuid at *testcases/kernel/syscalls/setresuid. *You may need to include/port setresuid16 on similar lines to sendfile64 and fadvise64. 47) chown16 Comments: We have chown at *testcases/kernel/syscalls/chown. *You may need to include/port chown16 on similar lines to sendfile64 and fadvise64. 48) fcntl64 Comments: We have fcntl at *testcases/kernel/syscalls/fcntl. *You may need to include/port fcntl64 on similar lines to sendfile64 and fadvise64. 49) move_pages: This is not in LTP. But work for this has been already initiated. Need not consider this. 50) statfs64 Comments: We have statfs at *testcases/kernel/syscalls/statfs. *You may need to include/port statfs64 on similar lines to sendfile64 and fadvise64. 51) clock_getres Comments: We need this from Crackerjack as we do not have this in LTP anywhere. 52) lchown16 Comments: We have lchown at *testcases/kernel/syscalls/lchown. *You may need to include/port lchown16 on similar lines to sendfile64 and fadvise64. 53) setreuid16 Comments: We have setreuid at *testcases/kernel/syscalls/setreuid. *You may need to include/port setreuid16 on similar lines to sendfile64 and fadvise64. 54) fgetxattr Comments: We need this from Crackerjack as we do not have this in LTP anywhere. 55) lgetxattr Comments: We need this from Crackerjack as we do not have this in LTP anywhere. 56) mq_getsetattr Comments: We need this from Crackerjack as we do not have this in LTP anywhere. 57) unshare Comments: We need this from Crackerjack as we do not have this in LTP anywhere. 58) clock_nanosleep Comments: We need this from Crackerjack as we do not have this in LTP anywhere. 59) flistxattr Comments: We need this from Crackerjack as we do not have this in LTP anywhere. 60) mq_notify Comments: We need this from Crackerjack as we do not have this in LTP anywhere. 61) ppoll Comments: We need this from Crackerjack as we do not have this in LTP anywhere. 62) useclock_settime Comments: We need this from Crackerjack as we do not have this in LTP anywhere. 63) mq_open Comments: We need this from Crackerjack as we do not have this in LTP anywhere. 64) set_thread_area Comments: We need this from Crackerjack as we do not have this in LTP anywhere. 65) listxattr Comments: We need this from Crackerjack as we do not have this in LTP anywhere. 66) mq_timedreceive Comments: We need this from Crackerjack as we do not have this in LTP anywhere. 67) pread64 Comments: We have pread at *testcases/kernel/syscalls/pread. *You may need to include/port pread64 on similar lines to sendfile64 and fadvise64. 68) set_tid_address Comments: We need this from Crackerjack as we do not have this in LTP anywhere. 69) fremovexattr Comments: We need this from Crackerjack as we do not have this in LTP anywhere. 70) llistxatt Comments: We need this from Crackerjack as we do not have this in LTP anywhere. 71) mq_timedsend Comments: We need this from Crackerjack as we do not have this in LTP anywhere. 72) pselect6 Comments: We have pselect at *testcases/kernel/syscalls/paelect. *You may need to include/port pselect6 on similar lines to sendfile64 and fadvise64. 73) utimes Comments: We need this from Crackerjack as we do not have this in LTP anywhere. 74) fsetxattr Comments: We need this from Crackerjack as we do not have this in LTP anywhere. 75) mq_unlink Comments: We need this from Crackerjack as we do not have this in LTP anywhere. 76) gettid Comments: We need this from Crackerjack as we do not have this in LTP anywhere. 77) lremovexattr Comments: We need this from Crackerjack as we do not have this in LTP anywhere. 78) quotactl Comments: We need this from Crackerjack as we do not have this in LTP anywhere. 79) setuid16 Comments: We have setuid at *testcases/kernel/syscalls/setuid* and * testcases/kernel/syscalls/mount**. *You may need to include/port setuid16 on similar lines to sendfile64 and fadvise64. 80) fstat64 Comments: We have fstat at *testcases/kernel/syscalls/fstat**. *You may need to include/port fstat64 on similar lines to sendfile64 and fadvise64. 81) setxattr Comments: We need this from Crackerjack as we do not have this in LTP anywhere. 82) fstatat64 Comments: We have fstatat at *testcases/kernel/syscalls/fstatat**. *You may need to include/port fstatat64 on similar lines to sendfile64 and fadvise64. 83) lsetxattr Comments: We need this from Crackerjack as we do not have this in LTP anywhere. 84) readahead Comments: We need this from Crackerjack as we do not have this in LTP anywhere. 85) setfsgid16 Comments: We have setfsgid at *testcases/kernel/syscalls/setfsgid**. *You may need to include/port setfsgid16 on similar lines to sendfile64 and fadvise64. 86) sgetmask Comments: We need this from Crackerjack as we do not have this in LTP anywhere. 87) waitid Comments: We need this from Crackerjack as we do not have this in LTP anywhere. 88) fstatfs64 Comments: We have fstatfs at *testcases/kernel/syscalls/fstatfs**. *You may need to include/port fstatfs64 on similar lines to sendfile64 and fadvise64. 89) getuid16 Comments: We have getuid at *testcases/kernel/syscalls/getuid**. *You may need to include/port getuid16 on similar lines to sendfile64 and fadvise64. 90) getxattr Comments: We need this from Crackerjack as we do not have this in LTP anywhere. 91) lstat64 Comments: We have lstat at *testcases/kernel/syscalls/lstat**. *You may need to include/port lstat64 on similar lines to sendfile64 and fadvise64. 92) setfsuid16 Comments: We have setfsuid at *testcases/kernel/syscalls/setfsuid**. *You may need to include/port setfsuid16 on similar lines to sendfile64 and fadvise64. Ooopss......!! Trying to find out the whereabouts of these test cases is really tough. But once decided we can keep going. So, we have the following summary: i) *61* of the syscall tests require direct porting and creation of new directories in LTP, ii) Once this is over, *29* of testcases needs to be ported for their 16-bit or 64-bit versions. This will not involve creation of new directories in LTP, but will be placed in the existing directories. These may either add new source files or update the existing ones similar to the sendfile64 and fadvise64, iii) Remaining test cases like shmget, shmat, msgsnd, shmctl, shmdt, mmap2 can be investigated later, and once all the above i) and ii) are done. We can see whether there is something additional functionality testing in Crackerjack for them. If yes, we can then include 1/2 tests for each of them. But that is at the end. Yamato, hope this list helps you in your work. Regards-- Subrata > > > Masatake > > ------------------------------------------------------------------------- > Check out the new SourceForge.net Marketplace. > It's the best place to buy or sell services for > just about anything Open Source. > http://sourceforge.net/services/buy/index.php > _______________________________________________ > Ltp-list mailing list > Ltp...@li... > https://lists.sourceforge.net/lists/listinfo/ltp-list > -- Regards & Thanks-- Subrata |
From: Masatake Y. <ya...@re...> - 2008-06-17 03:31:13
|
> Oh Yes. You are correct. I did not look in to that aspect at all. I just did > a comparison of whatever we have at *testcases/kernel/syscalls* and what > they have in *Crackerjack*. I am listing down my comments here: > > 1) ftruncate64 > Comments: We have ftruncate at *testcases/kernel/syscalls/ftruncate/*. But > this may not be doing the ftruncate64() test. You may need to include/port > ftruncate64 on similar lines to sendfile64 and fadvise64. How do you think add this list to ltp source tree? The list should be kept untill we complete the porting. This list may be helpful to avoid conflicts between contributors who wants to port testcases. Here is the update: 76) gettid Comments: We need this from Crackerjack as we do not have this in LTP anywhere. (Done by Masatake YAMATO <ya...@re...>). |
From: Masatake Y. <ya...@re...> - 2008-06-16 10:12:04
|
> Oh Yes. You are correct. I did not look in to that aspect at all. I just did > a comparison of whatever we have at *testcases/kernel/syscalls* and what > they have in *Crackerjack*. I am listing down my comments here: O.k. I'll focus on *testcases/kernel/syscalls*. So the porting task becomes much simpler. > 91) lstat64 > Comments: We have lstat at *testcases/kernel/syscalls/lstat**. *You may need > to include/port lstat64 on similar lines to sendfile64 and fadvise64. > > 92) setfsuid16 > Comments: We have setfsuid at *testcases/kernel/syscalls/setfsuid**. *You > may need to include/port setfsuid16 on similar lines to sendfile64 and > fadvise64. Really grate list. BTW, > 92) setfsuid16 > Comments: We have setfsuid at *testcases/kernel/syscalls/setfsuid**. *You > may need to include/port setfsuid16 on similar lines to sendfile64 and > fadvise64. You wrote "sendfile64 and fadvise64". I think you mean setfsuid instead, don't you? I coundn't understand why you wrote "sendfile64 and fadvise64" here. Masatake |
From: Subrata M. <tos...@gm...> - 2008-06-16 10:18:36
|
On Mon, Jun 16, 2008 at 3:41 PM, Masatake YAMATO <ya...@re...> wrote: > > Oh Yes. You are correct. I did not look in to that aspect at all. I just > did > > a comparison of whatever we have at *testcases/kernel/syscalls* and what > > they have in *Crackerjack*. I am listing down my comments here: > > O.k. I'll focus on *testcases/kernel/syscalls*. So the porting task becomes > much simpler. > > > 91) lstat64 > > Comments: We have lstat at *testcases/kernel/syscalls/lstat**. *You may > need > > to include/port lstat64 on similar lines to sendfile64 and fadvise64. > > > > 92) setfsuid16 > > Comments: We have setfsuid at *testcases/kernel/syscalls/setfsuid**. *You > > may need to include/port setfsuid16 on similar lines to sendfile64 and > > fadvise64. > > > Really grate list. Thanks. We will keep this as a standard one to review our progress against what is done and what still needs to be done. > > BTW, > > > 92) setfsuid16 > > Comments: We have setfsuid at *testcases/kernel/syscalls/setfsuid**. *You > > may need to include/port setfsuid16 on similar lines to sendfile64 and > > fadvise64. > > You wrote "sendfile64 and fadvise64". > I think you mean setfsuid instead, don't you? > > I coundn't understand why you wrote "sendfile64 and fadvise64" here. Oh yes. What i actually wanted to mean is, you can port setfsuid16 from Crackerjack to LTP, in the same way you have done for sendfile64 and fadvise64 earlier. You can either include the code from Crackerjackś setfsuid16 into ltp/testcases/kernel/syscalls/setfsuid/setfsuid*.c, or, you can have them as seperate directory as ltp/testcases/kernel/syscalls/setfsuid16. This can be the way for all the 16-bit and 64-bit versions of the syscalls available in Crackerjack. Regards-- Subrata > > Masatake > -- Regards & Thanks-- Subrata |
From: Masatake Y. <ya...@re...> - 2008-06-16 05:50:47
|
(More people are added to Cc by Subrata, so I'll keep the field in spite of my last mail.) > And here goes the list of syscall test cases that are additionally in > Crackerjack and can be included in LTP. Please verify it once again. > > ftruncate64 > ... > shmat > msgsnd > ... > shmctl > shmdt > ... Could you tell me more about "can be included"? e.g. shmat, msgsnd, shmctl, and shmdt are tested in ltp/testcases/kernel/ipc/ipc_stress, aren't they? Masatake |
From: <his...@hi...> - 2008-06-11 03:41:06
|
Hi, Subrata, This is Hisashi Hashimoto, I am working as project leader of Crackerjack. Sorry for getting back to you late. We would like to collaborate you regarding test cases. We are discussing about next step, enhancing test cases, improving user interface, etc. We have the plan to visit Linux World in San Francisco, this August. If you are there, we will find the time to meet you, and wanna talk to you regarding colloaboration and our plan. Your regards, Hashimoto Hisashi >送信者 : Subrata Modak <su...@li...> >主題 : Re: [Crackerjack-devel] [LTP] Crackerjack and Linux Test Project >受信日 :08/06/09 15:02 >属性 : なし > >I find out that most of the test cases in Crackerjack has been written >by you. I have earlier sent a mail regarding leveraging Cracker Syscall >tests for LTP. I would like to bridge the gap in no. of test cases >available in LTP, by importing the same from Crackerjack project. Since, >most of these tests in Crackerjack are licensed under GPLv2, i hope you >will not have much problem in donating those useful tests for LTP. > >Regards-- >Subrata > >On Tue, 2008-05-27 at 20:46 +0530, Subrata Modak wrote: >> Hi Crackerjack Users/Developers, >> >> I was happening to browse through your home page: >> >> https://sourceforge.net/projects/crackerjack, >> >> and also went through your OLS 2007 paper on >> >> Regression Test Framework and Kernel Execution Coverage. >> (http://ols.108.redhat.com/2007/Reprints/yoshioka-Reprint.pdf) >> >> Your paper mentions that you can work with LTP to bring Regression >> Testing to more greater heights. I would also be interested for it. Let >> me know how we can start and move forward on this. I would also be >> interested to see if we can leverage your tests for LTP. >> >> Regards-- >> Subrata >> >> >> ------------------------------------------------------------------------- >> This SF.net email is sponsored by: Microsoft >> Defy all challenges. Microsoft(R) Visual Studio 2008. >> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ >> _______________________________________________ >> Ltp-list mailing list >> Ltp...@li... >> https://lists.sourceforge.net/lists/listinfo/ltp-list > > >------------------------------------------------------------------------- >Check out the new SourceForge.net Marketplace. >It's the best place to buy or sell services for >just about anything Open Source. >http://sourceforge.net/services/buy/index.php >_______________________________________________ >Crackerjack-devel mailing list >Cra...@li... >https://lists.sourceforge.net/lists/listinfo/crackerjack-devel > Hisashi Hashimoto Section Manager Open Source Software Promotion Center Platform Software Hitachi, Ltd., Software Division Tel : +81-45-862-8424, Fax : +81-45-862-9047 his...@hi... |
From: <his...@hi...> - 2008-06-13 03:42:30
|
Unfortunatelly, I will not be on OLS. Could I visit your office and discuss while I am in US in August ? Let me know your idea. Thanks in advance, Hashimoto Hisashi >送信者 : Subrata Modak <su...@li...> >主題 : Re: Re[2]: [Crackerjack-devel] [LTP] Crackerjack and Linux TestProject >受信日 :08/06/11 19:56 >属性 : なし > >On Wed, 2008-06-11 at 12:40 +0900, his...@hi... >wrote: >> Hi, Subrata, >> >> This is Hisashi Hashimoto, I am working as project leader >> of Crackerjack. Sorry for getting back to you late. >> We would like to collaborate you regarding test cases. >> We are discussing about next step, enhancing test cases, improving >> user interface, etc. >> We have the plan to visit Linux World in San Francisco, this August. >> If you are there, we will find the time to meet you, and wanna talk >> to you regarding colloaboration and our plan. > >I will not be there as you know it is difficult to generate travel >funding for all Linux conferences. However i will be there to present my >Paper on LTP in OLS 2008 at Ottowa. If some of you are there, then >definitely we can discuss things and move forward. > >Meanwhile i believe that the test cases that you have written will help >us a lot if we can import from you under the GPLv2. The present >challenge before us is to find out ways to test the untested parts of >kernel. We need test cases for them. We are not adding even fraction of >test cases of what is going in to the kernel code. So, fresh test cases >(as can be found from your project) is of great help. I have also read >in your OLS 2007 paper, the future work that you have envisaged in this >regard. We would appreciate that as it will unleash another round of >fresh test cases for testing the Linux kernel. > >Regards-- >Subrata > >> >> Your regards, >> >> Hashimoto Hisashi >> >> >送信者 : Subrata Modak <su...@li...> >> >主題 : Re: [Crackerjack-devel] [LTP] Crackerjack and Linux Test Project >> >受信日 :08/06/09 15:02 >> >属性 : なし >> > >> >I find out that most of the test cases in Crackerjack has been written >> >by you. I have earlier sent a mail regarding leveraging Cracker Syscall >> >tests for LTP. I would like to bridge the gap in no. of test cases >> >available in LTP, by importing the same from Crackerjack project. Since, >> >most of these tests in Crackerjack are licensed under GPLv2, i hope you >> >will not have much problem in donating those useful tests for LTP. >> > >> >Regards-- >> >Subrata >> > >> >On Tue, 2008-05-27 at 20:46 +0530, Subrata Modak wrote: >> >> Hi Crackerjack Users/Developers, >> >> >> >> I was happening to browse through your home page: >> >> >> >> https://sourceforge.net/projects/crackerjack, >> >> >> >> and also went through your OLS 2007 paper on >> >> >> >> Regression Test Framework and Kernel Execution Coverage. >> >> (http://ols.108.redhat.com/2007/Reprints/yoshioka-Reprint.pdf) >> >> >> >> Your paper mentions that you can work with LTP to bring Regression >> >> Testing to more greater heights. I would also be interested for it. Let >> >> me know how we can start and move forward on this. I would also be >> >> interested to see if we can leverage your tests for LTP. >> >> >> >> Regards-- >> >> Subrata >> >> >> >> >> >> ------------------------------------------------------------------------- >> >> This SF.net email is sponsored by: Microsoft >> >> Defy all challenges. Microsoft(R) Visual Studio 2008. >> >> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ >> >> _______________________________________________ >> >> Ltp-list mailing list >> >> Ltp...@li... >> >> https://lists.sourceforge.net/lists/listinfo/ltp-list >> > >> > >> >------------------------------------------------------------------------- >> >Check out the new SourceForge.net Marketplace. >> >It's the best place to buy or sell services for >> >just about anything Open Source. >> >http://sourceforge.net/services/buy/index.php >> >_______________________________________________ >> >Crackerjack-devel mailing list >> >Cra...@li... >> >https://lists.sourceforge.net/lists/listinfo/crackerjack-devel >> > >> Hisashi Hashimoto >> Section Manager >> Open Source Software Promotion Center >> Platform Software >> Hitachi, Ltd., Software Division >> Tel : +81-45-862-8424, Fax : +81-45-862-9047 >> his...@hi... > > Hisashi Hashimoto Section Manager Open Source Software Promotion Center Platform Software Hitachi, Ltd., Software Division Tel : +81-45-862-8424, Fax : +81-45-862-9047 his...@hi... |
From: Subrata M. <su...@li...> - 2008-06-14 06:47:20
|
On Fri, 2008-06-13 at 12:42 +0900, his...@hi... wrote: > Unfortunatelly, > I will not be on OLS. > Could I visit your office and discuss while I am in You can visit my office/home. > US in August ? Alas, i am not in US. I work from Bangalore, India. Regards-- Subrata > > Let me know your idea. > > Thanks in advance, > > Hashimoto Hisashi > > >送信者 : Subrata Modak <su...@li...> > >主題 : Re: Re[2]: [Crackerjack-devel] [LTP] Crackerjack and Linux TestProject > >受信日 :08/06/11 19:56 > >属性 : なし > > > >On Wed, 2008-06-11 at 12:40 +0900, his...@hi... > >wrote: > >> Hi, Subrata, > >> > >> This is Hisashi Hashimoto, I am working as project leader > >> of Crackerjack. Sorry for getting back to you late. > >> We would like to collaborate you regarding test cases. > >> We are discussing about next step, enhancing test cases, improving > >> user interface, etc. > >> We have the plan to visit Linux World in San Francisco, this August. > >> If you are there, we will find the time to meet you, and wanna talk > >> to you regarding colloaboration and our plan. > > > >I will not be there as you know it is difficult to generate travel > >funding for all Linux conferences. However i will be there to present my > >Paper on LTP in OLS 2008 at Ottowa. If some of you are there, then > >definitely we can discuss things and move forward. > > > >Meanwhile i believe that the test cases that you have written will help > >us a lot if we can import from you under the GPLv2. The present > >challenge before us is to find out ways to test the untested parts of > >kernel. We need test cases for them. We are not adding even fraction of > >test cases of what is going in to the kernel code. So, fresh test cases > >(as can be found from your project) is of great help. I have also read > >in your OLS 2007 paper, the future work that you have envisaged in this > >regard. We would appreciate that as it will unleash another round of > >fresh test cases for testing the Linux kernel. > > > >Regards-- > >Subrata > > > >> > >> Your regards, > >> > >> Hashimoto Hisashi > >> > >> >送信者 : Subrata Modak <su...@li...> > >> >主題 : Re: [Crackerjack-devel] [LTP] Crackerjack and Linux Test Project > >> >受信日 :08/06/09 15:02 > >> >属性 : なし > >> > > >> >I find out that most of the test cases in Crackerjack has been written > >> >by you. I have earlier sent a mail regarding leveraging Cracker Syscall > >> >tests for LTP. I would like to bridge the gap in no. of test cases > >> >available in LTP, by importing the same from Crackerjack project. Since, > >> >most of these tests in Crackerjack are licensed under GPLv2, i hope you > >> >will not have much problem in donating those useful tests for LTP. > >> > > >> >Regards-- > >> >Subrata > >> > > >> >On Tue, 2008-05-27 at 20:46 +0530, Subrata Modak wrote: > >> >> Hi Crackerjack Users/Developers, > >> >> > >> >> I was happening to browse through your home page: > >> >> > >> >> https://sourceforge.net/projects/crackerjack, > >> >> > >> >> and also went through your OLS 2007 paper on > >> >> > >> >> Regression Test Framework and Kernel Execution Coverage. > >> >> (http://ols.108.redhat.com/2007/Reprints/yoshioka-Reprint.pdf) > >> >> > >> >> Your paper mentions that you can work with LTP to bring Regression > >> >> Testing to more greater heights. I would also be interested for it. Let > >> >> me know how we can start and move forward on this. I would also be > >> >> interested to see if we can leverage your tests for LTP. > >> >> > >> >> Regards-- > >> >> Subrata > >> >> > >> >> > >> >> ------------------------------------------------------------------------- > >> >> This SF.net email is sponsored by: Microsoft > >> >> Defy all challenges. Microsoft(R) Visual Studio 2008. > >> >> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > >> >> _______________________________________________ > >> >> Ltp-list mailing list > >> >> Ltp...@li... > >> >> https://lists.sourceforge.net/lists/listinfo/ltp-list > >> > > >> > > >> >------------------------------------------------------------------------- > >> >Check out the new SourceForge.net Marketplace. > >> >It's the best place to buy or sell services for > >> >just about anything Open Source. > >> >http://sourceforge.net/services/buy/index.php > >> >_______________________________________________ > >> >Crackerjack-devel mailing list > >> >Cra...@li... > >> >https://lists.sourceforge.net/lists/listinfo/crackerjack-devel > >> > > >> Hisashi Hashimoto > >> Section Manager > >> Open Source Software Promotion Center > >> Platform Software > >> Hitachi, Ltd., Software Division > >> Tel : +81-45-862-8424, Fax : +81-45-862-9047 > >> his...@hi... > > > > > Hisashi Hashimoto > Section Manager > Open Source Software Promotion Center > Platform Software > Hitachi, Ltd., Software Division > Tel : +81-45-862-8424, Fax : +81-45-862-9047 > his...@hi... |
From: <his...@hi...> - 2008-06-17 03:08:12
|
>Alas, i am not in US. I work from Bangalore, India. Oh, I feel sorry I can not be in India. I will summarize our internal discussion, and send you in soon. Regards, Hashimoto Hisashi >送信者 : Subrata Modak <su...@li...> >主題 : Re: Re[2]: Re[2]: [Crackerjack-devel] [LTP] Crackerjack and LinuxTestProject >受信日 :08/06/14 15:47 >属性 : なし > >On Fri, 2008-06-13 at 12:42 +0900, his...@hi... >wrote: >> Unfortunatelly, >> I will not be on OLS. >> Could I visit your office and discuss while I am in > >You can visit my office/home. > >> US in August ? > >Alas, i am not in US. I work from Bangalore, India. > >Regards-- >Subrata > >> >> Let me know your idea. >> >> Thanks in advance, >> >> Hashimoto Hisashi >> >> >送信者 : Subrata Modak <su...@li...> >> >主題 : Re: Re[2]: [Crackerjack-devel] [LTP] Crackerjack and Linux TestProject >> >受信日 :08/06/11 19:56 >> >属性 : なし >> > >> >On Wed, 2008-06-11 at 12:40 +0900, his...@hi... >> >wrote: >> >> Hi, Subrata, >> >> >> >> This is Hisashi Hashimoto, I am working as project leader >> >> of Crackerjack. Sorry for getting back to you late. >> >> We would like to collaborate you regarding test cases. >> >> We are discussing about next step, enhancing test cases, improving >> >> user interface, etc. >> >> We have the plan to visit Linux World in San Francisco, this August. >> >> If you are there, we will find the time to meet you, and wanna talk >> >> to you regarding colloaboration and our plan. >> > >> >I will not be there as you know it is difficult to generate travel >> >funding for all Linux conferences. However i will be there to present my >> >Paper on LTP in OLS 2008 at Ottowa. If some of you are there, then >> >definitely we can discuss things and move forward. >> > >> >Meanwhile i believe that the test cases that you have written will help >> >us a lot if we can import from you under the GPLv2. The present >> >challenge before us is to find out ways to test the untested parts of >> >kernel. We need test cases for them. We are not adding even fraction of >> >test cases of what is going in to the kernel code. So, fresh test cases >> >(as can be found from your project) is of great help. I have also read >> >in your OLS 2007 paper, the future work that you have envisaged in this >> >regard. We would appreciate that as it will unleash another round of >> >fresh test cases for testing the Linux kernel. >> > >> >Regards-- >> >Subrata >> > >> >> >> >> Your regards, >> >> >> >> Hashimoto Hisashi >> >> >> >> >送信者 : Subrata Modak <su...@li...> >> >> >主題 : Re: [Crackerjack-devel] [LTP] Crackerjack and Linux Test Project >> >> >受信日 :08/06/09 15:02 >> >> >属性 : なし >> >> > >> >> >I find out that most of the test cases in Crackerjack has been written >> >> >by you. I have earlier sent a mail regarding leveraging Cracker Syscall >> >> >tests for LTP. I would like to bridge the gap in no. of test cases >> >> >available in LTP, by importing the same from Crackerjack project. Since, >> >> >most of these tests in Crackerjack are licensed under GPLv2, i hope you >> >> >will not have much problem in donating those useful tests for LTP. >> >> > >> >> >Regards-- >> >> >Subrata >> >> > >> >> >On Tue, 2008-05-27 at 20:46 +0530, Subrata Modak wrote: >> >> >> Hi Crackerjack Users/Developers, >> >> >> >> >> >> I was happening to browse through your home page: >> >> >> >> >> >> https://sourceforge.net/projects/crackerjack, >> >> >> >> >> >> and also went through your OLS 2007 paper on >> >> >> >> >> >> Regression Test Framework and Kernel Execution Coverage. >> >> >> (http://ols.108.redhat.com/2007/Reprints/yoshioka-Reprint.pdf) >> >> >> >> >> >> Your paper mentions that you can work with LTP to bring Regression >> >> >> Testing to more greater heights. I would also be interested for it. Let >> >> >> me know how we can start and move forward on this. I would also be >> >> >> interested to see if we can leverage your tests for LTP. >> >> >> >> >> >> Regards-- >> >> >> Subrata >> >> >> >> >> >> >> >> >> ------------------------------------------------------------------------- >> >> >> This SF.net email is sponsored by: Microsoft >> >> >> Defy all challenges. Microsoft(R) Visual Studio 2008. >> >> >> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ >> >> >> _______________________________________________ >> >> >> Ltp-list mailing list >> >> >> Ltp...@li... >> >> >> https://lists.sourceforge.net/lists/listinfo/ltp-list >> >> > >> >> > >> >> >------------------------------------------------------------------------- >> >> >Check out the new SourceForge.net Marketplace. >> >> >It's the best place to buy or sell services for >> >> >just about anything Open Source. >> >> >http://sourceforge.net/services/buy/index.php >> >> >_______________________________________________ >> >> >Crackerjack-devel mailing list >> >> >Cra...@li... >> >> >https://lists.sourceforge.net/lists/listinfo/crackerjack-devel >> >> > >> >> Hisashi Hashimoto >> >> Section Manager >> >> Open Source Software Promotion Center >> >> Platform Software >> >> Hitachi, Ltd., Software Division >> >> Tel : +81-45-862-8424, Fax : +81-45-862-9047 >> >> his...@hi... >> > >> > >> Hisashi Hashimoto >> Section Manager >> Open Source Software Promotion Center >> Platform Software >> Hitachi, Ltd., Software Division >> Tel : +81-45-862-8424, Fax : +81-45-862-9047 >> his...@hi... > > Hisashi Hashimoto Section Manager Open Source Software Promotion Center Platform Software Hitachi, Ltd., Software Division Tel : +81-45-862-8424, Fax : +81-45-862-9047 his...@hi... |
From: Subrata M. <su...@li...> - 2008-06-17 09:08:26
|
On Tue, 2008-06-17 at 12:07 +0900, his...@hi... wrote: > >Alas, i am not in US. I work from Bangalore, India. > Oh, I feel sorry I can not be in India. > I will summarize our internal discussion, > and send you in soon. That will really be great. Regards-- Subrata > > Regards, > > Hashimoto Hisashi > > >送信者 : Subrata Modak <su...@li...> > >主題 : Re: Re[2]: Re[2]: [Crackerjack-devel] [LTP] Crackerjack and LinuxTestProject > >受信日 :08/06/14 15:47 > >属性 : なし > > > >On Fri, 2008-06-13 at 12:42 +0900, his...@hi... > >wrote: > >> Unfortunatelly, > >> I will not be on OLS. > >> Could I visit your office and discuss while I am in > > > >You can visit my office/home. > > > >> US in August ? > > > >Alas, i am not in US. I work from Bangalore, India. > > > >Regards-- > >Subrata > > > >> > >> Let me know your idea. > >> > >> Thanks in advance, > >> > >> Hashimoto Hisashi > >> > >> >送信者 : Subrata Modak <su...@li...> > >> >主題 : Re: Re[2]: [Crackerjack-devel] [LTP] Crackerjack and Linux TestProject > >> >受信日 :08/06/11 19:56 > >> >属性 : なし > >> > > >> >On Wed, 2008-06-11 at 12:40 +0900, his...@hi... > >> >wrote: > >> >> Hi, Subrata, > >> >> > >> >> This is Hisashi Hashimoto, I am working as project leader > >> >> of Crackerjack. Sorry for getting back to you late. > >> >> We would like to collaborate you regarding test cases. > >> >> We are discussing about next step, enhancing test cases, improving > >> >> user interface, etc. > >> >> We have the plan to visit Linux World in San Francisco, this August. > >> >> If you are there, we will find the time to meet you, and wanna talk > >> >> to you regarding colloaboration and our plan. > >> > > >> >I will not be there as you know it is difficult to generate travel > >> >funding for all Linux conferences. However i will be there to present my > >> >Paper on LTP in OLS 2008 at Ottowa. If some of you are there, then > >> >definitely we can discuss things and move forward. > >> > > >> >Meanwhile i believe that the test cases that you have written will help > >> >us a lot if we can import from you under the GPLv2. The present > >> >challenge before us is to find out ways to test the untested parts of > >> >kernel. We need test cases for them. We are not adding even fraction of > >> >test cases of what is going in to the kernel code. So, fresh test cases > >> >(as can be found from your project) is of great help. I have also read > >> >in your OLS 2007 paper, the future work that you have envisaged in this > >> >regard. We would appreciate that as it will unleash another round of > >> >fresh test cases for testing the Linux kernel. > >> > > >> >Regards-- > >> >Subrata > >> > > >> >> > >> >> Your regards, > >> >> > >> >> Hashimoto Hisashi > >> >> > >> >> >送信者 : Subrata Modak <su...@li...> > >> >> >主題 : Re: [Crackerjack-devel] [LTP] Crackerjack and Linux Test Project > >> >> >受信日 :08/06/09 15:02 > >> >> >属性 : なし > >> >> > > >> >> >I find out that most of the test cases in Crackerjack has been written > >> >> >by you. I have earlier sent a mail regarding leveraging Cracker Syscall > >> >> >tests for LTP. I would like to bridge the gap in no. of test cases > >> >> >available in LTP, by importing the same from Crackerjack project. Since, > >> >> >most of these tests in Crackerjack are licensed under GPLv2, i hope you > >> >> >will not have much problem in donating those useful tests for LTP. > >> >> > > >> >> >Regards-- > >> >> >Subrata > >> >> > > >> >> >On Tue, 2008-05-27 at 20:46 +0530, Subrata Modak wrote: > >> >> >> Hi Crackerjack Users/Developers, > >> >> >> > >> >> >> I was happening to browse through your home page: > >> >> >> > >> >> >> https://sourceforge.net/projects/crackerjack, > >> >> >> > >> >> >> and also went through your OLS 2007 paper on > >> >> >> > >> >> >> Regression Test Framework and Kernel Execution Coverage. > >> >> >> (http://ols.108.redhat.com/2007/Reprints/yoshioka-Reprint.pdf) > >> >> >> > >> >> >> Your paper mentions that you can work with LTP to bring Regression > >> >> >> Testing to more greater heights. I would also be interested for it. Let > >> >> >> me know how we can start and move forward on this. I would also be > >> >> >> interested to see if we can leverage your tests for LTP. > >> >> >> > >> >> >> Regards-- > >> >> >> Subrata > >> >> >> > >> >> >> > >> >> >> ------------------------------------------------------------------------- > >> >> >> This SF.net email is sponsored by: Microsoft > >> >> >> Defy all challenges. Microsoft(R) Visual Studio 2008. > >> >> >> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > >> >> >> _______________________________________________ > >> >> >> Ltp-list mailing list > >> >> >> Ltp...@li... > >> >> >> https://lists.sourceforge.net/lists/listinfo/ltp-list > >> >> > > >> >> > > >> >> >------------------------------------------------------------------------- > >> >> >Check out the new SourceForge.net Marketplace. > >> >> >It's the best place to buy or sell services for > >> >> >just about anything Open Source. > >> >> >http://sourceforge.net/services/buy/index.php > >> >> >_______________________________________________ > >> >> >Crackerjack-devel mailing list > >> >> >Cra...@li... > >> >> >https://lists.sourceforge.net/lists/listinfo/crackerjack-devel > >> >> > > >> >> Hisashi Hashimoto > >> >> Section Manager > >> >> Open Source Software Promotion Center > >> >> Platform Software > >> >> Hitachi, Ltd., Software Division > >> >> Tel : +81-45-862-8424, Fax : +81-45-862-9047 > >> >> his...@hi... > >> > > >> > > >> Hisashi Hashimoto > >> Section Manager > >> Open Source Software Promotion Center > >> Platform Software > >> Hitachi, Ltd., Software Division > >> Tel : +81-45-862-8424, Fax : +81-45-862-9047 > >> his...@hi... > > > > > Hisashi Hashimoto > Section Manager > Open Source Software Promotion Center > Platform Software > Hitachi, Ltd., Software Division > Tel : +81-45-862-8424, Fax : +81-45-862-9047 > his...@hi... |
From: <his...@hi...> - 2008-07-11 04:53:25
|
Subrata, Sorry for long time no reply. We had internal meeting of our project regarding how to proceed. Current our plan is: (1) Unfortunattelly, our test cases do not cover all system call yet. Total number of system call in Kernel 2.6.26 is almost 326. But, we developed around 299. Therefore, we would like to start developing in soon. The starting date would be some time in August. (2) Some of our developed test cases do not meet our quality criteria. As you know, we set our quality goal as covarage ratio more than yours. We are now checking all tests again. And if necessary, we need to enhance test cases. (3) Through our acitivity (1) and (2), we would like to exchange information and technical/engineering discussion on this mailing list. Yesterday, 4 Maintainers(Andrew Morton, Paul Moor, James Morris, Thomas Gleixner) visit Japan to make presentation regarding the latest kernel/security activity. I had opportunity to make presentation to them about our current status. I present that (1) We released test cases 300/326. (2) We plan to make complete set and enhance quality. (3) We continue to talk with other test project(LTP as you, AutoTest:Martin Bleigh), and exchange information and status, and others. Maintainers said/recommend to expand the target, to continue the effort and collaborate with LTP and so on. I hope you and we can do the collaboration work on test project. Best, Hashimoto Hisashi >送信者 : Subrata Modak <su...@li...> >主題 : Re: Re[2]: Re[2]: Re[2]: [Crackerjack-devel] [LTP] Crackerjack andLinuxTestProje >受信日 :08/06/18 17:43 >属性 : なし > >On Tue, 2008-06-17 at 12:07 +0900, his...@hi... >wrote: >> >Alas, i am not in US. I work from Bangalore, India. >> Oh, I feel sorry I can not be in India. >> I will summarize our internal discussion, >> and send you in soon. > >That will really be great. > >Regards-- >Subrata > >> >> Regards, >> >> Hashimoto Hisashi >> >> >送信者 : Subrata Modak <su...@li...> >> >主題 : Re: Re[2]: Re[2]: [Crackerjack-devel] [LTP] Crackerjack and LinuxTestProject >> >受信日 :08/06/14 15:47 >> >属性 : なし >> > >> >On Fri, 2008-06-13 at 12:42 +0900, his...@hi... >> >wrote: >> >> Unfortunatelly, >> >> I will not be on OLS. >> >> Could I visit your office and discuss while I am in >> > >> >You can visit my office/home. >> > >> >> US in August ? >> > >> >Alas, i am not in US. I work from Bangalore, India. >> > >> >Regards-- >> >Subrata >> > >> >> >> >> Let me know your idea. >> >> >> >> Thanks in advance, >> >> >> >> Hashimoto Hisashi >> >> >> >> >送信者 : Subrata Modak <su...@li...> >> >> >主題 : Re: Re[2]: [Crackerjack-devel] [LTP] Crackerjack and Linux TestProject >> >> >受信日 :08/06/11 19:56 >> >> >属性 : なし >> >> > >> >> >On Wed, 2008-06-11 at 12:40 +0900, his...@hi... >> >> >wrote: >> >> >> Hi, Subrata, >> >> >> >> >> >> This is Hisashi Hashimoto, I am working as project leader >> >> >> of Crackerjack. Sorry for getting back to you late. >> >> >> We would like to collaborate you regarding test cases. >> >> >> We are discussing about next step, enhancing test cases, improving >> >> >> user interface, etc. >> >> >> We have the plan to visit Linux World in San Francisco, this August. >> >> >> If you are there, we will find the time to meet you, and wanna talk >> >> >> to you regarding colloaboration and our plan. >> >> > >> >> >I will not be there as you know it is difficult to generate travel >> >> >funding for all Linux conferences. However i will be there to present my >> >> >Paper on LTP in OLS 2008 at Ottowa. If some of you are there, then >> >> >definitely we can discuss things and move forward. >> >> > >> >> >Meanwhile i believe that the test cases that you have written will help >> >> >us a lot if we can import from you under the GPLv2. The present >> >> >challenge before us is to find out ways to test the untested parts of >> >> >kernel. We need test cases for them. We are not adding even fraction of >> >> >test cases of what is going in to the kernel code. So, fresh test cases >> >> >(as can be found from your project) is of great help. I have also read >> >> >in your OLS 2007 paper, the future work that you have envisaged in this >> >> >regard. We would appreciate that as it will unleash another round of >> >> >fresh test cases for testing the Linux kernel. >> >> > >> >> >Regards-- >> >> >Subrata >> >> > >> >> >> >> >> >> Your regards, >> >> >> >> >> >> Hashimoto Hisashi >> >> >> >> >> >> >送信者 : Subrata Modak <su...@li...> >> >> >> >主題 : Re: [Crackerjack-devel] [LTP] Crackerjack and Linux Test Project >> >> >> >受信日 :08/06/09 15:02 >> >> >> >属性 : なし >> >> >> > >> >> >> >I find out that most of the test cases in Crackerjack has been written >> >> >> >by you. I have earlier sent a mail regarding leveraging Cracker Syscall >> >> >> >tests for LTP. I would like to bridge the gap in no. of test cases >> >> >> >available in LTP, by importing the same from Crackerjack project. Since, >> >> >> >most of these tests in Crackerjack are licensed under GPLv2, i hope you >> >> >> >will not have much problem in donating those useful tests for LTP. >> >> >> > >> >> >> >Regards-- >> >> >> >Subrata >> >> >> > >> >> >> >On Tue, 2008-05-27 at 20:46 +0530, Subrata Modak wrote: >> >> >> >> Hi Crackerjack Users/Developers, >> >> >> >> >> >> >> >> I was happening to browse through your home page: >> >> >> >> >> >> >> >> https://sourceforge.net/projects/crackerjack, >> >> >> >> >> >> >> >> and also went through your OLS 2007 paper on >> >> >> >> >> >> >> >> Regression Test Framework and Kernel Execution Coverage. >> >> >> >> (http://ols.108.redhat.com/2007/Reprints/yoshioka-Reprint.pdf) >> >> >> >> >> >> >> >> Your paper mentions that you can work with LTP to bring Regression >> >> >> >> Testing to more greater heights. I would also be interested for it. Let >> >> >> >> me know how we can start and move forward on this. I would also be >> >> >> >> interested to see if we can leverage your tests for LTP. >> >> >> >> >> >> >> >> Regards-- >> >> >> >> Subrata >> >> >> >> >> >> >> >> >> >> >> >> ------------------------------------------------------------------------- >> >> >> >> This SF.net email is sponsored by: Microsoft >> >> >> >> Defy all challenges. Microsoft(R) Visual Studio 2008. >> >> >> >> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ >> >> >> >> _______________________________________________ >> >> >> >> Ltp-list mailing list >> >> >> >> Ltp...@li... >> >> >> >> https://lists.sourceforge.net/lists/listinfo/ltp-list >> >> >> > >> >> >> > >> >> >> >------------------------------------------------------------------------- >> >> >> >Check out the new SourceForge.net Marketplace. >> >> >> >It's the best place to buy or sell services for >> >> >> >just about anything Open Source. >> >> >> >http://sourceforge.net/services/buy/index.php >> >> >> >_______________________________________________ >> >> >> >Crackerjack-devel mailing list >> >> >> >Cra...@li... >> >> >> >https://lists.sourceforge.net/lists/listinfo/crackerjack-devel >> >> >> > >> >> >> Hisashi Hashimoto >> >> >> Section Manager >> >> >> Open Source Software Promotion Center >> >> >> Platform Software >> >> >> Hitachi, Ltd., Software Division >> >> >> Tel : +81-45-862-8424, Fax : +81-45-862-9047 >> >> >> his...@hi... >> >> > >> >> > >> >> Hisashi Hashimoto >> >> Section Manager >> >> Open Source Software Promotion Center >> >> Platform Software >> >> Hitachi, Ltd., Software Division >> >> Tel : +81-45-862-8424, Fax : +81-45-862-9047 >> >> his...@hi... >> > >> > >> Hisashi Hashimoto >> Section Manager >> Open Source Software Promotion Center >> Platform Software >> Hitachi, Ltd., Software Division >> Tel : +81-45-862-8424, Fax : +81-45-862-9047 >> his...@hi... > > Hisashi Hashimoto Section Manager Open Source Software Promotion Center Platform Software Hitachi, Ltd., Software Division Tel : +81-45-862-8424, Fax : +81-45-862-9047 his...@hi... |
From: Subrata M. <su...@li...> - 2008-07-11 19:48:29
|
Dear Hisashi Hashimoto San, Thank you for writing back to us. We are delighted to colloborate on all fronts to make Linux testing better. My comments embedded in the following lines. On Fri, 2008-07-11 at 13:53 +0900, his...@hi... wrote: > Subrata, > > Sorry for long time no reply. > We had internal meeting of our project > regarding how to proceed. > Current our plan is: > (1) Unfortunattelly, our test cases do not cover all system call yet. > Total number of system call in Kernel 2.6.26 is almost 326. > But, we developed around 299. Masatake is already working to port the missing syscall test cases in LTP from the Crackerjack Project to LTP. And it is already a huge effort. I think the idea here is great as we are not re-inventing the same wheel. We can use your test cases released under GPLv2 and port to LTP format. We gain by getting more kernel test cases in LTP without the need to massively write things. And you retain your goal of providing a regression comparisn framework. In future, whenever the no. of syscalls goes increasing in kernels, and as soon as you write them ,we will try to port from you, unless somebody in LTP community has already submitted a test case in that regard. Or, even we can concentrate to write test cases in some other are as well. Presently, the first job is to port the existing ones from you to us. > Therefore, we would like to start developing in soon. > The starting date would be some time in August. > (2) Some of our developed test cases do not meet our quality criteria. > As you know, we set our quality goal as covarage ratio more than > yours. > We are now checking all tests again. And if necessary, we need to > enhance test cases. Good to know that. As and when you do this, we might also relook into ours and change accordingly. > (3) Through our acitivity (1) and (2), we would like to exchange > information and technical/engineering discussion on this mailing list. Definitely yes. All of you can subscribe to the ltp mailing list: https://lists.sourceforge.net/mailman/listinfo/ltp-list And please add us as CC to all the technical discussion you have. Guys in ltp mailing list will need to subscribe yours if they want to reply back to you. > > Yesterday, 4 Maintainers(Andrew Morton, Paul Moor, James Morris, Thomas Gleixner) > visit Japan to make presentation regarding the latest kernel/security activity. > I had opportunity to make presentation to them about our current status. > I present that > (1) We released test cases 300/326. > (2) We plan to make complete set and enhance quality. > (3) We continue to talk with other test project(LTP as you, AutoTest:Martin Bleigh), > and exchange information and status, and others. > Maintainers said/recommend to expand the target, to continue the effort > and collaborate with LTP and so on. We will be collaborating as far as we can, as all of us striving to Linux better. Regards-- Subrata > > I hope you and we can do the collaboration work on test project. > > Best, > > Hashimoto Hisashi |
From: <his...@hi...> - 2008-07-14 04:40:10
|
Subrata, >Masatake is already working to port the missing syscall test cases in >LTP from the Crackerjack Project to LTP. And it is already a huge I am glad to hear that. >effort. I think the idea here is great as we are not re-inventing the Yes, you can do this and we can use our time to develope more test cases. >to port from you, unless somebody in LTP community has already submitted In that case, we will refer your cases and make them better. >https://lists.sourceforge.net/mailman/listinfo/ltp-list I joined now. I am very surprised that your community are working so active. I will check and I will post anything I can collaborate. >We will be collaborating as far as we can, as all of us striving to >Linux better. Yes, agree. We will continue this effort for our purpose, Linux better. Please post anything to both our mailing list and cc-d your mailing list when you have to contact me. Thanks, Hisashi >送信者 : Subrata Modak <su...@li...> >主題 : Re: Re[2]: Re[2]: Re[2]: Re[2]: [Crackerjack-devel] [LTP]Crackerjack andLinuxTes >受信日 :08/07/12 04:10 >属性 : なし > >Dear Hisashi Hashimoto San, > >Thank you for writing back to us. We are delighted to colloborate on all >fronts to make Linux testing better. My comments embedded in the >following lines. > >On Fri, 2008-07-11 at 13:53 +0900, his...@hi... >wrote: >> Subrata, >> >> Sorry for long time no reply. >> We had internal meeting of our project >> regarding how to proceed. >> Current our plan is: >> (1) Unfortunattelly, our test cases do not cover all system call yet. >> Total number of system call in Kernel 2.6.26 is almost 326. >> But, we developed around 299. > >Masatake is already working to port the missing syscall test cases in >LTP from the Crackerjack Project to LTP. And it is already a huge >effort. I think the idea here is great as we are not re-inventing the >same wheel. We can use your test cases released under GPLv2 and port to >LTP format. > >We gain by getting more kernel test cases in LTP without the need to >massively write things. And you retain your goal of providing a >regression comparisn framework. In future, whenever the no. of syscalls >goes increasing in kernels, and as soon as you write them ,we will try >to port from you, unless somebody in LTP community has already submitted >a test case in that regard. Or, even we can concentrate to write test >cases in some other are as well. > >Presently, the first job is to port the existing ones from you to us. > >> Therefore, we would like to start developing in soon. >> The starting date would be some time in August. >> (2) Some of our developed test cases do not meet our quality criteria. >> As you know, we set our quality goal as covarage ratio more than >> yours. >> We are now checking all tests again. And if necessary, we need to >> enhance test cases. > >Good to know that. As and when you do this, we might also relook into >ours and change accordingly. > >> (3) Through our acitivity (1) and (2), we would like to exchange >> information and technical/engineering discussion on this mailing list. > >Definitely yes. All of you can subscribe to the ltp mailing list: >https://lists.sourceforge.net/mailman/listinfo/ltp-list > >And please add us as CC to all the technical discussion you have. Guys >in ltp mailing list will need to subscribe yours if they want to reply >back to you. > >> >> Yesterday, 4 Maintainers(Andrew Morton, Paul Moor, James Morris, Thomas Gleixner) >> visit Japan to make presentation regarding the latest kernel/security activity. >> I had opportunity to make presentation to them about our current status. >> I present that >> (1) We released test cases 300/326. >> (2) We plan to make complete set and enhance quality. >> (3) We continue to talk with other test project(LTP as you, AutoTest:Martin Bleigh), >> and exchange information and status, and others. >> Maintainers said/recommend to expand the target, to continue the effort >> and collaborate with LTP and so on. > >We will be collaborating as far as we can, as all of us striving to >Linux better. > > >Regards-- >Subrata > >> >> I hope you and we can do the collaboration work on test project. >> >> Best, >> >> Hashimoto Hisashi > > > Hisashi Hashimoto Section Manager Open Source Software Promotion Center Platform Software Hitachi, Ltd., Software Division Tel : +81-45-862-8424, Fax : +81-45-862-9047 his...@hi... |
From: Masatake Y. <ya...@re...> - 2008-07-16 09:24:30
|
> From now on, I'll be agitating more to get man pages provided more with new > syscalls and ther kernel-userland interfaces. That will mean either I twist > developers arms to write pages ;-), or I write them myself, with help from > them. I do think that man-pages, if well written, are often sufficient as > (or at least a very good base for) a test specification. Here's an example > that I did with the timerfd API, finding two bugs in the process: > http://thread.gmane.org/gmane.linux.kernel/613442 . I did something similar > while writing the utimensat(2) man page, finding 5 or 6 different bugs in > the end, see > http://linux-man-pages.blogspot.com/2008/06/whats-wrong-with-kernel-userland_30.html And from now on, I'll be agitating much more to report a mistake in man pages if you, a test case auther, found it during writing test cases. Generally we can expect a test case auther reads man pages very carefully. Such a person may have much chance to find mistake in man page (than kernel developers:-) If a kernel developer writes both test cases, and man pages, it is very nice. However, checking each other by independent teams like test case authors and man page authors is also good. When I received a bug report about my test case and I confirmed that there were no bug in my test case itself, I had to inspect both the kernel/libc code and man page. This is the most exciting experience during working on LTP for me. Once I concluded to send a patch to LKML: http://www.opensubscriber.com/message/lin...@vg.../8342264.html Once I concluded to report a mistake to Michael: http://www.mail-archive.com/ltp...@li.../msg02730.html How about opposite direction? Tracking all discussion in LKML is hard. However, tracking changes in the section 2 of man pages are easier than tracking LKML. If the page in the section is changed, it may have impact on test cases for the system call. I hit on a term "the division of the three powers." Masatake YAMATO |
From: Michael K. <mtk...@go...> - 2008-07-16 09:32:30
|
On Wed, Jul 16, 2008 at 11:23 AM, Masatake YAMATO <ya...@re...> wrote: >> From now on, I'll be agitating more to get man pages provided more with new >> syscalls and ther kernel-userland interfaces. That will mean either I twist >> developers arms to write pages ;-), or I write them myself, with help from >> them. I do think that man-pages, if well written, are often sufficient as >> (or at least a very good base for) a test specification. Here's an example >> that I did with the timerfd API, finding two bugs in the process: >> http://thread.gmane.org/gmane.linux.kernel/613442 . I did something similar >> while writing the utimensat(2) man page, finding 5 or 6 different bugs in >> the end, see >> http://linux-man-pages.blogspot.com/2008/06/whats-wrong-with-kernel-userland_30.html > > > And from now on, I'll be agitating much more to report a mistake in > man pages if you, a test case auther, found it during writing test > cases. Yes, please! Now that I have more time for man-pages, I should usually be able to respond quickly to such reports. > Generally we can expect a test case auther reads man pages very carefully. > Such a person may have much chance to find mistake in man page (than kernel > developers:-) Yes. > If a kernel developer writes both test cases, and man pages, it is very nice. > However, checking each other by independent teams like test case authors and > man page authors is also good. Yes; indeed it is better. An implementer can be inclined to make assumptions about their own code, and then not test those asumptions; implementers are also sometimes just lazy about testing. Having other people involved in testing counteracts those problems. > When I received a bug report about my test case and I confirmed that there > were no bug in my test case itself, I had to inspect both the kernel/libc > code and man page. This is the most exciting experience during working on > LTP for me. > > Once I concluded to send a patch to LKML: > > http://www.opensubscriber.com/message/lin...@vg.../8342264.html > > Once I concluded to report a mistake to Michael: > > http://www.mail-archive.com/ltp...@li.../msg02730.html > > How about opposite direction? > Tracking all discussion in LKML is hard. Yes, it is. > However, tracking changes in > the section 2 of man pages are easier than tracking LKML. If the page > in the section is changed, it may have impact on test cases for the > system call. This is true. Of course, I'm still trying to solve the problem of how *I* find out about all of the changes in the kernel so that the man pages can be updated accordingly. > I hit on a term "the division of the three powers." ;-) Cheers, Michael -- Michael Kerrisk Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/ man-pages online: http://www.kernel.org/doc/man-pages/online_pages.html Found a bug? http://www.kernel.org/doc/man-pages/reporting_bugs.html |
From: Serge E. H. <se...@us...> - 2008-07-16 13:46:31
|
Quoting Michael Kerrisk (mtk...@go...): > On Wed, Jul 16, 2008 at 11:23 AM, Masatake YAMATO <ya...@re...> wrote: > >> From now on, I'll be agitating more to get man pages provided more with new > >> syscalls and ther kernel-userland interfaces. That will mean either I twist > >> developers arms to write pages ;-), or I write them myself, with help from > >> them. I do think that man-pages, if well written, are often sufficient as > >> (or at least a very good base for) a test specification. Here's an example > >> that I did with the timerfd API, finding two bugs in the process: > >> http://thread.gmane.org/gmane.linux.kernel/613442 . I did something similar > >> while writing the utimensat(2) man page, finding 5 or 6 different bugs in > >> the end, see > >> http://linux-man-pages.blogspot.com/2008/06/whats-wrong-with-kernel-userland_30.html > > > > > > And from now on, I'll be agitating much more to report a mistake in > > man pages if you, a test case auther, found it during writing test > > cases. > > Yes, please! Now that I have more time for man-pages, I should > usually be able to respond quickly to such reports. > > > Generally we can expect a test case auther reads man pages very carefully. > > Such a person may have much chance to find mistake in man page (than kernel > > developers:-) > > Yes. > > > If a kernel developer writes both test cases, and man pages, it is very nice. > > However, checking each other by independent teams like test case authors and > > man page authors is also good. > > Yes; indeed it is better. An implementer can be inclined to make > assumptions about their own code, and then not test those asumptions; > implementers are also sometimes just lazy about testing. Having other > people involved in testing counteracts those problems. > > > When I received a bug report about my test case and I confirmed that there > > were no bug in my test case itself, I had to inspect both the kernel/libc > > code and man page. This is the most exciting experience during working on > > LTP for me. > > > > Once I concluded to send a patch to LKML: > > > > http://www.opensubscriber.com/message/lin...@vg.../8342264.html > > > > Once I concluded to report a mistake to Michael: > > > > http://www.mail-archive.com/ltp...@li.../msg02730.html > > > > How about opposite direction? > > Tracking all discussion in LKML is hard. > > Yes, it is. > > > However, tracking changes in > > the section 2 of man pages are easier than tracking LKML. If the page > > in the section is changed, it may have impact on test cases for the > > system call. > > This is true. Of course, I'm still trying to solve the problem of how > *I* find out about all of the changes in the kernel so that the man > pages can be updated accordingly. It might help to lobby for an addition to Documentation/SubmitChecklist or SubmittingPatches to mention checking whether changes to manpages are necessary. -serge |
From: Subrata M. <su...@li...> - 2008-07-14 11:05:25
|
Dear Hisashi San, Since we have already embarked on our path to mutual co-operation in Linux testing and test cases development, i would like to discuss some of my ideas, which i think is capable of adding shot-in-the arm for linux testing. I want to talk about Test Driven Development for Linux Kernel Testing, which can bring us very early effective results and can catch Bug(s), if any, in the kernel code much much early, and even before the code makes it´s way through the stable kernel release. The idea is to submit the test cases to LTP, for certain kernel features, which makes its way through the kernel-mm tree. When a developer submits his/her feature to mm-kernel, he gives a specification of the features. Depending on the feature specification available either from the developer himself, or, from manpages, if it is available that soon, tests cases will be written. Now, we have an inherent question, who writes and submits those test cases ?? Either it can be the developer himself, or, somebody from the test community. Since, the developer is not bound to submit any test case, we can assume we need to initiate doing something in this regard. Having said that, i want to propose that, since in your project you have already taken up the Job of making available the system call test cases up-to-date, we can score a point here. Whenever a new system call code enters the mm-tree in the kernel, somebody in your project can write the test case(s) immediately and make it available to: 1) Your Project, 2) LTP, 3) LKML, We will use the test case(s) to test that particular syscall feature and report Bug(s) to the developer(s) and LKML. Meanwhile, updates to the test cases will also go on depending on feedback from community. With that we will address the following issues: 1) Catch bug at the earliest for the new syscalls in the kernel, so that the cost of Bug fixing, when something is discovered later, can be hugely reduced, 2) We have a regular and systematic flow of test cases for both LTP and Crackerjack. This process becomes institutional and gurantees us maximum and continued code coverage for syscalls in future, 3) LTP-mm tree is born, 4) Others/developers get motivated to submit test cases(to LTP and Crackerjack) at the time of submission of kernel features to kernel-mm-tree, if we show them the benefit of early testing and it´s effectiveness of controlling more bugs to seep in before it is too late to fix. For this to happen, we need to make our test cases ready when the corresponding kernel features are in -mm tree. Let me know your ideas on this. Regards-- Subrata On Mon, 2008-07-14 at 13:39 +0900, his...@hi... wrote: > Subrata, > > >Masatake is already working to port the missing syscall test cases in > >LTP from the Crackerjack Project to LTP. And it is already a huge > I am glad to hear that. > >effort. I think the idea here is great as we are not re-inventing the > Yes, you can do this and we can use our time to develope more test cases. > >to port from you, unless somebody in LTP community has already submitted > In that case, we will refer your cases and make them better. > >https://lists.sourceforge.net/mailman/listinfo/ltp-list > I joined now. I am very surprised that your community are > working so active. I will check and I will post anything > I can collaborate. > >We will be collaborating as far as we can, as all of us striving to > >Linux better. > Yes, agree. We will continue this effort for our purpose, Linux better. > Please post anything to both our mailing list and cc-d your mailing list > when you have to contact me. > > Thanks, > > Hisashi > > >送信者 : Subrata Modak <su...@li...> > >主題 : Re: Re[2]: Re[2]: Re[2]: Re[2]: [Crackerjack-devel] [LTP]Crackerjack andLinuxTes > >受信日 :08/07/12 04:10 > >属性 : なし > > > >Dear Hisashi Hashimoto San, > > > >Thank you for writing back to us. We are delighted to colloborate on all > >fronts to make Linux testing better. My comments embedded in the > >following lines. > > > >On Fri, 2008-07-11 at 13:53 +0900, his...@hi... > >wrote: > >> Subrata, > >> > >> Sorry for long time no reply. > >> We had internal meeting of our project > >> regarding how to proceed. > >> Current our plan is: > >> (1) Unfortunattelly, our test cases do not cover all system call yet. > >> Total number of system call in Kernel 2.6.26 is almost 326. > >> But, we developed around 299. > > > >Masatake is already working to port the missing syscall test cases in > >LTP from the Crackerjack Project to LTP. And it is already a huge > >effort. I think the idea here is great as we are not re-inventing the > >same wheel. We can use your test cases released under GPLv2 and port to > >LTP format. > > > >We gain by getting more kernel test cases in LTP without the need to > >massively write things. And you retain your goal of providing a > >regression comparisn framework. In future, whenever the no. of syscalls > >goes increasing in kernels, and as soon as you write them ,we will try > >to port from you, unless somebody in LTP community has already submitted > >a test case in that regard. Or, even we can concentrate to write test > >cases in some other are as well. > > > >Presently, the first job is to port the existing ones from you to us. > > > >> Therefore, we would like to start developing in soon. > >> The starting date would be some time in August. > >> (2) Some of our developed test cases do not meet our quality criteria. > >> As you know, we set our quality goal as covarage ratio more than > >> yours. > >> We are now checking all tests again. And if necessary, we need to > >> enhance test cases. > > > >Good to know that. As and when you do this, we might also relook into > >ours and change accordingly. > > > >> (3) Through our acitivity (1) and (2), we would like to exchange > >> information and technical/engineering discussion on this mailing list. > > > >Definitely yes. All of you can subscribe to the ltp mailing list: > >https://lists.sourceforge.net/mailman/listinfo/ltp-list > > > >And please add us as CC to all the technical discussion you have. Guys > >in ltp mailing list will need to subscribe yours if they want to reply > >back to you. > > > >> > >> Yesterday, 4 Maintainers(Andrew Morton, Paul Moor, James Morris, Thomas Gleixner) > >> visit Japan to make presentation regarding the latest kernel/security activity. > >> I had opportunity to make presentation to them about our current status. > >> I present that > >> (1) We released test cases 300/326. > >> (2) We plan to make complete set and enhance quality. > >> (3) We continue to talk with other test project(LTP as you, AutoTest:Martin Bleigh), > >> and exchange information and status, and others. > >> Maintainers said/recommend to expand the target, to continue the effort > >> and collaborate with LTP and so on. > > > >We will be collaborating as far as we can, as all of us striving to > >Linux better. > > > > > >Regards-- > >Subrata > > > >> > >> I hope you and we can do the collaboration work on test project. > >> > >> Best, > >> > >> Hashimoto Hisashi > > > > > > > Hisashi Hashimoto > Section Manager > Open Source Software Promotion Center > Platform Software > Hitachi, Ltd., Software Division > Tel : +81-45-862-8424, Fax : +81-45-862-9047 > his...@hi... |
From: <his...@hi...> - 2008-07-16 01:53:28
|
Subrata, Michael, Thank you for proposing. ideas >> The idea is to submit the test cases to LTP, for certain kernel >> features, which makes its way through the kernel-mm tree. When a >> developer submits his/her feature to mm-kernel, he gives a specification >> of the features. Depending on the feature specification available either >> from the developer himself, or, from manpages, if it is available that >> soon, tests cases will be written. Basically, I agree your idea. >> Having said that, i want to propose that, since in your project you have >> already taken up the Job of making available the system call test cases >> up-to-date, we can score a point here. Whenever a new system call code >> enters the mm-tree in the kernel, somebody in your project can write the >> test case(s) immediately and make it available to: >> >> 1) Your Project, >> 2) LTP, >> 3) LKML, Agreed. I will negotiate our members in soon. >From now on, I'll be agitating more to get man pages provided more with new >syscalls and ther kernel-userland interfaces. That will mean either I twist Great, These will help us to develop/enhance our test cases better. I will check your pages. >calls. What list do I need to subscribe to? CrackerJack has two mailing list. cra...@li... https://lists.sourceforge.net/lists/listinfo/crackerjack-users cra...@li... https://lists.sourceforge.net/lists/listinfo/crackerjack-devel >I can't help but feel that part of the solution here is to agitate more so >that the developer is (strongly) encouraged to submit test cases along with >patches that change the kernel-userspace interface. As things stand, there >are far too many bugs in released interrfaces. I can't help neither, but I will do talk developers importance of providing/using test cases when new interface has been released. Every users wants to use less bugs kernel, wants to know what difference happens if it exists. Hisashi >送信者 : Michael Kerrisk <mtk...@go...> >主題 : Re: [LTP] Crackerjack and Linux Test Project >受信日 :08/07/15 21:39 >属性 : なし > >On Mon, Jul 14, 2008 at 1:05 PM, Subrata Modak <su...@li...> >wrote: > >> Dear Hisashi San, >> >> Since we have already embarked on our path to mutual co-operation in >> Linux testing and test cases development, i would like to discuss some >> of my ideas, which i think is capable of adding shot-in-the arm for >> linux testing. >> >> I want to talk about Test Driven Development for Linux Kernel Testing, >> which can bring us very early effective results and can catch Bug(s), if >> any, in the kernel code much much early, and even before the code makes >> it´s way through the stable kernel release. >> >> The idea is to submit the test cases to LTP, for certain kernel >> features, which makes its way through the kernel-mm tree. When a >> developer submits his/her feature to mm-kernel, he gives a specification >> of the features. Depending on the feature specification available either >> from the developer himself, or, from manpages, if it is available that >> soon, tests cases will be written. > > >From now on, I'll be agitating more to get man pages provided more with new >syscalls and ther kernel-userland interfaces. That will mean either I twist >developers arms to write pages ;-), or I write them myself, with help from >them. I do think that man-pages, if well written, are often sufficient as >(or at least a very good base for) a test specification. Here's an example >that I did with the timerfd API, finding two bugs in the process: >http://thread.gmane.org/gmane.linux.kernel/613442 . I did something similar >while writing the utimensat(2) man page, finding 5 or 6 different bugs in >the end, see >http://linux-man-pages.blogspot.com/2008/06/whats-wrong-with-kernel-userland_30.html > . > > >> Now, we have an inherent question, >> who writes and submits those test cases ?? Either it can be the >> developer himself, or, somebody from the test community. > > >Or, IMO, preferably both, working together. > > >> Since, the >> developer is not bound to submit any test case, > > >I can't help but feel that part of the solution here is to agitate more so >that the developer is (strongly) encouraged to submit test cases along with >patches that change the kernel-userspace interface. As things stand, there >are far too many bugs in released interrfaces. > > >> we can assume we need to >> initiate doing something in this regard. >> >> Having said that, i want to propose that, since in your project you have >> already taken up the Job of making available the system call test cases >> up-to-date, we can score a point here. Whenever a new system call code >> enters the mm-tree in the kernel, somebody in your project can write the >> test case(s) immediately and make it available to: >> >> 1) Your Project, >> 2) LTP, >> 3) LKML, > > >I would also be very happy to see any test cases you produce for new system >calls. What list do I need to subscribe to? > > >> We will use the test case(s) to test that particular syscall feature and >> report Bug(s) to the developer(s) and LKML. Meanwhile, updates to the >> test cases will also go on depending on feedback from community. With >> that we will address the following issues: >> 1) Catch bug at the earliest for the new syscalls in the kernel, so that >> the cost of Bug fixing, when something is discovered later, can be >> hugely reduced, > > >Bug fixing is perhaps the lesser problem. The greater problem is that if >there are bugs, we must change the ABI to fix them, and if that is done >after a stable kernel release, then userland programmer's can end up having >to do things like special case their code according to the kernel version. > > >> 2) We have a regular and systematic flow of test cases for both LTP and >> Crackerjack. This process becomes institutional and gurantees us maximum >> and continued code coverage for syscalls in future, >> 3) LTP-mm tree is born, >> 4) Others/developers get motivated to submit test cases(to LTP and >> Crackerjack) at the time of submission of kernel features to >> kernel-mm-tree, if we show them the benefit of early testing and it´s >> effectiveness of controlling more bugs to seep in before it is too late >> to fix. >> >> For this to happen, we need to make our test cases ready when the >> corresponding kernel features are in -mm tree. Let me know your ideas on >> this. > > >Part of the problem here is knowing when interface changes have occurred. >Seeing new system calls in a release is easy. But, for the project to be >truly effective, all kernel-userland interfacechanges need to be checked >for, including > >system calls >ioctl()s >netlink >/proc >/sys (okay -- that's a big ask at the moment) >virtual /dev interfaces used for kernel-uerland communication >And probably other things I forgot right now. > >Cheers, > >Michael > > >On Mon, 2008-07-14 at 13:39 +0900, his...@hi... >wrote: >> Subrata, >> >> >Masatake is already working to port the missing syscall test cases in >> >LTP from the Crackerjack Project to LTP. And it is already a huge >> I am glad to hear that. >> >effort. I think the idea here is great as we are not re-inventing the >> Yes, you can do this and we can use our time to develope more test cases. >> >to port from you, unless somebody in LTP community has already submitted >> In that case, we will refer your cases and make them better. >> >https://lists.sourceforge.net/mailman/listinfo/ltp-list >> I joined now. I am very surprised that your community are >> working so active. I will check and I will post anything >> I can collaborate. >> >We will be collaborating as far as we can, as all of us striving to >> >Linux better. >> Yes, agree. We will continue this effort for our purpose, Linux better. >> Please post anything to both our mailing list and cc-d your mailing list >> when you have to contact me. >> >> Thanks, >> >> Hisashi >> >> >送信者 : Subrata Modak <su...@li...> >> >主題 : Re: Re[2]: Re[2]: Re[2]: Re[2]: [Crackerjack-devel] [LTP]Crackerjack >andLinuxTes >> >受信日 :08/07/12 04:10 >> >属性 : なし >> > >> >Dear Hisashi Hashimoto San, >> > >> >Thank you for writing back to us. We are delighted to colloborate on all >> >fronts to make Linux testing better. My comments embedded in the >> >following lines. >> > >> >On Fri, 2008-07-11 at 13:53 +0900, his...@hi... >> >wrote: >> >> Subrata, >> >> >> >> Sorry for long time no reply. >> >> We had internal meeting of our project >> >> regarding how to proceed. >> >> Current our plan is: >> >> (1) Unfortunattelly, our test cases do not cover all system call yet. >> >> Total number of system call in Kernel 2.6.26 is almost 326. >> >> But, we developed around 299. >> > >> >Masatake is already working to port the missing syscall test cases in >> >LTP from the Crackerjack Project to LTP. And it is already a huge >> >effort. I think the idea here is great as we are not re-inventing the >> >same wheel. We can use your test cases released under GPLv2 and port to >> >LTP format. >> > >> >We gain by getting more kernel test cases in LTP without the need to >> >massively write things. And you retain your goal of providing a >> >regression comparisn framework. In future, whenever the no. of syscalls >> >goes increasing in kernels, and as soon as you write them ,we will try >> >to port from you, unless somebody in LTP community has already submitted >> >a test case in that regard. Or, even we can concentrate to write test >> >cases in some other are as well. >> > >> >Presently, the first job is to port the existing ones from you to us. >> > >> >> Therefore, we would like to start developing in soon. >> >> The starting date would be some time in August. >> >> (2) Some of our developed test cases do not meet our quality criteria. >> >> As you know, we set our quality goal as covarage ratio more than >> >> yours. >> >> We are now checking all tests again. And if necessary, we need >to >> >> enhance test cases. >> > >> >Good to know that. As and when you do this, we might also relook into >> >ours and change accordingly. >> > >> >> (3) Through our acitivity (1) and (2), we would like to exchange >> >> information and technical/engineering discussion on this mailing >list. >> > >> >Definitely yes. All of you can subscribe to the ltp mailing list: >> >https://lists.sourceforge.net/mailman/listinfo/ltp-list >> > >> >And please add us as CC to all the technical discussion you have. Guys >> >in ltp mailing list will need to subscribe yours if they want to reply >> >back to you. >> > >> >> >> >> Yesterday, 4 Maintainers(Andrew Morton, Paul Moor, James Morris, Thomas >Gleixner) >> >> visit Japan to make presentation regarding the latest kernel/security >activity. >> >> I had opportunity to make presentation to them about our current >status. >> >> I present that >> >> (1) We released test cases 300/326. >> >> (2) We plan to make complete set and enhance quality. >> >> (3) We continue to talk with other test project(LTP as you, >AutoTest:Martin Bleigh), >> >> and exchange information and status, and others. >> >> Maintainers said/recommend to expand the target, to continue the effort >> >> and collaborate with LTP and so on. >> > >> >We will be collaborating as far as we can, as all of us striving to >> >Linux better. >> > >> > >> >Regards-- >> >Subrata >> > >> >> >> >> I hope you and we can do the collaboration work on test project. >> >> >> >> Best, >> >> >> >> Hashimoto Hisashi >> > >> > >> > >> Hisashi Hashimoto >> Section Manager >> Open Source Software Promotion Center >> Platform Software >> Hitachi, Ltd., Software Division >> Tel : +81-45-862-8424, Fax : +81-45-862-9047 >> his...@hi... > > > > >-- >Michael Kerrisk >Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/ >man-pages online: http://www.kernel.org/doc/man-pages/online_pages.html >Found a bug? http://www.kernel.org/doc/man-pages/reporting_bugs.html > Hisashi Hashimoto Section Manager Open Source Software Promotion Center Platform Software Hitachi, Ltd., Software Division Tel : +81-45-862-8424, Fax : +81-45-862-9047 his...@hi... |