Thread: Re: [Pyret-devel] [HOW TO] Examples

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 422-6466

Currently the closest thing we have to that is the README file.

Jeremy

Sorry only sent this to Jeremy
---------- Forwarded message ----------
From: Parse Exception <par...@gm...>
Date: Aug 8, 2006 8:55 AM
Subject: Re: [Pyret-devel] [HOW TO] Examples
To: Jeremy Wilde <toa...@gm...>

Does anyone have the bandwidth to whip one up pretty quick?
maybe Eduardo could help, perhaps with a list of questions,
maybe we could build a FAQ out of it? Thoughts?

-James

hello all,

   Sorry for this delay.

    I'm trying to reproduce the Linux Kernel Regression Hunting and
I'm facing a problem with the patching and building phases. First, I'm
using only one machine. Here are my files:

master.conf:

###Paths PyReT needs to know about###
sfs=/nfsdrives/var_bubs/
jobs=/nfsdrives/var_bubs/jobs/
jobsDead=/nfsdrives/var_bubs/jobs/dead/
jobsError=/nfsdrives/var_bubs/jobs/error/
jobsFinished=/nfsdrives/var_bubs/jobs/finished/
hunters=/nfsdrives/var_bubs/hunters/
hunterLogs=/nfsdrives/var_bubs/logs/hunters
masterLogs=/nfsdrives/var_bubs/logs/master/
tests=/nfsdrives/var_bubs/tests/
modules=/nfsdrives/var_bubs/modules/
searches=/nfsdrives/var_bubs/searches/
repository=/nfsdrives/var_bubs/repository/

###Modules###
FileSystem=modules.system.LinuxFS
#
# 'ConfigurationFile' is now depricated in the 'master.conf' file
#
#ConfigurationFile=modules.system.LinuxCF

masterLog=modules.system.ASCIILog {
        fileName = master.log
}

###Intervals###
jobInterval=2
searchInterval=4
retryLimit=3

###Hunter Interval###
deadTime=5

###Logging####
loggingLevel=5

### max hunters to assign for a search space ###
maxHunters=50000

hunter.conf

###Computer Info###
name=strongbad

###Shared File System###
base=/nfsdrives/var_bubs/
hunters=hunters/
hunterLogs=logs/hunters/
jobs=jobs/
ExecutablePath=/home/ebv/diversos/ols/pyret/PyretHunter.py

###Modules###
FileSystem=LinuxFS
TextFile=TextFile
FileAccess=FileAccess
ConfigurationFile=LinuxCF
System=DebianSystem

HunterLog=ASCIILog
LogFileName=hunter.log

###Logging####
loggingLevel=5

searchspace

  Search = BinaryRS {
  }
    MasterPatchSource = LinuxKernelMasterPatchSource {
        ChangesetStart = e3071039a1389fa767113ef4472040df507f3b02
        ChangesetEnd = 51775f2daed1fd12472b48335286b7d251ac8cea
        Dest = /nfsdrives/var_bubs/repository/cgrepo/linux2/
        PatchDest = /nfsdrives/var_bubs/repository/patches/linux2/
    }

job01

  Search = BinaryRS {
  }
    MasterPatchSource = LinuxKernelMasterPatchSource {
        ChangesetStart = e3071039a1389fa767113ef4472040df507f3b02
        ChangesetEnd = 51775f2daed1fd12472b48335286b7d251ac8cea
        Dest = /nfsdrives/var_bubs/repository/cgrepo/linux2/
        PatchDest = /nfsdrives/var_bubs/repository/patches/linux2/
    }

ebv@neumann:~/diversos/ols/pyret$ cat job01
  Copy = LinuxKernelCopy {
      From = /nfsdrives/var_bubs/repository/cgrepo/linux2/
      To = /usr/src/linux2/
      Regex=(.)*
      CleanUp=False
  }

  HunterPatchSource = LinuxKernelHunterPatchSource {
      PatchDest = /nfsdrives/var_bubs/repository/patches/linux2/
      PatchID = 0a05d26784046b1fde1f4c2aef040c7864478410
      HunterDest = /usr/src/linux2/
  }

  Build = LinuxKernelBuild {
      Input = /usr/src/linux2/
      Arch = i386
  }

and here is my nfsdrives tree:

ebv@neumann:/nfsdrives$ tree
.
`-- var_bubs
    |-- hunters
    |   `-- strongbad.idle
    |-- jobs
    |   |-- dead
    |   |-- error
    |   |-- finished
    |-- logs
    |   |-- hunters
    |   |   `-- strongbad.log
    |   `-- master
    |       |-- master.log
    |       `-- searchspace.log
    |-- modules
    |-- repository
    |   |-- cgrepo
    |   `-- patches
    |       `-- linux2
    |           |-- 0a05d26784046b1fde1f4c2aef040c7864478410
    |           |-- 51775f2daed1fd12472b48335286b7d251ac8cea
    |           |-- 829702a4980b5c13dc14f4556c513bc62e4aaec1
    |           |-- 9c3d91f81f7b467c468971136a231d8160f1ebb6
    |           `-- f16bdc72dc139e4413e75ef5b91690e9ad8e1b05
    |-- searches
    |   `-- searchspace.proc
    `-- tests

    So, when I copy the searchspace file to the "searches" directory,
it seams to work.
However, when I copy the job01 file to the  "jobs" directory, it does
not do the build phase.
And if copy a job file like:
  Copy = LinuxKernelCopy {
      From = /nfsdrives/var_bubs/repository/cgrepo/linux2/
      To = /usr/src/linux2/
      Regex=(.)*
      CleanUp=False
  }

  HunterPatchSource = LinuxKernelHunterPatchSource {
      PatchDest = /nfsdrives/var_bubs/repository/patches/linux2/
      PatchID = 0a05d26784046b1fde1f4c2aef040c7864478410
      HunterDest = /usr/src/linux2/
  }

   it patched the tree even though the hunter ends with an error
(creates a job.error file)

   So, what should I do to accomplish the build phase?

BR,
---
Eduardo Valentin

On 8/8/06, Parse Exception <par...@gm...> wrote:
>
> Sorry only sent this to Jeremy
> ---------- Forwarded message ----------
> From: Parse Exception <par...@gm... >
> Date: Aug 8, 2006 8:55 AM
> Subject: Re: [Pyret-devel] [HOW TO] Examples
> To: Jeremy Wilde <toa...@gm...>
>
>
>
>  Does anyone have the bandwidth to whip one up pretty quick?
> maybe Eduardo could help, perhaps with a list of questions,
> maybe we could build a FAQ out of it? Thoughts?
>
> -James
>
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job
> easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
>
> _______________________________________________
> Pyret-devel mailing list
> Pyr...@li...
> https://lists.sourceforge.net/lists/listinfo/pyret-devel
>
>
>

-- 
Eduardo Bezerra Valentin

Eduardo,

what was in job.error?

Hello Parse,

    I forgot to mention/explain that the file created was
job01.strongbad.error inside the
"jobs" directory. It contains exatly the same contents of my initial job file.
 Copy = LinuxKernelCopy {
     From = /nfsdrives/var_bubs/repository/cgrepo/linux2/
     To = /usr/src/linux2/
     Regex=(.)*
     CleanUp=False
 }

 HunterPatchSource = LinuxKernelHunterPatchSource {
     PatchDest = /nfsdrives/var_bubs/repository/patches/linux2/
     PatchID = 0a05d26784046b1fde1f4c2aef040c7864478410
     HunterDest = /usr/src/linux2/
 }

and here is my log for my hunter process:
Status @time="20061091642": Hunter is changing status to copying
Status @time="20061091642": Hunter is changing status to patching
Status @time="20061091642": Hunter is changing status to building
Jog Info @time="2006109161521": Build completed sucessfully
Status @time="2006109161521": Hunter is changing status to installing
Status @time="2006109161521": Hunter is changing status to testing
Status @time="2006109161521": Hunter is changing status to cleanup
Status @time="2006109161521": Hunter is changing status to idle
Job @time="2006101010832": found job - /nfsdrives/var_bubs/jobs/job03.strongbad
Status @time="2006101010832": Hunter is changing status to copying
Info @time="2006101010930": Copy succesfull
Job Info @time="2006101010930": Copy completed sucessfully
Status @time="2006101010930": Hunter is changing status to patching
Info @time="2006101010930": Applying patch using COMMAND:patch -p1 -i
/nfsdrives/var_bubs/repository/patches/linux2/0a05d26784046b1fde1f4c2aef040c7864478410Status
@time="2006101010930": Hunter is changing status to idle

  Any ideas?

Thanks in advance

BR,

---
Eduardo Valentin

On 10/10/06, Parse Exception <par...@gm...> wrote:
> Eduardo,
>
> what was in job.error?
>
>
>
> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share your
> opinions on IT & business topics through brief surveys -- and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
>
> _______________________________________________
> Pyret-devel mailing list
> Pyr...@li...
> https://lists.sourceforge.net/lists/listinfo/pyret-devel
>
>
>

-- 
Eduardo Bezerra Valentin

Hello,

    Alright, after some permission problems I could do something
useful. But I have some
questions and some unsolved problems. So, I think that I'm missing
something into my
configuration files. I used the same master.conf and hunter.conf files
I already sent.
What's more, I'm executing the PyretMaster.py and PyretHunter.py as
root user. Therefore,
I'm able to execute the following job file:

  Copy = LinuxKernelCopy {
      From = /nfsdrives/var_bubs/repository/cgrepo/linux2/
      To = /usr/src/linux2/
      Regex=(.)*
      CleanUp=False
  }

  Build = LinuxKernelBuild {
      Input = /usr/src/linux2/
      Arch = i386
  }

  Install = LinuxKernelInstall {
  }

  Test = LinuxKernelTest {
      LTPFolderPath = /home/ebv/diversos/ols/pyret/ltp-full-20060717/
      Scenario = syscalls
      Duration = 0s
  }

    But, I needed to change the install module for Linux Kernels as follows:

  modules/install/LinuxKernelInstall.py:
kernel = "kernel          /vmlinuz-PyReT root="+kernel[start+5:];

to

kernel = "kernel          /boot/vmlinuz-PyReT root="+kernel[start+5:];

    Also, there is a little mistake in the search space example file
(docs/searchspace.example):

It is:
#  Test = LinuxKernelTest {
#      LTPFolderPath = /usr/local/ltp-full-20060105/
#      TestName = syscalls
#      TimeOut = 0s
#  }

and should be:
#  Test = LinuxKernelTest {
#      LTPFolderPath = /usr/local/ltp-full-20060105/
#      Scenario = syscalls
#      Duration = 0s
#  }

    However, what I really couldn't setup is the error messaging
system. My log files do not
print the error messages. There is no useful info there. If something
goes wrong, I get something like:
Job @time="2006101012107": found job - /nfsdrives/var_bubs/jobs/job03.strongbad
Status @time="2006101012107": Hunter is changing status to copying
Status @time="2006101012107": Hunter is changing status to patching
Status @time="2006101012107": Hunter is changing status to building
Status @time="2006101012107": Hunter is changing status to installing
Status @time="2006101012107": Hunter is changing status to testing
Status @time="2006101012107": Hunter is changing status to idle

    It just says that is changing from state to state until reach the
idle state, whereas the error messages are not displayed (if one
parameter is not present/configured for example).

    Furthermore, I couldn't accomplish a patching phase.
    Actually, I still couldn't even understand how the automation
process works. Specifically,
how the patch control is done. I passed a range in my search space:

  Search = BinaryRS {
  }
    MasterPatchSource = LinuxKernelMasterPatchSource {
        ChangesetStart = e3071039a1389fa767113ef4472040df507f3b02
        ChangesetEnd = 51775f2daed1fd12472b48335286b7d251ac8cea
        Dest = /nfsdrives/var_bubs/repository/cgrepo/linux2/
        PatchDest = /nfsdrives/var_bubs/repository/patches/linux2/
    }

   I thought that as I specified a patch range (ChangesetStart and
ChangesetEnd variables),
it would do the search automatically. But I need to specify a patch ID
in my job file:
  HunterPatchSource = LinuxKernelHunterPatchSource {
      PatchDest = /nfsdrives/var_bubs/repository/patches/linux2/
      PatchID = 0a05d26784046b1fde1f4c2aef040c7864478410
      HunterDest = /usr/src/linux2/
  }

   So, how does the automation process work? What wrong am I doing ?
Why my hunters does not log error messages?

Thanks in  advance,

BR,
---
Eduardo Valentin

On 10/10/06, Eduardo Valentin <edu...@gm...> wrote:
> Hello Parse,
>
>
>     I forgot to mention/explain that the file created was
> job01.strongbad.error inside the
> "jobs" directory. It contains exatly the same contents of my initial job file.
>  Copy = LinuxKernelCopy {
>      From = /nfsdrives/var_bubs/repository/cgrepo/linux2/
>      To = /usr/src/linux2/
>      Regex=(.)*
>      CleanUp=False
>  }
>
>  HunterPatchSource = LinuxKernelHunterPatchSource {
>      PatchDest = /nfsdrives/var_bubs/repository/patches/linux2/
>      PatchID = 0a05d26784046b1fde1f4c2aef040c7864478410
>      HunterDest = /usr/src/linux2/
>  }
>
> and here is my log for my hunter process:
> Status @time="20061091642": Hunter is changing status to copying
> Status @time="20061091642": Hunter is changing status to patching
> Status @time="20061091642": Hunter is changing status to building
> Jog Info @time="2006109161521": Build completed sucessfully
> Status @time="2006109161521": Hunter is changing status to installing
> Status @time="2006109161521": Hunter is changing status to testing
> Status @time="2006109161521": Hunter is changing status to cleanup
> Status @time="2006109161521": Hunter is changing status to idle
> Job @time="2006101010832": found job - /nfsdrives/var_bubs/jobs/job03.strongbad
> Status @time="2006101010832": Hunter is changing status to copying
> Info @time="2006101010930": Copy succesfull
> Job Info @time="2006101010930": Copy completed sucessfully
> Status @time="2006101010930": Hunter is changing status to patching
> Info @time="2006101010930": Applying patch using COMMAND:patch -p1 -i
> /nfsdrives/var_bubs/repository/patches/linux2/0a05d26784046b1fde1f4c2aef040c7864478410Status
> @time="2006101010930": Hunter is changing status to idle
>
>
>
>   Any ideas?
>
>
> Thanks in advance
>
> BR,
>
> ---
> Eduardo Valentin
>
>
> On 10/10/06, Parse Exception <par...@gm...> wrote:
> > Eduardo,
> >
> > what was in job.error?
> >
> >
> >
> > -------------------------------------------------------------------------
> > Take Surveys. Earn Cash. Influence the Future of IT
> > Join SourceForge.net's Techsay panel and you'll get the chance to share your
> > opinions on IT & business topics through brief surveys -- and earn cash
> > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> >
> > _______________________________________________
> > Pyret-devel mailing list
> > Pyr...@li...
> > https://lists.sourceforge.net/lists/listinfo/pyret-devel
> >
> >
> >
>
>
> --
> Eduardo Bezerra Valentin
>

-- 
Eduardo Bezerra Valentin

Hello guys,

     I would ask you to send master and hunter configuration files and
searchspace
and jobs files which are known as working. In this way I can compare
with my files.

    BTW, in doc/searchspace.example there is an entry for search
module configuration:

#  Search = MultiHunterRS {
#  }

   Is it "Search = MultiHunterRS" or "RegressionSearch =
MultiHunterRS" ? I'm asking
this because the Python class is named RegressionSearch, as you can
see at the modules/search directory.

BR,
---
Eduardo Valentin

On 10/11/06, Jason Ruesch <jr...@ja...> wrote:
> Eduardo,
>
> Sorry to hear you're having so many problems.  We definitely didn't
> make the project easy to setup!
>
> I believe there still may be another log file that we started using
> and never got rid of completely in the /var/log folder that may
> include the errors you are having.
>
> --Jason Ruesch
>
> On Oct 11, 2006, at 6:35 AM, Eduardo Valentin wrote:
>
> > Hello,
> >
> >
> >
> >
> >     Alright, after some permission problems I could do something
> > useful. But I have some
> > questions and some unsolved problems. So, I think that I'm missing
> > something into my
> > configuration files. I used the same master.conf and hunter.conf files
> > I already sent.
> > What's more, I'm executing the PyretMaster.py and PyretHunter.py as
> > root user. Therefore,
> > I'm able to execute the following job file:
> >
> >   Copy = LinuxKernelCopy {
> >       From = /nfsdrives/var_bubs/repository/cgrepo/linux2/
> >       To = /usr/src/linux2/
> >       Regex=(.)*
> >       CleanUp=False
> >   }
> >
> >   Build = LinuxKernelBuild {
> >       Input = /usr/src/linux2/
> >       Arch = i386
> >   }
> >
> >   Install = LinuxKernelInstall {
> >   }
> >
> >
> >   Test = LinuxKernelTest {
> >       LTPFolderPath = /home/ebv/diversos/ols/pyret/ltp-full-20060717/
> >       Scenario = syscalls
> >       Duration = 0s
> >   }
> >
> >
> >     But, I needed to change the install module for Linux Kernels as
> > follows:
> >
> >   modules/install/LinuxKernelInstall.py:
> > kernel = "kernel          /vmlinuz-PyReT root="+kernel[start+5:];
> >
> > to
> >
> > kernel = "kernel          /boot/vmlinuz-PyReT root="+kernel[start+5:];
> >
> >     Also, there is a little mistake in the search space example file
> > (docs/searchspace.example):
> >
> > It is:
> > #  Test = LinuxKernelTest {
> > #      LTPFolderPath = /usr/local/ltp-full-20060105/
> > #      TestName = syscalls
> > #      TimeOut = 0s
> > #  }
> >
> > and should be:
> > #  Test = LinuxKernelTest {
> > #      LTPFolderPath = /usr/local/ltp-full-20060105/
> > #      Scenario = syscalls
> > #      Duration = 0s
> > #  }
> >
> >     However, what I really couldn't setup is the error messaging
> > system. My log files do not
> > print the error messages. There is no useful info there. If something
> > goes wrong, I get something like:
> > Job @time="2006101012107": found job - /nfsdrives/var_bubs/jobs/
> > job03.strongbad
> > Status @time="2006101012107": Hunter is changing status to copying
> > Status @time="2006101012107": Hunter is changing status to patching
> > Status @time="2006101012107": Hunter is changing status to building
> > Status @time="2006101012107": Hunter is changing status to installing
> > Status @time="2006101012107": Hunter is changing status to testing
> > Status @time="2006101012107": Hunter is changing status to idle
> >
> >
> >     It just says that is changing from state to state until reach the
> > idle state, whereas the error messages are not displayed (if one
> > parameter is not present/configured for example).
> >
> >     Furthermore, I couldn't accomplish a patching phase.
> >     Actually, I still couldn't even understand how the automation
> > process works. Specifically,
> > how the patch control is done. I passed a range in my search space:
> >
> >   Search = BinaryRS {
> >   }
> >     MasterPatchSource = LinuxKernelMasterPatchSource {
> >         ChangesetStart = e3071039a1389fa767113ef4472040df507f3b02
> >         ChangesetEnd = 51775f2daed1fd12472b48335286b7d251ac8cea
> >         Dest = /nfsdrives/var_bubs/repository/cgrepo/linux2/
> >         PatchDest = /nfsdrives/var_bubs/repository/patches/linux2/
> >     }
> >
> >    I thought that as I specified a patch range (ChangesetStart and
> > ChangesetEnd variables),
> > it would do the search automatically. But I need to specify a patch ID
> > in my job file:
> >   HunterPatchSource = LinuxKernelHunterPatchSource {
> >       PatchDest = /nfsdrives/var_bubs/repository/patches/linux2/
> >       PatchID = 0a05d26784046b1fde1f4c2aef040c7864478410
> >       HunterDest = /usr/src/linux2/
> >   }
> >
> >    So, how does the automation process work? What wrong am I doing ?
> > Why my hunters does not log error messages?
> >
> > Thanks in  advance,
> >
> > BR,
> > ---
> > Eduardo Valentin
> >
> > On 10/10/06, Eduardo Valentin <edu...@gm...> wrote:
> >> Hello Parse,
> >>
> >>
> >>     I forgot to mention/explain that the file created was
> >> job01.strongbad.error inside the
> >> "jobs" directory. It contains exatly the same contents of my
> >> initial job file.
> >>  Copy = LinuxKernelCopy {
> >>      From = /nfsdrives/var_bubs/repository/cgrepo/linux2/
> >>      To = /usr/src/linux2/
> >>      Regex=(.)*
> >>      CleanUp=False
> >>  }
> >>
> >>  HunterPatchSource = LinuxKernelHunterPatchSource {
> >>      PatchDest = /nfsdrives/var_bubs/repository/patches/linux2/
> >>      PatchID = 0a05d26784046b1fde1f4c2aef040c7864478410
> >>      HunterDest = /usr/src/linux2/
> >>  }
> >>
> >> and here is my log for my hunter process:
> >> Status @time="20061091642": Hunter is changing status to copying
> >> Status @time="20061091642": Hunter is changing status to patching
> >> Status @time="20061091642": Hunter is changing status to building
> >> Jog Info @time="2006109161521": Build completed sucessfully
> >> Status @time="2006109161521": Hunter is changing status to installing
> >> Status @time="2006109161521": Hunter is changing status to testing
> >> Status @time="2006109161521": Hunter is changing status to cleanup
> >> Status @time="2006109161521": Hunter is changing status to idle
> >> Job @time="2006101010832": found job - /nfsdrives/var_bubs/jobs/
> >> job03.strongbad
> >> Status @time="2006101010832": Hunter is changing status to copying
> >> Info @time="2006101010930": Copy succesfull
> >> Job Info @time="2006101010930": Copy completed sucessfully
> >> Status @time="2006101010930": Hunter is changing status to patching
> >> Info @time="2006101010930": Applying patch using COMMAND:patch -p1 -i
> >> /nfsdrives/var_bubs/repository/patches/
> >> linux2/0a05d26784046b1fde1f4c2aef040c7864478410Status
> >> @time="2006101010930": Hunter is changing status to idle
> >>
> >>
> >>
> >>   Any ideas?
> >>
> >>
> >> Thanks in advance
> >>
> >> BR,
> >>
> >> ---
> >> Eduardo Valentin
> >>
> >>
> >> On 10/10/06, Parse Exception <par...@gm...> wrote:
> >>> Eduardo,
> >>>
> >>> what was in job.error?
> >>>
> >>>
> >>>
> >>> --------------------------------------------------------------------
> >>> -----
> >>> Take Surveys. Earn Cash. Influence the Future of IT
> >>> Join SourceForge.net's Techsay panel and you'll get the chance to
> >>> share your
> >>> opinions on IT & business topics through brief surveys -- and
> >>> earn cash
> >>> http://www.techsay.com/default.php?
> >>> page=join.php&p=sourceforge&CID=DEVDEV
> >>>
> >>> _______________________________________________
> >>> Pyret-devel mailing list
> >>> Pyr...@li...
> >>> https://lists.sourceforge.net/lists/listinfo/pyret-devel
> >>>
> >>>
> >>>
> >>
> >>
> >> --
> >> Eduardo Bezerra Valentin
> >>
> >
> >
> > --
> > Eduardo Bezerra Valentin
> >
> > ----------------------------------------------------------------------
> > ---
> > Using Tomcat but need to do more? Need to support web services,
> > security?
> > Get stuff done quickly with pre-integrated technology to make your
> > job easier
> > Download IBM WebSphere Application Server v.1.0.1 based on Apache
> > Geronimo
> > http://sel.as-us.falkag.net/sel?
> > cmd=lnk&kid=120709&bid=263057&dat=121642
> > _______________________________________________
> > Pyret-devel mailing list
> > Pyr...@li...
> > https://lists.sourceforge.net/lists/listinfo/pyret-devel
>
>
>

-- 
Eduardo Bezerra Valentin