[r9922]: / test / trunk / ReadMe.first  Maximize  Restore  History

Download this file

274 lines (185 with data), 9.5 kB

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273

                  ooRexxUnit 4.2.0 snapshot 06 Quick Start
                  ========================================

This is the read me file explaining how the ooTest framework is used by
the ooRexx team to test ooRexx.

The quick start notes below should allow one to quickly run the current test
suite.  After installation, the doc subdirectory contains a quick start
tutorial in PDF format.  This file: ooTestQuick.pdf provides additional
useful information and is a brief tutorial on how to write a test case.

NOTE: You need to match the ooRexxUnit version level with the version of
the ooRexx interpreter you are running.  To use the ooRexxUnit 4.2.0 test
framework you must have the 4.2.0 version of the interpreter running.

Installation Notes:
===================

* The test suite framework must be able to write temporary files in its
sub-directory structure.  Do not install this snapshot into a directory
structure that may restict file writing privileges.  On Windows Vista and
later, the typical user can not write files to the C:\Program Files
sub-directory tree.

* In general, it is not advised to install the snapshot into a sub-directory
tree that will have spaces in the path name.

PLEASE NOTE:

All example text shown here, such as:

C:\ooTest-4.2.0-snapshot06>testOORexx -X native_api

is, of course, dependent on where you have installed the teste suite.  For
the purpose of writing this read me file, the test suite was installed at:

C:\ooTest-4.2.0-snapshot06

Gotchas for the General User:
=============================

Some test cases may make assumptions that cause the test to fail when the
test suite is run in an unexpected environment.  Some of these are known and
listed below.  Most likely there are other circumstances not as yet known.

Windows Platform:
-----------------

The SpecialFolders.testGroup assumes an English languge version of Windows.
Test cases in this test group will fail if run under a different language
version of Windows.

The WindowsEventLog.testGroup has tests in it that will fail if the test
suite is not executed with Administrator privileges.

Several tests in the SysCurPos.testgroup will fail if the output of
exectuing the test suite is redirected to a file.  There is an additional
test case in the CHAROUT.testGroup that will fail.  The test suite is
intened to be run in a command shell with no redirection of its output.


Unix-like Platforms:
-------------------

There is a test case that will fail if the test suite is executed as root.


Quick Start to execute tests:
=============================


    1.)  Simple command line, running the entire test suite at one time.
    --------------------------------------------------------------------

NOTE: Starting with ooRexx 4.0.0, the test suite contains tests that require
external executables.  These are tests to exercise the ooRexx native API.
Running these tests makes things slightly more complex.  There are 3 options
here, simplest to hardest.

1.) Skip the native API tests altogether.  The best option if you are just
getting familiar with the test suite.

2.) Download the pre-compiled external binaries for your system.  Details on
picking and installing the proper set are in the Release.notes in the doc
directory.

NOTE: at this time the pre-compiled binareies are NOT available.  If you are
interested enough to want the pre-compiled binaries, post a note to either
the ooRexx developers list or the ooRexx users list requesting the binaries
and one of the developers will make thenm available.

3.) If you understand, somewhat, compiling C / C++ files you can compile
your own binaries as part of running the automated test suite.  Details for
this step have not been written, just post a question on the ooRexx
developers list or the ooRexx users list.

Start in the root directory, the directory that contains this file.  The
Rexx program that runs the tests is: testOORexx.rex

That is the only file you need to use to begin with.  We will assume that
you are using option 1 above, skipping the tests for the native API
binaries.  The program file will run the ooRexx test suite.  Depending on
the horse power in your system this could take 5 or 6 minutes, maybe longer
on a real old system.

Once you invoke testOORexx everything runs automatically and you will see
the results printed on the screen when the test finishes.

The argument: '-X native_api' in the following examples tells the framework
to eXclude the native API tests.

Windows
-------
If you have done a normal install of the interpreter so that file
associations are set up (done automatically by the installer) you can
simply use the following command line.

C:\ooTest-4.2.0-snapshot06>testOORexx -X native_api

of course you can always use the full invocation the interpreter like this:

C:\ooTest-4.2.0-snapshot06>rexx testOORexx.rex -X native_api

Linux
-----
If you have a normal install of the interpreter with the executable in the
path then use this command line:

# ./testOORexx.rex -X native_API

AIX
---
If you have a normal install of the interpreter with the executable in the
path then use this command line:

# ./setTestEnv.sh
# ./testOORexx.rex -X native_api

After Test Completion on Your OS
--------------------------------
Look at the file: Expected.Results in the same directory as this file to see
some typical console output when executing the entire test suite.  This file
is only to give you an idea of the output.  The output on your system could
be different.


               2.)  Running just a subset of the test suite.
               ---------------------------------------------

The examples from here on out will be from a Windows box.  The ooTest
framework works the same way on Linux, you just need to translate the
slashes in directory names.

Help
----

This command will print out some help:

C:\ooTest-4.2.0-snapshot06>testOORexx.rex /?
testOORexx version 1.1.0 ooTest Framework version 1.0.0_4.0.0

usage: testOORexx [OPTIONS]
Try 'testOORexx --help' for more information.

C:\ooTest-4.2.0-snapshot06>

This command will print out the syntax of testOORexx:

C:\ooTest-4.2.0-snapshot06>testOORexx.rex --help

Note that not all command line options are implemented yet.  You will get a
polite message if you use an un-implemented option.

To get started and get a feel for how a test group works, you will probably
want to run just one or two test groups at a time.  The simplest way to do
that is with args to testOORexx.

All test groups are in the subdirectory: ooRexx

This is the default root.  To run a subset of the test you use the -R option
and name a new root.  Browse the ooRexx directory to get an idea of the
directory structure.

All test groups from the root on down are gathered together to be run for
one invocation of testOORexx.  This command line will run only the test
groups in the ooRexx\base\expressions subdirectory.

C:\ooTest-4.2.0-snapshot06>testOORexx.rex -R ooRexx\base\expressions

There will be 10 test groups executed.

You can execute only one test group by using a combination of the -R option
and the -f option.  The -f option takes a file name part and only executes
test groups that match the file name part.

This command line will run a single test group, the ADDITION.testGroup:

C:\ooTest-4.2.0-snapshot06>testOORexx.rex -R ooRexx\base\expressions -f Addition

and you will get output like this:

C:\ooTest-4.2.0-snapshot06>testOORexx.rex -R ooRexx\base\expressions -f Addition
Searching for test containers..
Executing automated test suite..

ooTest Framework - Automated Test of the ooRexx Interpreter


Interpreter:     REXX-ooRexx_4.2.0(MT)_64-bit 6.04 30 Jan 2014
Addressing Mode: 64
ooRexxUnit:      2.0.0_3.2.0    ooTest: 1.0.0_4.0.0

Tests ran:           200
Assertions:          304
Failures:            0
Errors:              0
Skipped files:       0

File search:        00:00:00.078000
Suite construction: 00:00:00.000000
Test execution:     00:00:00.093000
Total time:         00:00:01.155000


C:\ooTest-4.2.0-snapshot06>


   3.)  Writing your first test group to contribute to the ooRexx project
   ----------------------------------------------------------------------

In the misc subdirectory there is a template file.  Simply copy it to a
subdirectory that might be appropriate, renaming it to the name of your test
group.  For example:

C:\ooTest-4.2.0-snapshot06>copy misc\template.testGroup ooRexx\MyTests.testGroup

Edit the file and start writing your test cases.

PLEASE ask questions on one of the Rexx lists, as this Quick Start does not
go much further.  In addition, the ooTestQuick.pdf doc will walk you through
the process of creating test cases.

You can learn a lot by examining some other test groups.

The template.testGroup file will execute as is and contains all the
boilerplate needed.  By simply copying it to a new file name, a working test
group is created:

C:\ooTest-4.2.0-snapshot06>testOORexx.rex ooRexx\MyTests.testGroup
Searching for test containers..
Executing automated test suite..

ooTest Framework - Automated Test of the ooRexx Interpreter


Interpreter:     REXX-ooRexx_4.2.0(MT)_64-bit 6.04 30 Jan 2014
Addressing Mode: 64
ooRexxUnit:      2.0.0_3.2.0    ooTest: 1.0.0_4.0.0

Tests ran:           4
Assertions:          2
Failures:            0
Errors:              0
Skipped files:       0

File search:        00:00:00.078000
Suite construction: 00:00:00.000000
Test execution:     00:00:00.078000
Total time:         00:00:01.154000


C:\ooTest-4.2.0-snapshot06>