[6566df]: doc / html / about_pyke / modifying_pyke.html Maximize Restore History

Download this file

modifying_pyke.html    371 lines (357 with data), 20.4 kB

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en">
<head>
<title>Modifying Pyke</title>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<link rel="stylesheet" href="../stylesheets/pyke.css" type="text/css" />
</head>
<body>
<table id="page-table">
<thead class="head">
<tr id="header1"><th id="header" colspan="3">
&nbsp;
</th></tr>
<tr id="header2">
<th id="crumb-left"></th>
<th id="crumb-line">
<div id="nav">
<ul>
<li><a href="../index.html">Home</a></li>
<li>&gt;</li>
<li><a href="index.html">About Pyke</a></li>
<li>&gt;</li>
<li>Modifying Pyke</li>
</ul>
</div>
</th>
<th id="crumb-right"></th>
</tr>
</thead>
<tbody id="body">
<tr id="body-tr">
<td id="left-nav">
<div id="left-nav-div">
<div class="title-nav"><a href="../index.html">Home</a></div><div class="nav-branch">
<div class="title-nav"><a href="index.html">About Pyke</a></div><div class="nav-branch">
<div class="normal-nav"><a href="cooking_functions.html">Cooking Functions</a></div>
<div class="normal-nav"><a href="what_is_pyke.html">What is Pyke?</a></div>
<div class="normal-nav"><a href="steps_to_using_pyke.html">Steps to Using Pyke</a></div>
<div class="normal-nav"><a href="installing_pyke.html">Installing Pyke</a></div>
<div class="normal-nav"><a href="modifying_pyke.html">Modifying Pyke</a></div>
</div>
<div class="normal-nav"><a href="../logic_programming/index.html">Logic Programming</a></div>
<div class="normal-nav"><a href="../knowledge_bases/index.html">Knowledge Bases</a></div>
<div class="normal-nav"><a href="../pyke_syntax/index.html">Pyke Syntax</a></div>
<div class="normal-nav"><a href="../using_pyke/index.html">Using Pyke</a></div>
<div class="normal-nav"><a href="../examples.html">Examples</a></div>
<div class="normal-nav"><a href="../PyCon2008-paper.html">PyCon 2008 Paper</a></div>
</div>
</div>
<div id="icons">
<div id="project-page">
<a href="http://sourceforge.net/projects/pyke/">Pyke Project Page</a>
</div>
Please Make a Donation:<br />
<a href="http://sourceforge.net/donate/index.php?group_id=207724">
<img src="http://images.sourceforge.net/images/project-support.jpg"
width="88" height="32" border="0"
alt="Support This Project" /> </a> <br /><br />
Hosted by: <br />
<a href="http://sourceforge.net/projects/pyke">
<img src="http://sflogo.sourceforge.net/sflogo.php?group_id=207724&amp;type=14"
width="150" height="40"
alt="Get Python Knowledge Engine (PyKE) at SourceForge.net. Fast, secure and Free Open Source software downloads" /></a>
</div>
</td>
<td id="main-td">
<div id="main">
<a name="startcontent" id="startcontent"></a>
<div class="document" id="modifying-pyke">
<h1 class="title">Modifying Pyke</h1>
<div class="section" id="index-to-this-page">
<h2>Index to This Page</h2>
<ul class="simple">
<li><a class="reference internal" href="#mercurial-repositories">Mercurial Repositories</a><ul>
<li><a class="reference internal" href="#mercurial-keyword-extension">Mercurial Keyword Extension</a></li>
<li><a class="reference internal" href="#which-repository-do-i-use">Which Repository Do I Use?</a></li>
</ul>
</li>
<li><a class="reference internal" href="#compiling-ply-tables-files">Compiling PLY Tables Files</a></li>
<li><a class="reference internal" href="#compiling-the-compiler-krb-file">Compiling the Compiler.krb File</a></li>
<li><a class="reference external" href="modifying_pyke.html#running-unit-tests">Running Unit Tests</a></li>
<li><a class="reference external" href="modifying_pyke.html#rebuilding-the-html-documentation">Rebuilding the HTML Documentation</a></li>
</ul>
</div>
<div class="section" id="mercurial-repositories">
<h2>Mercurial Repositories</h2>
<p>With <a class="reference external" href="http://mercurial.selenic.com/wiki/">Mercurial</a>, you clone the entire repository locally on your computer.
Then you can make changes and commit those changes to your local repository.
If you think those changes might be interesting to everybody, make your local
repository (or a clone of it) publicly available (either on your own server,
or on one of the <a class="reference external" href="http://mercurial.selenic.com/wiki/MercurialHosting">Mercurial Hosting Sites</a>) and send me an email. I will
pull your changes, examine them, and push them to the master repository on
sourceforge.</p>
<div class="section" id="mercurial-keyword-extension">
<h3>Mercurial Keyword Extension</h3>
<p>The Pyke sources use the Mercurial <a class="reference external" href="http://mercurial.selenic.com/wiki/KeywordExtension">Keyword Extension</a> as a holdover from
when the repository used Subversion rather than Mercurial.</p>
<p>The <tt class="docutils literal"><span class="pre">hgrc_keywords</span></tt> file has been provided to enable and configure this
extension for Pyke use. You can append this file to either your personal
.hgrc configuration file (which would then apply to all of your Mercurial
projects) or the project .hg/hgrc file (see <a class="reference external" href="http://www.selenic.com/mercurial/hgrc.5.html">hgrc</a> in the Mercurial wiki).</p>
<p>If you use a <tt class="docutils literal"><span class="pre">post-clone</span></tt> <a class="reference external" href="http://www.selenic.com/mercurial/hgrc.5.html#hooks">Mercurial hook</a>, or append <tt class="docutils literal"><span class="pre">hgrc_keywords</span></tt>
manually after cloning, the keywords won't be expanded properly when the
project is first cloned. But they will be expanded properly if the clone is
done with the -U option and then an <tt class="docutils literal"><span class="pre">hg</span> <span class="pre">update</span></tt> done in the newly cloned
repository (after the changes to .hg/hgrc have been made).</p>
<p>The keyword expansions are only used by the tools that generate the html
documentation (see <a class="reference external" href="modifying_pyke.html#rebuilding-the-html-documentation">Rebuilding the HTML Documentation</a>, below).</p>
</div>
<div class="section" id="which-repository-do-i-use">
<h3>Which Repository Do I Use?</h3>
<p>Normally, you will clone one of the following four repositories locally to
make a master copy of what's on sourceforge. Then you would clone your master
copy (which is very fast) to make separate clones for each development task
that you are working on for Pyke.</p>
<p>So it is best to keep all of these clones together in a common directory.</p>
<p>There are four repositories on sourceforge that you can start with:</p>
<dl class="docutils">
<dt>release_1</dt>
<dd>Use this for bug fixes, code and documentation cleanup, and anything else
that would go into a point release for release 1. I merge the changes made
here into all of the other repositories. So this code goes into both the
Python2.x and Python3.x versions of Pyke.</dd>
<dt>pyke</dt>
<dd>Use this for major new features that would result in a major new release
(e.g., release 1.2). I merge the changes made in release_1 into the pyke
repository (but maybe not the other way around). And I merge the changes
made in the pyke repository into the pre_2to3 repository. So the code here
goes into both the Python2.x and Python3.x future versions of Pyke.</dd>
<dt>pre_2to3_r1</dt>
<dd><p class="first">Use this for bug fixes, code and documentation cleanup, and anything else
that would go into a point release for release 1, but only apply to the
Python3.x version of Pyke. I merge the changes made in release_1 into the
pre_2to3_r1 repository (but not the other way around). And I merge the
changes made in the pre_2to3_r1 repository into the pre_2to3 repository.
So changes here only go into the next point release of the Python3.x version
of Pyke.</p>
<div class="warning last">
<p class="first admonition-title">Warning</p>
<p>This code is maintained in a state just prior to running Python's
<a class="reference external" href="http://docs.python.org/library/2to3.html">2to3</a> tool on it. So you can't just run the code here directly.</p>
<p>The <tt class="docutils literal"><span class="pre">run_2to3</span></tt> script runs 2to3 on the current copy of the sources. Do
<strong>not</strong> run this in a repository clone that you still want to use to do
commits! Instead, commit all of your changes, then clone the repository
and do <tt class="docutils literal"><span class="pre">run_2to3</span></tt> in the clone. If anything doesn't work, go back to
the first repository to fix it, delete the clone, and repeat the whole
process. This was done to minimize merge conflicts caused by the 2to3
changes.</p>
<p>The <tt class="docutils literal"><span class="pre">run_pre_test</span></tt> script will:</p>
<ul class="simple">
<li>clone the current repository</li>
<li>then in the clone do:<ul>
<li><tt class="docutils literal"><span class="pre">run_2to3</span></tt></li>
<li><tt class="docutils literal"><span class="pre">testpyke</span></tt> -3.1</li>
<li>python setup.py -q sdist --formats zip</li>
<li>insert '3' after 'pyke' in the name of the source distribution zip
file.</li>
</ul>
</li>
</ul>
<p class="last"><tt class="docutils literal"><span class="pre">Run_pre_test</span></tt> assumes that you either have the keywording options set
in your personal .hgrc file, or have clone hooks in place to copy these
into the .hg/hgrc file of all clones within your pyke work area. See
<a class="reference internal" href="#mercurial-keyword-extension">Mercurial Keyword Extension</a>, above.</p>
</div>
</dd>
<dt>pre_2to3</dt>
<dd>Normally I merge changes from the pyke repository and the pre_2to3_r1
repository into pre_2to3 so that nothing needs to be done in this repository.
Most major new features would be developed in the <tt class="docutils literal"><span class="pre">pyke</span></tt> repository and
merged into pre_2to3. Making changes to pre_2to3 directly would only be
done when those changes are for major new features that only apply to the
Python3.x version of Pyke.</dd>
</dl>
<p>So, for example, if you wanted to work on the <tt class="docutils literal"><span class="pre">release_1</span></tt> repository, you
would:</p>
<pre class="literal-block">
$ mkdir pyke_repos
$ cd pyke_repos
$ hg clone -U http://pyke.hg.sourceforge.net:8000/hgroot/pyke/release_1 master
$ hg clone master task_1
$ cd task_1
</pre>
<div class="note">
<p class="first admonition-title">Note</p>
<p class="last">This assumes that you've added the <a class="reference external" href="http://pyke.hg.sourceforge.net/hgweb/pyke/release_1/raw-file/tip/hgrc_keywords">hgrc_keywords</a> file to your ~/.hgrc
file. See <a class="reference internal" href="#mercurial-keyword-extension">Mercurial Keyword Extension</a>, above.</p>
</div>
</div>
</div>
<div class="section" id="compiling-ply-tables-files">
<h2>Compiling PLY Tables Files</h2>
<p>Pyke uses <a class="reference external" href="http://www.dabeaz.com/ply/">PLY</a> (Python Lex and Yacc) as it's parser generator. PLY compiles
the Pyke grammars into a set of three tables files:</p>
<ul class="simple">
<li>kfbparser_tables.py (from kfbparser.py)</li>
<li>krbparser_tables.py (from krbparser.py)</li>
<li>scanner_tables.py (from scanner.py)</li>
</ul>
<p>A copy of PLY is included in the source directory (pyke/krb_compiler/ply) so
that there there can be no version mismatch between the version of PLY used to
compile these tables files and the version of PLY installed on your machine.</p>
<p>To regenerate these tables files, at the top-level source directory:</p>
<pre class="literal-block">
$ python
&gt;&gt;&gt; from pyke.krb_compiler import kfbparser, krbparser, scanner
&gt;&gt;&gt; scanner.init(scanner, 0, True)
&gt;&gt;&gt; krbparser.init(krbparser, True)
&gt;&gt;&gt; kfbparser.init(kfbparser, True)
</pre>
<p>or just run the &quot;testall.py&quot; program from the doctest-tools package:</p>
<pre class="literal-block">
$ cd pyke/krb_compiler
$ testall.py
</pre>
</div>
<div class="section" id="compiling-the-compiler-krb-file">
<h2>Compiling the Compiler.krb File</h2>
<p>Pyke uses itself to compile your <a class="reference external" href="../knowledge_bases/rule_bases.html">rule base</a> sources (<a class="reference external" href="../pyke_syntax/krb_syntax/index.html">.krb</a> files) into
Python source (<tt class="docutils literal"><span class="pre">.py</span></tt>) files.</p>
<p>The knowledge base file that Pyke uses for this is
pyke/krb_compiler/compiler.krb. This gets compiled into compiler_bc.py, which
is stored in the source code repository.</p>
<!-- this code is hidden and will create the pyke/krb_compiler/compiled_krb
directory, if needed, for the code section following:
>>> import os, os.path
>>> os.chdir('../../..')
>>> root='pyke/krb_compiler'
>>> dir=root + '/compiled_krb'
>>> os.path.isdir(root)
True
>>> if not os.path.isdir(dir): os.mkdir(dir) -->
<p>To recompile the compiler_bc.py file, from the top-level source directory:</p>
<pre class="literal-block">
$ mkdir pyke/krb_compiler/compiled_krb
$ python
&gt;&gt;&gt; from pyke import krb_compiler
&gt;&gt;&gt; krb_compiler.compile_krb('compiler', 'pyke.krb_compiler.compiled_krb',
... 'pyke/krb_compiler/compiled_krb',
... 'pyke/krb_compiler/compiler.krb')
['compiler_bc.py']
$ mv pyke/krb_compiler/compiled_krb/compiler_bc.py pyke/krb_compiler
</pre>
<!-- this code is also hidden and deletes the
pyke/krb_compiler/compiled_krb/compiler_bc.py file and
pyke/krb_compiler/compiled_krb directory created above.
>>> os.path.isdir(root)
True
>>> os.remove(dir + '/compiler_bc.py')
>>> os.rmdir(dir) -->
</div>
<div class="section" id="running-unit-tests">
<h2>Running Unit Tests</h2>
<p>The <a class="reference external" href="http://code.google.com/p/doctest-tools/">doctest-tools</a> package is required to run the unit tests (see
<a class="reference external" href="installing_pyke.html#other-required-packages">Other Required Packages</a> for more details).</p>
<p>The <tt class="docutils literal"><span class="pre">testall.py</span></tt> and <tt class="docutils literal"><span class="pre">testdoc.py</span></tt> scripts from <tt class="docutils literal"><span class="pre">doctest-tools</span></tt> can be run
anywhere.</p>
<p>In addition, the top-level directory contains a <tt class="docutils literal"><span class="pre">testpyke</span></tt> script that will
delete all of the compiled_krb directories, then run <tt class="docutils literal"><span class="pre">testall.py</span></tt> twice. The
first run must recompile all of the <a class="reference external" href="../knowledge_bases/index.html">knowledge base</a> sources (<a class="reference external" href="../pyke_syntax/krb_syntax/index.html">.krb</a>,
<a class="reference external" href="../pyke_syntax/kfb_syntax.html">.kfb</a> and <a class="reference external" href="../pyke_syntax/kqb_syntax.html">.kqb</a> files) into the compiled_krb directories in order to run
the tests. The second run reuses the files compiled in the first run. This
makes sure that all of the tests run properly whether they have to compile the
knowledge base sources or not.</p>
</div>
<div class="section" id="rebuilding-the-html-documentation">
<h2>Rebuilding the HTML Documentation</h2>
<p>The <tt class="docutils literal"><span class="pre">doc/html</span></tt> directory contains all of the documents that you are reading
now. These are ready to browse directly from your hard drive if you'd like.</p>
<p>The documentation is generated using the <a class="reference external" href="http://sourceforge.net/projects/rest2web">rest2web</a> package, which uses
<a class="reference external" href="http://sourceforge.net/projects/docutils">docutils</a> (see <a class="reference external" href="installing_pyke.html#other-required-packages">Other Required Packages</a> for more details).</p>
<p>The sources for the documentation are in <tt class="docutils literal"><span class="pre">doc/source</span></tt>. Each .txt file there
is converted into an .html file in the doc/html directory by running:</p>
<pre class="literal-block">
$ cd doc/source
$ bin/gen_html
</pre>
<p>This takes about 9 seconds. It:</p>
<ol class="arabic simple">
<li>Temporarily appends hyperlink references onto all of the *.txt files.</li>
<li>Runs <tt class="docutils literal"><span class="pre">r2w</span></tt> to regenerate the files in <tt class="docutils literal"><span class="pre">doc/html</span></tt><ul>
<li>except for those in <tt class="docutils literal"><span class="pre">doc/html/stylesheets</span></tt> and <tt class="docutils literal"><span class="pre">doc/html/images</span></tt>.</li>
</ul>
</li>
<li>Strips all of the hyperlink references from the *.txt files.</li>
<li>Creates a new sitemap.xml file with all of the dates that the files were
last modified.</li>
</ol>
<div class="note">
<p class="first admonition-title">Note</p>
<p class="last">This process uses the date information expanded by the Mercurial <a class="reference external" href="http://mercurial.selenic.com/wiki/KeywordExtension">Keyword
Extension</a>. See <a class="reference internal" href="#mercurial-keyword-extension">Mercurial Keyword Extension</a>, above.</p>
</div>
<p>I've gone ahead and placed the generated html files in the source repository
so that you can browse the documentation locally without having to run
<tt class="docutils literal"><span class="pre">bin/gen_html</span></tt>. So you only need these procedures if you change the
documentation (i.e., change the .txt files in doc/source).</p>
<p>To test all of the code examples in the documents, use the <tt class="docutils literal"><span class="pre">testall.py</span></tt>
command from the <a class="reference external" href="http://code.google.com/p/doctest-tools/">doctest-tools</a> package:</p>
<pre class="literal-block">
$ cd doc/source
$ testall.py
</pre>
<!-- ADD_LINKS MARKER -->
</div>
</div>
<!-- <div id="return-to-top">
<a href="#">Return to Top</a>
</div>
-->
</div>
</td>
<td id="right-nav">
<div id="right-nav-div">
<h3>More:</h3>
<div class="right-item"><a href="cooking_functions.html">Cooking Python Functions</a><p>Explanation of how Pyke &quot;cooks&quot; Python functions.</p>
</div>
<div class="right-item"><a href="what_is_pyke.html">What is Pyke?</a><p>An overview of Pyke's features.</p>
</div>
<div class="right-item"><a href="steps_to_using_pyke.html">Steps to Using Pyke</a><p>A brief list of the steps involved in programming in Pyke (with lots
of links).</p>
</div>
<div class="right-item"><a href="installing_pyke.html">Installing Pyke</a><p>System Requirements and installing Pyke.</p>
</div>
<div class="right-item"><a href="modifying_pyke.html">Modifying Pyke</a><p>Which source code repository to use. And the other tools that you'll
need run the units tests, and rebuild the html documentation.</p>
</div>
</div>
</td>
</tr>
</tbody>
<tfoot id="foot">
<tr id="foot2">
<td id="copyright" colspan="3">
Copyright &copy; 2007-2009 Bruce Frederiksen
</td>
</tr>
</tfoot>
</table>
<div id="last-modified">
Page last modified
Thu, Mar 11 2010.
</div>
<script type="text/javascript">
var gaJsHost = (("https:" == document.location.protocol) ?
"https://ssl." : "http://www.");
document.write(unescape("%3Cscript src='" + gaJsHost +
"google-analytics.com/ga.js' type='text/javascript'%3E%3C/script%3E"));
</script>
<script type="text/javascript">
try {
var pageTracker = _gat._getTracker("UA-6310805-1");
pageTracker._trackPageview();
} catch(err) {}
</script>
</body>
</html>