Menu
▾
▴
#79 Missing snapshot files: automated builds for Reduce.jl
Hello, I am the creator of a package for the Julia language for interfacing with Reduce PSL.
Reduce.jl: https://github.com/chakravala/Reduce.jl
The issue I have is with the selection of downloads available in the snapshots. In order to make sure that a generic Julia language user can use the Reduce.jl package on any computer, a version of Reduce PSL must be downloaded. To take care of this, I have created an automated build script that can download and place the files so that Julia language can access Reduce PSL.
This is the build script: https://github.com/chakravala/Reduce.jl/blob/master/deps/build.jl
The "reduce-psl" linux tar file is excellent for my purposes (it's a small download and easy to extract and work with for use).
For OSX, I use the dmg file and extract only the reduce-psl component from it, this works fine (however it would be convenient if the user would not have to wait to download all 100+mb of it, instead a snapshot dmg file with only the reduce-psl variant inside would make the download much more efficient and fast).
For Windows, the Reduce-Setup exe download is used. I have tested it, and a User is able to use the automated build script to fetch Reduce from Julia on windows. However, there is a big inconvenience here: the User must click through an installation program in order to have the Reduce PSL files available. This greatly disrupts the streamlined and robust automated build ability for Windows users, unlike for OSX and Linux users.
Therefore, I would like to request that an additional Windows binary or zip or cab file be included in snapshots, one that only contains the Reduce PSL binaries for Windows users and does not require an installation program to be run. I simply want to be able to copy the Reduce PSL binaries to a folder that the Julia package has write access to, so that the Reduce.jl package build script for Julia can also function completely automated on Windows.
Having such an extra windows binary available in the snapshot is absolutely essential for the successful automated build & testing on Windows of the Reduce.jl package for the Julia language. (Linux and OSX function fine, although a dmg file with only PSL would be nice).
Please let me know if this could be worked out? Regards,
I view your post as really good fun. Having it possible to access some of
Reduce's facility via Julia seems excellent to me. From the Reduce
perspective it would be really good if you could either provide us to a
pointer to the best sectio nof some Julia material that explains and
illustrates it, or create a page that can go on the Reduce web pages
(which are at present being re-deveoloped actively, so now is an
especially useful time) so that we can publicise your work and in doing
make people who look at Reduce see it being of interest to you!
Making a PSL-only Mac .dmg and making a simple .zip file for Windows will
in principle be straightforward. There is liable to be a week or so delay
before getting to it, because the snapshot building system is being
maintained right now and als because various of us are pretty much in the
middle of other projects that we breakl into at risk of losing our
concentration. But it sounds reasonable.
Thank you for your Julia work - and it will be good to see a bit of blurb
showing how Reduce can be used within JUlia - both why Julia might be
interesting and relevant to those of us who start with a Reduce background
and who existing Julia users who have not spotted to Reduce interface
there might like it.
Arthur
On Thu, 31 Aug 2017, Dream Scatter wrote:
Thanks for enjoying Reduce.jl, it was quite fun to work on it.
It originally spawned out of an idea I had explained in this thread: here
Background on Julia and Reduce.jl
Julia language is a high performance language for numerical computation. It uses an abstract syntax tree to pre-compile Julia expression objects (when needed) to machine code using LLVM so that the Julia polymorphic multiple-dispatch algorithm can select the best compiled function / algorithm to use based on type-inference. In other words, it is extremely efficient and quick at searching a huge selection of available specialized and general function defintions, then selecting the best one based on an inference of argument types. The key to manipulating this efficiency is the syntax of the Julia language expression objects themselves.
In julia, it is possible to do metaprogramming (the link explains what an Expr object in julia is. I also recommend looking at other parts of the julia docs to learn about julia language.
With the Reduce.jl package, we are applying Julia's metaprogramming facility. Julia does not have a built in symbolic computation like Reduce does. It does have access to some packages that provide this features externally, such as SymPy for example. However, these are not high performance packages. One solution is to use Reduce PSL and communicate with a pipe of character streams between Julia and Reduce. Within Julia, the Reduce.jl package allows a julia Expr object to be parsed into an RExpr object, which is essentially an object that encapsulates strings of Reduce syntax. Then Reduce.jl can send those Reduce syntax strings via character stream to redpsl via a function called rcall and the output is read back from the character stream and interpreted by Reduce.jl into either an Expr object or RExpr object or LaTeX, etc.
What would be even better if Julia and Reduce could be directly combined into each other, skipping over the requirement for a character stream link between them. The character stream somewhat limits the bandwidth of communication when extremely long input or output is being transmitted as of now. Otherwise, the communication is quite quick and there can be ~10,000 calls to Reduce per second on a small laptop.
The README of Reduce.jl provides some examples of how Reduce.jl can be used to parse julia expressions and reduce expressions. However, I agree with you that some sort of extra explanations could be put together somewhere to introduce these ideas to non-julia or non-reduce users. I've also got other things to concentrate on, but I will make some contributions towards that. If you also have a specific question about it, please state it so that I can make sure to include the answer for future readers.
Windows binary
One more point on the windows binary: in order to extract the Windows binary for Reduce.jl it would be nice if it was in cab format. Would it be possible for your snapshot build system to produce such a cab file? I have an example of one here
The reason is, I experimented with using zip files to extract, but in the end it worked out better with a cab file format because its extract functionality is natively built into microsoft windows software. This is because running commands from Julia into the CLI on windows is not as nice as on Unix-like OS. It requires me to run a bat script with the following command:
where redpsl.cab is the file I want to download from the snapshot (for which I provided an example above). If this kind of cab file could be provided in the snapshot, it would help make sure that every windows user can extract it.
Please let me know if the cab format is possible or if you have another proposal.
When I tried a direct reply to just you, not the whole group, I got
A message that you sent could not be delivered to one or more of its
recipients. This is a permanent error. The following address's) failed:
chakravala@users.sf.net
host mx.sourceforge.net [216.34.181.68]
SMTP error from remote mail server after RCPT
TO:chakravala@users.sf.net:
550 unknown user
Some things make sense to have on full public record - finer details are
perhaps best done directly, so can you please investigate your mail
options and sourceforge and I will try to resend the lost message.
Arthur
I'm afraid that changing the build process for the installation programs is not a priority. None of us have the resources to look into it.
Have you tried starting the Windows installation program with the /? option? A combination of the /SILENT and /DIR options should do what you want.
I don't know much about the MacOS stuff, not owning a Mac myself.
Rainer
That should be fixed now.
Is there any progress on separating the OSX redpsl build from the full distribution and using the gcab for making a portable Windows redpsl for the next release cycle?
If it's possible to do this with CSL too, please let me know.
Regards, Michael