sprog-users — This list is for both users and developers of Sprog.

Re: [sprog-users] Make Test and Apache LogRegex module

[Gavin H. <gh...@fe...>]

---------- Original Message -----------
From: Grant McLean <gr...@mc...>
To: Sprog Users <spr...@li...>
Sent: Sun, 26 Jun 2005 20:41:42 +1200
Subject: Re: [sprog-users] Make Test and Apache LogRegex module

> On Sun, 2005-06-26 at 08:59 +0100, Gavin Henry wrote:
> > ---------- Original Message -----------
> > From: Grant McLean <gr...@mc...>
> > > I really need someone knowledgeable about repositories 
> > > for their distribution to act as packaging maintainer.
> >
> > ok, Ill start working on it and getting the extra modules
> > needed turned into RPMs too.
> 
> Great.  Once you have some details, I'll put them up on the Sprog
> installation page.  Now I'd better get back to putting a release
> together.

The rpm is just done. I'll send it over once it is approved by the extras team.

> 
> Cheers
> Grant
> 
> -------------------------------------------------------
> SF.Net email is sponsored by: Discover Easy Linux Migration 
> Strategies from IBM. Find simple to follow Roadmaps, straightforward 
> articles, informative Webcasts and more! Get everything you need to 
> get up to speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
> _______________________________________________
> sprog-users mailing list
> spr...@li...
> https://lists.sourceforge.net/lists/listinfo/sprog-users
------- End of Original Message -------

Re: [sprog-users] Re: DBI Gear

[Matthew K. <df...@ya...>]

--- Grant McLean <gr...@mc...> wrote:

> On Fri, 2005-07-01 at 21:42 +1000, Matthew Keene
> wrote:
> > --- Grant McLean <gr...@mc...> wrote:

> 
> And likewise, I'd appreciate comments from people
> who are 
> building gears and have questions about how things
> are meant 
> to work (or why) or comments about how things could
> be better.

The obvious thing is more complete documentation.  A
'How To' guide for gear developers would be excellent
- the gear internals guide that you've got is a good
start, but I found it a little lacking in detail (and
I've just read the scheduler internals document and
found some more stuff in there which now starts to
make a bit more sense).  I was planning on making a
start on a step by step guide when I finished writing
the DBI gear, sort of like a 'Gear Development for
Dummies'.  (We're not all rocket scientists, you know
;-)).  Obviously I could only put in the little I know
so far and you have to fill in the rest....

My other main complaint is not related to developing
gears, but a tool usability issue.  When you right
click on a gear on the palette the current behaviour
is that the selected item is executed when you release
the right mouse button.  This isn't my expectation of
how context menus normally work, which is that a right
mouse click simply displays the context menu, and
selecting an item from the menu is a separate mouse
operation (ie when the right button is released the
menu stays displayed until a selection is made or the
menu is dismissed).  This behaviour is particularly
annoying because the default selected menu item is
Delete.  I spent my first five or ten minutes of
playing with the tool cursing at it because the gears
kept disappearing every time I tried to modify their
properties.  Even after I figured it out it still
catches me sometimes.  Can you either change the
context menu behaviour to be more consistent with
other applications, or at the very least can you make
the default selected item something other than Delete
?
> 
> Here are some thoughts from browsing through your
> code ...
> 
> Package Namespace:
> 
> You might not have got the memo, but all
> 'third-party' extensions
> to Sprog should come under the SprogEx namespace. 

You're right, I must have missed that memo.  I must
have stern words with the mail room.
> 
> The 'engage' Method:
> 
> Your gear does all its work in the 'engage' method. 
> The result 
> is that a query which returns a million row result
> set will dump 
> a million messages in the input queue of the next
> gear - which 
> isn't how I intended it too be used.
> 

No problems - I've made the change as you suggested. 
As I suggested, better documentation would have made
this clearer.  I found that it was a little difficult
to work out which methods in the various gears were
related to the framework (ie overriding behaviours frm
the base class) and which were for that gear (or it's
parent classes) only.
> 
> Password Input:
> 
> You can set the password text to be obscured (eg:
> ******), if 
> you want to.  In Glade, select the password Entry
> widget and 
> toggle the 'Text Visible' button from 'Yes' to 'No'.
> 
Yep, although it's still in clear text in the sprog
file.
> 
> POD:
> 
> You obviously haven't gotten around to doing the POD
> yet, but you really
> out to put your own name in the copyright section at
> the very least.
>

Doesn't everybody do the documentation last, after the
fun part's finished ? 

> 

One thing that I'm still a little confused by when
writing the non-blocking version of this gear is how
to give control back to the framework so that other
events can be processed.  The difficulty here is that
I'm going to basically have a single event (executing
the query) which will potentially consume a large
amount of time before any data becomes available.  I'm
planning to call can_read (or some equivalent) on the
IPC file handle to determine whether there the worker
has produced any data to be processed, should I just
return from send_data without sending any messages and
let the scheduler handle the looping, or should I loop
in my send_data method and make some other call in
order to let the scheduler process other events ?  How
often will the scheduler call the input gear's
send_data method if it hasn't yet started producing
any data ?  I'm a little conscious of consuming CPU
needlessly cycling around waiting for data if it's
going to take some time to arrive, and I had been
planning on putting a second or two's sleep time in
the loop just to make sure that it doesn't put a hard
head lock on the CPU.


	

	
		
____________________________________________________ 
Do you Yahoo!? 
Try Yahoo! Photomail Beta: Send up to 300 photos in one email! 
http://au.photomail.mail.yahoo.com

Re: [sprog-users] Re: DBI Gear

[Grant M. <gr...@mc...>]

On Fri, 2005-07-01 at 21:42 +1000, Matthew Keene wrote:
> --- Grant McLean <gr...@mc...> wrote:
> 
> > On Thu, 2005-06-30 at 19:45 +1000, Matthew Keene
> > wrote:
> > > >Obviously Sprog needs the ability to source data
> > from
> > > >a database via DBI. 
> > > 
> > > I'm trying to write one at the moment.
> > 
> > Great!  (See I said it was an obvious need).
> > 
> I've got a synchronous DBIGear working, 

Excellent.  I just tried it and had no problems.


> I'd appreciate any comments (I'll attach it here).

And likewise, I'd appreciate comments from people who are 
building gears and have questions about how things are meant 
to work (or why) or comments about how things could be better.

Here are some thoughts from browsing through your code ...

Package Namespace:

You might not have got the memo, but all 'third-party' extensions
to Sprog should come under the SprogEx namespace.  Also, in your 
case the word 'Gear' appears redundantly.  I'd suggest changing 
from:

  package Sprog::Gear::DBIGear;

to something like:

  package SprogEx::Gear::DBIQuery;


The 'engage' Method:

Your gear does all its work in the 'engage' method.  The result 
is that a query which returns a million row result set will dump 
a million messages in the input queue of the next gear - which 
isn't how I intended it too be used.

The framework relies on cooperative multitasking, so if any one 
method is working for a long time and not returning, no other 
gear is getting a chance to process its incoming messages.  By 
sending messages in chunks, machines can work with arbitrarily 
large data sets.  

It's not usually a consideration for filter gears since they 
just process each message as it is received.  Data source gears 
which generate messages are expected to be a bit 'nicer'.

The Sprog scheduler will call the 'send_data' method of a data 
source gear whenever it is ready for the next chunk of data.  The 
gear may respond by sending more than one message, but shouldn't 
send an arbitrarily large number.  I'd suggest restructuring your 
code so that the connect/prepare/execute happen in the engage 
method, but the fetch happens in the send_data method.  Something 
like this:

sub send_data {
  my $self = shift;

  if(my $hash_ref = $self->{_sth}->fetchrow_hashref) {
    $self->msg_out(record => $hash_ref) ;
    return;
  } 

  $self->{_dbh}->disconnect ;
  $self->disengage ;
}


Password Input:

You can set the password text to be obscured (eg: ******), if 
you want to.  In Glade, select the password Entry widget and 
toggle the 'Text Visible' button from 'Yes' to 'No'.


POD:

You obviously haven't gotten around to doing the POD yet, but you really
out to put your own name in the copyright section at the very least.


> I'm working on a design which will allow the query to
> run asynchronously, I'm going to start a worker
> process which will use sockets to communicate back to
> the caller.  This is apparently what POE does to
> simulate asynchronous database calls, and I figure
> that it might be a bit easier to have structured (and
> potentially binary) communication than IPC::Open*.

The only reason I was thinking of IPC::Open3 was because it is 
meant to handle spawning another process properly on Win32 (without
using fork emulation).

> I'm a bit concerned that starting a worker process and
> using interprocess communication (of some form) is
> quite cumbersome and error-prone (not to mention
> slower)

Ultimately, I intend to have more support in the Sprog framework 
for doing this type of thing.  It will be needed for other data 
sources (and sinks) as well.

> and I'll probably end up making it a
> user-selectable option to run it asynchronously or
> not.  This gives the user the ability to run it in
> process if they're confident that the select statement
> will return in a reasonable amount of time or are
> willing to wait for it.  

Personally, I don't see the need for two different approaches, 
but if you want to then the simplest approach is to just make 
two gears.  But whether the user chooses a different gear or a 
checkbox in a dialog, it's not clear to me what they'd base 
their choice on.

> It seems a bit like the tail
> wagging the dog if the whole thing has to run out of
> process just so someone can click the stop button,
> when this is probably the exceptional case.  Your
> thoughts ?

It is more complex than the equivalent Perl script you'd run 
from the command line.  But that's the nature of working in an 
event driven environment.

Cheers
Grant

Re: [sprog-users] Re: DBI Gear

[Matthew K. <df...@ya...>]

--- Grant McLean <gr...@mc...> wrote:

> On Thu, 2005-06-30 at 19:45 +1000, Matthew Keene
> wrote:
> > >Obviously Sprog needs the ability to source data
> from
> > >a database via DBI. 
> > 
> > I'm trying to write one at the moment.
> 
> Great!  (See I said it was an obvious need).
> 
I've got a synchronous DBIGear working, I'd appreciate
any comments (I'll attach it here).

I'm working on a design which will allow the query to
run asynchronously, I'm going to start a worker
process which will use sockets to communicate back to
the caller.  This is apparently what POE does to
simulate asynchronous database calls, and I figure
that it might be a bit easier to have structured (and
potentially binary) communication than IPC::Open*.

I'm a bit concerned that starting a worker process and
using interprocess communication (of some form) is
quite cumbersome and error-prone (not to mention
slower), and I'll probably end up making it a
user-selectable option to run it asynchronously or
not.  This gives the user the ability to run it in
process if they're confident that the select statement
will return in a reasonable amount of time or are
willing to wait for it.  It seems a bit like the tail
wagging the dog if the whole thing has to run out of
process just so someone can click the stop button,
when this is probably the exceptional case.  Your
thoughts ?


	

	
		
____________________________________________________ 
Do you Yahoo!? 
Try Yahoo! Photomail Beta: Send up to 300 photos in one email! 
http://au.photomail.mail.yahoo.com

Re: [sprog-users] Re: DBI Gear

[Matthew K. <df...@ya...>]

--- Matthew Keene <df...@ya...> wrote:

> 
> > Yes that's just a problem with the palette - I've
> > only added 
> > mini-icons for the gear shapes I've need so far. 
> > I've just 
> > checked a new version of
> lib/Sprog/GtkView/Chrome.pm
> > into CVS
> > to fix those two.
> 
> Thanks for that, I'll get it from CVS now and let
> you
> know how I go.
> 
>
Cool, new icons work fine.  

Send instant messages to your online friends http://au.messenger.yahoo.com 

Re: [sprog-users] Re: DBI Gear

[Matthew K. <df...@ya...>]

> Yes that's just a problem with the palette - I've
> only added 
> mini-icons for the gear shapes I've need so far. 
> I've just 
> checked a new version of lib/Sprog/GtkView/Chrome.pm
> into CVS
> to fix those two.

Thanks for that, I'll get it from CVS now and let you
know how I go.


	

	
		
____________________________________________________ 
Do you Yahoo!? 
Try Yahoo! Photomail Beta: Send up to 300 photos in one email! 
http://au.photomail.mail.yahoo.com

Re: [sprog-users] Re: DBI Gear

[Matthew K. <df...@ya...>]

--- Grant McLean <gr...@mc...> wrote:


> 
> Except that no one would describe Perl threads as
> elegant.
> The Gtk2 libraries are also reputed to have 'issues'
> with 
> threads.  I haven't tried using them together but I
> was 
> advised not to by someone who should know :-)
> 
Yes !  I just tried it and found those very 'issues'. 
There's also the even bigger problem of the fact that
there's no 'kill' or 'destroy' method available for
threads, which kind of defeats the purpose.
> 
> 
> I intended to use IPC::Open2/3.  Obviously that
> would use 
> fork on *nix, but on Windows it seems to use the
> native 
> spawn functions.
> 

I'll try out IPC::Open* and see how I go.  I'll also
have a look at fork() (which I've found out does have
an 'emulation' on Windows) and see if that also might
help, although I don't think we want to fork the
entire sprog process.  I'll let you know how I go.

> 
> Yes :-)  But you're not the only one.
> 
> If you want to connect gear B to the output of gear
> A, then 
> drag B and drop it on A.  The secret is that when
> you drop it, 
> the top left corner of B must be within the outline
> of A.

I thought I'd tried all combinations of that, but I'll
give it another go.

> 
> I plan to redo that area and add support snapping to
> the closest
> connector as well as for moving gears with the
> keyboard.
> 

Yep, I'd have to say it's not particularly intuitive
right now.



	

	
		
____________________________________________________ 
Do you Yahoo!? 
Try Yahoo! Photomail Beta: Send up to 300 photos in one email! 
http://au.photomail.mail.yahoo.com

Re: [sprog-users] Re: DBI Gear

[Grant M. <gr...@mc...>]

On Thu, 2005-06-30 at 20:01 +1000, Matthew Keene wrote:
> One other question - I had envisaged that the gear
> would have null input (ie be a starting gear) and hash
> output (so that the values returned by the query would
> be passed in a hash keyed by the appropriate column
> names).  

Sounds reasonable.

> When I set the type_in to _ and the type_out
> to H, sprog doesn't seem to recognise it (it doesn't
> show the appropriate icon on the gear).  

Yes that's just a problem with the palette - I've only added 
mini-icons for the gear shapes I've need so far.  I've just 
checked a new version of lib/Sprog/GtkView/Chrome.pm into CVS
to fix those two.

Cheers
Grant

Re: [sprog-users] Re: DBI Gear

[Grant M. <gr...@mc...>]

On Thu, 2005-06-30 at 19:45 +1000, Matthew Keene wrote:
> >Obviously Sprog needs the ability to source data from
> >a database via DBI. 
> 
> I'm trying to write one at the moment.

Great!  (See I said it was an obvious need).

> >The tricky part of writing data source gears is the
> >need for non-blocking IO.  If someone submits a query that
> >takes several minutes to complete, then we obviously want
> >the "Stop" button to be functional during that time.
>  
> > Does anyone know if there are any hooks in DBI for
> > using it asynchronously?
> 
> I'm not aware of anything built into DBI directly, but
> if the application needs to remain responsive while
> the query's executing then I would think that threads
> would be the best way to go.  This is a bit more
> elegant than using fork(), 

Except that no one would describe Perl threads as elegant.
The Gtk2 libraries are also reputed to have 'issues' with 
threads.  I haven't tried using them together but I was 
advised not to by someone who should know :-)

> particularly on platforms
> where Perl's support for fork is a bit of a kludge 

I intended to use IPC::Open2/3.  Obviously that would use 
fork on *nix, but on Windows it seems to use the native 
spawn functions.

> One unrelated question:  I haven't yet figured out how
> you're meant to modify a machine once you've
> constructed it (ie add or remove gears).  The gears
> seem to connect OK when you drag them from the list on
> the left, but I haven't yet found a way of
> reconnecting gears which become disconnected in the
> machine (due to deleting gears), or to connect new
> gears in the middle of an existing machine.  Is there
> something I'm missing ?

Yes :-)  But you're not the only one.

If you want to connect gear B to the output of gear A, then 
drag B and drop it on A.  The secret is that when you drop it, 
the top left corner of B must be within the outline of A.

I plan to redo that area and add support snapping to the closest
connector as well as for moving gears with the keyboard.

Cheers
Grant

Re: [sprog-users] Re: DBI Gear

[Matthew K. <df...@ya...>]

One other question - I had envisaged that the gear
would have null input (ie be a starting gear) and hash
output (so that the values returned by the query would
be passed in a hash keyed by the appropriate column
names).  When I set the type_in to _ and the type_out
to H, sprog doesn't seem to recognise it (it doesn't
show the appropriate icon on the gear).  It does the
same thing for type_in _ and type_out A.  Is this
supported at the moment, or is it just something that
the GUI doesn't recognise ?

Send instant messages to your online friends http://au.messenger.yahoo.com 

[sprog-users] Re: DBI Gear

[Matthew K. <df...@ya...>]

>Obviously Sprog needs the ability to source data from
a database via
> DBI. 

I'm trying to write one at the moment.  

>The tricky part of writing data source gears is the
need for
> non-blocking IO.  If someone submits a query that
takes several minutes
> to complete, then we obviously want the "Stop"
button to be functional
> during that time.
 
> Does anyone know if there are any hooks in DBI for
using it
> asynchronously?

I'm not aware of anything built into DBI directly, but
if the application needs to remain responsive while
the query's executing then I would think that threads
would be the best way to go.  This is a bit more
elegant than using fork(), particularly on platforms
where Perl's support for fork is a bit of a kludge (I
know it doesn't currently run on Windows, but I guess
there shouldn't be anything that precludes it).  The
only drawback is that it makes this gear dependent on
having Perl 5.8 installed.

Anyway, I've got it about 50% written, when I'm close
to finished I wouldn't mind getting some feedback or
suggestions about it.

One unrelated question:  I haven't yet figured out how
you're meant to modify a machine once you've
constructed it (ie add or remove gears).  The gears
seem to connect OK when you drag them from the list on
the left, but I haven't yet found a way of
reconnecting gears which become disconnected in the
machine (due to deleting gears), or to connect new
gears in the middle of an existing machine.  Is there
something I'm missing ?


	

	
		
____________________________________________________ 
Do you Yahoo!? 
Try Yahoo! Photomail Beta: Send up to 300 photos in one email! 
http://au.photomail.mail.yahoo.com

[sprog-users] Announce: LDIF to Record

[Chris B. <ch...@je...>]

Hi all,

This is the initial public release of a Gear to turn LDIF stanzas (as
produced by e.g. OpenLDAP's ldapsearch) into records.

It depends on Sprog::Gear::InputByPara to read \n\n-delimited stanzas.

Available from http://jesmond.demon.co.uk/Sprog/

------------------
NAME
       Sprog::Gear::LdifToRecord - parses an LDIF stream into records

DESCRIPTION
       This gear parses an LDIF stream into records.  Records should be sepa-
       rated by a blank line.  This module assumes that the attribute names
       are separated from the value by ': '.  The attributes of the entry are
       parsed into keys in the the hash record.

       Multi-valued attributes are stored in an array-reference which can be
       used in the Template gear like other template array variables:

               Alternate addresses: [%  mailAlternateAddress.join %]
               [% FOREACH m IN mailAlternateAddress %]
               Alternate address: [% m; END %]
------------------

A useful combination is:

Text Input (with the string to search for)
->	Find and Replace (s/\s*$/\n/)	# to ensure the string ends in \n
->	Run Filter Command (ldapsearch -LLLx -f- 'cn=%s')
->	LDIF to Record
->	Apply Template ([% attributes %])
->	Text Window

Best wishes
-- 
Chris Benson

[sprog-users] Announce: Read Excel Version 0.04

[Marc S. <mar...@on...>]

The reworked Excel gear has been posted to:

http://osdn.whapps.com/files/Sprog-Gear-ReadExcel-0.04.tar.gz

New in this revision:
* Reads Excel data via 'pipe' input, allowing the Read File/Retrieve 
URL/Run Command gears to all pass Excel data
* Outputs 'list' data allowing the user to skip the CSV step needed with 
the last release

Thanks for all of the suggestions for improving the module.  Hopefully 
this release is a bit more useful than the last one.  I'm planning on 
working in support for selecting which sheet from the Excel data you 
want to read.  I'm not sure when this will happen, hopefully soon. 

Marc Slagle

Re: [sprog-users] Suggestion/Request: runable .sprog files?

[Grant M. <gr...@mc...>]

On Wed, 2005-06-29 at 01:47 +0100, Chris Benson wrote:
> Hi all,
> 
> I'm starting to build a library (family?) of .sprogs.  I was wondering how
> feasible it would be to make the saved .sprog files executable?
> That is: start the the saved files with some command like:
> 	#!/usr/bin/sprog --nogui --run -

Yes, I also considered that idea and didn't reach a conclusion as to
whether it was desirable or not.  I wondered if emailing such a file to
someone would constitute a security risk and decided it probably
wouldn't (at least no worse than mailing a .sh, .exe or .bat).

> Adding that to the .sprog file doesn't seem to interfere with the YAML.
> But it looks like the .sprog is run by the shell:
>
> $ ./t1.sprog
> ./t1.sprog: line 2: -: command not found
> ./t1.sprog: line 3: -: command not found
> ./t1.sprog: line 4: -: command not found

Yes, I vaguely recall that you can't use an interpreted program as the
interpreter in a shebang line.  Or if you can, some non-portable tricks
are required.

What you could do is compile a C wrapper like the following
(runsprog.c):

    #include <stdio.h>
    #include <unistd.h>

    int main(int argc, char** argv) {
        char *args[] = {"sprog", "--nogui", "", NULL};

        args[2] = argv[argc-1];
        execvp("/usr/bin/sprog", args);
        return(1);
    } 

Compile it with 'make runsprog' and install it as say 
/usr/local/bin/runsprog and then use the shebang line:

  #!/usr/local/bin/runsprog

Note, any options in the shebang line would be ignored by this simple
wrapper.  Options in shebang lines are treated differently by different
systems so for expediency I ignored them and hardcoded '--nogui'.

That small snippet severely tested my C 'skills' but if someone wanted
to turn it into a more general solution then go for it.

Regards
Grant

[sprog-users] Suggestion/Request: runable .sprog files?

[Chris B. <ch...@je...>]

Hi all,

I'm starting to build a library (family?) of .sprogs.  I was wondering how
feasible it would be to make the saved .sprog files executable?
That is: start the the saved files with some command like:
	#!/usr/bin/sprog --nogui --run -

Adding that to the .sprog file doesn't seem to interfere with the YAML.
But it looks like the .sprog is run by the shell:

$ type sprog
sprog is hashed (/usr/bin/sprog)
$ head -4 t1.sprog
#!/usr/bin/sprog -n -r -
- Sprog
- 1
- run_on_drop: 0
$ ls -l ./t1.sprog
-rwxr-xr-x  1 chrisb users 652 2005-06-29 01:44 ./t1.sprog
$ ./t1.sprog
./t1.sprog: line 2: -: command not found
./t1.sprog: line 3: -: command not found
./t1.sprog: line 4: -: command not found
...

$ strace ./t1.sprog
execve("./t1.sprog", ["./t1.sprog"], [/* 31 vars */]) = 0
strace: exec: Exec format error

So how can I tell sprog to read its source from the file?
-- 
Chris Benson

[sprog-users] Re: [ sprog-Feature Requests-1228729 ] Merge file readers/writers

[Grant M. <gr...@mc...>]

I've posted my reply to this feature request to the list because I don't
think the Sourceforge tracker system is a useful mechanism for debating
interface changes.

On Mon, 2005-06-27 at 19:25 -0700, SourceForge.net wrote:
> Feature Requests item #1228729, was opened at 2005-06-28 12:25
> Message generated for change (Tracker Item Submitted) made by Item Submitter
> Submitted By: Tim Nelson (wayland)
> Summary: Merge file readers/writers
> 
> Initial Comment:
> The number of gears is going to become unmanageable as
> soon as people really start using this.  

I'm not sure I agree with that as a starting proposition.  The palette
already supports filtering so you can limit the view by connector type
or keyword matching or both.

It's true that having one long list of all the gears wouldn't be very
useful.  That's just the default setting at the moment while the list of
core gears is small.

I am definitely open to ideas on how to make the palette more usable.  I
built it as a floating palette window and may make that an option in the
future if there's any demand.  I happen to like the way the Gimp uses
floating windows, but I know a lot of people hate it and find it
confusing.  I'm aiming for simple.

> I'd suggest having slightly more generic gears with more properties.  

That idea is definitely moving in the opposite direction from where I'm
heading.  I don't mind if people write gears that are complex and have
lots of settings, but I won't be doing that with the core gears.  The
successful Unix philosphy of using small focussed utilities and
connecting them together underlies my thinking.

> In this particular instance, I'm suggesting that the
> file reading/writing gears have a "file format" option,
> so for example, you could choose the format "Apache
> Log", and then you would no longer need the gear for
> "Parse Apache Log".  

Except then your Apache logs parsing is tied to reading local files.  
I want people to be able to source their data independently of how
they're going to process it.  We already have support for HTTP, HTTPS
and FTP via the Retrieve URL gear.  Although it would make sense to use
these protocols to retrieve Apache log files, it doesn't make sense to
build log parsing into a file transfer gear.

> You could also set an output format for the files.  eg.
> you could set the  output format to "CSV" (as a
> property), and then you could eliminate the "list to
> CSV" transform.

But what if someone needs to select an output format that we haven't
thought of (MIME encoded GZIPed XML for example)?  By building
components to do the individual steps, we can allow people to assemble
them in ways we never envisaged.

> Unfortunately that would mean that the above would need
> to be polymorphic as to the type of connector allowed,

Yes, I think that's a fatal flaw right there.

> but if you were using the "pipes" idea, you could
> colour-code the types of connector allowed, and allow
> multiple types to connect to the one thing (and use
> perl's "ref" to figure out which was which).

Colour-coding is an interesting idea.  I've given a little thought to
questions like "should somebody be able to connect an HTML data source
to a CSV parser?".  Ultimately, I think the answer to that one is yes.
Even though we're pretty sure in that example it won't be useful.

The different connector shapes that I've built so far indicate a
different API rather than a different file format.  It's kind of
analagous to you can't plug your phone into the hot water tap - it just
wouldn't make sense.  I'm not planning on having lots more connector
types but it'll be interesting to see what needs develop.

Regards
Grant

Re: [sprog-users] Sprog on Mac OS X

[Christian R. <cr...@we...>]

Hi Grant,

> Ok, that's pretty clear then (if a little surprising).  Can you =
confirm

Seems that the form you used is fairly recent =96 the Exporter docs for=20=

5.8.1 don't have it.

>   use base qw(Exporter);

That works fine.

Greetings,
    Christian

Re: [sprog-users] Sprog on Mac OS X

[Grant M. <gr...@mc...>]

On Mon, 2005-06-27 at 08:33 +0200, Christian Renz wrote:
> Hi Grant,
> 
> > Can you tell me what error message you get with the original code?
> 
> $ perl -e 'use Exporter qw(import);'
> "import" is not exported by the Exporter module

Ok, that's pretty clear then (if a little surprising).  Can you confirm
that it works if you change the original line from:

  use Exporter qw(import);

to:

  use base qw(Exporter);

If so, I'll check that in.

Regards
Grant

Re: [sprog-users] Sprog on Mac OS X

[Christian R. <cr...@we...>]

Hi Grant,

> Can you tell me what error message you get with the original code?

$ perl -e 'use Exporter qw(import);'
"import" is not exported by the Exporter module

It works with 5.8.4 on another machine.

> Cool! Can I put that up in the screenshots section of the web site?

Sure! Let me know if you need other screenshots.

> Are you running the latest version with the configurable background
> colour or is that pale yellowish colour some GTK/X under MacOS

Yes, I configured the background; the standard colour was green.

Greetings,
    Christian

Re: [sprog-users] Sprog on Mac OS X

[Grant M. <gr...@mc...>]

On Sun, 2005-06-26 at 19:37 +0200, Christian Renz wrote:
> Hi all,
>=20
> I stumbled upon sprog via the perl.com article, and I really like t=
he=20
> idea! Even though sprog is young, I think it has a lot of potential=
.
>=20
> To make it work on Mac OS X 10.3 (=B4Panther=A1, using Perl 5.8.1),=
 I had=20
> to change Sprog::ClassFactory to read
>      use Exporter;
>      our @ISA =3D qw(Exporter);
> instead of
>      use Exporter qw(import);

Can you tell me what error message you get with the original code?

> I made myself a little =B4Read URL from Param=A1 gear and made a li=
ttle=20
> sprog to retrieve today's dilbert comic strip. Here's a screenshot:

Cool! Can I put that up in the screenshots section of the web site?

Are you running the latest version with the configurable background
colour or is that pale yellowish colour some GTK/X under MacOS
weirdness?

Cheers
Grant

[sprog-users] Sprog on Mac OS X

[Christian R. <cr...@we...>]

Hi all,

I stumbled upon sprog via the perl.com article, and I really like the=20
idea! Even though sprog is young, I think it has a lot of potential.

To make it work on Mac OS X 10.3 (=93Panther=94, using Perl 5.8.1), I =
had=20
to change Sprog::ClassFactory to read
     use Exporter;
     our @ISA =3D qw(Exporter);
instead of
     use Exporter qw(import);

I made myself a little =93Read URL from Param=94 gear and made a little=20=

sprog to retrieve today's dilbert comic strip. Here's a screenshot:

[sprog-users] Announce: Sprog 0.12

[Grant M. <gr...@mc...>]

Hi All

Just a quick note to say release 0.12 of Sprog is out now ...

The primary focus of this release is fixing some bugs that became
apparent when lots of people started downloading and installing Sprog.
Thanks for the reports! The installation dependencies should be fixed
now, as well as the crash when deleting a flashing gear.

There's a new preferences dialog which allows you to define a 'Personal
Gear Folder' - a place to stick all those gears you're creating or
downloading. You can also use the preferences dialog to change the
colour of the Sprog workspace.

The Sourceforge site has links to the source download as well as Debian
and RPM packages:

  http://sprog.sourceforge.net/

Regards
Grant

Re: [sprog-users] Make Test and Apache LogRegex module

[Grant M. <gr...@mc...>]

On Sun, 2005-06-26 at 08:59 +0100, Gavin Henry wrote:
> ---------- Original Message -----------
> From: Grant McLean <gr...@mc...>
> > I really need someone knowledgeable about repositories 
> > for their distribution to act as packaging maintainer.
>
> ok, Ill start working on it and getting the extra modules
> needed turned into RPMs too.

Great.  Once you have some details, I'll put them up on the Sprog
installation page.  Now I'd better get back to putting a release
together.

Cheers
Grant

Re: [sprog-users] Make Test and Apache LogRegex module

[Gavin H. <gh...@fe...>]

---------- Original Message -----------
From: Grant McLean <gr...@mc...>
To: Sprog Users <spr...@li...>
Sent: Sun, 26 Jun 2005 08:27:16 +1200
Subject: Re: [sprog-users] Make Test and Apache LogRegex module

> On Sat, 2005-06-25 at 20:36 +0100, Gavin Henry wrote:
> > Have you any thoughts about getting it into Fedora Extras?
> 
> That sounds like a great idea.  I'm a Debian user so I've sorted out 
> a Debian package and repository.  Gavin Brown helped me set up a process
> for creating a generic RPM package on my Debian box, but I really 
> need someone knowledgeable about repositories for their distribution 
> to act as packaging maintainer.



ok, Ill start working on it and getting the extra modules needed turned into
RPMs too.

> 
> Cheers
> Grant
> 
> -------------------------------------------------------
> SF.Net email is sponsored by: Discover Easy Linux Migration 
> Strategies from IBM. Find simple to follow Roadmaps, straightforward 
> articles, informative Webcasts and more! Get everything you need to 
> get up to speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
> _______________________________________________
> sprog-users mailing list
> spr...@li...
> https://lists.sourceforge.net/lists/listinfo/sprog-users
------- End of Original Message -------

Re: [sprog-users] Gear Repository

[Tony B. <ton...@gm...>]

> >PS I'm curious as to why you didn't use
> >Spreadsheet::ParseExcel::Simple for this
> I would have used that module, but I have run into trouble using it when
> there are calculated values in the columns. Usually I get better
> results using this more nasty looking approach.

Any examples of the problems here would be appreciated so I can fix them!

Thanks,

Tony