sprog-users

[sprog-users] Gear Repository

[Marc S. <mar...@on...>]

Is there a place where I can post or download additional gears?  If not 
I would assume that CPAN is the place to go.  I have written an Excel to 
CSV gear, but wanted to see if there was another one floating around 
somewhere before trying to refine the one I'm working on.

Thanks

Marc Slagle
Online-Rewards
2900 Jefferson Ave.
Cincinnati, OH 45219

Re: [sprog-users] Gear Repository

[Grant M. <gr...@mc...>]

Hi Marc

On Fri, 2005-06-24 at 14:52 -0400, Marc Slagle wrote:
> Is there a place where I can post or download additional gears?  If not 
> I would assume that CPAN is the place to go.  I have written an Excel to 
> CSV gear, but wanted to see if there was another one floating around 
> somewhere before trying to refine the one I'm working on.

Welcome to the bleeding edge.  To my knowledge, you are the first person
in the world apart from myself to successfully build a Sprog gear :-)
Congratulations!

Obviously my master plan for Sprog includes lots of different gears
written and maintained by lots of different people.  I have vaguely
considered the need for a gear repository (GearPAN?) but haven't given
much thought yet to how that would work.

In the long term, I want to build repository support directly into Sprog
so installing new gears is as easy as dragging a link from your browser
and dropping it into your palette.  Or a Find Gear dialog that searches
one or more repositories and lets you select gears for installation
directly from the search results.

In the immediate short term we need a way to share gear files manually.
I can add a page to the Sprog web site to provide links to user
contributed gears.  Do you have web space somewhere I could link to?  If
not, I could put files directly on the project site but that would make
it harder for you to upload new versions.

Right now, (as you know) Sprog only recognises gears in the
Sprog::Gear::* namespace.  It might be an idea if I added support for
the SprogEx::Gear::* namespace as well to distinguish contributed gears
from ones in the core distribution.  If anyone has a better idea then
speak up.

Probably best to hold off on uploading to CPAN until the namespace thing
is sorted.

Regards
Grant

Re: [sprog-users] Gear Repository

[Marc S. <mar...@on...>]

>In the long term, I want to build repository support directly into Sprog
>so installing new gears is as easy as dragging a link from your browser
>and dropping it into your palette.  Or a Find Gear dialog that searches
>one or more repositories and lets you select gears for installation
>directly from the search results.
>  
>
There is probably a way to tie this in to the CPAN modules available, I 
can look into this if you haven't got the time to devote to it.

>In the immediate short term we need a way to share gear files manually.
>I can add a page to the Sprog web site to provide links to user
>contributed gears.  Do you have web space somewhere I could link to?  If
>not, I could put files directly on the project site but that would make
>it harder for you to upload new versions.
>  
>
http://osdn.whapps.com/files/Sprog-Gear-ReadExcel-0.03.tar.gz

I can probably place other versions there as I go until there is a 
decision on where to put the repository. 

>Right now, (as you know) Sprog only recognises gears in the
>Sprog::Gear::* namespace.  It might be an idea if I added support for
>the SprogEx::Gear::* namespace as well to distinguish contributed gears
>from ones in the core distribution.  If anyone has a better idea then
>speak up.
>  
>
This is fine with me, I'll be happy to post to whichever namespace makes 
the most sense to the community.  I would suggest that another directive 
for the author or homepage of the gear is added to the POD where the 
name is set. 

I can see tons of uses for this utility, and plan on releasing back to 
the community anything that seems like it will be of  use.  Let me know 
if I can be of more assistance.  Thanks for the app!

Marc Slagle
Online-Rewards

Re: [sprog-users] Gear Repository

[Tony B. <ton...@gm...>]

On 24/06/05, Marc Slagle <mar...@on...> wrote:
> I have written an Excel to CSV gear, but wanted to see if there was anoth=
er=20
> one floating around somewhere before trying to refine the one I'm working=
 on.

You might want to extend this to allow users to specify which
worksheet they wish to use. If I'm reading the code right it's
hardwired to use the first sheet? (Would also be useful to document
this).

I'm also wondering whether mapping this to CSV is the best approach.
It's obviously workable, as Sprog already has the ability to handle
CSV files, and users can therefore translate into whatever they want,
but I suspect that more often than not the next step will be to
translate back out of CSV. So perhaps it might be better to just have
it output a list of lists and have the ability to translate that into
whatever is required (including CSV). Thoughts?

Anyway nice work!

Tony

PS I'm curious as to why you didn't use
Spreadsheet::ParseExcel::Simple for this - would simplify that code a
lot ... :)

Re: [sprog-users] Gear Repository

[Marc S. <mar...@on...>]

>
>PS I'm curious as to why you didn't use
>Spreadsheet::ParseExcel::Simple for this - would simplify that code a
>lot ... :)
>
>
>  
>
I would have used that module, but I have run into trouble using it when 
there are calculated values in the columns.  Usually I get better 
results using this more nasty looking approach.

Marc

Re: [sprog-users] Gear Repository

[Tony B. <ton...@gm...>]

> >PS I'm curious as to why you didn't use
> >Spreadsheet::ParseExcel::Simple for this
> I would have used that module, but I have run into trouble using it when
> there are calculated values in the columns. Usually I get better
> results using this more nasty looking approach.

Any examples of the problems here would be appreciated so I can fix them!

Thanks,

Tony

[sprog-users] Re: Sprog::Gear::ReadExcel.pm Code Review

[Marc S. <mar...@on...>]

Grant McLean wrote:

>I've also been working through your ReadExcel.pm code and coming up with
>some suggestions for making it both better and simpler.  I hope the
>suggestions don't come across as critical because they're not intended
>to be.  I really do appreciate the fact that you've slogged through my
>code enough to get something working.  Looking at your code has given me
>some ideas for how I can make things easier for gear-writers (including
>me).
>
>* I'd recommend moving the 'use strict' and related lines to after
>  the metadata block.  When Sprog is scanning @INC for gear classes
>  it 'parses' the start of each .pm file.  Any extraneous code
>  between the package name and the metadata section runs the risk of
>  confusing the simple parser.
>
>  
>
This was an oversight by me.  Normally I always use this, but since I 
was in "Play Mode" with the module I didn't go too far down the path of 
making it super-tight.  A rework of this code already going on in the 
background.

>* I'm guessing you based your code on the TextInput gear and some
>  bits of code are artifacts from that.  For example you declare a
>  property called 'text' which isn't really needed; an accessor
>  called gear_view that isn't used at all; and an accessor called
>  filename that really ought to be a property.  
>
>  
>
Yes, I did start that way.  Actually, the first thing I tried was to 
"use base qw(Sprog::Gear::ReadFile)" and override the _open_file 
subroutine.  I got that to work, but couldn't get the IO::Scalar fake 
filehandle to work.  After that I went with the TextInput gear since its 
straightforward.  There are probably several strange bits of code that 
don't belong there. I figure that as I get better with the framework 
I'll start writing code that is simpler.

>
>* The file_start and file_end events both send an argument of undef.
>  It really ought to be the filename.
>  
>
Next version will do this.

>* As Tony Bowden observed, your gear probably shouldn't be outputting
>  CSV data.  The 'List' output connector is probably a better choice.
>  People can always use the ListToCSV gear if that's what they want.
>  To use the list connector, just change the value for 'type_out' in
>  the metadata from 'P' to 'A', then instead of calling:
>  
>    $self->msg_out(data => $self->text);
>
>  to send all the CSV data, call:
>
>    $self->msg_out(row => \@cell_values);
>
>  once for each row in the spreadsheet - regardless of whether it's a
>  header or not.
>  
>
>* I'd also suggest that your gear shouldn't read from a file at all.
>  Instead, it should have a type 'P' input connector and simply
>  accumulate all 'data' events.  Then in the file_end event it can
>  call $XL->Parse(), passing it a reference to a scalar containing
>  all the accumulated data instead of a filename.  See ParseHTMLTable.pm
>  for example code.
>  
>
I thought of that as soon as I sent the link last night.  No problems, I 
will update the code to handle input this way.

>
>  Another advantage is that your gear would be simpler.  It wouldn't
>  need the Text::CSV_XS dependency, or the routine for accepting
>  drag-n-drop and it would even need a dialog box at all.  Unless you
>  decided to implement Tony's suggestion of allowing the user to select
>  which sheet to parse.  In which case, I'd suggest a dialog like the
>  ParseHTMLTable gear - one simple entry box where the user can type
>  either the sheet number or the name.
>
>  
>
I would love to implement this.  I'll probably be needing it soon 
anyway.  I have one question about the XML dialogs.  It my be 
documented, and if it is tell me to RTFM.  I would like to have the 
system pop up the dialog box with a pulldown list of the sheets on the 
excel file if they have not selected one.  I know how to make the 
dialog, but didn't know if there was an event I could call to pop up the 
dialog myself from the gear.  I'm not sure its even the best way to do it.

Perhaps even another way would be to put in a gear that splits the excel 
file into a list connector, one sheet for each record.  Any thoughts on 
the preferred methodology?

I don't take any of these things critically, I'm always looking to learn 
something new or improve something I'm working on.  After this excel 
module is put to rest I plan on moving into an XSLT gear, since I 
already have a need for that.  I'm also considering touching MySQL data 
input/output gears, since being able to shuffle spreadsheets on my 
desktop and either read or write records to my database would also make 
life a lot easier.

Thanks!
Marc

Re: [sprog-users] Re: Sprog::Gear::ReadExcel.pm Code Review

[Grant M. <gr...@mc...>]

On Sat, 2005-06-25 at 12:35 -0400, Marc Slagle wrote:
> Grant McLean wrote:
> >* The file_start and file_end events both send an argument of undef.
> >  It really ought to be the filename.
>
> Next version will do this.

Actually if you switch to not reading from a file then you can leave out
the file_start, file_end stuff altogether - it will be handled
automatically by the ReadFile gear.

> >* I'd also suggest that your gear shouldn't read from a file at all.
> >
> I will update the code to handle input this way.

I've just checked in to CVS a SlurpFile mixin module which handles the
details of file_start/end so all your gear needs to do is:

  use base qw(
    Sprog::Gear::SlurpFile
    Sprog::Gear
  );

  sub file_data {
    my($self, $data, $filename) = @_;

    # do stuff with $data and call $self->msg_out
  }

Expect a release in a day or two.

> I have one question about the XML dialogs.  It my be 
> documented, and if it is tell me to RTFM.  I would like to have the 
> system pop up the dialog box with a pulldown list of the sheets on the 
> excel file if they have not selected one.  

Hmm that's tricky from many angles.

* I haven't implemented drop downs in properties dialogs yet because 
  I haven't needed them (yet).

* You can't do it in a normal properties dialog anyway because the
  ParseExcel gear won't know the filename until run-time.

* When I implement the ReadFiles gear, someone could drag-n-drop
  multiple files onto a machine.

* In theory, you could pop-up a dialog at run-time, but you'd need to
  implement that in a custom gear view class.  None of the existing
  gears use dialogs at run time.

> After this excel 
> module is put to rest I plan on moving into an XSLT gear, since I 
> already have a need for that.

That one is on my list too.  Not that I'd discourage you from taking it
on.

> I'm also considering touching MySQL data 
> input/output gears, since being able to shuffle spreadsheets on my 
> desktop and either read or write records to my database would also make 
> life a lot easier.

Yes, DBI connectivity is a much needed feature.  See my other email for
a workaround you might be able to use now.

Cheers
Grant