pinky-develop — PiNKY Developer Discussion

[Pinky-develop] Filtering function

[Randolph K. <rs...@10...>]

Hi team,

We are getting ready to release NK 3.2 which will include PiNKY.

I started to build a simple demo this morning and I realized that we  
are missing a filtering function. The demo I am building will  
aggregate a bunch of feeds and look for mentions of "NetKernel",  
"1060", etc. (The demo will be posted on demo.1060.org).

Is anyone working on a filtering function for PiNKY? Please let me  
know if you are and I'll incorporate it into the PiNKY module and NK 3.2

Thanks!

Randy

[Pinky-develop] Pinky 0.2.0

[Peter R. <dr...@us...>]

Hi Pinksters  - seems like a long time since there was any Pinky news.

I've finished the address space refactor.  All accessors are now of the
form, active:feedXXXX.   The unit tests are converted over and I think I
converted every reference in the docs.  I've updated the build version
number to 0.2.0 to reflect the changes.  Let me know if you find
anything that I missed.  Once its been reviewed we can post a 0.2.0
build on SF.

FYI we're in the final stages of preparing NK 3.2 (hence the long period
of silence).  One of the libs we'd like to make available is PiNKy. If
you're currently working on a Pinky accessor that you'd like to see in
the distro then let me know and we'll make sure to work it into the
build schedule.

Cheers,

Pete

Re: [Pinky-develop] Union updated

[Peter R. <dr...@us...>]

OK. So we should plan an address-space refactor.  I might get chance to
do this before the weekend. Otherwise it will be next week ... or
alternatively if someone fancies taking it on - not difficult just a
chore ;-) ?

Either way we'll hold off doing another public build until after the
refactor.

P.

Mircea Cocosila wrote:
> Using "feed" as a prefix for the PiNKY accessors seems perfect for me.
> 
> --Mircea
> 
> On 2/27/07, *Peter Rodgers* < dr...@us...
> <mailto:dr...@us...>> wrote:
> 
>     Maybe active:pinkyXXXX is still too opaque.  What about using the
>     resource model as the prefix?
> 
>     active:feedUnion
>     active:feedForEachEntry
>     active:feedFetch
> 
>     etc.
> 
>     -------------------------------------------------------------------------
>     Take Surveys. Earn Cash. Influence the Future of IT
>     Join SourceForge.net's Techsay panel and you'll get the chance to
>     share your
>     opinions on IT & business topics through brief surveys-and earn cash
>     http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
>     <http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV>
>     _______________________________________________
>     Pinky-develop mailing list
>     Pin...@li...
>     <mailto:Pin...@li...>
>     https://lists.sourceforge.net/lists/listinfo/pinky-develop
> 
> 
> 
> 
> -- 
> Cheers,
> --Mircea
> 
> 
> ------------------------------------------------------------------------
> 
> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share your
> opinions on IT & business topics through brief surveys-and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> Pinky-develop mailing list
> Pin...@li...
> https://lists.sourceforge.net/lists/listinfo/pinky-develop

Re: [Pinky-develop] Union updated

[Mircea C. <mi...@ne...>]

Using "feed" as a prefix for the PiNKY accessors seems perfect for me.

--Mircea

On 2/27/07, Peter Rodgers <dr...@us...> wrote:
>
> Maybe active:pinkyXXXX is still too opaque.  What about using the
> resource model as the prefix?
>
> active:feedUnion
> active:feedForEachEntry
> active:feedFetch
>
> etc.
>
> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share
> your
> opinions on IT & business topics through brief surveys-and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> _______________________________________________
> Pinky-develop mailing list
> Pin...@li...
> https://lists.sourceforge.net/lists/listinfo/pinky-develop
>



-- 
Cheers,
--Mircea

Re: [Pinky-develop] Planet Pinky Example / Tolerant Design Discussion

[Peter R. <dr...@us...>]

Just realised the sort and truncate criteria can be extracted and
provided declaratively so I've just updated the example to use a feed
config like this:

<feeds>
	<!--The info format is as documented for SetFeedInfo-->
	<info>
		<title>Planet BBC Sports Feeds (Football, Cricket, Golf, Tennis)</title>
		<date>now</date>
	</info>
	<!--The number of entries to provide in the final aggregate-->
	<number>20</number>
	<!--The sorting criteria-->
	<sort>
		<rdate/>
	</sort>
	<!--The feeds to be aggregated - name is not currently used-->
	<feed>
		<name>Football</name>
	
<url>http://newsrss.bbc.co.uk/rss/sportonline_uk_edition/football/rss.xml</url>
	</feed>
	<feed>
		<name>Cricket</name>
	
<url>http://newsrss.bbc.co.uk/rss/sportonline_uk_edition/cricket/rss.xml</url>
	</feed>
	<feed>
		<name>Golf</name>
	
<url>http://newsrss.bbc.co.uk/rss/sportonline_uk_edition/golf/rss.xml</url>
	</feed>
	<feed>
		<name>Tennis</name>
	
<url>http://newsrss.bbc.co.uk/rss/sportonline_uk_edition/tennis/rss.xml</url>
	</feed>
</feeds>


The planet engine is now fully declaratively programmable with no hard
coded values.

P.

[Pinky-develop] Planet Pinky Example / Tolerant Design Discussion

[Peter R. <dr...@us...>]

So the motivation for the recent accessor additions I've been making is I've
been playing with a 'Planet Pinky' application. I've checked in an
implementation to  /test/examples/planet/planet.idoc in the tests
- it runs as the 3rd test in the example tests.

I wanted to have a declaratively configurable "planet engine" to
aggregate an arbitrary mix of RSS and Atom feeds into a single uniform
feed and visual site (cf planetapache.org etc etc) .

The implementation uses a declarative config that looks like this:

<feeds>
	<!--The info format is as documented for SetFeedInfo-->
	<info>
		<title>Planet BBC Sports Feeds (Football, Cricket, Golf, Tennis)</title>
		<date>now</date>
	</info>
	<!--The feeds to be aggregated - name is not currently used-->
	<feed>
		<name>Football</name>
	
<url>http://newsrss.bbc.co.uk/rss/sportonline_uk_edition/football/rss.xml</url>
	</feed>
	<feed>
		<name>Cricket</name>
	
<url>http://newsrss.bbc.co.uk/rss/sportonline_uk_edition/cricket/rss.xml</url>
	</feed>
	<feed>
		<name>Golf</name>
	
<url>http://newsrss.bbc.co.uk/rss/sportonline_uk_edition/golf/rss.xml</url>
	</feed>
	<feed>
		<name>Tennis</name>
	
<url>http://newsrss.bbc.co.uk/rss/sportonline_uk_edition/tennis/rss.xml</url>
	</feed>
</feeds>

The feed engine processes the config into a dynamically generated DPML
script to perform the union. It then sorts, limits to 20 entries, and
sets feed metadata according to the info section of the config.  All in
just 5 instructions!

The nice thing is that the HTTP TTL for each feed provides the cache
lifetime for the aggregate feed - this automagically provides an optimal
global feed polling time since as each dependent feed expires the
aggregate will automatically get rebuilt and refresh the relevant
expired feed resource - upshot is individual feed polling is global minimum.

The only thing missing is feed failure handling - eg a feed goes down or
has a 404 URL etc.  My preference to solve this is to  review the pinky
accessors and have them default to tolerant behaviour - that is we
default to produce a resource that is processible whenever this makes
sense rather than throw exceptions and break the whole pipeline.  This
is one of those 'philosophical' points of NetKernel - we tend to write
accessors to default to be tolerant and then augment with strict
variants (cf xrl, xrl-tolerant). This is in line with the Construct,
Compose, Constrain development model.

So my suggestion is we have, for example...

active:feedUnion   		(default tolerant, if any feed argument is missing
carry-on doing the best it can).
active:feedUnion-strict      (throws exception if a feed argument cannot
be sourced).

Other accessors may not need this differentiation and may naturally fit
with a strict or tolerant design.  For example I have implemented the
Sort accessor to be tolerant of variants between RSS and Atom feeds - if
the comparison cannot be made then it still sorts based on fields that
are there and effectively drops the missing sort criteria.  Equally if
you request to sort on author and the feed doesn't have author it just
sorts on the other valid criteria.

P.

PS I've not changed the service URIs to active:feedXXX yet - still
waiting to hear if this meets with general approval?

[Pinky-develop] Feed Sort Accessor

[Peter R. <dr...@us...>]

I've added a sort accessor.  It sorts (and reverse sorts) by title,
date, description, author, update and category in any combination using
a declarative sort specification like:

<sort>
	<rdate/>
	<title/>
	<category/>
</sort>

This example sorts by date (rdate is reverse sort ie youngest first),
then title, then category.   Note an 'r' prefix on a search term
indicates a reverse sort (so 'rtitle' is reverse alphabetic title sort
etc etc).

P.

Re: [Pinky-develop] Union updated

[Peter R. <dr...@us...>]

Maybe active:pinkyXXXX is still too opaque.  What about using the
resource model as the prefix?

active:feedUnion
active:feedForEachEntry
active:feedFetch

etc.

Re: [Pinky-develop] Union updated

[Brian S. <br...@bo...>]

On Feb 27, 2007, at 11:31 AM, Peter Rodgers wrote:

> active:Union  -> active:pinkyUnion
> active:ForEachEntry -> active:pinkyForEachEntry
>
> The reason for this is that a prefix can be useful to indicate the
> interoperability of a tool ecosystem.  See, for example, the
> active:sqlXXXX set of accessors or  active:rdfXXXX etc.

I like this idea.

[Pinky-develop] Union updated

[Peter R. <dr...@us...>]

I've added support for n-way multiple unions to active:Union.  Old tests
pass and I've added a new multi-union test.

Also I added a change.log to the base pinky/ directory so that we can
keep a record of significant updates which will help with tracking
changes between releases.  If you think you've committed a significant
change please record it in the log.

P.

PS Mircea thanks for fixing the typos!  Looking forward to seeing the
Flickr accessor.  We might want to refactor the accessors   package at
some point - but the first decision should be whether we have the URI
logical space right first - that is the significant thing from a users
point of view.  I'm wondering if we should prefix the pinky collection
so we'd have something like

active:Union  -> active:pinkyUnion
active:ForEachEntry -> active:pinkyForEachEntry

The reason for this is that a prefix can be useful to indicate the
interoperability of a tool ecosystem.  See, for example, the
active:sqlXXXX set of accessors or  active:rdfXXXX etc.

[Pinky-develop] Yahoo! Pipes tutorials

[Mircea C. <mi...@ne...>]

Nice Y! Pipes videos -
http://usefulvideo.blogspot.com/2007/02/yahoo-pipes-tutorials.html

Cheers,
--Mircea

[Pinky-develop] Pipes: Suddenly, Everything Is Mashable

[Mircea C. <mi...@ne...>]

Hi,

Just in case you didn't get to this:
http://developer.yahoo.net/blog/archives/2007/02/yahoo_pipes_sud.html. I
just discovered it.

Cheers,
--Mircea

[Pinky-develop] Working on the Flickr accessor

[Mircea C. <mi...@ne...>]

Hi,

Just to let you know that I'm working on the Flickr accessor.

Cheers,
--Mircea

[Pinky-develop] Organizing accessors in categories

[Mircea C. <mi...@ne...>]

Hi,

I was thinking that it'd be good for us to group accessors by functionality
as Yahoo does. As you know, they have groups such as: sources, user inputs,
operators, url, string, date.

Example: We should have packages like: org.pinkypipes.accessor.sources,
org.pinkypipes.accessor.operators and so on.

Cheers,
--Mircea

Re: [Pinky-develop] Package Imports/Style was Re: PiNKY 0.1.0 Released

[Brian S. <br...@bo...>]

On Feb 23, 2007, at 3:35 PM, Peter Rodgers wrote:

> So for flexibility/malleability we tend to use a * imports of packages
> in NK module classes rather than class by class.
>
> I can live with class-by-class if that's the consensus, just  
> thought I'd
> raise a general point.

<shrug>

> PS Butterfield Condensed Bracket Notation eg
>
> method()
> {	blah;
> 	blah;
> }
>
> is the in-house 1060 style.  This originates from Mr B. who is an
> obsessive normalizer and will not tolerate any waste of space -  
> probably
> 'cos he learnt his craft developing z80 assembler games using peek/ 
> poke.

I don't care what the style is, I just want it to be consistent. In- 
house 1060 style works for me.

Do we have any guidelines about language features? 1.4 only? 1.5? 1.3?

[Pinky-develop] Package Imports/Style was Re: PiNKY 0.1.0 Released

[Peter R. <dr...@us...>]

> Additionally, as this is a dual-purpose module to demonstrate how to do
> these things, I would like to suggest that we don't use the .* import
> notation since that makes it more difficult for people not using IDEs to
> find where things come from.

Interesting.  I usually find the opposite - developing without code
completion/IDE-support using a variety of languages I find it much more
tolerant and flexible to a do package import (*) of Java classes.  Class
by class is very hard to manage without an IDE.

For the record, it is safe to do * imports in a class in an NK module
since each module's class package space is dynamically managed by the
module classloader. This classloader isolation means its very very rare
to get class collisions.

So for flexibility/malleability we tend to use a * imports of packages
in NK module classes rather than class by class.

I can live with class-by-class if that's the consensus, just thought I'd
raise a general point.

> Congrats on the first release.

Congrats all round.  Incidentally do you think we should have a
contributor page on pinkypipes.org to give all contributors credit?

> I'm going to demonstrate Pinky as part of a new Data integration talk at
> the NFJS shows this year.

That's great!

Pete

PS Butterfield Condensed Bracket Notation eg

method()
{	blah;
	blah;
}

is the in-house 1060 style.  This originates from Mr B. who is an
obsessive normalizer and will not tolerate any waste of space - probably
'cos he learnt his craft developing z80 assembler games using peek/poke.

[Pinky-develop] PiNKY Blog Makeover

[Peter R. <dr...@us...>]

PiNKY blog is now publicly accessible. It's also had a makeover...

http://www.1060.org/blogxter/publish/9

Its a multi author blog - if you've got something to say its still
looking for bloggers.

Pete

Re: [Pinky-develop] PiNKY 0.1.0 Released

[Brian S. <br...@bo...>]

I would like to bring up the point about style. Right now we have  
multiple formatting styles including Randy's own... uhm...  
interesting style that he applied to the CountAccessor when he added  
types to it. :)

We should probably standardize on something idiommatic and apply  
CheckStyle.

Additionally, as this is a dual-purpose module to demonstrate how to  
do these things, I would like to suggest that we don't use the .*  
import notation since that makes it more difficult for people not  
using IDEs to find where things come from.

I plan on being aggressive with Eclipse's Organize Imports feature.  
Just want to throw that out there.

Congrats on the first release.

I'm going to demonstrate Pinky as part of a new Data integration talk  
at the NFJS shows this year.

[Pinky-develop] PiNKY 0.1.0 Released

[Peter R. <dr...@us...>]

PiNKY 0.1.0 has been released.  It is available as a sourceforge
download and is also installable within NetKernel using the module installer

Download Install Instructions:

1) download mod-pinky-0.1.0.jar
2) edit NK's etc/deployedModules.xml and add an entry pointing to the jar.
3) boot or cold-restart NetKernel
4) Search for PiNKY to find the documentation

Repeat for test-pinky-0.1.0.jar which contains the unit tests and usage
examples.  To execute tests after install go to
xunit in the developer panel.

NetKernel Module Installer Instructions

1) Go to the module manager:
http://localhost:1066/ep+name@app_install_installer
2) Choose "Install new applications and library modules"
3) Choose "Install new modules from 1060 Mirror."
4) Select your preferred mirror and make sure to check "Display
libraries" (it can take up to 24hrs to be mirrored so use 'Primary
Mirror' today).
5) Select PiNKY and PiNKY Tests from module list.
6) After restart search for 'PiNKY' to find the documentation

Release Notes:

This is the first public build of PiNKY. It implements a large set of
composable feed processing services for high-level feed pipeline
processes.  In addition it contains a resource object model for
constructing and manipulating feeds using either native Java or the
procedural dynamic scripting languages of NetKernel.

Breaking News:

Added a new SetFeedInfo accessor that provides a declarative way to
configure feed metadata.
Added usage examples to the test module in /test/examples/
Wrote a fairly comprehensive guide - search for 'pinky guide'

Enjoy...

The PiNKY Team

[Pinky-develop] Pinky Release Prep

[Peter R. <dr...@us...>]

In preparation for a release I've updated the license references
throughout.  Added licensing documentation. And given Pinky a bit of
style with a module icon.

I've updated the website with the icon...

http://pinkypipes.org/

The SVG source is with the icon in the root of the module.

Will build and release tomorrow.

P.

[Pinky-develop] Automatic check-in notification mechanism for a sourceforge project?

[Mircea C. <mi...@ne...>]

Hi,

Do you know of the existence of an automatic check-in notification mechanism
for a sourceforge project? It'd be useful to have that feedback for PiNKY.
Not a showstopper, though.

Cheers,
--Mircea

Re: [Pinky-develop] Public Module Snapshot + License Question

[Peter R. <dr...@us...>]

Thanks everyone.  The dual-license has now been approved by everyone
that has contributed code.

I'll update the license information in CVS and post builds tomorrow to
both SF and NK mirrors.

Cheers,

Pete

Mircea Cocosila wrote:
> Hi Peter,
> 
> The reason I didn't reply to this message was that the question was
> asked as: "
> 
> Would anyone object if we  dual
> licensed PiNKY"
> 
> I considered that a reply was not needed because I didn't have any
> objection.
> 
> So, YES, I agree with a dual license for PiNKY.
> 
> Cheers,
> --Mircea
> 
> Peter Rodgers wrote:
>> PiNKY is making good progress. In fact we've just had a request from a
>> customer to make it an installable module from the NK module manager
>> service.   I think we should probably start to do a weekly snapshot
>> build on sourceforge with a less frequent and more formal stable release
>> through the NK module installer service?
>>
>> But before I do that I want to make a request.  I started the project
>> with the GPL v2.0 license and I think it's important to keep the code
>> open with this license, but for folks who want to use PiNKY in
>> production the GPL can cause issues.  Would anyone object if we  dual
>> licensed PiNKY and issued the installable module under the 1060 Public
>> License so that it is in line with other module installs and gives
>> solution builders freedom to choose their application licensing?
>>
>> If this dual license approach is acceptable I will create an initial
>> public release build on SF and create a 0.1.0 release on the NK mirrors.
>>
>> Cheers,
>>
>> Pete
>>
>> -------------------------------------------------------------------------
>> Take Surveys. Earn Cash. Influence the Future of IT
>> Join SourceForge.net's Techsay panel and you'll get the chance to
>> share your
>> opinions on IT & business topics through brief surveys-and earn cash
>> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
>> _______________________________________________
>> Pinky-develop mailing list
>> Pin...@li...
>> https://lists.sourceforge.net/lists/listinfo/pinky-develop
>>
>>   
> 

Re: [Pinky-develop] Public Module Snapshot + License Question

[Mircea C. <mi...@ne...>]

Hi Peter,

The reason I didn't reply to this message was that the question was 
asked as: "

Would anyone object if we  dual
licensed PiNKY"

I considered that a reply was not needed because I didn't have any objection.

So, YES, I agree with a dual license for PiNKY.

Cheers,
--Mircea

Peter Rodgers wrote:
> PiNKY is making good progress. In fact we've just had a request from a
> customer to make it an installable module from the NK module manager
> service.   I think we should probably start to do a weekly snapshot
> build on sourceforge with a less frequent and more formal stable release
> through the NK module installer service?
>
> But before I do that I want to make a request.  I started the project
> with the GPL v2.0 license and I think it's important to keep the code
> open with this license, but for folks who want to use PiNKY in
> production the GPL can cause issues.  Would anyone object if we  dual
> licensed PiNKY and issued the installable module under the 1060 Public
> License so that it is in line with other module installs and gives
> solution builders freedom to choose their application licensing?
>
> If this dual license approach is acceptable I will create an initial
> public release build on SF and create a 0.1.0 release on the NK mirrors.
>
> Cheers,
>
> Pete
>
> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share your
> opinions on IT & business topics through brief surveys-and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> _______________________________________________
> Pinky-develop mailing list
> Pin...@li...
> https://lists.sourceforge.net/lists/listinfo/pinky-develop
>
>   

Re: [Pinky-develop] Public Module Snapshot + License Question

[Randolph K. <rs...@10...>]

> Would anyone object if we  dual
> licensed PiNKY and issued the installable module under the 1060 Public
> License so that it is in line with other module installs and gives
> solution builders freedom to choose their application licensing?
>

+1

I have no objection.

Randy

Re: [Pinky-develop] Public Module Snapshot + License Question

[Brian S. <br...@bo...>]

On Feb 21, 2007, at 11:34 AM, Peter Rodgers wrote:

> Would anyone object if we  dual
> licensed PiNKY and issued the installable module under the 1060 Public
> License so that it is in line with other module installs and gives
> solution builders freedom to choose their application licensing?

+1