ng4j-namedgraphs — Discussion of Named Graphs in RDF

Re: [namedgraphs] Base64 encoding

[Tim P. <tim...@ou...>]

Hi Jennifer,

The change was, indeed, made on a best guess basis, only motivated by 
removing warnings from the build, so please do as you see fit. 

yours
Tim


On Thursday 17 September 2009 20:50:09 Jennifer Cormier wrote:
> Hi everyone,
> 
> The Base64 encoding is working slightly different since the change away
> from the sun libraries.  (Note: I have no issues with the change, and
> thank Tim for making it!  We needed to move away from those libs.)
> 
> If you run the SWPExample (in src/de.fuberlin.wiwiss.ng4j.examples) and
> compare the functionality from the prior release, it can be seen that the
> sun libraries didn't chunk the data when encoding.  (i.e. no carriage
> return is added after 76 characters.)
> 
> The new code (e.g. in SWPSignatureUtilities.calculateDigest) is using
> Base64.encodeBase64Chunked so it does insert the carriage return.
> 
> >From what I've read, it's actually standard to include the carriage
> return.
> 
> HOWEVER I'm not sure we want to do that for the digest in NG4J.  The
> digest is then added to the warrantGraph as a literal (as the object in an
> RDF statement about the digest).  Perhaps that's legal, but it does look
> odd when the RDF is written in whatever format.
> 
> I'm assuming that change was made unintentionally and we could go back to
> the chunk-less style?  i.e. change to use Base64.encodeBase64 in all
> places?
> 
> Jennifer
> 
> Jennifer P. Cormier
> Sr. Computer Scientist
> ATC-NY
> www.atc-nycorp.com
>   
> 
> ------------------------------------------------------------------------------
> Come build with us! The BlackBerry® Developer Conference in SF, CA
> is the only developer event you need to attend this year. Jumpstart your
> developing skills, take BlackBerry mobile applications to market and stay 
> ahead of the curve. Join us from November 9-12, 2009. Register now!
> http://p.sf.net/sfu/devconf
> _______________________________________________
> ng4j-namedgraphs mailing list
> ng4...@li...
> https://lists.sourceforge.net/lists/listinfo/ng4j-namedgraphs
> 

[namedgraphs] Base64 encoding

[Jennifer C. <jen...@at...>]

Hi everyone,

The Base64 encoding is working slightly different since the change away
from the sun libraries.  (Note: I have no issues with the change, and
thank Tim for making it!  We needed to move away from those libs.)

If you run the SWPExample (in src/de.fuberlin.wiwiss.ng4j.examples) and
compare the functionality from the prior release, it can be seen that the
sun libraries didn't chunk the data when encoding.  (i.e. no carriage
return is added after 76 characters.)

The new code (e.g. in SWPSignatureUtilities.calculateDigest) is using
Base64.encodeBase64Chunked so it does insert the carriage return.

>From what I've read, it's actually standard to include the carriage
return.

HOWEVER I'm not sure we want to do that for the digest in NG4J.  The
digest is then added to the warrantGraph as a literal (as the object in an
RDF statement about the digest).  Perhaps that's legal, but it does look
odd when the RDF is written in whatever format.

I'm assuming that change was made unintentionally and we could go back to
the chunk-less style?  i.e. change to use Base64.encodeBase64 in all
places?

Jennifer

Jennifer P. Cormier
Sr. Computer Scientist
ATC-NY
www.atc-nycorp.com
  

Re: [namedgraphs] What about a new release?

[Tim P. <tim...@ou...>]

On Wednesday 26 Aug 2009 19:11:48 Olaf Hartig wrote:
> On Wednesday 26 August 2009 17:46:04 Tim Pizey wrote:
> > [...]
> > I have just deployed to maven repository at http://melati.og/maven2/ and
> > deployed the maven generated project web site at
> > http://paneris.net/ng4j/index.html
> 
> Looks good. I added a link to this page to the NG4J Web site and to our wiki.
> You should add Jennifer to the project team members.
> 
> Olaf
Committed, redeployed.

Jen I just called you Jen as per your Sourceforge page. 


Tim

Re: [namedgraphs] What about a new release?

[Olaf H. <ha...@in...>]

On Wednesday 26 August 2009 17:46:04 Tim Pizey wrote:
> [...]
> I have just deployed to maven repository at http://melati.og/maven2/ and
> deployed the maven generated project web site at
> http://paneris.net/ng4j/index.html

Looks good. I added a link to this page to the NG4J Web site and to our wiki.
You should add Jennifer to the project team members.

Olaf

Re: [namedgraphs] What about a new release?

[Tim P. <tim...@ou...>]

On Thursday 20 Aug 2009 14:06:53 Olaf Hartig wrote:
> Hey,
> 
> I just released 0.9.2!

Congratulations!

I have just deployed to maven repository at http://melati.og/maven2/ and 
deployed the maven generated project web site at http://paneris.net/ng4j/index.html

Tim

Re: [namedgraphs] What about a new release?

[Olaf H. <ha...@in...>]

Hey,

I just released 0.9.2!

Thanks again for your support.
Cheers!

Olaf

Re: [namedgraphs] SNAPSHOTs

[Tim P. <tim...@ou...>]

On Monday 17 Aug 2009 17:11:58 Olaf Hartig wrote:
> Hey Tim,
> 
> On Monday 17 August 2009 17:36:20 Tim Pizey wrote:
> > Hi,
> >
> >
> > I appear to have passed on the wrong advice with regard to -SNAPSHOT
> > builds.
> >
> > Apparently -SNAPSHOT builds precede release candidates and releases.
> >
> > So the sequence would be:
> >
> > ng4j-0.9.2-SNAPSHOT
> > ng4j-0.9.2-RC1
> > ng4j-0.9.2-RC2
> > ng4j-0.9.2
> > [...]
> 
> The most recent release of NG4J was 0.9.1, current CVS builds ng4j-0.9.2-
> SNAPSHOT, and we will release 0.9.2 this week. Where's the problem?
> 
> Olaf


You have it right, only my advice and even the instructions in pom.xml are sufficiently unclear as not to be wrong.
I will update pom.xml with the above example.

the only place where there is a mistake, apart from in my head, is in ReleaseProcedure.txt, 
which I will amend now. 

cheers
Tim

Re: [namedgraphs] SNAPSHOTs

[Olaf H. <ha...@in...>]

Hey Tim,

On Monday 17 August 2009 17:36:20 Tim Pizey wrote:
> Hi,
>
>
> I appear to have passed on the wrong advice with regard to -SNAPSHOT
> builds.
>
> Apparently -SNAPSHOT builds precede release candidates and releases.
>
> So the sequence would be:
>
> ng4j-0.9.2-SNAPSHOT
> ng4j-0.9.2-RC1
> ng4j-0.9.2-RC2
> ng4j-0.9.2
> [...]

The most recent release of NG4J was 0.9.1, current CVS builds ng4j-0.9.2-
SNAPSHOT, and we will release 0.9.2 this week. Where's the problem?

Olaf

[namedgraphs] SNAPSHOTs

[Tim P. <tim...@ou...>]

Hi, 


I appear to have passed on the wrong advice with regard to -SNAPSHOT builds. 

Apparently -SNAPSHOT builds precede release candidates and releases. 

So the sequence would be:

ng4j-0.9.2-SNAPSHOT
ng4j-0.9.2-RC1
ng4j-0.9.2-RC2
ng4j-0.9.2
ng4j-0.9.3-SNAPSHOT
ng4j-0.9.3-RC1
ng4j-0.9.3-RC2
ng4j-0.9.3
ng4j-0.9.4-SNAPSHOT
ng4j-0.9.4-RC1
ng4j-0.9.4-RC2
ng4j-0.9.4


I am sorry and a little annoyed, as I did research this and was told on IRC the opposite, 
but I was corrected, with citation, by Andy Seabourne. 


I prefer our current sequence, and think it makes more sense, 
but configuration management is about conformity not clarity. 

cheers
Tim

Re: [namedgraphs] What about a new release?

[Tim P. <tim...@ou...>]

On Monday 17 Aug 2009 07:26:59 Olaf Hartig wrote:
> Hey,
> 
> I added RDFa support to the SemWeb Client Lib. This is a great new feature. 
> Furthermore, NG4J uses the latest version of Jena/ARQ thanks to the great work 
> by Tim. I think we should release v.0.9.2 of NG4J.
> 
> Does anyone has something else that should go in this relase?
> 
> Any objection if I prepare the release by the end of this work week?
 
I have added nekohtml to the dependencies, and versioned the .jar file in /lib/

We should now be ready to create a new release. 

cheers
TimP

Re: [namedgraphs] What about a new release?

[Tim P. <tim...@ou...>]

On Monday 17 Aug 2009 15:33:55 Jennifer Cormier wrote:
> 
> I finally pulled down the latest code Friday afternoon and started testing
> but haven't finished yet.  I did notice one small typo in the
> modifications but haven't had a chance to commit - could you or Tim
> confirm the following and make the change?
> 
> In the tests, in de.fuberlin.wiwiss.ng4j.swp.SWPNamedGraphSetTest.java,
> method getAuthority, the "keystoreP" passed in should be used.  (Instead
> "keystore" is used.)

Good catch, sorry. 
Committed change. 

Tim

Re: [namedgraphs] What about a new release?

[Jennifer C. <jen...@at...>]

Hi Olaf,

No objections here re a new release.  It's great that NG4J has been
upgraded to the latest Jena/ARQ - thanks, Tim!  Also the RDFa support that
you added.

I finally pulled down the latest code Friday afternoon and started testing
but haven't finished yet.  I did notice one small typo in the
modifications but haven't had a chance to commit - could you or Tim
confirm the following and make the change?

In the tests, in de.fuberlin.wiwiss.ng4j.swp.SWPNamedGraphSetTest.java,
method getAuthority, the "keystoreP" passed in should be used.  (Instead
"keystore" is used.)

Thanks!

-Jen
 

-----Original Message-----
From: Olaf Hartig [mailto:ha...@in...] 
Sent: Monday, August 17, 2009 2:27 AM
To: ng4...@li...
Subject: [namedgraphs] What about a new release?

Hey,

I added RDFa support to the SemWeb Client Lib. This is a great new
feature. 
Furthermore, NG4J uses the latest version of Jena/ARQ thanks to the great
work 
by Tim. I think we should release v.0.9.2 of NG4J.

Does anyone has something else that should go in this relase?

Any objection if I prepare the release by the end of this work week?

Greetings,
Olaf

[namedgraphs] What about a new release?

[Olaf H. <ha...@in...>]

Hey,

I added RDFa support to the SemWeb Client Lib. This is a great new feature. 
Furthermore, NG4J uses the latest version of Jena/ARQ thanks to the great work 
by Tim. I think we should release v.0.9.2 of NG4J.

Does anyone has something else that should go in this relase?

Any objection if I prepare the release by the end of this work week?

Greetings,
Olaf

Re: [namedgraphs] cvs tag v0_9_2_for_Jena_2_6_0

[Olaf H. <ha...@in...>]

Hey Tim,

On Thursday 30 July 2009 18:29:22 Tim Pizey wrote:
> Hi,
>
> I have now tagged to v0_9_2_for_Jena_2_6_0

Great - things seem to work with the latest Jena/ARQ now. Thanks for your 
efforts!!!

> This means I have followed the procedure in ReleaseProcedure.txt
> except I have not uploaded the jar.
>
> I have also, locally the changes needed to run with jena 2.6.2-SNAPSHOT
> but I have yet to commit.

The latest official release is Jena 2.6.0. We should not rely on unreleased 
snapshots.

Greetings,
Olaf

Re: [namedgraphs] Tag v0_9_1_for_Jena_2_5_7

[Olaf H. <ha...@in...>]

Hey Tim,

On Thursday 30 July 2009 17:02:29 Tim Pizey wrote:
> [...]
> There is one remaining warning in the ant build:
>
> src/de/fuberlin/wiwiss/ng4j/impl/idbased/IdBasedQueryPlan.java:26: warning
> - Tag @link: reference not found: IdBasedQueryIterator

Fixed this. Since the class IdBasedQueryIterator was neither protected nor 
public it was not considered during the generation of the javadocs. It's 
protected now.

Greetings,
Olaf

[namedgraphs] cvs tag v0_9_2_for_Jena_2_6_0

[Tim P. <tim...@ou...>]

Hi, 

I have now tagged to v0_9_2_for_Jena_2_6_0

This means I have followed the procedure in ReleaseProcedure.txt  
except I have not uploaded the jar. 

I have also, locally the changes needed to run with jena 2.6.2-SNAPSHOT 
but I have yet to commit. 

cheers
Tim

[namedgraphs] Tag v0_9_1_for_Jena_2_5_7

[Tim P. <tim...@ou...>]

Hi, 

I have tagged the tree with v0_9_1_for_Jena_2_5_7 
which is a version which builds in ant and maven with jena 2.5.7

I have cleared all warnings in Eclipse (for the Eclipse configuration  
files in the CVS tree).

There is one remaining warning in the ant build: 

src/de/fuberlin/wiwiss/ng4j/impl/idbased/IdBasedQueryPlan.java:26: warning - Tag @link: reference not found: IdBasedQueryIterator

which I cannot understand. Maybe we need to put the class in a separate file.


I am now going to commit the changes needed to build with the current jena.


Tim

Re: [namedgraphs] Moving target

[Tim P. <tim...@ou...>]

On Thursday 30 Jul 2009 12:48:36 Olaf Hartig wrote:
> 
> I don't want to stifle your enthusiasm but I don't see the justification for a 
> new release. What would be the contribution of such a release except 'it 
> works with Jena 2.5.7'? (Jena 2.6 would be something different, BTW)
> Apart from this doubt I must admit I didn't had the time yet to take a look at 
> your work, unfortunately, and this won't change until the end of this week. 
> I'm really sorry for this!
> 
> I propose you tag the current tree as v0_9_1_for_Jena_2_5_7 and then you 
> commit your Jena 2.6.2 compatible work. Would this be okay for you?

Perfect, this means that I can carry on and get everything committed before I go away. 

thanks 
Tim

 

Re: [namedgraphs] Moving target

[Olaf H. <ha...@in...>]

Hey Tim,

On Thursday 30 July 2009 12:36:46 Tim Pizey wrote:
> On Thursday 30 Jul 2009 10:30:53 you wrote:
> > On Thursday 30 July 2009 11:20:26 Tim Pizey wrote:
> > > I nearly had everything compiling with jena 2.6.0 but now loads of
> > > compile problems with 2.6.2!
> >
> > I guess that's due to API changes in Jena or ARQ.
> > Grrr. These permanent changes are very annoying.
>
> Wow, I seem to have fixed it!
>
> I now have ng4j building with Maven and latest and tests passing.

Great job! Thanks!

> I will go back  over what I have done and look.
>
> Before I commit we need to tag the current tree as 'working with 2.5.7'
>
> This could even be a small release, however I do wish to keep up the
> momentum, I go on holiday at the end of this week for two weeks, so it
> would be good to commit today or tomorrow.
>
> What about:
>
> I tag current tree and make a 0.9.3 release, then commit my work so that we
> can have a 0.9.3-SNAPSHOT for a while, during which we remove grddl.

I don't want to stifle your enthusiasm but I don't see the justification for a 
new release. What would be the contribution of such a release except 'it 
works with Jena 2.5.7'? (Jena 2.6 would be something different, BTW)
Apart from this doubt I must admit I didn't had the time yet to take a look at 
your work, unfortunately, and this won't change until the end of this week. 
I'm really sorry for this!

I propose you tag the current tree as v0_9_1_for_Jena_2_5_7 and then you 
commit your Jena 2.6.2 compatible work. Would this be okay for you?

Greetings,
Olaf

Re: [namedgraphs] Moving target

[Olaf H. <ha...@in...>]

On Thursday 30 July 2009 11:20:26 Tim Pizey wrote:
> I nearly had everything compiling with jena 2.6.0 but now loads of compile
> problems with 2.6.2!

I guess that's due to API changes in Jena or ARQ.
Grrr. These permanent changes are very annoying.

Olaf

Re: [namedgraphs] Base64 issues and roadmap

[Olaf H. <ha...@in...>]

Hey,

On Wednesday 29 July 2009 18:34:15 Tim Pizey wrote:
> [...]
> The RoadMap
> https://sourceforge.net/apps/mediawiki/ng4j/index.php?title=Post_0.9_Releas
>e_Plans shows the next thing to do is remove grddl, nekohtml and tagsoup.
> Would you like me to have a go at this?

Feel free but please keep nekohtml.jar. It is needed not just for the GRDDL 
support.

Olaf

[namedgraphs] Moving target

[Tim P. <tim...@ou...>]

I nearly had everything compiling with jena 2.6.0 but now loads of compile problems with 2.6.2!

[namedgraphs] Base64 issues and roadmap

[Tim P. <tim...@ou...>]

Hi, 

I can find out little about the sun package
however http://www.lumentier.com/java/?pth=/java/jdk-7-bld1/sun.misc/BASE64Encoder/class-javadoc.lmtr
seems to give java7 source. 

http://java.sun.com/products/jdk/faq/faq-sun-packages.html
details the problems with using sun packages. 

The uses in CreateTestKeystore were incidental, that is they were just used to print out informational messages. 
so I have been able to change to the Apache Base64Encoder without problem. 

The remaining uses were in SWPSignatureUtilities. 
I tried to write a test which checked that the my change had no effect.
I have found this difficult however. 

Using http://www.lumentier.com/java/?pth=/java/jdk-7-bld1/sun.misc/BASE64Encoder/class-javadoc.lmtr 
and clicking the source link seems to suggest that the sun.misc.CharacterEncoder assumes 8859_1 
character set. 

The apache equivalent method inserts "\r\n" rather than just "\n" as the sun method does. 
to get around this I stripped both line-end chars in the tests before comparison. 

I do not know whether we really want to be formatting the Base64 strings at all. 

We now have the sun.misc removed and banned from the Eclipse class path.

The RoadMap https://sourceforge.net/apps/mediawiki/ng4j/index.php?title=Post_0.9_Release_Plans
shows the next thing to do is remove grddl, nekohtml and tagsoup. 
Would you like me to have a go at this?

I think we should move CreateTestKeyStore to the test hierarchy so that it does not 
show up in the test coverage reports (see http://paneris.net/ng4j/cobertura/index.html)


cheers
Tim 

Re: [namedgraphs] Nearly done

[Jennifer C. <jen...@at...>]

Hi Tim,

That's great news!

Re de.fuberlin.wiwiss.ng4j.swp.setup.CreateTestKeystore:  Rather than
being part of the tests, this is meant to be run very infrequently - for
example when the certificate in the keystore expires and thus causes
errors.  It appeared that originally the keystore was created manually.
That class was my attempt to provide an automated way to create the
keystore and its contents.

Re de/fuberlin/wiwiss/ng4j/impl/NamedGraphSetImpl.java:  What's the hack
that you're referring to?  Is it what you mentioned on 6/19 "However there
appears to be a hack where we treat List<NamedGraph> and List<Graph> as
covariant which will no longer work."?  Could you provide a line number
for the problem?  (Doesn't need to be all occurrences, an example should
suffice.)

Thanks!

Jennifer

[namedgraphs] Nearly done

[Tim P. <tim...@ou...>]

Hi, 

I have updated the lib jars and the LIBRARIES.txt. 
I have added an Eclipse configuration to allow tabs. 

I have started to evict the Sun Base64 encoder in favour of the 
apache commons one. 
The Sun method formats the Base64 output so I have used 
the equivalent apache method. I wrote a test to make sure I broke nothing. 

http://paneris.net/ng4j/cobertura/de.fuberlin.wiwiss.ng4j.swp.impl.SWPAuthorityImpl.html
shows addDescriptionToGraph is now exercised by a new test (it also shows why tabs are evil ;) 

de.fuberlin.wiwiss.ng4j.swp.setup.CreateTestKeystore could perhaps be converted to a unit test?
Currently it outputs nothing as the logger is not set to ALL. 
It does create files in the test hierarchy and alter the results of the test I have just committed. 
I have not understood it yet. 

Can I have blanket permission to change 
				// TODO Auto-generated catch block
				e.printStackTrace();
to 
                                throw new RuntimeException(e);

The Maven and ant builds both work. 

I am keen to move on the the latest jena, though I have yet to figure out exactly how to 
fix the hack in de/fuberlin/wiwiss/ng4j/impl/NamedGraphSetImpl.java

I hope I have not broken anything or trodden on any toes. 


Tim