VIVO

Brian and Eliza,
     Let's get a merged copy of your code into the harvester repo. 
Brian, it seems your new jar is missing some of Eliza's code -- 
specifically the CSV Map:

The method processCsvMap(String, String, String) is undefined for the 
type D2rProcessor    D2RMapFetch.java    
/Harvester-hg/src/main/java/org/vivoweb/harvester/fetch    line 113    
Java Problem

Eliza, I know you have commit rights to SVN -- Brian do you? Eliza can 
you check in your source for D2RMap, and then have Brian add his changes 
in as another commit, etc. That would be really awesome. I'm sure others 
who are using it would be excited to be able to commit fixes and updates 
as well.

Right now we're still working on restoring the SVN from the mercurial 
commits we did. As soon as that's done, we can get your commits in.

Thanks!

Nicholas


On 7/1/11 2:43 PM, Eliza Chan wrote:
> You are right, we should have a repository for the d2rmap source code. 
> Since now you have a modified copy, and I also have one. It's better 
> to merge them.
>
>
> From: "Keese, Brian W" <bkeese@... <mailto:bkeese@...>>
> Reply-To: VIVO Development List <vivo-dev-all@... 
> <mailto:vivo-dev-all@...>>
> Date: Fri, 1 Jul 2011 14:37:11 -0400
> To: VIVO Development List <vivo-dev-all@... 
> <mailto:vivo-dev-all@...>>
> Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster
>
> Sure. Let me know how I can help with the docs.
>
> For the bugs, one of them was fixed before (the oracle semi-colon 
> thing, Map.java around line 94).
>
> The other bug has to do with null values for instance identifiers. In 
> Map.java around line 126, a list of new instances is being compiled. 
> Unique instances are defined by values in a discreet set of fields. If 
> one those values happens to be null, this line would give a null 
> pointer exception:
>
> if (!currentTuple.get(fieldName).equals(lastTuple.get(fieldName)))
>
> So, I changed the logic to this:
>
> if (!(currentTuple.get(fieldName)==null && 
> lastTuple.get(fieldName)==null) &&
>
>                       (currentTuple.get(fieldName)==null || 
> !currentTuple.get(fieldName).equals(lastTuple.get(fieldName))))
>
> I also added some logging info to help with diagnosing problems with 
> the harvest. And I removed some logging info which seemed to me to be 
> not useful.
>
> We don’t have an svn repository for the d2rmap source code do we? 
> Maybe we should set that up.
>
> Brian
>
> Brian Keese
>
> Indiana University Digital Library Program
>
> Herman B Wells Library W501
>
> 812-856-3865
>
> *From:*Eliza Chan [mailto:elc2013@...]
> *Sent:* Friday, July 01, 2011 2:22 PM
> *To:* VIVO Development List
> *Subject:* Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster
>
> Hi Brian,
>
> Thanks for making the changes.
>
> It'd be nice to have d2rmap harvests (db and csv) being supported in 
> future releases, since there are people who use the tools on a regular 
> basis.
>
> I have been trying to update the documentation on d2rmap but it has 
> not been able to keep up with so many changes with Harvester. :-(
>
> Would you be able to document some of it? Also, would you be able to 
> list the bugs that you fixed?
>
> Thanks,
>
> Eliza
>
> *From: *"Keese, Brian W" <bkeese@... <mailto:bkeese@...>>
> *Reply-To: *VIVO Development List <vivo-dev-all@... 
> <mailto:vivo-dev-all@...>>
> *Date: *Fri, 1 Jul 2011 14:05:23 -0400
> *To: *VIVO Development List <vivo-dev-all@... 
> <mailto:vivo-dev-all@...>>
> *Subject: *[Vivo-dev-all] d2rmap harvesting, was oracle for harverster
>
> Nick, Thanks for the info on Oracle. Makes sense.
>
> Here’s another thing. I have made a couple of changes to the d2rmap 
> processor that is included with the harvester. I have uploaded the new 
> jar file to 
> https://www.slashtmp.iu.edu/files/download?FILE=bkeese%2F94328zp0Qy2 
> (the password is ‘d2r4vivo’).
>
> It fixes a couple of bugs, and adds some status info to the logs for 
> d2rmap fetch.
>
> I noticed that there is no example script for d2rmap in the latest 
> release. I hope support for d2r harvests will be continued.
>
> Brian
>
> Brian Keese
>
> Indiana University Digital Library Program
>
> Herman B Wells Library W501
>
> 812-856-3865
>
> *From:*Nicholas Skaggs [mailto:nskaggs@...]
> *Sent:* Friday, July 01, 2011 10:51 AM
> *To:* vivo-dev-all@... 
> <mailto:vivo-dev-all@...>
> *Subject:* Re: [Vivo-dev-all] Vivo development call / meeting reminder 
> for today.
>
> Great catch Brian. The trouble with oracle is that it requires a 
> specific jdbc connector for each release -- including minor point 
> release versions.
>
> http://www.oracle.com/technetwork/database/features/jdbc/index-091264.html
>
> We can't package every version of the connector with the harvester 
> unfortunately. I have heard of a so called "universal" connector for 
> oracle that connects to the databases through lowest common 
> denominator SQL. I've never been able to find anything additional 
> about this, nor a download for it -- so I can't verify it exists or 
> not.  In addition, the connectors are released by Oracle under the 
> Oracle Technology Network Development and Distribution License. This 
> prevents us from distributing these files by packaging them directly 
> without complying with the license terms. Specifically, the 
> redistribution clause of the license causes trouble:
>
> /Program Distribution
> We grant you a nonexclusive, nontransferable right to copy and 
> distribute the programs to your end users provided that you do not 
> charge your end users for use of the programs and provided your end 
> users may only use the programs to run your applications for their 
> business operations. Prior to distributing the programs you shall 
> require your end users to execute an agreement binding them to terms 
> consistent with those contained in this section and the sections of 
> this agreement entitled "License Rights," "Ownership and 
> Restrictions," "Export," "Disclaimer of Warranties and Exclusive 
> Remedies," "No Technical Support," "End of Agreement," "Relationship 
> Between the Parties," and "Open Source." You must also include a 
> provision stating that your end users shall have no right to 
> distribute the programs, and a provision specifying us as a third 
> party beneficiary of the agreement. You are responsible for obtaining 
> these agreements with your end users./
>
> You can read the entire license here:
>
> http://www.oracle.com/technetwork/licenses/distribution-license-152002.html
>
> If you have an idea about how we can distribute something to help the 
> harvester work out of the box with oracle, please let us know. Thanks,
>
> Nicholas
>
> On 7/1/11 10:39 AM, Keese, Brian W wrote:
>
> The attachment didn't work. The driver can be downloaded from Oracle here:
>   
> http://www.oracle.com/technetwork/database/enterprise-edition/jdbc-10201-088211.html
> Brian
> ________________________________________
> From: Keese, Brian W [bkeese@...  <mailto:bkeese@...>]
> Sent: Friday, July 01, 2011 10:36 AM
> To:jrpence@...  <mailto:jrpence@...>; VIVO Development List
> Subject: Re: [Vivo-dev-all] Vivo development call / meeting reminder for today.
>   
> Hey Harvester Guys,
> I noticed that the oracle jdbc driver was never added (or was added, then removed) to the harvester bin/dependencies folder for the latest release. Could you add that in please?
> It is ojdbc14.jar (attached to this email).
>   
> Thanks,
> Brian
>   
>   
> ________________________________________
> From: James Pence [jrpence@...  <mailto:jrpence@...>]
> Sent: Thursday, June 30, 2011 1:23 PM
> To:vivo-dev-all@...  <mailto:vivo-dev-all@...>
> Subject: [Vivo-dev-all] Vivo development call / meeting reminder for today.
>   
> Hello,
>   
>      Today the vivo developers, and maybe you, are having a goto-meeting at 3:00pm EST (UTC-4 hours)
>   
>      We have the beginnings of an agenda at<http://sourceforge.net/apps/mediawiki/vivo/index.php?title=2011-06-30_Development_Leads_Call>
>   
> If you have any agenda items to add to the meeting please let me know (jrpence@...  <mailto:jrpence@...>    ).
>   
>     Since we only have 25 connections for the goto meeting please consolidate where possible.
>   
>     1. Please join my meeting, Thursday, June 30 at 3:00 PM Eastern DaylightTime.
>     https://www1.gotomeeting.com/join/486839241
>   
>     2. Use your microphone and speakers (VoIP) - a headset is recommended.
>   
>     Or, call in using your telephone.
>     Dial +1 (215) 383-1013
>   
>     Access Code: 486-839-241
>     Audio PIN: Shown after joining the meeting
>     Meeting ID: 486-839-241
>   
>     GoToMeeting®
>     Online Meetings Made Easy™
>   
>   
>   
> --
> James R. Pence
> jrpence@...  <mailto:jrpence@...>
> Software Engineer
> Communicore C2-12
> (352)294-5154 (ex. 45154)
> Enterprise Software Engineering
> University of Florida
> Academic Health Center IT
> Developer on the VIVO Grant
>   
>   
> ------------------------------------------------------------------------------
> All of the data generated in your IT infrastructure is seriously valuable.
> Why? It contains a definitive record of application performance, security
> threats, fraudulent activity, and more. Splunk takes this data and makes
> sense of it. IT sense. And common sense.
> http://p.sf.net/sfu/splunk-d2d-c2
> _______________________________________________
> Vivo-dev-all mailing list
> Vivo-dev-all@...  <mailto:Vivo-dev-all@...>
> https://lists.sourceforge.net/lists/listinfo/vivo-dev-all
>   
> ------------------------------------------------------------------------------
> All of the data generated in your IT infrastructure is seriously valuable.
> Why? It contains a definitive record of application performance, security
> threats, fraudulent activity, and more. Splunk takes this data and makes
> sense of it. IT sense. And common sense.
> http://p.sf.net/sfu/splunk-d2d-c2
> _______________________________________________
> Vivo-dev-all mailing list
> Vivo-dev-all@...  <mailto:Vivo-dev-all@...>
> https://lists.sourceforge.net/lists/listinfo/vivo-dev-all
>

Nicholas,
As far as I know I do not have SVN commit rights.  Your plan sounds good to me. I have only one or two small changes to make to Eliza's version.
Brian

From: Nicholas Skaggs [mailto:nskaggs@...]
Sent: Thursday, July 07, 2011 11:10 AM
To: vivo-dev-all@...
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Brian and Eliza,
    Let's get a merged copy of your code into the harvester repo. Brian, it seems your new jar is missing some of Eliza's code -- specifically the CSV Map:

The method processCsvMap(String, String, String) is undefined for the type D2rProcessor    D2RMapFetch.java    /Harvester-hg/src/main/java/org/vivoweb/harvester/fetch    line 113    Java Problem

Eliza, I know you have commit rights to SVN -- Brian do you? Eliza can you check in your source for D2RMap, and then have Brian add his changes in as another commit, etc. That would be really awesome. I'm sure others who are using it would be excited to be able to commit fixes and updates as well.

Right now we're still working on restoring the SVN from the mercurial commits we did. As soon as that's done, we can get your commits in.

Thanks!

Nicholas


On 7/1/11 2:43 PM, Eliza Chan wrote:
You are right, we should have a repository for the d2rmap source code. Since now you have a modified copy, and I also have one. It's better to merge them.


From: "Keese, Brian W" <bkeese@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Fri, 1 Jul 2011 14:37:11 -0400
To: VIVO Development List <vivo-dev-all@...>>
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Sure. Let me know how I can help with the docs.

For the bugs, one of them was fixed before (the oracle semi-colon thing, Map.java around line 94).

The other bug has to do with null values for instance identifiers. In Map.java around line 126, a list of new instances is being compiled. Unique instances are defined by values in a discreet set of fields. If one those values happens to be null, this line would give a null pointer exception:

if (!currentTuple.get(fieldName).equals(lastTuple.get(fieldName)))

So, I changed the logic to this:

if (!(currentTuple.get(fieldName)==null && lastTuple.get(fieldName)==null) &&
                      (currentTuple.get(fieldName)==null || !currentTuple.get(fieldName).equals(lastTuple.get(fieldName))))

I also added some logging info to help with diagnosing problems with the harvest. And I removed some logging info which seemed to me to be not useful.

We don't have an svn repository for the d2rmap source code do we? Maybe we should set that up.

Brian

Brian Keese
Indiana University Digital Library Program
Herman B Wells Library W501
812-856-3865

From: Eliza Chan [mailto:elc2013@...]
Sent: Friday, July 01, 2011 2:22 PM
To: VIVO Development List
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Hi Brian,

Thanks for making the changes.
It'd be nice to have d2rmap harvests (db and csv) being supported in future releases, since there are people who use the tools on a regular basis.
I have been trying to update the documentation on d2rmap but it has not been able to keep up with so many changes with Harvester. :-(
Would you be able to document some of it? Also, would you be able to list the bugs that you fixed?

Thanks,
Eliza

From: "Keese, Brian W" <bkeese@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Fri, 1 Jul 2011 14:05:23 -0400
To: VIVO Development List <vivo-dev-all@...>>
Subject: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Nick, Thanks for the info on Oracle. Makes sense.

Here's another thing. I have made a couple of changes to the d2rmap processor that is included with the harvester. I have uploaded the new jar file to https://www.slashtmp.iu.edu/files/download?FILE=bkeese%2F94328zp0Qy2 (the password is 'd2r4vivo').
It fixes a couple of bugs, and adds some status info to the logs for d2rmap fetch.

I noticed that there is no example script for d2rmap in the latest release. I hope support for d2r harvests will be continued.

Brian

Brian Keese
Indiana University Digital Library Program
Herman B Wells Library W501
812-856-3865

From: Nicholas Skaggs [mailto:nskaggs@...]
Sent: Friday, July 01, 2011 10:51 AM
To: vivo-dev-all@...>
Subject: Re: [Vivo-dev-all] Vivo development call / meeting reminder for today.

Great catch Brian. The trouble with oracle is that it requires a specific jdbc connector for each release -- including minor point release versions.

http://www.oracle.com/technetwork/database/features/jdbc/index-091264.html

We can't package every version of the connector with the harvester unfortunately. I have heard of a so called "universal" connector for oracle that connects to the databases through lowest common denominator SQL. I've never been able to find anything additional about this, nor a download for it -- so I can't verify it exists or not.  In addition, the connectors are released by Oracle under the Oracle Technology Network Development and Distribution License. This prevents us from distributing these files by packaging them directly without complying with the license terms. Specifically, the redistribution clause of the license causes trouble:

Program Distribution
We grant you a nonexclusive, nontransferable right to copy and distribute the programs to your end users provided that you do not charge your end users for use of the programs and provided your end users may only use the programs to run your applications for their business operations. Prior to distributing the programs you shall require your end users to execute an agreement binding them to terms consistent with those contained in this section and the sections of this agreement entitled "License Rights," "Ownership and Restrictions," "Export," "Disclaimer of Warranties and Exclusive Remedies," "No Technical Support," "End of Agreement," "Relationship Between the Parties," and "Open Source." You must also include a provision stating that your end users shall have no right to distribute the programs, and a provision specifying us as a third party beneficiary of the agreement. You are responsible for obtaining these agreements with your end users.

You can read the entire license here:

http://www.oracle.com/technetwork/licenses/distribution-license-152002.html

If you have an idea about how we can distribute something to help the harvester work out of the box with oracle, please let us know. Thanks,

Nicholas

On 7/1/11 10:39 AM, Keese, Brian W wrote:

The attachment didn't work. The driver can be downloaded from Oracle here:



http://www.oracle.com/technetwork/database/enterprise-edition/jdbc-10201-088211.html

Brian

________________________________________

From: Keese, Brian W [bkeese@...>]

Sent: Friday, July 01, 2011 10:36 AM

To: jrpence@...>; VIVO Development List

Subject: Re: [Vivo-dev-all] Vivo development call / meeting reminder for today.



Hey Harvester Guys,

I noticed that the oracle jdbc driver was never added (or was added, then removed) to the harvester bin/dependencies folder for the latest release. Could you add that in please?

It is ojdbc14.jar (attached to this email).



Thanks,

Brian





________________________________________

From: James Pence [jrpence@...>]

Sent: Thursday, June 30, 2011 1:23 PM

To: vivo-dev-all@...>

Subject: [Vivo-dev-all] Vivo development call / meeting reminder for today.



Hello,



    Today the vivo developers, and maybe you, are having a goto-meeting at 3:00pm EST (UTC-4 hours)



    We have the beginnings of an agenda at<http://sourceforge.net/apps/mediawiki/vivo/index.php?title=2011-06-30_Development_Leads_Call><http://sourceforge.net/apps/mediawiki/vivo/index.php?title=2011-06-30_Development_Leads_Call>



If you have any agenda items to add to the meeting please let me know (jrpence@...>   ).



   Since we only have 25 connections for the goto meeting please consolidate where possible.



   1. Please join my meeting, Thursday, June 30 at 3:00 PM Eastern DaylightTime.

   https://www1.gotomeeting.com/join/486839241



   2. Use your microphone and speakers (VoIP) - a headset is recommended.



   Or, call in using your telephone.

   Dial +1 (215) 383-1013



   Access Code: 486-839-241

   Audio PIN: Shown after joining the meeting

   Meeting ID: 486-839-241



   GoToMeeting(r)

   Online Meetings Made Easy(tm)







--

James R. Pence

jrpence@...>

Software Engineer

Communicore C2-12

(352)294-5154 (ex. 45154)

Enterprise Software Engineering

University of Florida

Academic Health Center IT

Developer on the VIVO Grant





------------------------------------------------------------------------------

All of the data generated in your IT infrastructure is seriously valuable.

Why? It contains a definitive record of application performance, security

threats, fraudulent activity, and more. Splunk takes this data and makes

sense of it. IT sense. And common sense.

http://p.sf.net/sfu/splunk-d2d-c2

_______________________________________________

Vivo-dev-all mailing list

Vivo-dev-all@...>

https://lists.sourceforge.net/lists/listinfo/vivo-dev-all



------------------------------------------------------------------------------

All of the data generated in your IT infrastructure is seriously valuable.

Why? It contains a definitive record of application performance, security

threats, fraudulent activity, and more. Splunk takes this data and makes

sense of it. IT sense. And common sense.

http://p.sf.net/sfu/splunk-d2d-c2

_______________________________________________

Vivo-dev-all mailing list

Vivo-dev-all@...>

https://lists.sourceforge.net/lists/listinfo/vivo-dev-all

Hi Nicholas and Brian,

I can definitely commit D2RMap to sourceforge.
Just a policy question before doing that.
As you are probably aware, D2RMap is a sourceforge project: https://sourceforge.net/projects/d2r-map/
Do we need to do anything extra (whatever that is) before we post their modified copy under our project?

Thanks,
Eliza

From: Nicholas Skaggs <nskaggs@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Thu, 7 Jul 2011 11:10:13 -0400
To: "vivo-dev-all@...>" <vivo-dev-all@...>>
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Brian and Eliza,
    Let's get a merged copy of your code into the harvester repo. Brian, it seems your new jar is missing some of Eliza's code -- specifically the CSV Map:

The method processCsvMap(String, String, String) is undefined for the type D2rProcessor    D2RMapFetch.java    /Harvester-hg/src/main/java/org/vivoweb/harvester/fetch    line 113    Java Problem

Eliza, I know you have commit rights to SVN -- Brian do you? Eliza can you check in your source for D2RMap, and then have Brian add his changes in as another commit, etc. That would be really awesome. I'm sure others who are using it would be excited to be able to commit fixes and updates as well.

Right now we're still working on restoring the SVN from the mercurial commits we did. As soon as that's done, we can get your commits in.

Thanks!

Nicholas


On 7/1/11 2:43 PM, Eliza Chan wrote:
You are right, we should have a repository for the d2rmap source code. Since now you have a modified copy, and I also have one. It's better to merge them.


From: "Keese, Brian W" <bkeese@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Fri, 1 Jul 2011 14:37:11 -0400
To: VIVO Development List <vivo-dev-all@...>>
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Sure. Let me know how I can help with the docs.

For the bugs, one of them was fixed before (the oracle semi-colon thing, Map.java around line 94).

The other bug has to do with null values for instance identifiers. In Map.java around line 126, a list of new instances is being compiled. Unique instances are defined by values in a discreet set of fields. If one those values happens to be null, this line would give a null pointer exception:

if (!currentTuple.get(fieldName).equals(lastTuple.get(fieldName)))

So, I changed the logic to this:

if (!(currentTuple.get(fieldName)==null && lastTuple.get(fieldName)==null) &&
                      (currentTuple.get(fieldName)==null || !currentTuple.get(fieldName).equals(lastTuple.get(fieldName))))

I also added some logging info to help with diagnosing problems with the harvest. And I removed some logging info which seemed to me to be not useful.

We don’t have an svn repository for the d2rmap source code do we? Maybe we should set that up.

Brian

Brian Keese
Indiana University Digital Library Program
Herman B Wells Library W501
812-856-3865

From: Eliza Chan [mailto:elc2013@...]
Sent: Friday, July 01, 2011 2:22 PM
To: VIVO Development List
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Hi Brian,

Thanks for making the changes.
It'd be nice to have d2rmap harvests (db and csv) being supported in future releases, since there are people who use the tools on a regular basis.
I have been trying to update the documentation on d2rmap but it has not been able to keep up with so many changes with Harvester. :-(
Would you be able to document some of it? Also, would you be able to list the bugs that you fixed?

Thanks,
Eliza

From: "Keese, Brian W" <bkeese@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Fri, 1 Jul 2011 14:05:23 -0400
To: VIVO Development List <vivo-dev-all@...>>
Subject: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Nick, Thanks for the info on Oracle. Makes sense.

Here’s another thing. I have made a couple of changes to the d2rmap processor that is included with the harvester. I have uploaded the new jar file to https://www.slashtmp.iu.edu/files/download?FILE=bkeese%2F94328zp0Qy2 (the password is ‘d2r4vivo’).
It fixes a couple of bugs, and adds some status info to the logs for d2rmap fetch.

I noticed that there is no example script for d2rmap in the latest release. I hope support for d2r harvests will be continued.

Brian

Brian Keese
Indiana University Digital Library Program
Herman B Wells Library W501
812-856-3865

From: Nicholas Skaggs [mailto:nskaggs@...]
Sent: Friday, July 01, 2011 10:51 AM
To: vivo-dev-all@...>
Subject: Re: [Vivo-dev-all] Vivo development call / meeting reminder for today.

Great catch Brian. The trouble with oracle is that it requires a specific jdbc connector for each release -- including minor point release versions.

http://www.oracle.com/technetwork/database/features/jdbc/index-091264.html

We can't package every version of the connector with the harvester unfortunately. I have heard of a so called "universal" connector for oracle that connects to the databases through lowest common denominator SQL. I've never been able to find anything additional about this, nor a download for it -- so I can't verify it exists or not.  In addition, the connectors are released by Oracle under the Oracle Technology Network Development and Distribution License. This prevents us from distributing these files by packaging them directly without complying with the license terms. Specifically, the redistribution clause of the license causes trouble:

Program Distribution
We grant you a nonexclusive, nontransferable right to copy and distribute the programs to your end users provided that you do not charge your end users for use of the programs and provided your end users may only use the programs to run your applications for their business operations. Prior to distributing the programs you shall require your end users to execute an agreement binding them to terms consistent with those contained in this section and the sections of this agreement entitled "License Rights," "Ownership and Restrictions," "Export," "Disclaimer of Warranties and Exclusive Remedies," "No Technical Support," "End of Agreement," "Relationship Between the Parties," and "Open Source." You must also include a provision stating that your end users shall have no right to distribute the programs, and a provision specifying us as a third party beneficiary of the agreement. You are responsible for obtaining these agreements with your end users.

You can read the entire license here:

http://www.oracle.com/technetwork/licenses/distribution-license-152002.html

If you have an idea about how we can distribute something to help the harvester work out of the box with oracle, please let us know. Thanks,

Nicholas

On 7/1/11 10:39 AM, Keese, Brian W wrote:

The attachment didn't work. The driver can be downloaded from Oracle here:



http://www.oracle.com/technetwork/database/enterprise-edition/jdbc-10201-088211.html

Brian

________________________________________

From: Keese, Brian W [bkeese@...>]

Sent: Friday, July 01, 2011 10:36 AM

To: jrpence@...>; VIVO Development List

Subject: Re: [Vivo-dev-all] Vivo development call / meeting reminder for today.



Hey Harvester Guys,

I noticed that the oracle jdbc driver was never added (or was added, then removed) to the harvester bin/dependencies folder for the latest release. Could you add that in please?

It is ojdbc14.jar (attached to this email).



Thanks,

Brian





________________________________________

From: James Pence [jrpence@...>]

Sent: Thursday, June 30, 2011 1:23 PM

To: vivo-dev-all@...>

Subject: [Vivo-dev-all] Vivo development call / meeting reminder for today.



Hello,



    Today the vivo developers, and maybe you, are having a goto-meeting at 3:00pm EST (UTC-4 hours)



    We have the beginnings of an agenda at<http://sourceforge.net/apps/mediawiki/vivo/index.php?title=2011-06-30_Development_Leads_Call><http://sourceforge.net/apps/mediawiki/vivo/index.php?title=2011-06-30_Development_Leads_Call>



If you have any agenda items to add to the meeting please let me know (jrpence@...>   ).



   Since we only have 25 connections for the goto meeting please consolidate where possible.



   1. Please join my meeting, Thursday, June 30 at 3:00 PM Eastern DaylightTime.

   https://www1.gotomeeting.com/join/486839241



   2. Use your microphone and speakers (VoIP) - a headset is recommended.



   Or, call in using your telephone.

   Dial +1 (215) 383-1013



   Access Code: 486-839-241

   Audio PIN: Shown after joining the meeting

   Meeting ID: 486-839-241



   GoToMeeting®

   Online Meetings Made Easy™







--

James R. Pence

jrpence@...>

Software Engineer

Communicore C2-12

(352)294-5154 (ex. 45154)

Enterprise Software Engineering

University of Florida

Academic Health Center IT

Developer on the VIVO Grant





------------------------------------------------------------------------------

All of the data generated in your IT infrastructure is seriously valuable.

Why? It contains a definitive record of application performance, security

threats, fraudulent activity, and more. Splunk takes this data and makes

sense of it. IT sense. And common sense.

http://p.sf.net/sfu/splunk-d2d-c2

_______________________________________________

Vivo-dev-all mailing list

Vivo-dev-all@...>

https://lists.sourceforge.net/lists/listinfo/vivo-dev-all



------------------------------------------------------------------------------

All of the data generated in your IT infrastructure is seriously valuable.

Why? It contains a definitive record of application performance, security

threats, fraudulent activity, and more. Splunk takes this data and makes

sense of it. IT sense. And common sense.

http://p.sf.net/sfu/splunk-d2d-c2

_______________________________________________

Vivo-dev-all mailing list

Vivo-dev-all@...>

https://lists.sourceforge.net/lists/listinfo/vivo-dev-all

Eliza;

That's a very interesting question, and it opens an big can of worms.

I am not a lawyer, but it appears that D2RMap is covered under the GNU Lesser General Public License (LGPL), which is a modification of the GNU General Public License, which appears to cover this scenario in section 6, paragraph d.

If I understand this correctly, if you offer the object code (i.e., JAR files or CLASS files) for public download as part of your product, you are required to offer the source code (i.e. JAVA files) for public download on the same server, or an equivalent server operated by you or by a third party.

So if our software is available for download from SourceForge, and the D2RMap source code is available from SourceForge, have we satisfied the terms of the license?

Have I mentioned that I have no legal training or expertise?

I wonder whether the VIVO project is in violation of the terms of many additional open source licenses. For exmaple, some licenses require that we cite them in all of our end-user documentation. Do we do that?

But I'm just taking wild guesses, here. Can anybody shed more light on this?

Jim


From: Eliza Chan [mailto:elc2013@...]
Sent: Thursday, July 07, 2011 11:59 AM
To: VIVO Development List
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Hi Nicholas and Brian,

I can definitely commit D2RMap to sourceforge.
Just a policy question before doing that.
As you are probably aware, D2RMap is a sourceforge project: https://sourceforge.net/projects/d2r-map/
Do we need to do anything extra (whatever that is) before we post their modified copy under our project?

Thanks,
Eliza

From: Nicholas Skaggs <nskaggs@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Thu, 7 Jul 2011 11:10:13 -0400
To: "vivo-dev-all@...>" <vivo-dev-all@...>>
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Brian and Eliza,
    Let's get a merged copy of your code into the harvester repo. Brian, it seems your new jar is missing some of Eliza's code -- specifically the CSV Map:

The method processCsvMap(String, String, String) is undefined for the type D2rProcessor    D2RMapFetch.java    /Harvester-hg/src/main/java/org/vivoweb/harvester/fetch    line 113    Java Problem

Eliza, I know you have commit rights to SVN -- Brian do you? Eliza can you check in your source for D2RMap, and then have Brian add his changes in as another commit, etc. That would be really awesome. I'm sure others who are using it would be excited to be able to commit fixes and updates as well.

Right now we're still working on restoring the SVN from the mercurial commits we did. As soon as that's done, we can get your commits in.

Thanks!

Nicholas


On 7/1/11 2:43 PM, Eliza Chan wrote:
You are right, we should have a repository for the d2rmap source code. Since now you have a modified copy, and I also have one. It's better to merge them.


From: "Keese, Brian W" <bkeese@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Fri, 1 Jul 2011 14:37:11 -0400
To: VIVO Development List <vivo-dev-all@...>>
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Sure. Let me know how I can help with the docs.

For the bugs, one of them was fixed before (the oracle semi-colon thing, Map.java around line 94).

The other bug has to do with null values for instance identifiers. In Map.java around line 126, a list of new instances is being compiled. Unique instances are defined by values in a discreet set of fields. If one those values happens to be null, this line would give a null pointer exception:

if (!currentTuple.get(fieldName).equals(lastTuple.get(fieldName)))

So, I changed the logic to this:

if (!(currentTuple.get(fieldName)==null && lastTuple.get(fieldName)==null) &&
                      (currentTuple.get(fieldName)==null || !currentTuple.get(fieldName).equals(lastTuple.get(fieldName))))

I also added some logging info to help with diagnosing problems with the harvest. And I removed some logging info which seemed to me to be not useful.

We don't have an svn repository for the d2rmap source code do we? Maybe we should set that up.

Brian

Brian Keese
Indiana University Digital Library Program
Herman B Wells Library W501
812-856-3865

From: Eliza Chan [mailto:elc2013@...]
Sent: Friday, July 01, 2011 2:22 PM
To: VIVO Development List
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Hi Brian,

Thanks for making the changes.
It'd be nice to have d2rmap harvests (db and csv) being supported in future releases, since there are people who use the tools on a regular basis.
I have been trying to update the documentation on d2rmap but it has not been able to keep up with so many changes with Harvester. :-(
Would you be able to document some of it? Also, would you be able to list the bugs that you fixed?

Thanks,
Eliza

From: "Keese, Brian W" <bkeese@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Fri, 1 Jul 2011 14:05:23 -0400
To: VIVO Development List <vivo-dev-all@...>>
Subject: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Nick, Thanks for the info on Oracle. Makes sense.

Here's another thing. I have made a couple of changes to the d2rmap processor that is included with the harvester. I have uploaded the new jar file to https://www.slashtmp.iu.edu/files/download?FILE=bkeese%2F94328zp0Qy2 (the password is 'd2r4vivo').
It fixes a couple of bugs, and adds some status info to the logs for d2rmap fetch.

I noticed that there is no example script for d2rmap in the latest release. I hope support for d2r harvests will be continued.

Brian

Brian Keese
Indiana University Digital Library Program
Herman B Wells Library W501
812-856-3865

From: Nicholas Skaggs [mailto:nskaggs@...]
Sent: Friday, July 01, 2011 10:51 AM
To: vivo-dev-all@...>
Subject: Re: [Vivo-dev-all] Vivo development call / meeting reminder for today.

Great catch Brian. The trouble with oracle is that it requires a specific jdbc connector for each release -- including minor point release versions.

http://www.oracle.com/technetwork/database/features/jdbc/index-091264.html

We can't package every version of the connector with the harvester unfortunately. I have heard of a so called "universal" connector for oracle that connects to the databases through lowest common denominator SQL. I've never been able to find anything additional about this, nor a download for it -- so I can't verify it exists or not.  In addition, the connectors are released by Oracle under the Oracle Technology Network Development and Distribution License. This prevents us from distributing these files by packaging them directly without complying with the license terms. Specifically, the redistribution clause of the license causes trouble:

Program Distribution
We grant you a nonexclusive, nontransferable right to copy and distribute the programs to your end users provided that you do not charge your end users for use of the programs and provided your end users may only use the programs to run your applications for their business operations. Prior to distributing the programs you shall require your end users to execute an agreement binding them to terms consistent with those contained in this section and the sections of this agreement entitled "License Rights," "Ownership and Restrictions," "Export," "Disclaimer of Warranties and Exclusive Remedies," "No Technical Support," "End of Agreement," "Relationship Between the Parties," and "Open Source." You must also include a provision stating that your end users shall have no right to distribute the programs, and a provision specifying us as a third party beneficiary of the agreement. You are responsible for obtaining these agreements with your end users.

You can read the entire license here:

http://www.oracle.com/technetwork/licenses/distribution-license-152002.html

If you have an idea about how we can distribute something to help the harvester work out of the box with oracle, please let us know. Thanks,

Nicholas

On 7/1/11 10:39 AM, Keese, Brian W wrote:

The attachment didn't work. The driver can be downloaded from Oracle here:



http://www.oracle.com/technetwork/database/enterprise-edition/jdbc-10201-088211.html

Brian

________________________________________

From: Keese, Brian W [bkeese@...>]

Sent: Friday, July 01, 2011 10:36 AM

To: jrpence@...>; VIVO Development List

Subject: Re: [Vivo-dev-all] Vivo development call / meeting reminder for today.



Hey Harvester Guys,

I noticed that the oracle jdbc driver was never added (or was added, then removed) to the harvester bin/dependencies folder for the latest release. Could you add that in please?

It is ojdbc14.jar (attached to this email).



Thanks,

Brian





________________________________________

From: James Pence [jrpence@...>]

Sent: Thursday, June 30, 2011 1:23 PM

To: vivo-dev-all@...>

Subject: [Vivo-dev-all] Vivo development call / meeting reminder for today.



Hello,



    Today the vivo developers, and maybe you, are having a goto-meeting at 3:00pm EST (UTC-4 hours)



    We have the beginnings of an agenda at<http://sourceforge.net/apps/mediawiki/vivo/index.php?title=2011-06-30_Development_Leads_Call><http://sourceforge.net/apps/mediawiki/vivo/index.php?title=2011-06-30_Development_Leads_Call>



If you have any agenda items to add to the meeting please let me know (jrpence@...>   ).



   Since we only have 25 connections for the goto meeting please consolidate where possible.



   1. Please join my meeting, Thursday, June 30 at 3:00 PM Eastern DaylightTime.

   https://www1.gotomeeting.com/join/486839241



   2. Use your microphone and speakers (VoIP) - a headset is recommended.



   Or, call in using your telephone.

   Dial +1 (215) 383-1013



   Access Code: 486-839-241

   Audio PIN: Shown after joining the meeting

   Meeting ID: 486-839-241



   GoToMeeting(r)

   Online Meetings Made Easy(tm)







--

James R. Pence

jrpence@...>

Software Engineer

Communicore C2-12

(352)294-5154 (ex. 45154)

Enterprise Software Engineering

University of Florida

Academic Health Center IT

Developer on the VIVO Grant





------------------------------------------------------------------------------

All of the data generated in your IT infrastructure is seriously valuable.

Why? It contains a definitive record of application performance, security

threats, fraudulent activity, and more. Splunk takes this data and makes

sense of it. IT sense. And common sense.

http://p.sf.net/sfu/splunk-d2d-c2

_______________________________________________

Vivo-dev-all mailing list

Vivo-dev-all@...>

https://lists.sourceforge.net/lists/listinfo/vivo-dev-all



------------------------------------------------------------------------------

All of the data generated in your IT infrastructure is seriously valuable.

Why? It contains a definitive record of application performance, security

threats, fraudulent activity, and more. Splunk takes this data and makes

sense of it. IT sense. And common sense.

http://p.sf.net/sfu/splunk-d2d-c2

_______________________________________________

Vivo-dev-all mailing list

Vivo-dev-all@...>

https://lists.sourceforge.net/lists/listinfo/vivo-dev-all

Hi Jim,

Thanks for looking up.
In this case, committing the D2RMap source code to our project is not only Ok, it's required.

Eliza

From: Jim Blake <jeb228@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Thu, 7 Jul 2011 13:41:20 -0400
To: VIVO Development List <vivo-dev-all@...>>
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Eliza;

That’s a very interesting question, and it opens an big can of worms.

I am not a lawyer, but it appears that D2RMap is covered under the GNU Lesser General Public License (LGPL), which is a modification of the GNU General Public License, which appears to cover this scenario in section 6, paragraph d.

If I understand this correctly, if you offer the object code (i.e., JAR files or CLASS files) for public download as part of your product, you are required to offer the source code (i.e. JAVA files) for public download on the same server, or an equivalent server operated by you or by a third party.

So if our software is available for download from SourceForge, and the D2RMap source code is available from SourceForge, have we satisfied the terms of the license?

Have I mentioned that I have no legal training or expertise?

I wonder whether the VIVO project is in violation of the terms of many additional open source licenses. For exmaple, some licenses require that we cite them in all of our end-user documentation. Do we do that?

But I’m just taking wild guesses, here. Can anybody shed more light on this?

Jim


From: Eliza Chan [mailto:elc2013@...]
Sent: Thursday, July 07, 2011 11:59 AM
To: VIVO Development List
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Hi Nicholas and Brian,

I can definitely commit D2RMap to sourceforge.
Just a policy question before doing that.
As you are probably aware, D2RMap is a sourceforge project: https://sourceforge.net/projects/d2r-map/
Do we need to do anything extra (whatever that is) before we post their modified copy under our project?

Thanks,
Eliza

From: Nicholas Skaggs <nskaggs@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Thu, 7 Jul 2011 11:10:13 -0400
To: "vivo-dev-all@...>" <vivo-dev-all@...>>
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Brian and Eliza,
    Let's get a merged copy of your code into the harvester repo. Brian, it seems your new jar is missing some of Eliza's code -- specifically the CSV Map:

The method processCsvMap(String, String, String) is undefined for the type D2rProcessor    D2RMapFetch.java    /Harvester-hg/src/main/java/org/vivoweb/harvester/fetch    line 113    Java Problem

Eliza, I know you have commit rights to SVN -- Brian do you? Eliza can you check in your source for D2RMap, and then have Brian add his changes in as another commit, etc. That would be really awesome. I'm sure others who are using it would be excited to be able to commit fixes and updates as well.

Right now we're still working on restoring the SVN from the mercurial commits we did. As soon as that's done, we can get your commits in.

Thanks!

Nicholas


On 7/1/11 2:43 PM, Eliza Chan wrote:
You are right, we should have a repository for the d2rmap source code. Since now you have a modified copy, and I also have one. It's better to merge them.


From: "Keese, Brian W" <bkeese@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Fri, 1 Jul 2011 14:37:11 -0400
To: VIVO Development List <vivo-dev-all@...>>
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Sure. Let me know how I can help with the docs.

For the bugs, one of them was fixed before (the oracle semi-colon thing, Map.java around line 94).

The other bug has to do with null values for instance identifiers. In Map.java around line 126, a list of new instances is being compiled. Unique instances are defined by values in a discreet set of fields. If one those values happens to be null, this line would give a null pointer exception:

if (!currentTuple.get(fieldName).equals(lastTuple.get(fieldName)))

So, I changed the logic to this:

if (!(currentTuple.get(fieldName)==null && lastTuple.get(fieldName)==null) &&
                      (currentTuple.get(fieldName)==null || !currentTuple.get(fieldName).equals(lastTuple.get(fieldName))))

I also added some logging info to help with diagnosing problems with the harvest. And I removed some logging info which seemed to me to be not useful.

We don’t have an svn repository for the d2rmap source code do we? Maybe we should set that up.

Brian

Brian Keese
Indiana University Digital Library Program
Herman B Wells Library W501
812-856-3865

From: Eliza Chan [mailto:elc2013@...]
Sent: Friday, July 01, 2011 2:22 PM
To: VIVO Development List
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Hi Brian,

Thanks for making the changes.
It'd be nice to have d2rmap harvests (db and csv) being supported in future releases, since there are people who use the tools on a regular basis.
I have been trying to update the documentation on d2rmap but it has not been able to keep up with so many changes with Harvester. :-(
Would you be able to document some of it? Also, would you be able to list the bugs that you fixed?

Thanks,
Eliza

From: "Keese, Brian W" <bkeese@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Fri, 1 Jul 2011 14:05:23 -0400
To: VIVO Development List <vivo-dev-all@...>>
Subject: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Nick, Thanks for the info on Oracle. Makes sense.

Here’s another thing. I have made a couple of changes to the d2rmap processor that is included with the harvester. I have uploaded the new jar file to https://www.slashtmp.iu.edu/files/download?FILE=bkeese%2F94328zp0Qy2 (the password is ‘d2r4vivo’).
It fixes a couple of bugs, and adds some status info to the logs for d2rmap fetch.

I noticed that there is no example script for d2rmap in the latest release. I hope support for d2r harvests will be continued.

Brian

Brian Keese
Indiana University Digital Library Program
Herman B Wells Library W501
812-856-3865

From: Nicholas Skaggs [mailto:nskaggs@...]
Sent: Friday, July 01, 2011 10:51 AM
To: vivo-dev-all@...>
Subject: Re: [Vivo-dev-all] Vivo development call / meeting reminder for today.

Great catch Brian. The trouble with oracle is that it requires a specific jdbc connector for each release -- including minor point release versions.

http://www.oracle.com/technetwork/database/features/jdbc/index-091264.html

We can't package every version of the connector with the harvester unfortunately. I have heard of a so called "universal" connector for oracle that connects to the databases through lowest common denominator SQL. I've never been able to find anything additional about this, nor a download for it -- so I can't verify it exists or not.  In addition, the connectors are released by Oracle under the Oracle Technology Network Development and Distribution License. This prevents us from distributing these files by packaging them directly without complying with the license terms. Specifically, the redistribution clause of the license causes trouble:

Program Distribution
We grant you a nonexclusive, nontransferable right to copy and distribute the programs to your end users provided that you do not charge your end users for use of the programs and provided your end users may only use the programs to run your applications for their business operations. Prior to distributing the programs you shall require your end users to execute an agreement binding them to terms consistent with those contained in this section and the sections of this agreement entitled "License Rights," "Ownership and Restrictions," "Export," "Disclaimer of Warranties and Exclusive Remedies," "No Technical Support," "End of Agreement," "Relationship Between the Parties," and "Open Source." You must also include a provision stating that your end users shall have no right to distribute the programs, and a provision specifying us as a third party beneficiary of the agreement. You are responsible for obtaining these agreements with your end users.

You can read the entire license here:

http://www.oracle.com/technetwork/licenses/distribution-license-152002.html

If you have an idea about how we can distribute something to help the harvester work out of the box with oracle, please let us know. Thanks,

Nicholas

On 7/1/11 10:39 AM, Keese, Brian W wrote:

The attachment didn't work. The driver can be downloaded from Oracle here:



http://www.oracle.com/technetwork/database/enterprise-edition/jdbc-10201-088211.html

Brian

________________________________________

From: Keese, Brian W [bkeese@...>]

Sent: Friday, July 01, 2011 10:36 AM

To: jrpence@...>; VIVO Development List

Subject: Re: [Vivo-dev-all] Vivo development call / meeting reminder for today.



Hey Harvester Guys,

I noticed that the oracle jdbc driver was never added (or was added, then removed) to the harvester bin/dependencies folder for the latest release. Could you add that in please?

It is ojdbc14.jar (attached to this email).



Thanks,

Brian





________________________________________

From: James Pence [jrpence@...>]

Sent: Thursday, June 30, 2011 1:23 PM

To: vivo-dev-all@...>

Subject: [Vivo-dev-all] Vivo development call / meeting reminder for today.



Hello,



    Today the vivo developers, and maybe you, are having a goto-meeting at 3:00pm EST (UTC-4 hours)



    We have the beginnings of an agenda at<http://sourceforge.net/apps/mediawiki/vivo/index.php?title=2011-06-30_Development_Leads_Call><http://sourceforge.net/apps/mediawiki/vivo/index.php?title=2011-06-30_Development_Leads_Call>



If you have any agenda items to add to the meeting please let me know (jrpence@...>   ).



   Since we only have 25 connections for the goto meeting please consolidate where possible.



   1. Please join my meeting, Thursday, June 30 at 3:00 PM Eastern DaylightTime.

   https://www1.gotomeeting.com/join/486839241



   2. Use your microphone and speakers (VoIP) - a headset is recommended.



   Or, call in using your telephone.

   Dial +1 (215) 383-1013



   Access Code: 486-839-241

   Audio PIN: Shown after joining the meeting

   Meeting ID: 486-839-241



   GoToMeeting®

   Online Meetings Made Easy™







--

James R. Pence

jrpence@...>

Software Engineer

Communicore C2-12

(352)294-5154 (ex. 45154)

Enterprise Software Engineering

University of Florida

Academic Health Center IT

Developer on the VIVO Grant





------------------------------------------------------------------------------

All of the data generated in your IT infrastructure is seriously valuable.

Why? It contains a definitive record of application performance, security

threats, fraudulent activity, and more. Splunk takes this data and makes

sense of it. IT sense. And common sense.

http://p.sf.net/sfu/splunk-d2d-c2

_______________________________________________

Vivo-dev-all mailing list

Vivo-dev-all@...>

https://lists.sourceforge.net/lists/listinfo/vivo-dev-all



------------------------------------------------------------------------------

All of the data generated in your IT infrastructure is seriously valuable.

Why? It contains a definitive record of application performance, security

threats, fraudulent activity, and more. Splunk takes this data and makes

sense of it. IT sense. And common sense.

http://p.sf.net/sfu/splunk-d2d-c2

_______________________________________________

Vivo-dev-all mailing list

Vivo-dev-all@...>

https://lists.sourceforge.net/lists/listinfo/vivo-dev-all

Jim, yes we must consider these. For distributing a binary, we can 
simply disclose it's license as GPL, and do nothing more.  LGPL and GPL 
allow for different things in terms of linking and non-open source 
software, but we don't need to get into that here. BSD is "compatible" 
with the GPL/LGPL. In this case, Eliza/Brian has modified GPL/LGPL 
source (they appear to have it dual licensed). We can distribute this of 
course, but we do need to make those modifications available in source 
form upon request. This can be done thru writing and physical media (we 
can even charge the cost of distribution), etc, but the simplest and 
most common is to host the source in a freely downloadable place.

So, I didn't realize D2RMap was an existing SF project. We essentially 
have 2 options. We can fork there project and host it ourselves with our 
modifications, or we can contribute them back upstream and see if we can 
get them to adopt them. I would suggest starting with the latter. 
Depending on the outcome, in the meantime, we will be maintaining a 
"fork" of the project with our modifications as part of the harvester 
since we want to utilize it. Now, the best practice for Eliza and Brian 
would be to make there modifications back to the D2RMap project as 
GPL/LGPL (there current license format). If they licensed it as BSD or 
something else, the project would have part of the code under a 
different license which is no good, and would make it harder for them to 
accept.

The ramifications then for the harvester is we must ship the GPL license 
with our project and state that D2RMap is under that license. Not a big 
deal, it's an external library; but we should make note of it.

In summary we should:

License modifications as GPL/LGPL
Push patchsets to D2RMap community
Host our modified version of the code on SF (seperate sub-project?)
Use the modified binary in the harvester, noting the GPL license

Nicholas

On 7/7/11 1:50 PM, Eliza Chan wrote:
> Hi Jim,
>
> Thanks for looking up.
> In this case, committing the D2RMap source code to our project is not 
> only Ok, it's required.
>
> Eliza
>
> From: Jim Blake <jeb228@... <mailto:jeb228@...>>
> Reply-To: VIVO Development List <vivo-dev-all@... 
> <mailto:vivo-dev-all@...>>
> Date: Thu, 7 Jul 2011 13:41:20 -0400
> To: VIVO Development List <vivo-dev-all@... 
> <mailto:vivo-dev-all@...>>
> Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster
>
> Eliza;
>
> That’s a very interesting question, and it opens an big can of worms.
>
> I am not a lawyer, but it appears that D2RMap is covered under the GNU 
> Lesser General Public License (LGPL), which is a modification of the 
> GNU General Public License, which appears to cover this scenario in 
> section 6, paragraph d.
>
> If I understand this correctly, if you offer the object code (i.e., 
> JAR files or CLASS files) for public download as part of your product, 
> you are required to offer the source code (i.e. JAVA files) for public 
> download on the same server, or an equivalent server operated by you 
> or by a third party.
>
> So if our software is available for download from SourceForge, and the 
> D2RMap source code is available from SourceForge, have we satisfied 
> the terms of the license?
>
> Have I mentioned that I have no legal training or expertise?
>
> I wonder whether the VIVO project is in violation of the terms of many 
> additional open source licenses. For exmaple, some licenses require 
> that we cite them in all of our end-user documentation. Do we do that?
>
> But I’m just taking wild guesses, here. Can anybody shed more light on 
> this?
>
> Jim
>
> *From:*Eliza Chan [mailto:elc2013@...]
> *Sent:* Thursday, July 07, 2011 11:59 AM
> *To:* VIVO Development List
> *Subject:* Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster
>
> Hi Nicholas and Brian,
>
> I can definitely commit D2RMap to sourceforge.
>
> Just a policy question before doing that.
>
> As you are probably aware, D2RMap is a sourceforge project: 
> https://sourceforge.net/projects/d2r-map/
>
> Do we need to do anything extra (whatever that is) before we post 
> their modified copy under our project?
>
> Thanks,
>
> Eliza
>
> *From: *Nicholas Skaggs <nskaggs@... <mailto:nskaggs@...>>
> *Reply-To: *VIVO Development List <vivo-dev-all@... 
> <mailto:vivo-dev-all@...>>
> *Date: *Thu, 7 Jul 2011 11:10:13 -0400
> *To: *"vivo-dev-all@... 
> <mailto:vivo-dev-all@...>" 
> <vivo-dev-all@... 
> <mailto:vivo-dev-all@...>>
> *Subject: *Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster
>
> Brian and Eliza,
>     Let's get a merged copy of your code into the harvester repo. 
> Brian, it seems your new jar is missing some of Eliza's code -- 
> specifically the CSV Map:
>
> The method processCsvMap(String, String, String) is undefined for the 
> type D2rProcessor    D2RMapFetch.java    
> /Harvester-hg/src/main/java/org/vivoweb/harvester/fetch    line 113    
> Java Problem
>
> Eliza, I know you have commit rights to SVN -- Brian do you? Eliza can 
> you check in your source for D2RMap, and then have Brian add his 
> changes in as another commit, etc. That would be really awesome. I'm 
> sure others who are using it would be excited to be able to commit 
> fixes and updates as well.
>
> Right now we're still working on restoring the SVN from the mercurial 
> commits we did. As soon as that's done, we can get your commits in.
>
> Thanks!
>
> Nicholas
>
>
> On 7/1/11 2:43 PM, Eliza Chan wrote:
>
> You are right, we should have a repository for the d2rmap source code. 
> Since now you have a modified copy, and I also have one. It's better 
> to merge them.
>
> *From: *"Keese, Brian W" <bkeese@... <mailto:bkeese@...>>
> *Reply-To: *VIVO Development List <vivo-dev-all@... 
> <mailto:vivo-dev-all@...>>
> *Date: *Fri, 1 Jul 2011 14:37:11 -0400
> *To: *VIVO Development List <vivo-dev-all@... 
> <mailto:vivo-dev-all@...>>
> *Subject: *Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster
>
> Sure. Let me know how I can help with the docs.
>
> For the bugs, one of them was fixed before (the oracle semi-colon 
> thing, Map.java around line 94).
>
> The other bug has to do with null values for instance identifiers. In 
> Map.java around line 126, a list of new instances is being compiled. 
> Unique instances are defined by values in a discreet set of fields. If 
> one those values happens to be null, this line would give a null 
> pointer exception:
>
> if (!currentTuple.get(fieldName).equals(lastTuple.get(fieldName)))
>
> So, I changed the logic to this:
>
> if (!(currentTuple.get(fieldName)==null && 
> lastTuple.get(fieldName)==null) &&
>
>                       (currentTuple.get(fieldName)==null || 
> !currentTuple.get(fieldName).equals(lastTuple.get(fieldName))))
>
> I also added some logging info to help with diagnosing problems with 
> the harvest. And I removed some logging info which seemed to me to be 
> not useful.
>
> We don’t have an svn repository for the d2rmap source code do we? 
> Maybe we should set that up.
>
> Brian
>
> Brian Keese
>
> Indiana University Digital Library Program
>
> Herman B Wells Library W501
>
> 812-856-3865
>
> *From:*Eliza Chan [mailto:elc2013@...]
> *Sent:* Friday, July 01, 2011 2:22 PM
> *To:* VIVO Development List
> *Subject:* Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster
>
> Hi Brian,
>
> Thanks for making the changes.
>
> It'd be nice to have d2rmap harvests (db and csv) being supported in 
> future releases, since there are people who use the tools on a regular 
> basis.
>
> I have been trying to update the documentation on d2rmap but it has 
> not been able to keep up with so many changes with Harvester. :-(
>
> Would you be able to document some of it? Also, would you be able to 
> list the bugs that you fixed?
>
> Thanks,
>
> Eliza
>
> *From: *"Keese, Brian W" <bkeese@... <mailto:bkeese@...>>
> *Reply-To: *VIVO Development List <vivo-dev-all@... 
> <mailto:vivo-dev-all@...>>
> *Date: *Fri, 1 Jul 2011 14:05:23 -0400
> *To: *VIVO Development List <vivo-dev-all@... 
> <mailto:vivo-dev-all@...>>
> *Subject: *[Vivo-dev-all] d2rmap harvesting, was oracle for harverster
>
> Nick, Thanks for the info on Oracle. Makes sense.
>
> Here’s another thing. I have made a couple of changes to the d2rmap 
> processor that is included with the harvester. I have uploaded the new 
> jar file to 
> https://www.slashtmp.iu.edu/files/download?FILE=bkeese%2F94328zp0Qy2 
> (the password is ‘d2r4vivo’).
>
> It fixes a couple of bugs, and adds some status info to the logs for 
> d2rmap fetch.
>
> I noticed that there is no example script for d2rmap in the latest 
> release. I hope support for d2r harvests will be continued.
>
> Brian
>
> Brian Keese
>
> Indiana University Digital Library Program
>
> Herman B Wells Library W501
>
> 812-856-3865
>
> *From:*Nicholas Skaggs [mailto:nskaggs@...]
> *Sent:* Friday, July 01, 2011 10:51 AM
> *To:* vivo-dev-all@... 
> <mailto:vivo-dev-all@...>
> *Subject:* Re: [Vivo-dev-all] Vivo development call / meeting reminder 
> for today.
>
> Great catch Brian. The trouble with oracle is that it requires a 
> specific jdbc connector for each release -- including minor point 
> release versions.
>
> http://www.oracle.com/technetwork/database/features/jdbc/index-091264.html
>
> We can't package every version of the connector with the harvester 
> unfortunately. I have heard of a so called "universal" connector for 
> oracle that connects to the databases through lowest common 
> denominator SQL. I've never been able to find anything additional 
> about this, nor a download for it -- so I can't verify it exists or 
> not.  In addition, the connectors are released by Oracle under the 
> Oracle Technology Network Development and Distribution License. This 
> prevents us from distributing these files by packaging them directly 
> without complying with the license terms. Specifically, the 
> redistribution clause of the license causes trouble:
>
> /Program Distribution
> We grant you a nonexclusive, nontransferable right to copy and 
> distribute the programs to your end users provided that you do not 
> charge your end users for use of the programs and provided your end 
> users may only use the programs to run your applications for their 
> business operations. Prior to distributing the programs you shall 
> require your end users to execute an agreement binding them to terms 
> consistent with those contained in this section and the sections of 
> this agreement entitled "License Rights," "Ownership and 
> Restrictions," "Export," "Disclaimer of Warranties and Exclusive 
> Remedies," "No Technical Support," "End of Agreement," "Relationship 
> Between the Parties," and "Open Source." You must also include a 
> provision stating that your end users shall have no right to 
> distribute the programs, and a provision specifying us as a third 
> party beneficiary of the agreement. You are responsible for obtaining 
> these agreements with your end users./
>
> You can read the entire license here:
>
> http://www.oracle.com/technetwork/licenses/distribution-license-152002.html
>
> If you have an idea about how we can distribute something to help the 
> harvester work out of the box with oracle, please let us know. Thanks,
>
> Nicholas
>
> On 7/1/11 10:39 AM, Keese, Brian W wrote:
>
> The attachment didn't work. The driver can be downloaded from Oracle here:
>   
> http://www.oracle.com/technetwork/database/enterprise-edition/jdbc-10201-088211.html
> Brian
> ________________________________________
> From: Keese, Brian W [bkeese@...  <mailto:bkeese@...>]
> Sent: Friday, July 01, 2011 10:36 AM
> To:jrpence@...  <mailto:jrpence@...>; VIVO Development List
> Subject: Re: [Vivo-dev-all] Vivo development call / meeting reminder for today.
>   
> Hey Harvester Guys,
> I noticed that the oracle jdbc driver was never added (or was added, then removed) to the harvester bin/dependencies folder for the latest release. Could you add that in please?
> It is ojdbc14.jar (attached to this email).
>   
> Thanks,
> Brian
>   
>   
> ________________________________________
> From: James Pence [jrpence@...  <mailto:jrpence@...>]
> Sent: Thursday, June 30, 2011 1:23 PM
> To:vivo-dev-all@...  <mailto:vivo-dev-all@...>
> Subject: [Vivo-dev-all] Vivo development call / meeting reminder for today.
>   
> Hello,
>   
>      Today the vivo developers, and maybe you, are having a goto-meeting at 3:00pm EST (UTC-4 hours)
>   
>      We have the beginnings of an agenda at<http://sourceforge.net/apps/mediawiki/vivo/index.php?title=2011-06-30_Development_Leads_Call>
>   
> If you have any agenda items to add to the meeting please let me know (jrpence@...  <mailto:jrpence@...>    ).
>   
>     Since we only have 25 connections for the goto meeting please consolidate where possible.
>   
>     1. Please join my meeting, Thursday, June 30 at 3:00 PM Eastern DaylightTime.
>     https://www1.gotomeeting.com/join/486839241
>   
>     2. Use your microphone and speakers (VoIP) - a headset is recommended.
>   
>     Or, call in using your telephone.
>     Dial +1 (215) 383-1013
>   
>     Access Code: 486-839-241
>     Audio PIN: Shown after joining the meeting
>     Meeting ID: 486-839-241
>   
>     GoToMeeting®
>     Online Meetings Made Easy™
>   
>   
>   
> --
> James R. Pence
> jrpence@...  <mailto:jrpence@...>
> Software Engineer
> Communicore C2-12
> (352)294-5154 (ex. 45154)
> Enterprise Software Engineering
> University of Florida
> Academic Health Center IT
> Developer on the VIVO Grant
>   
>   
> ------------------------------------------------------------------------------
> All of the data generated in your IT infrastructure is seriously valuable.
> Why? It contains a definitive record of application performance, security
> threats, fraudulent activity, and more. Splunk takes this data and makes
> sense of it. IT sense. And common sense.
> http://p.sf.net/sfu/splunk-d2d-c2
> _______________________________________________
> Vivo-dev-all mailing list
> Vivo-dev-all@...  <mailto:Vivo-dev-all@...>
> https://lists.sourceforge.net/lists/listinfo/vivo-dev-all
>   
> ------------------------------------------------------------------------------
> All of the data generated in your IT infrastructure is seriously valuable.
> Why? It contains a definitive record of application performance, security
> threats, fraudulent activity, and more. Splunk takes this data and makes
> sense of it. IT sense. And common sense.
> http://p.sf.net/sfu/splunk-d2d-c2
> _______________________________________________
> Vivo-dev-all mailing list
> Vivo-dev-all@...  <mailto:Vivo-dev-all@...>
> https://lists.sourceforge.net/lists/listinfo/vivo-dev-all
>

Given the shortcomings of email, I don't want to be mis-interpreted on 
my reply.

I would like to preface this email with "In my opinion", and "I think we 
should' :-)

Thanks,

Nicholas

On 7/7/11 1:58 PM, Nicholas Skaggs wrote:
> Jim, yes we must consider these. For distributing a binary, we can 
> simply disclose it's license as GPL, and do nothing more.  LGPL and 
> GPL allow for different things in terms of linking and non-open source 
> software, but we don't need to get into that here. BSD is "compatible" 
> with the GPL/LGPL. In this case, Eliza/Brian has modified GPL/LGPL 
> source (they appear to have it dual licensed). We can distribute this 
> of course, but we do need to make those modifications available in 
> source form upon request. This can be done thru writing and physical 
> media (we can even charge the cost of distribution), etc, but the 
> simplest and most common is to host the source in a freely 
> downloadable place.
>
> So, I didn't realize D2RMap was an existing SF project. We essentially 
> have 2 options. We can fork there project and host it ourselves with 
> our modifications, or we can contribute them back upstream and see if 
> we can get them to adopt them. I would suggest starting with the 
> latter. Depending on the outcome, in the meantime, we will be 
> maintaining a "fork" of the project with our modifications as part of 
> the harvester since we want to utilize it. Now, the best practice for 
> Eliza and Brian would be to make there modifications back to the 
> D2RMap project as GPL/LGPL (there current license format). If they 
> licensed it as BSD or something else, the project would have part of 
> the code under a different license which is no good, and would make it 
> harder for them to accept.
>
> The ramifications then for the harvester is we must ship the GPL 
> license with our project and state that D2RMap is under that license. 
> Not a big deal, it's an external library; but we should make note of it.
>
> In summary we should:
>
> License modifications as GPL/LGPL
> Push patchsets to D2RMap community
> Host our modified version of the code on SF (seperate sub-project?)
> Use the modified binary in the harvester, noting the GPL license
>
> Nicholas
>
> On 7/7/11 1:50 PM, Eliza Chan wrote:
>> Hi Jim,
>>
>> Thanks for looking up.
>> In this case, committing the D2RMap source code to our project is not 
>> only Ok, it's required.
>>
>> Eliza
>>
>> From: Jim Blake <jeb228@... <mailto:jeb228@...>>
>> Reply-To: VIVO Development List <vivo-dev-all@... 
>> <mailto:vivo-dev-all@...>>
>> Date: Thu, 7 Jul 2011 13:41:20 -0400
>> To: VIVO Development List <vivo-dev-all@... 
>> <mailto:vivo-dev-all@...>>
>> Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster
>>
>> Eliza;
>>
>> That’s a very interesting question, and it opens an big can of worms.
>>
>> I am not a lawyer, but it appears that D2RMap is covered under the 
>> GNU Lesser General Public License (LGPL), which is a modification of 
>> the GNU General Public License, which appears to cover this scenario 
>> in section 6, paragraph d.
>>
>> If I understand this correctly, if you offer the object code (i.e., 
>> JAR files or CLASS files) for public download as part of your 
>> product, you are required to offer the source code (i.e. JAVA files) 
>> for public download on the same server, or an equivalent server 
>> operated by you or by a third party.
>>
>> So if our software is available for download from SourceForge, and 
>> the D2RMap source code is available from SourceForge, have we 
>> satisfied the terms of the license?
>>
>> Have I mentioned that I have no legal training or expertise?
>>
>> I wonder whether the VIVO project is in violation of the terms of 
>> many additional open source licenses. For exmaple, some licenses 
>> require that we cite them in all of our end-user documentation. Do we 
>> do that?
>>
>> But I’m just taking wild guesses, here. Can anybody shed more light 
>> on this?
>>
>> Jim
>>
>> *From:*Eliza Chan [mailto:elc2013@...]
>> *Sent:* Thursday, July 07, 2011 11:59 AM
>> *To:* VIVO Development List
>> *Subject:* Re: [Vivo-dev-all] d2rmap harvesting, was oracle for 
>> harverster
>>
>> Hi Nicholas and Brian,
>>
>> I can definitely commit D2RMap to sourceforge.
>>
>> Just a policy question before doing that.
>>
>> As you are probably aware, D2RMap is a sourceforge project: 
>> https://sourceforge.net/projects/d2r-map/
>>
>> Do we need to do anything extra (whatever that is) before we post 
>> their modified copy under our project?
>>
>> Thanks,
>>
>> Eliza
>>
>> *From: *Nicholas Skaggs <nskaggs@... <mailto:nskaggs@...>>
>> *Reply-To: *VIVO Development List <vivo-dev-all@... 
>> <mailto:vivo-dev-all@...>>
>> *Date: *Thu, 7 Jul 2011 11:10:13 -0400
>> *To: *"vivo-dev-all@... 
>> <mailto:vivo-dev-all@...>" 
>> <vivo-dev-all@... 
>> <mailto:vivo-dev-all@...>>
>> *Subject: *Re: [Vivo-dev-all] d2rmap harvesting, was oracle for 
>> harverster
>>
>> Brian and Eliza,
>>     Let's get a merged copy of your code into the harvester repo. 
>> Brian, it seems your new jar is missing some of Eliza's code -- 
>> specifically the CSV Map:
>>
>> The method processCsvMap(String, String, String) is undefined for the 
>> type D2rProcessor    D2RMapFetch.java    
>> /Harvester-hg/src/main/java/org/vivoweb/harvester/fetch    line 
>> 113    Java Problem
>>
>> Eliza, I know you have commit rights to SVN -- Brian do you? Eliza 
>> can you check in your source for D2RMap, and then have Brian add his 
>> changes in as another commit, etc. That would be really awesome. I'm 
>> sure others who are using it would be excited to be able to commit 
>> fixes and updates as well.
>>
>> Right now we're still working on restoring the SVN from the mercurial 
>> commits we did. As soon as that's done, we can get your commits in.
>>
>> Thanks!
>>
>> Nicholas
>>
>>
>> On 7/1/11 2:43 PM, Eliza Chan wrote:
>>
>> You are right, we should have a repository for the d2rmap source 
>> code. Since now you have a modified copy, and I also have one. It's 
>> better to merge them.
>>
>> *From: *"Keese, Brian W" <bkeese@... <mailto:bkeese@...>>
>> *Reply-To: *VIVO Development List <vivo-dev-all@... 
>> <mailto:vivo-dev-all@...>>
>> *Date: *Fri, 1 Jul 2011 14:37:11 -0400
>> *To: *VIVO Development List <vivo-dev-all@... 
>> <mailto:vivo-dev-all@...>>
>> *Subject: *Re: [Vivo-dev-all] d2rmap harvesting, was oracle for 
>> harverster
>>
>> Sure. Let me know how I can help with the docs.
>>
>> For the bugs, one of them was fixed before (the oracle semi-colon 
>> thing, Map.java around line 94).
>>
>> The other bug has to do with null values for instance identifiers. In 
>> Map.java around line 126, a list of new instances is being compiled. 
>> Unique instances are defined by values in a discreet set of fields. 
>> If one those values happens to be null, this line would give a null 
>> pointer exception:
>>
>> if (!currentTuple.get(fieldName).equals(lastTuple.get(fieldName)))
>>
>> So, I changed the logic to this:
>>
>> if (!(currentTuple.get(fieldName)==null && 
>> lastTuple.get(fieldName)==null) &&
>>
>>                       (currentTuple.get(fieldName)==null || 
>> !currentTuple.get(fieldName).equals(lastTuple.get(fieldName))))
>>
>> I also added some logging info to help with diagnosing problems with 
>> the harvest. And I removed some logging info which seemed to me to be 
>> not useful.
>>
>> We don’t have an svn repository for the d2rmap source code do we? 
>> Maybe we should set that up.
>>
>> Brian
>>
>> Brian Keese
>>
>> Indiana University Digital Library Program
>>
>> Herman B Wells Library W501
>>
>> 812-856-3865
>>
>> *From:*Eliza Chan [mailto:elc2013@...]
>> *Sent:* Friday, July 01, 2011 2:22 PM
>> *To:* VIVO Development List
>> *Subject:* Re: [Vivo-dev-all] d2rmap harvesting, was oracle for 
>> harverster
>>
>> Hi Brian,
>>
>> Thanks for making the changes.
>>
>> It'd be nice to have d2rmap harvests (db and csv) being supported in 
>> future releases, since there are people who use the tools on a 
>> regular basis.
>>
>> I have been trying to update the documentation on d2rmap but it has 
>> not been able to keep up with so many changes with Harvester. :-(
>>
>> Would you be able to document some of it? Also, would you be able to 
>> list the bugs that you fixed?
>>
>> Thanks,
>>
>> Eliza
>>
>> *From: *"Keese, Brian W" <bkeese@... <mailto:bkeese@...>>
>> *Reply-To: *VIVO Development List <vivo-dev-all@... 
>> <mailto:vivo-dev-all@...>>
>> *Date: *Fri, 1 Jul 2011 14:05:23 -0400
>> *To: *VIVO Development List <vivo-dev-all@... 
>> <mailto:vivo-dev-all@...>>
>> *Subject: *[Vivo-dev-all] d2rmap harvesting, was oracle for harverster
>>
>> Nick, Thanks for the info on Oracle. Makes sense.
>>
>> Here’s another thing. I have made a couple of changes to the d2rmap 
>> processor that is included with the harvester. I have uploaded the 
>> new jar file to 
>> https://www.slashtmp.iu.edu/files/download?FILE=bkeese%2F94328zp0Qy2 
>> (the password is ‘d2r4vivo’).
>>
>> It fixes a couple of bugs, and adds some status info to the logs for 
>> d2rmap fetch.
>>
>> I noticed that there is no example script for d2rmap in the latest 
>> release. I hope support for d2r harvests will be continued.
>>
>> Brian
>>
>> Brian Keese
>>
>> Indiana University Digital Library Program
>>
>> Herman B Wells Library W501
>>
>> 812-856-3865
>>
>> *From:*Nicholas Skaggs [mailto:nskaggs@...]
>> *Sent:* Friday, July 01, 2011 10:51 AM
>> *To:* vivo-dev-all@... 
>> <mailto:vivo-dev-all@...>
>> *Subject:* Re: [Vivo-dev-all] Vivo development call / meeting 
>> reminder for today.
>>
>> Great catch Brian. The trouble with oracle is that it requires a 
>> specific jdbc connector for each release -- including minor point 
>> release versions.
>>
>> http://www.oracle.com/technetwork/database/features/jdbc/index-091264.html
>>
>> We can't package every version of the connector with the harvester 
>> unfortunately. I have heard of a so called "universal" connector for 
>> oracle that connects to the databases through lowest common 
>> denominator SQL. I've never been able to find anything additional 
>> about this, nor a download for it -- so I can't verify it exists or 
>> not.  In addition, the connectors are released by Oracle under the 
>> Oracle Technology Network Development and Distribution License. This 
>> prevents us from distributing these files by packaging them directly 
>> without complying with the license terms. Specifically, the 
>> redistribution clause of the license causes trouble:
>>
>> /Program Distribution
>> We grant you a nonexclusive, nontransferable right to copy and 
>> distribute the programs to your end users provided that you do not 
>> charge your end users for use of the programs and provided your end 
>> users may only use the programs to run your applications for their 
>> business operations. Prior to distributing the programs you shall 
>> require your end users to execute an agreement binding them to terms 
>> consistent with those contained in this section and the sections of 
>> this agreement entitled "License Rights," "Ownership and 
>> Restrictions," "Export," "Disclaimer of Warranties and Exclusive 
>> Remedies," "No Technical Support," "End of Agreement," "Relationship 
>> Between the Parties," and "Open Source." You must also include a 
>> provision stating that your end users shall have no right to 
>> distribute the programs, and a provision specifying us as a third 
>> party beneficiary of the agreement. You are responsible for obtaining 
>> these agreements with your end users./
>>
>> You can read the entire license here:
>>
>> http://www.oracle.com/technetwork/licenses/distribution-license-152002.html
>>
>> If you have an idea about how we can distribute something to help the 
>> harvester work out of the box with oracle, please let us know. Thanks,
>>
>> Nicholas
>>
>> On 7/1/11 10:39 AM, Keese, Brian W wrote:
>>
>> The attachment didn't work. The driver can be downloaded from Oracle here:
>>   
>> http://www.oracle.com/technetwork/database/enterprise-edition/jdbc-10201-088211.html
>> Brian
>> ________________________________________
>> From: Keese, Brian W [bkeese@...  <mailto:bkeese@...>]
>> Sent: Friday, July 01, 2011 10:36 AM
>> To:jrpence@...  <mailto:jrpence@...>; VIVO Development List
>> Subject: Re: [Vivo-dev-all] Vivo development call / meeting reminder for today.
>>   
>> Hey Harvester Guys,
>> I noticed that the oracle jdbc driver was never added (or was added, then removed) to the harvester bin/dependencies folder for the latest release. Could you add that in please?
>> It is ojdbc14.jar (attached to this email).
>>   
>> Thanks,
>> Brian
>>   
>>   
>> ________________________________________
>> From: James Pence [jrpence@...  <mailto:jrpence@...>]
>> Sent: Thursday, June 30, 2011 1:23 PM
>> To:vivo-dev-all@...  <mailto:vivo-dev-all@...>
>> Subject: [Vivo-dev-all] Vivo development call / meeting reminder for today.
>>   
>> Hello,
>>   
>>      Today the vivo developers, and maybe you, are having a goto-meeting at 3:00pm EST (UTC-4 hours)
>>   
>>      We have the beginnings of an agenda at<http://sourceforge.net/apps/mediawiki/vivo/index.php?title=2011-06-30_Development_Leads_Call>
>>   
>> If you have any agenda items to add to the meeting please let me know (jrpence@...  <mailto:jrpence@...>    ).
>>   
>>     Since we only have 25 connections for the goto meeting please consolidate where possible.
>>   
>>     1. Please join my meeting, Thursday, June 30 at 3:00 PM Eastern DaylightTime.
>>     https://www1.gotomeeting.com/join/486839241
>>   
>>     2. Use your microphone and speakers (VoIP) - a headset is recommended.
>>   
>>     Or, call in using your telephone.
>>     Dial +1 (215) 383-1013
>>   
>>     Access Code: 486-839-241
>>     Audio PIN: Shown after joining the meeting
>>     Meeting ID: 486-839-241
>>   
>>     GoToMeeting®
>>     Online Meetings Made Easy™
>>   
>>   
>>   
>> --
>> James R. Pence
>> jrpence@...  <mailto:jrpence@...>
>> Software Engineer
>> Communicore C2-12
>> (352)294-5154 (ex. 45154)
>> Enterprise Software Engineering
>> University of Florida
>> Academic Health Center IT
>> Developer on the VIVO Grant
>>   
>>   
>> ------------------------------------------------------------------------------
>> All of the data generated in your IT infrastructure is seriously valuable.
>> Why? It contains a definitive record of application performance, security
>> threats, fraudulent activity, and more. Splunk takes this data and makes
>> sense of it. IT sense. And common sense.
>> http://p.sf.net/sfu/splunk-d2d-c2
>> _______________________________________________
>> Vivo-dev-all mailing list
>> Vivo-dev-all@...  <mailto:Vivo-dev-all@...>
>> https://lists.sourceforge.net/lists/listinfo/vivo-dev-all
>>   
>> ------------------------------------------------------------------------------
>> All of the data generated in your IT infrastructure is seriously valuable.
>> Why? It contains a definitive record of application performance, security
>> threats, fraudulent activity, and more. Splunk takes this data and makes
>> sense of it. IT sense. And common sense.
>> http://p.sf.net/sfu/splunk-d2d-c2
>> _______________________________________________
>> Vivo-dev-all mailing list
>> Vivo-dev-all@...  <mailto:Vivo-dev-all@...>
>> https://lists.sourceforge.net/lists/listinfo/vivo-dev-all
>>
>

Hi Nicholas,

Please see my questions regarding your action items:

1. License modifications as GPL/LGPL

Did you mean adding GPL/LGPL to Licensing & Acknowledgements?
Or did you mean adding a license statement to the source code?

2. Push patchsets to D2RMap community

Under the license agreement, I don't see anywhere that we are required to push the patchsets back to the D2RMap community. Could someone confirm this please?
However, if we really want to check in the changes, Brian's patch seems to be more relevant because it's related to a bug fix. But my part was mainly for integrating D2RMap to VIVO, so I'm not sure if it's relevant to the D2RMap project.

3. Host our modified version of the code on SF (seperate sub-project?)

How can one create a sub-project in SF?

4. Use the modified binary in the harvester, noting the GPL license

Did you mean adding a LICENSE file somewhere in the Harvester source code?

5.
Under the "TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION",
section 6, paragraph b: You must cause the files modified to carry prominent notices stating that you changed the files and the date of any change.

Have we addressed this? I'm also not a lawyer. ;-)

Thanks,
Eliza

From: Nicholas Skaggs <nskaggs@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Thu, 7 Jul 2011 14:02:49 -0400
To: "vivo-dev-all@...>" <vivo-dev-all@...>>
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Given the shortcomings of email, I don't want to be mis-interpreted on my reply.

I would like to preface this email with "In my opinion", and "I think we should' :-)

Thanks,

Nicholas

On 7/7/11 1:58 PM, Nicholas Skaggs wrote:
Jim, yes we must consider these. For distributing a binary, we can simply disclose it's license as GPL, and do nothing more.  LGPL and GPL allow for different things in terms of linking and non-open source software, but we don't need to get into that here. BSD is "compatible" with the GPL/LGPL. In this case, Eliza/Brian has modified GPL/LGPL source (they appear to have it dual licensed). We can distribute this of course, but we do need to make those modifications available in source form upon request. This can be done thru writing and physical media (we can even charge the cost of distribution), etc, but the simplest and most common is to host the source in a freely downloadable place.

So, I didn't realize D2RMap was an existing SF project. We essentially have 2 options. We can fork there project and host it ourselves with our modifications, or we can contribute them back upstream and see if we can get them to adopt them. I would suggest starting with the latter. Depending on the outcome, in the meantime, we will be maintaining a "fork" of the project with our modifications as part of the harvester since we want to utilize it. Now, the best practice for Eliza and Brian would be to make there modifications back to the D2RMap project as GPL/LGPL (there current license format). If they licensed it as BSD or something else, the project would have part of the code under a different license which is no good, and would make it harder for them to accept.

The ramifications then for the harvester is we must ship the GPL license with our project and state that D2RMap is under that license. Not a big deal, it's an external library; but we should make note of it.

In summary we should:

License modifications as GPL/LGPL
Push patchsets to D2RMap community
Host our modified version of the code on SF (seperate sub-project?)
Use the modified binary in the harvester, noting the GPL license

Nicholas

On 7/7/11 1:50 PM, Eliza Chan wrote:
Hi Jim,

Thanks for looking up.
In this case, committing the D2RMap source code to our project is not only Ok, it's required.

Eliza

From: Jim Blake <jeb228@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Thu, 7 Jul 2011 13:41:20 -0400
To: VIVO Development List <vivo-dev-all@...>>
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Eliza;

That’s a very interesting question, and it opens an big can of worms.

I am not a lawyer, but it appears that D2RMap is covered under the GNU Lesser General Public License (LGPL), which is a modification of the GNU General Public License, which appears to cover this scenario in section 6, paragraph d.

If I understand this correctly, if you offer the object code (i.e., JAR files or CLASS files) for public download as part of your product, you are required to offer the source code (i.e. JAVA files) for public download on the same server, or an equivalent server operated by you or by a third party.

So if our software is available for download from SourceForge, and the D2RMap source code is available from SourceForge, have we satisfied the terms of the license?

Have I mentioned that I have no legal training or expertise?

I wonder whether the VIVO project is in violation of the terms of many additional open source licenses. For exmaple, some licenses require that we cite them in all of our end-user documentation. Do we do that?

But I’m just taking wild guesses, here. Can anybody shed more light on this?

Jim


From: Eliza Chan [mailto:elc2013@...]
Sent: Thursday, July 07, 2011 11:59 AM
To: VIVO Development List
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Hi Nicholas and Brian,

I can definitely commit D2RMap to sourceforge.
Just a policy question before doing that.
As you are probably aware, D2RMap is a sourceforge project: https://sourceforge.net/projects/d2r-map/
Do we need to do anything extra (whatever that is) before we post their modified copy under our project?

Thanks,
Eliza

From: Nicholas Skaggs <nskaggs@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Thu, 7 Jul 2011 11:10:13 -0400
To: "vivo-dev-all@...>" <vivo-dev-all@...>>
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Brian and Eliza,
    Let's get a merged copy of your code into the harvester repo. Brian, it seems your new jar is missing some of Eliza's code -- specifically the CSV Map:

The method processCsvMap(String, String, String) is undefined for the type D2rProcessor    D2RMapFetch.java    /Harvester-hg/src/main/java/org/vivoweb/harvester/fetch    line 113    Java Problem

Eliza, I know you have commit rights to SVN -- Brian do you? Eliza can you check in your source for D2RMap, and then have Brian add his changes in as another commit, etc. That would be really awesome. I'm sure others who are using it would be excited to be able to commit fixes and updates as well.

Right now we're still working on restoring the SVN from the mercurial commits we did. As soon as that's done, we can get your commits in.

Thanks!

Nicholas


On 7/1/11 2:43 PM, Eliza Chan wrote:
You are right, we should have a repository for the d2rmap source code. Since now you have a modified copy, and I also have one. It's better to merge them.


From: "Keese, Brian W" <bkeese@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Fri, 1 Jul 2011 14:37:11 -0400
To: VIVO Development List <vivo-dev-all@...>>
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Sure. Let me know how I can help with the docs.

For the bugs, one of them was fixed before (the oracle semi-colon thing, Map.java around line 94).

The other bug has to do with null values for instance identifiers. In Map.java around line 126, a list of new instances is being compiled. Unique instances are defined by values in a discreet set of fields. If one those values happens to be null, this line would give a null pointer exception:

if (!currentTuple.get(fieldName).equals(lastTuple.get(fieldName)))

So, I changed the logic to this:

if (!(currentTuple.get(fieldName)==null && lastTuple.get(fieldName)==null) &&
                      (currentTuple.get(fieldName)==null || !currentTuple.get(fieldName).equals(lastTuple.get(fieldName))))

I also added some logging info to help with diagnosing problems with the harvest. And I removed some logging info which seemed to me to be not useful.

We don’t have an svn repository for the d2rmap source code do we? Maybe we should set that up.

Brian

Brian Keese
Indiana University Digital Library Program
Herman B Wells Library W501
812-856-3865

From: Eliza Chan [mailto:elc2013@...]
Sent: Friday, July 01, 2011 2:22 PM
To: VIVO Development List
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Hi Brian,

Thanks for making the changes.
It'd be nice to have d2rmap harvests (db and csv) being supported in future releases, since there are people who use the tools on a regular basis.
I have been trying to update the documentation on d2rmap but it has not been able to keep up with so many changes with Harvester. :-(
Would you be able to document some of it? Also, would you be able to list the bugs that you fixed?

Thanks,
Eliza

From: "Keese, Brian W" <bkeese@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Fri, 1 Jul 2011 14:05:23 -0400
To: VIVO Development List <vivo-dev-all@...>>
Subject: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Nick, Thanks for the info on Oracle. Makes sense.

Here’s another thing. I have made a couple of changes to the d2rmap processor that is included with the harvester. I have uploaded the new jar file to https://www.slashtmp.iu.edu/files/download?FILE=bkeese%2F94328zp0Qy2 (the password is ‘d2r4vivo’).
It fixes a couple of bugs, and adds some status info to the logs for d2rmap fetch.

I noticed that there is no example script for d2rmap in the latest release. I hope support for d2r harvests will be continued.

Brian

Brian Keese
Indiana University Digital Library Program
Herman B Wells Library W501
812-856-3865

From: Nicholas Skaggs [mailto:nskaggs@...]
Sent: Friday, July 01, 2011 10:51 AM
To: vivo-dev-all@...>
Subject: Re: [Vivo-dev-all] Vivo development call / meeting reminder for today.

Great catch Brian. The trouble with oracle is that it requires a specific jdbc connector for each release -- including minor point release versions.

http://www.oracle.com/technetwork/database/features/jdbc/index-091264.html

We can't package every version of the connector with the harvester unfortunately. I have heard of a so called "universal" connector for oracle that connects to the databases through lowest common denominator SQL. I've never been able to find anything additional about this, nor a download for it -- so I can't verify it exists or not.  In addition, the connectors are released by Oracle under the Oracle Technology Network Development and Distribution License. This prevents us from distributing these files by packaging them directly without complying with the license terms. Specifically, the redistribution clause of the license causes trouble:

Program Distribution
We grant you a nonexclusive, nontransferable right to copy and distribute the programs to your end users provided that you do not charge your end users for use of the programs and provided your end users may only use the programs to run your applications for their business operations. Prior to distributing the programs you shall require your end users to execute an agreement binding them to terms consistent with those contained in this section and the sections of this agreement entitled "License Rights," "Ownership and Restrictions," "Export," "Disclaimer of Warranties and Exclusive Remedies," "No Technical Support," "End of Agreement," "Relationship Between the Parties," and "Open Source." You must also include a provision stating that your end users shall have no right to distribute the programs, and a provision specifying us as a third party beneficiary of the agreement. You are responsible for obtaining these agreements with your end users.

You can read the entire license here:

http://www.oracle.com/technetwork/licenses/distribution-license-152002.html

If you have an idea about how we can distribute something to help the harvester work out of the box with oracle, please let us know. Thanks,

Nicholas

On 7/1/11 10:39 AM, Keese, Brian W wrote:

The attachment didn't work. The driver can be downloaded from Oracle here:



http://www.oracle.com/technetwork/database/enterprise-edition/jdbc-10201-088211.html

Brian

________________________________________

From: Keese, Brian W [bkeese@...>]

Sent: Friday, July 01, 2011 10:36 AM

To: jrpence@...>; VIVO Development List

Subject: Re: [Vivo-dev-all] Vivo development call / meeting reminder for today.



Hey Harvester Guys,

I noticed that the oracle jdbc driver was never added (or was added, then removed) to the harvester bin/dependencies folder for the latest release. Could you add that in please?

It is ojdbc14.jar (attached to this email).



Thanks,

Brian





________________________________________

From: James Pence [jrpence@...>]

Sent: Thursday, June 30, 2011 1:23 PM

To: vivo-dev-all@...>

Subject: [Vivo-dev-all] Vivo development call / meeting reminder for today.



Hello,



    Today the vivo developers, and maybe you, are having a goto-meeting at 3:00pm EST (UTC-4 hours)



    We have the beginnings of an agenda at<http://sourceforge.net/apps/mediawiki/vivo/index.php?title=2011-06-30_Development_Leads_Call><http://sourceforge.net/apps/mediawiki/vivo/index.php?title=2011-06-30_Development_Leads_Call>



If you have any agenda items to add to the meeting please let me know (jrpence@...>   ).



   Since we only have 25 connections for the goto meeting please consolidate where possible.



   1. Please join my meeting, Thursday, June 30 at 3:00 PM Eastern DaylightTime.

   https://www1.gotomeeting.com/join/486839241



   2. Use your microphone and speakers (VoIP) - a headset is recommended.



   Or, call in using your telephone.

   Dial +1 (215) 383-1013



   Access Code: 486-839-241

   Audio PIN: Shown after joining the meeting

   Meeting ID: 486-839-241



   GoToMeeting®

   Online Meetings Made Easy™







--

James R. Pence

jrpence@...>

Software Engineer

Communicore C2-12

(352)294-5154 (ex. 45154)

Enterprise Software Engineering

University of Florida

Academic Health Center IT

Developer on the VIVO Grant





------------------------------------------------------------------------------

All of the data generated in your IT infrastructure is seriously valuable.

Why? It contains a definitive record of application performance, security

threats, fraudulent activity, and more. Splunk takes this data and makes

sense of it. IT sense. And common sense.

http://p.sf.net/sfu/splunk-d2d-c2

_______________________________________________

Vivo-dev-all mailing list

Vivo-dev-all@...>

https://lists.sourceforge.net/lists/listinfo/vivo-dev-all



------------------------------------------------------------------------------

All of the data generated in your IT infrastructure is seriously valuable.

Why? It contains a definitive record of application performance, security

threats, fraudulent activity, and more. Splunk takes this data and makes

sense of it. IT sense. And common sense.

http://p.sf.net/sfu/splunk-d2d-c2

_______________________________________________

Vivo-dev-all mailing list

Vivo-dev-all@...>

https://lists.sourceforge.net/lists/listinfo/vivo-dev-all

Eliza and Brian,
      we've created a d2rmap branch in the git repository for the 
harvester -- origin/D2rMapIntegration. It contains the latest code from 
the d2rmap repository. Can you two add your patchsets to this code to 
bring it up to date with what we want to ship with the harvester? We'll 
then need to update maven to build a jar for the harvester to use as 
part of the build for the harvester.

Secondly, we have the d2rmap example in the master branch on git. While 
the example doesn't give an error, no data is being added to VIVO. Can 
you take a look and figure out what's wrong? The team is ready to push 
out a 1.2 final of the harvester once this example is complete :-)

Now, onto your questions below:

1. I mean you should license your modifications to d2rmap as GPL/LGPL so 
that there project can accept them under than same license terms as 
d2rmap is licensed.

2. We are NOT required to followup with them, but I think it would be a 
good idea to let them know of our work. Once we have your code 
integrated, I would suggest sending the patchset to the d2rmap mailing 
list and/or letting them now we have changes in our git which are 
publically available.

3. The team seemed to favor including the code inline with the 
harvester, so no sub-project has been created.

4. Licensing file has been updated appropriately for this, so we're good.

5. I'm not sure where this is coming from, but making available the 
source, and making our changes available, as well as noting the license 
of the code as being GPL/LGPL meets the terms of the license.

Nicholas


On 7/8/11 11:26 AM, Eliza Chan wrote:
> Hi Nicholas,
>
> Please see my questions regarding your action items:
>
> 1. License modifications as GPL/LGPL
>
> Did you mean adding GPL/LGPL to Licensing & Acknowledgements?
> Or did you mean adding a license statement to the source code?
>
> 2. Push patchsets to D2RMap community
>
> Under the license agreement, I don't see anywhere that we are required 
> to push the patchsets back to the D2RMap community. Could someone 
> confirm this please?
> However, if we really want to check in the changes, Brian's patch 
> seems to be more relevant because it's related to a bug fix. But my 
> part was mainly for integrating D2RMap to VIVO, so I'm not sure if 
> it's relevant to the D2RMap project.
>
> 3. Host our modified version of the code on SF (seperate sub-project?)
>
> How can one create a sub-project in SF?
>
> 4. Use the modified binary in the harvester, noting the GPL license
>
> Did you mean adding a LICENSE file somewhere in the Harvester source code?
>
> 5.
> Under the "TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND 
> MODIFICATION",
> section 6, paragraph b: You must cause the files modified to carry 
> prominent notices stating that you changed the files and the date of 
> any change.
>
> Have we addressed this? I'm also not a lawyer. ;-)
>
> Thanks,
> Eliza
>
> From: Nicholas Skaggs <nskaggs@... <mailto:nskaggs@...>>
> Reply-To: VIVO Development List <vivo-dev-all@... 
> <mailto:vivo-dev-all@...>>
> Date: Thu, 7 Jul 2011 14:02:49 -0400
> To: "vivo-dev-all@... 
> <mailto:vivo-dev-all@...>" 
> <vivo-dev-all@... 
> <mailto:vivo-dev-all@...>>
> Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster
>
> Given the shortcomings of email, I don't want to be mis-interpreted on 
> my reply.
>
> I would like to preface this email with "In my opinion", and "I think 
> we should' :-)
>
> Thanks,
>
> Nicholas
>
> On 7/7/11 1:58 PM, Nicholas Skaggs wrote:
>> Jim, yes we must consider these. For distributing a binary, we can 
>> simply disclose it's license as GPL, and do nothing more.  LGPL and 
>> GPL allow for different things in terms of linking and non-open 
>> source software, but we don't need to get into that here. BSD is 
>> "compatible" with the GPL/LGPL. In this case, Eliza/Brian has 
>> modified GPL/LGPL source (they appear to have it dual licensed). We 
>> can distribute this of course, but we do need to make those 
>> modifications available in source form upon request. This can be done 
>> thru writing and physical media (we can even charge the cost of 
>> distribution), etc, but the simplest and most common is to host the 
>> source in a freely downloadable place.
>>
>> So, I didn't realize D2RMap was an existing SF project. We 
>> essentially have 2 options. We can fork there project and host it 
>> ourselves with our modifications, or we can contribute them back 
>> upstream and see if we can get them to adopt them. I would suggest 
>> starting with the latter. Depending on the outcome, in the meantime, 
>> we will be maintaining a "fork" of the project with our modifications 
>> as part of the harvester since we want to utilize it. Now, the best 
>> practice for Eliza and Brian would be to make there modifications 
>> back to the D2RMap project as GPL/LGPL (there current license 
>> format). If they licensed it as BSD or something else, the project 
>> would have part of the code under a different license which is no 
>> good, and would make it harder for them to accept.
>>
>> The ramifications then for the harvester is we must ship the GPL 
>> license with our project and state that D2RMap is under that license. 
>> Not a big deal, it's an external library; but we should make note of it.
>>
>> In summary we should:
>>
>> License modifications as GPL/LGPL
>> Push patchsets to D2RMap community
>> Host our modified version of the code on SF (seperate sub-project?)
>> Use the modified binary in the harvester, noting the GPL license
>>
>> Nicholas
>>
>> On 7/7/11 1:50 PM, Eliza Chan wrote:
>>> Hi Jim,
>>>
>>> Thanks for looking up.
>>> In this case, committing the D2RMap source code to our project is 
>>> not only Ok, it's required.
>>>
>>> Eliza
>>>
>>> From: Jim Blake <jeb228@... <mailto:jeb228@...>>
>>> Reply-To: VIVO Development List <vivo-dev-all@... 
>>> <mailto:vivo-dev-all@...>>
>>> Date: Thu, 7 Jul 2011 13:41:20 -0400
>>> To: VIVO Development List <vivo-dev-all@... 
>>> <mailto:vivo-dev-all@...>>
>>> Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster
>>>
>>> Eliza;
>>>
>>> That’s a very interesting question, and it opens an big can of worms.
>>>
>>> I am not a lawyer, but it appears that D2RMap is covered under the 
>>> GNU Lesser General Public License (LGPL), which is a modification of 
>>> the GNU General Public License, which appears to cover this scenario 
>>> in section 6, paragraph d.
>>>
>>> If I understand this correctly, if you offer the object code (i.e., 
>>> JAR files or CLASS files) for public download as part of your 
>>> product, you are required to offer the source code (i.e. JAVA files) 
>>> for public download on the same server, or an equivalent server 
>>> operated by you or by a third party.
>>>
>>> So if our software is available for download from SourceForge, and 
>>> the D2RMap source code is available from SourceForge, have we 
>>> satisfied the terms of the license?
>>>
>>> Have I mentioned that I have no legal training or expertise?
>>>
>>> I wonder whether the VIVO project is in violation of the terms of 
>>> many additional open source licenses. For exmaple, some licenses 
>>> require that we cite them in all of our end-user documentation. Do 
>>> we do that?
>>>
>>> But I’m just taking wild guesses, here. Can anybody shed more light 
>>> on this?
>>>
>>> Jim
>>>
>>> *From:*Eliza Chan [mailto:elc2013@...]
>>> *Sent:* Thursday, July 07, 2011 11:59 AM
>>> *To:* VIVO Development List
>>> *Subject:* Re: [Vivo-dev-all] d2rmap harvesting, was oracle for 
>>> harverster
>>>
>>> Hi Nicholas and Brian,
>>>
>>> I can definitely commit D2RMap to sourceforge.
>>>
>>> Just a policy question before doing that.
>>>
>>> As you are probably aware, D2RMap is a sourceforge project: 
>>> https://sourceforge.net/projects/d2r-map/
>>>
>>> Do we need to do anything extra (whatever that is) before we post 
>>> their modified copy under our project?
>>>
>>> Thanks,
>>>
>>> Eliza
>>>
>>> *From: *Nicholas Skaggs <nskaggs@... <mailto:nskaggs@...>>
>>> *Reply-To: *VIVO Development List 
>>> <vivo-dev-all@... 
>>> <mailto:vivo-dev-all@...>>
>>> *Date: *Thu, 7 Jul 2011 11:10:13 -0400
>>> *To: *"vivo-dev-all@... 
>>> <mailto:vivo-dev-all@...>" 
>>> <vivo-dev-all@... 
>>> <mailto:vivo-dev-all@...>>
>>> *Subject: *Re: [Vivo-dev-all] d2rmap harvesting, was oracle for 
>>> harverster
>>>
>>> Brian and Eliza,
>>>     Let's get a merged copy of your code into the harvester repo. 
>>> Brian, it seems your new jar is missing some of Eliza's code -- 
>>> specifically the CSV Map:
>>>
>>> The method processCsvMap(String, String, String) is undefined for 
>>> the type D2rProcessor    D2RMapFetch.java    
>>> /Harvester-hg/src/main/java/org/vivoweb/harvester/fetch    line 
>>> 113    Java Problem
>>>
>>> Eliza, I know you have commit rights to SVN -- Brian do you? Eliza 
>>> can you check in your source for D2RMap, and then have Brian add his 
>>> changes in as another commit, etc. That would be really awesome. I'm 
>>> sure others who are using it would be excited to be able to commit 
>>> fixes and updates as well.
>>>
>>> Right now we're still working on restoring the SVN from the 
>>> mercurial commits we did. As soon as that's done, we can get your 
>>> commits in.
>>>
>>> Thanks!
>>>
>>> Nicholas
>>>
>>>
>>> On 7/1/11 2:43 PM, Eliza Chan wrote:
>>>
>>> You are right, we should have a repository for the d2rmap source 
>>> code. Since now you have a modified copy, and I also have one. It's 
>>> better to merge them.
>>>
>>> *From: *"Keese, Brian W" <bkeese@... 
>>> <mailto:bkeese@...>>
>>> *Reply-To: *VIVO Development List 
>>> <vivo-dev-all@... 
>>> <mailto:vivo-dev-all@...>>
>>> *Date: *Fri, 1 Jul 2011 14:37:11 -0400
>>> *To: *VIVO Development List <vivo-dev-all@... 
>>> <mailto:vivo-dev-all@...>>
>>> *Subject: *Re: [Vivo-dev-all] d2rmap harvesting, was oracle for 
>>> harverster
>>>
>>> Sure. Let me know how I can help with the docs.
>>>
>>> For the bugs, one of them was fixed before (the oracle semi-colon 
>>> thing, Map.java around line 94).
>>>
>>> The other bug has to do with null values for instance identifiers. 
>>> In Map.java around line 126, a list of new instances is being 
>>> compiled. Unique instances are defined by values in a discreet set 
>>> of fields. If one those values happens to be null, this line would 
>>> give a null pointer exception:
>>>
>>> if (!currentTuple.get(fieldName).equals(lastTuple.get(fieldName)))
>>>
>>> So, I changed the logic to this:
>>>
>>> if (!(currentTuple.get(fieldName)==null && 
>>> lastTuple.get(fieldName)==null) &&
>>>
>>>                       (currentTuple.get(fieldName)==null || 
>>> !currentTuple.get(fieldName).equals(lastTuple.get(fieldName))))
>>>
>>> I also added some logging info to help with diagnosing problems with 
>>> the harvest. And I removed some logging info which seemed to me to 
>>> be not useful.
>>>
>>> We don’t have an svn repository for the d2rmap source code do we? 
>>> Maybe we should set that up.
>>>
>>> Brian
>>>
>>> Brian Keese
>>>
>>> Indiana University Digital Library Program
>>>
>>> Herman B Wells Library W501
>>>
>>> 812-856-3865
>>>
>>> *From:*Eliza Chan [mailto:elc2013@...]
>>> *Sent:* Friday, July 01, 2011 2:22 PM
>>> *To:* VIVO Development List
>>> *Subject:* Re: [Vivo-dev-all] d2rmap harvesting, was oracle for 
>>> harverster
>>>
>>> Hi Brian,
>>>
>>> Thanks for making the changes.
>>>
>>> It'd be nice to have d2rmap harvests (db and csv) being supported in 
>>> future releases, since there are people who use the tools on a 
>>> regular basis.
>>>
>>> I have been trying to update the documentation on d2rmap but it has 
>>> not been able to keep up with so many changes with Harvester. :-(
>>>
>>> Would you be able to document some of it? Also, would you be able to 
>>> list the bugs that you fixed?
>>>
>>> Thanks,
>>>
>>> Eliza
>>>
>>> *From: *"Keese, Brian W" <bkeese@... 
>>> <mailto:bkeese@...>>
>>> *Reply-To: *VIVO Development List 
>>> <vivo-dev-all@... 
>>> <mailto:vivo-dev-all@...>>
>>> *Date: *Fri, 1 Jul 2011 14:05:23 -0400
>>> *To: *VIVO Development List <vivo-dev-all@... 
>>> <mailto:vivo-dev-all@...>>
>>> *Subject: *[Vivo-dev-all] d2rmap harvesting, was oracle for harverster
>>>
>>> Nick, Thanks for the info on Oracle. Makes sense.
>>>
>>> Here’s another thing. I have made a couple of changes to the d2rmap 
>>> processor that is included with the harvester. I have uploaded the 
>>> new jar file to 
>>> https://www.slashtmp.iu.edu/files/download?FILE=bkeese%2F94328zp0Qy2 
>>> (the password is ‘d2r4vivo’).
>>>
>>> It fixes a couple of bugs, and adds some status info to the logs for 
>>> d2rmap fetch.
>>>
>>> I noticed that there is no example script for d2rmap in the latest 
>>> release. I hope support for d2r harvests will be continued.
>>>
>>> Brian
>>>
>>> Brian Keese
>>>
>>> Indiana University Digital Library Program
>>>
>>> Herman B Wells Library W501
>>>
>>> 812-856-3865
>>>
>>> *From:*Nicholas Skaggs [mailto:nskaggs@...]
>>> *Sent:* Friday, July 01, 2011 10:51 AM
>>> *To:* vivo-dev-all@... 
>>> <mailto:vivo-dev-all@...>
>>> *Subject:* Re: [Vivo-dev-all] Vivo development call / meeting 
>>> reminder for today.
>>>
>>> Great catch Brian. The trouble with oracle is that it requires a 
>>> specific jdbc connector for each release -- including minor point 
>>> release versions.
>>>
>>> http://www.oracle.com/technetwork/database/features/jdbc/index-091264.html
>>>
>>> We can't package every version of the connector with the harvester 
>>> unfortunately. I have heard of a so called "universal" connector for 
>>> oracle that connects to the databases through lowest common 
>>> denominator SQL. I've never been able to find anything additional 
>>> about this, nor a download for it -- so I can't verify it exists or 
>>> not.  In addition, the connectors are released by Oracle under the 
>>> Oracle Technology Network Development and Distribution License. This 
>>> prevents us from distributing these files by packaging them directly 
>>> without complying with the license terms. Specifically, the 
>>> redistribution clause of the license causes trouble:
>>>
>>> /Program Distribution
>>> We grant you a nonexclusive, nontransferable right to copy and 
>>> distribute the programs to your end users provided that you do not 
>>> charge your end users for use of the programs and provided your end 
>>> users may only use the programs to run your applications for their 
>>> business operations. Prior to distributing the programs you shall 
>>> require your end users to execute an agreement binding them to terms 
>>> consistent with those contained in this section and the sections of 
>>> this agreement entitled "License Rights," "Ownership and 
>>> Restrictions," "Export," "Disclaimer of Warranties and Exclusive 
>>> Remedies," "No Technical Support," "End of Agreement," "Relationship 
>>> Between the Parties," and "Open Source." You must also include a 
>>> provision stating that your end users shall have no right to 
>>> distribute the programs, and a provision specifying us as a third 
>>> party beneficiary of the agreement. You are responsible for 
>>> obtaining these agreements with your end users./
>>>
>>> You can read the entire license here:
>>>
>>> http://www.oracle.com/technetwork/licenses/distribution-license-152002.html
>>>
>>> If you have an idea about how we can distribute something to help 
>>> the harvester work out of the box with oracle, please let us know. 
>>> Thanks,
>>>
>>> Nicholas
>>>
>>> On 7/1/11 10:39 AM, Keese, Brian W wrote:
>>>
>>> The attachment didn't work. The driver can be downloaded from Oracle here:
>>>   
>>> http://www.oracle.com/technetwork/database/enterprise-edition/jdbc-10201-088211.html
>>> Brian
>>> ________________________________________
>>> From: Keese, Brian W [bkeese@...  <mailto:bkeese@...>]
>>> Sent: Friday, July 01, 2011 10:36 AM
>>> To:jrpence@...  <mailto:jrpence@...>; VIVO Development List
>>> Subject: Re: [Vivo-dev-all] Vivo development call / meeting reminder for today.
>>>   
>>> Hey Harvester Guys,
>>> I noticed that the oracle jdbc driver was never added (or was added, then removed) to the harvester bin/dependencies folder for the latest release. Could you add that in please?
>>> It is ojdbc14.jar (attached to this email).
>>>   
>>> Thanks,
>>> Brian
>>>   
>>>   
>>> ________________________________________
>>> From: James Pence [jrpence@...  <mailto:jrpence@...>]
>>> Sent: Thursday, June 30, 2011 1:23 PM
>>> To:vivo-dev-all@...  <mailto:vivo-dev-all@...>
>>> Subject: [Vivo-dev-all] Vivo development call / meeting reminder for today.
>>>   
>>> Hello,
>>>   
>>>      Today the vivo developers, and maybe you, are having a goto-meeting at 3:00pm EST (UTC-4 hours)
>>>   
>>>      We have the beginnings of an agenda at<http://sourceforge.net/apps/mediawiki/vivo/index.php?title=2011-06-30_Development_Leads_Call>
>>>   
>>> If you have any agenda items to add to the meeting please let me know (jrpence@...  <mailto:jrpence@...>    ).
>>>   
>>>     Since we only have 25 connections for the goto meeting please consolidate where possible.
>>>   
>>>     1. Please join my meeting, Thursday, June 30 at 3:00 PM Eastern DaylightTime.
>>>     https://www1.gotomeeting.com/join/486839241
>>>   
>>>     2. Use your microphone and speakers (VoIP) - a headset is recommended.
>>>   
>>>     Or, call in using your telephone.
>>>     Dial +1 (215) 383-1013
>>>   
>>>     Access Code: 486-839-241
>>>     Audio PIN: Shown after joining the meeting
>>>     Meeting ID: 486-839-241
>>>   
>>>     GoToMeeting®
>>>     Online Meetings Made Easy™
>>>   
>>>   
>>>   
>>> --
>>> James R. Pence
>>> jrpence@...  <mailto:jrpence@...>
>>> Software Engineer
>>> Communicore C2-12
>>> (352)294-5154 (ex. 45154)
>>> Enterprise Software Engineering
>>> University of Florida
>>> Academic Health Center IT
>>> Developer on the VIVO Grant
>>>   
>>>   
>>> ------------------------------------------------------------------------------
>>> All of the data generated in your IT infrastructure is seriously valuable.
>>> Why? It contains a definitive record of application performance, security
>>> threats, fraudulent activity, and more. Splunk takes this data and makes
>>> sense of it. IT sense. And common sense.
>>> http://p.sf.net/sfu/splunk-d2d-c2
>>> _______________________________________________
>>> Vivo-dev-all mailing list
>>> Vivo-dev-all@...  <mailto:Vivo-dev-all@...>
>>> https://lists.sourceforge.net/lists/listinfo/vivo-dev-all
>>>   
>>> ------------------------------------------------------------------------------
>>> All of the data generated in your IT infrastructure is seriously valuable.
>>> Why? It contains a definitive record of application performance, security
>>> threats, fraudulent activity, and more. Splunk takes this data and makes
>>> sense of it. IT sense. And common sense.
>>> http://p.sf.net/sfu/splunk-d2d-c2
>>> _______________________________________________
>>> Vivo-dev-all mailing list
>>> Vivo-dev-all@...  <mailto:Vivo-dev-all@...>
>>> https://lists.sourceforge.net/lists/listinfo/vivo-dev-all
>>>
>>
>

Hi Nicholas,

Thanks for creating a branch for d2rmap.
I was actually working on the examples but got interrupted by a bug fix on another project.
I'll add the patchset and fix the examples as soon as the bug fix is done. Hopefully by end of today.
Sorry about the delay.

Thanks,
Eliza

From: Nicholas Skaggs <nskaggs@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Tue, 19 Jul 2011 11:14:24 -0400
To: "vivo-dev-all@...>" <vivo-dev-all@...>>
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Eliza and Brian,
     we've created a d2rmap branch in the git repository for the harvester -- origin/D2rMapIntegration. It contains the latest code from the d2rmap repository. Can you two add your patchsets to this code to bring it up to date with what we want to ship with the harvester? We'll then need to update maven to build a jar for the harvester to use as part of the build for the harvester.

Secondly, we have the d2rmap example in the master branch on git. While the example doesn't give an error, no data is being added to VIVO. Can you take a look and figure out what's wrong? The team is ready to push out a 1.2 final of the harvester once this example is complete :-)

Now, onto your questions below:

1. I mean you should license your modifications to d2rmap as GPL/LGPL so that there project can accept them under than same license terms as d2rmap is licensed.

2. We are NOT required to followup with them, but I think it would be a good idea to let them know of our work. Once we have your code integrated, I would suggest sending the patchset to the d2rmap mailing list and/or letting them now we have changes in our git which are publically available.

3. The team seemed to favor including the code inline with the harvester, so no sub-project has been created.

4. Licensing file has been updated appropriately for this, so we're good.

5. I'm not sure where this is coming from, but making available the source, and making our changes available, as well as noting the license of the code as being GPL/LGPL meets the terms of the license.

Nicholas


On 7/8/11 11:26 AM, Eliza Chan wrote:
Hi Nicholas,

Please see my questions regarding your action items:

1. License modifications as GPL/LGPL

Did you mean adding GPL/LGPL to Licensing & Acknowledgements?
Or did you mean adding a license statement to the source code?

2. Push patchsets to D2RMap community

Under the license agreement, I don't see anywhere that we are required to push the patchsets back to the D2RMap community. Could someone confirm this please?
However, if we really want to check in the changes, Brian's patch seems to be more relevant because it's related to a bug fix. But my part was mainly for integrating D2RMap to VIVO, so I'm not sure if it's relevant to the D2RMap project.

3. Host our modified version of the code on SF (seperate sub-project?)

How can one create a sub-project in SF?

4. Use the modified binary in the harvester, noting the GPL license

Did you mean adding a LICENSE file somewhere in the Harvester source code?

5.
Under the "TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION",
section 6, paragraph b: You must cause the files modified to carry prominent notices stating that you changed the files and the date of any change.

Have we addressed this? I'm also not a lawyer. ;-)

Thanks,
Eliza

From: Nicholas Skaggs <nskaggs@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Thu, 7 Jul 2011 14:02:49 -0400
To: "vivo-dev-all@...>" <vivo-dev-all@...>>
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Given the shortcomings of email, I don't want to be mis-interpreted on my reply.

I would like to preface this email with "In my opinion", and "I think we should' :-)

Thanks,

Nicholas

On 7/7/11 1:58 PM, Nicholas Skaggs wrote:
Jim, yes we must consider these. For distributing a binary, we can simply disclose it's license as GPL, and do nothing more.  LGPL and GPL allow for different things in terms of linking and non-open source software, but we don't need to get into that here. BSD is "compatible" with the GPL/LGPL. In this case, Eliza/Brian has modified GPL/LGPL source (they appear to have it dual licensed). We can distribute this of course, but we do need to make those modifications available in source form upon request. This can be done thru writing and physical media (we can even charge the cost of distribution), etc, but the simplest and most common is to host the source in a freely downloadable place.

So, I didn't realize D2RMap was an existing SF project. We essentially have 2 options. We can fork there project and host it ourselves with our modifications, or we can contribute them back upstream and see if we can get them to adopt them. I would suggest starting with the latter. Depending on the outcome, in the meantime, we will be maintaining a "fork" of the project with our modifications as part of the harvester since we want to utilize it. Now, the best practice for Eliza and Brian would be to make there modifications back to the D2RMap project as GPL/LGPL (there current license format). If they licensed it as BSD or something else, the project would have part of the code under a different license which is no good, and would make it harder for them to accept.

The ramifications then for the harvester is we must ship the GPL license with our project and state that D2RMap is under that license. Not a big deal, it's an external library; but we should make note of it.

In summary we should:

License modifications as GPL/LGPL
Push patchsets to D2RMap community
Host our modified version of the code on SF (seperate sub-project?)
Use the modified binary in the harvester, noting the GPL license

Nicholas

On 7/7/11 1:50 PM, Eliza Chan wrote:
Hi Jim,

Thanks for looking up.
In this case, committing the D2RMap source code to our project is not only Ok, it's required.

Eliza

From: Jim Blake <jeb228@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Thu, 7 Jul 2011 13:41:20 -0400
To: VIVO Development List <vivo-dev-all@...>>
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Eliza;

That’s a very interesting question, and it opens an big can of worms.

I am not a lawyer, but it appears that D2RMap is covered under the GNU Lesser General Public License (LGPL), which is a modification of the GNU General Public License, which appears to cover this scenario in section 6, paragraph d.

If I understand this correctly, if you offer the object code (i.e., JAR files or CLASS files) for public download as part of your product, you are required to offer the source code (i.e. JAVA files) for public download on the same server, or an equivalent server operated by you or by a third party.

So if our software is available for download from SourceForge, and the D2RMap source code is available from SourceForge, have we satisfied the terms of the license?

Have I mentioned that I have no legal training or expertise?

I wonder whether the VIVO project is in violation of the terms of many additional open source licenses. For exmaple, some licenses require that we cite them in all of our end-user documentation. Do we do that?

But I’m just taking wild guesses, here. Can anybody shed more light on this?

Jim


From: Eliza Chan [mailto:elc2013@...]
Sent: Thursday, July 07, 2011 11:59 AM
To: VIVO Development List
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Hi Nicholas and Brian,

I can definitely commit D2RMap to sourceforge.
Just a policy question before doing that.
As you are probably aware, D2RMap is a sourceforge project: https://sourceforge.net/projects/d2r-map/
Do we need to do anything extra (whatever that is) before we post their modified copy under our project?

Thanks,
Eliza

From: Nicholas Skaggs <nskaggs@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Thu, 7 Jul 2011 11:10:13 -0400
To: "vivo-dev-all@...>" <vivo-dev-all@...>>
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Brian and Eliza,
    Let's get a merged copy of your code into the harvester repo. Brian, it seems your new jar is missing some of Eliza's code -- specifically the CSV Map:

The method processCsvMap(String, String, String) is undefined for the type D2rProcessor    D2RMapFetch.java    /Harvester-hg/src/main/java/org/vivoweb/harvester/fetch    line 113    Java Problem

Eliza, I know you have commit rights to SVN -- Brian do you? Eliza can you check in your source for D2RMap, and then have Brian add his changes in as another commit, etc. That would be really awesome. I'm sure others who are using it would be excited to be able to commit fixes and updates as well.

Right now we're still working on restoring the SVN from the mercurial commits we did. As soon as that's done, we can get your commits in.

Thanks!

Nicholas


On 7/1/11 2:43 PM, Eliza Chan wrote:
You are right, we should have a repository for the d2rmap source code. Since now you have a modified copy, and I also have one. It's better to merge them.


From: "Keese, Brian W" <bkeese@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Fri, 1 Jul 2011 14:37:11 -0400
To: VIVO Development List <vivo-dev-all@...>>
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Sure. Let me know how I can help with the docs.

For the bugs, one of them was fixed before (the oracle semi-colon thing, Map.java around line 94).

The other bug has to do with null values for instance identifiers. In Map.java around line 126, a list of new instances is being compiled. Unique instances are defined by values in a discreet set of fields. If one those values happens to be null, this line would give a null pointer exception:

if (!currentTuple.get(fieldName).equals(lastTuple.get(fieldName)))

So, I changed the logic to this:

if (!(currentTuple.get(fieldName)==null && lastTuple.get(fieldName)==null) &&
                      (currentTuple.get(fieldName)==null || !currentTuple.get(fieldName).equals(lastTuple.get(fieldName))))

I also added some logging info to help with diagnosing problems with the harvest. And I removed some logging info which seemed to me to be not useful.

We don’t have an svn repository for the d2rmap source code do we? Maybe we should set that up.

Brian

Brian Keese
Indiana University Digital Library Program
Herman B Wells Library W501
812-856-3865

From: Eliza Chan [mailto:elc2013@...]
Sent: Friday, July 01, 2011 2:22 PM
To: VIVO Development List
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Hi Brian,

Thanks for making the changes.
It'd be nice to have d2rmap harvests (db and csv) being supported in future releases, since there are people who use the tools on a regular basis.
I have been trying to update the documentation on d2rmap but it has not been able to keep up with so many changes with Harvester. :-(
Would you be able to document some of it? Also, would you be able to list the bugs that you fixed?

Thanks,
Eliza

From: "Keese, Brian W" <bkeese@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Fri, 1 Jul 2011 14:05:23 -0400
To: VIVO Development List <vivo-dev-all@...>>
Subject: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Nick, Thanks for the info on Oracle. Makes sense.

Here’s another thing. I have made a couple of changes to the d2rmap processor that is included with the harvester. I have uploaded the new jar file to https://www.slashtmp.iu.edu/files/download?FILE=bkeese%2F94328zp0Qy2 (the password is ‘d2r4vivo’).
It fixes a couple of bugs, and adds some status info to the logs for d2rmap fetch.

I noticed that there is no example script for d2rmap in the latest release. I hope support for d2r harvests will be continued.

Brian

Brian Keese
Indiana University Digital Library Program
Herman B Wells Library W501
812-856-3865

From: Nicholas Skaggs [mailto:nskaggs@...]
Sent: Friday, July 01, 2011 10:51 AM
To: vivo-dev-all@...>
Subject: Re: [Vivo-dev-all] Vivo development call / meeting reminder for today.

Great catch Brian. The trouble with oracle is that it requires a specific jdbc connector for each release -- including minor point release versions.

http://www.oracle.com/technetwork/database/features/jdbc/index-091264.html

We can't package every version of the connector with the harvester unfortunately. I have heard of a so called "universal" connector for oracle that connects to the databases through lowest common denominator SQL. I've never been able to find anything additional about this, nor a download for it -- so I can't verify it exists or not.  In addition, the connectors are released by Oracle under the Oracle Technology Network Development and Distribution License. This prevents us from distributing these files by packaging them directly without complying with the license terms. Specifically, the redistribution clause of the license causes trouble:

Program Distribution
We grant you a nonexclusive, nontransferable right to copy and distribute the programs to your end users provided that you do not charge your end users for use of the programs and provided your end users may only use the programs to run your applications for their business operations. Prior to distributing the programs you shall require your end users to execute an agreement binding them to terms consistent with those contained in this section and the sections of this agreement entitled "License Rights," "Ownership and Restrictions," "Export," "Disclaimer of Warranties and Exclusive Remedies," "No Technical Support," "End of Agreement," "Relationship Between the Parties," and "Open Source." You must also include a provision stating that your end users shall have no right to distribute the programs, and a provision specifying us as a third party beneficiary of the agreement. You are responsible for obtaining these agreements with your end users.

You can read the entire license here:

http://www.oracle.com/technetwork/licenses/distribution-license-152002.html

If you have an idea about how we can distribute something to help the harvester work out of the box with oracle, please let us know. Thanks,

Nicholas

On 7/1/11 10:39 AM, Keese, Brian W wrote:

The attachment didn't work. The driver can be downloaded from Oracle here:



http://www.oracle.com/technetwork/database/enterprise-edition/jdbc-10201-088211.html

Brian

________________________________________

From: Keese, Brian W [bkeese@...>]

Sent: Friday, July 01, 2011 10:36 AM

To: jrpence@...>; VIVO Development List

Subject: Re: [Vivo-dev-all] Vivo development call / meeting reminder for today.



Hey Harvester Guys,

I noticed that the oracle jdbc driver was never added (or was added, then removed) to the harvester bin/dependencies folder for the latest release. Could you add that in please?

It is ojdbc14.jar (attached to this email).



Thanks,

Brian





________________________________________

From: James Pence [jrpence@...>]

Sent: Thursday, June 30, 2011 1:23 PM

To: vivo-dev-all@...>

Subject: [Vivo-dev-all] Vivo development call / meeting reminder for today.



Hello,



    Today the vivo developers, and maybe you, are having a goto-meeting at 3:00pm EST (UTC-4 hours)



    We have the beginnings of an agenda at<http://sourceforge.net/apps/mediawiki/vivo/index.php?title=2011-06-30_Development_Leads_Call><http://sourceforge.net/apps/mediawiki/vivo/index.php?title=2011-06-30_Development_Leads_Call>



If you have any agenda items to add to the meeting please let me know (jrpence@...>   ).



   Since we only have 25 connections for the goto meeting please consolidate where possible.



   1. Please join my meeting, Thursday, June 30 at 3:00 PM Eastern DaylightTime.

   https://www1.gotomeeting.com/join/486839241



   2. Use your microphone and speakers (VoIP) - a headset is recommended.



   Or, call in using your telephone.

   Dial +1 (215) 383-1013



   Access Code: 486-839-241

   Audio PIN: Shown after joining the meeting

   Meeting ID: 486-839-241



   GoToMeeting®

   Online Meetings Made Easy™







--

James R. Pence

jrpence@...>

Software Engineer

Communicore C2-12

(352)294-5154 (ex. 45154)

Enterprise Software Engineering

University of Florida

Academic Health Center IT

Developer on the VIVO Grant





------------------------------------------------------------------------------

All of the data generated in your IT infrastructure is seriously valuable.

Why? It contains a definitive record of application performance, security

threats, fraudulent activity, and more. Splunk takes this data and makes

sense of it. IT sense. And common sense.

http://p.sf.net/sfu/splunk-d2d-c2

_______________________________________________

Vivo-dev-all mailing list

Vivo-dev-all@...>

https://lists.sourceforge.net/lists/listinfo/vivo-dev-all



------------------------------------------------------------------------------

All of the data generated in your IT infrastructure is seriously valuable.

Why? It contains a definitive record of application performance, security

threats, fraudulent activity, and more. Splunk takes this data and makes

sense of it. IT sense. And common sense.

http://p.sf.net/sfu/splunk-d2d-c2

_______________________________________________

Vivo-dev-all mailing list

Vivo-dev-all@...>

https://lists.sourceforge.net/lists/listinfo/vivo-dev-all

Hi Nicholas and Brian,

Changes have been committed to the d2rmap branch. A GPL/LGPL statement was added to D2rProcessor. Please make changes to the statement if needed, and copy to the other classes that have been modified. The revised classes are: D2rProcessor, Map, Instance, SourceControllerD2R.

Also, the d2rmap example in the master branch should work now. It's related to a bug in D2RMap that returned the column name (e.g. user_id) rather that the column label (e.g. UID) in a query with a "AS" clause (e.g. user_id AS UID). It seemed to affect H2 and not Oracle. Please give it a try and confirm.

Thanks,
Eliza

From: Nicholas Skaggs <nskaggs@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Tue, 19 Jul 2011 11:14:24 -0400
To: "vivo-dev-all@...>" <vivo-dev-all@...>>
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Eliza and Brian,
     we've created a d2rmap branch in the git repository for the harvester -- origin/D2rMapIntegration. It contains the latest code from the d2rmap repository. Can you two add your patchsets to this code to bring it up to date with what we want to ship with the harvester? We'll then need to update maven to build a jar for the harvester to use as part of the build for the harvester.

Secondly, we have the d2rmap example in the master branch on git. While the example doesn't give an error, no data is being added to VIVO. Can you take a look and figure out what's wrong? The team is ready to push out a 1.2 final of the harvester once this example is complete :-)

Now, onto your questions below:

1. I mean you should license your modifications to d2rmap as GPL/LGPL so that there project can accept them under than same license terms as d2rmap is licensed.

2. We are NOT required to followup with them, but I think it would be a good idea to let them know of our work. Once we have your code integrated, I would suggest sending the patchset to the d2rmap mailing list and/or letting them now we have changes in our git which are publically available.

3. The team seemed to favor including the code inline with the harvester, so no sub-project has been created.

4. Licensing file has been updated appropriately for this, so we're good.

5. I'm not sure where this is coming from, but making available the source, and making our changes available, as well as noting the license of the code as being GPL/LGPL meets the terms of the license.

Nicholas


On 7/8/11 11:26 AM, Eliza Chan wrote:
Hi Nicholas,

Please see my questions regarding your action items:

1. License modifications as GPL/LGPL

Did you mean adding GPL/LGPL to Licensing & Acknowledgements?
Or did you mean adding a license statement to the source code?

2. Push patchsets to D2RMap community

Under the license agreement, I don't see anywhere that we are required to push the patchsets back to the D2RMap community. Could someone confirm this please?
However, if we really want to check in the changes, Brian's patch seems to be more relevant because it's related to a bug fix. But my part was mainly for integrating D2RMap to VIVO, so I'm not sure if it's relevant to the D2RMap project.

3. Host our modified version of the code on SF (seperate sub-project?)

How can one create a sub-project in SF?

4. Use the modified binary in the harvester, noting the GPL license

Did you mean adding a LICENSE file somewhere in the Harvester source code?

5.
Under the "TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION",
section 6, paragraph b: You must cause the files modified to carry prominent notices stating that you changed the files and the date of any change.

Have we addressed this? I'm also not a lawyer. ;-)

Thanks,
Eliza

From: Nicholas Skaggs <nskaggs@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Thu, 7 Jul 2011 14:02:49 -0400
To: "vivo-dev-all@...>" <vivo-dev-all@...>>
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Given the shortcomings of email, I don't want to be mis-interpreted on my reply.

I would like to preface this email with "In my opinion", and "I think we should' :-)

Thanks,

Nicholas

On 7/7/11 1:58 PM, Nicholas Skaggs wrote:
Jim, yes we must consider these. For distributing a binary, we can simply disclose it's license as GPL, and do nothing more.  LGPL and GPL allow for different things in terms of linking and non-open source software, but we don't need to get into that here. BSD is "compatible" with the GPL/LGPL. In this case, Eliza/Brian has modified GPL/LGPL source (they appear to have it dual licensed). We can distribute this of course, but we do need to make those modifications available in source form upon request. This can be done thru writing and physical media (we can even charge the cost of distribution), etc, but the simplest and most common is to host the source in a freely downloadable place.

So, I didn't realize D2RMap was an existing SF project. We essentially have 2 options. We can fork there project and host it ourselves with our modifications, or we can contribute them back upstream and see if we can get them to adopt them. I would suggest starting with the latter. Depending on the outcome, in the meantime, we will be maintaining a "fork" of the project with our modifications as part of the harvester since we want to utilize it. Now, the best practice for Eliza and Brian would be to make there modifications back to the D2RMap project as GPL/LGPL (there current license format). If they licensed it as BSD or something else, the project would have part of the code under a different license which is no good, and would make it harder for them to accept.

The ramifications then for the harvester is we must ship the GPL license with our project and state that D2RMap is under that license. Not a big deal, it's an external library; but we should make note of it.

In summary we should:

License modifications as GPL/LGPL
Push patchsets to D2RMap community
Host our modified version of the code on SF (seperate sub-project?)
Use the modified binary in the harvester, noting the GPL license

Nicholas

On 7/7/11 1:50 PM, Eliza Chan wrote:
Hi Jim,

Thanks for looking up.
In this case, committing the D2RMap source code to our project is not only Ok, it's required.

Eliza

From: Jim Blake <jeb228@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Thu, 7 Jul 2011 13:41:20 -0400
To: VIVO Development List <vivo-dev-all@...>>
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Eliza;

That’s a very interesting question, and it opens an big can of worms.

I am not a lawyer, but it appears that D2RMap is covered under the GNU Lesser General Public License (LGPL), which is a modification of the GNU General Public License, which appears to cover this scenario in section 6, paragraph d.

If I understand this correctly, if you offer the object code (i.e., JAR files or CLASS files) for public download as part of your product, you are required to offer the source code (i.e. JAVA files) for public download on the same server, or an equivalent server operated by you or by a third party.

So if our software is available for download from SourceForge, and the D2RMap source code is available from SourceForge, have we satisfied the terms of the license?

Have I mentioned that I have no legal training or expertise?

I wonder whether the VIVO project is in violation of the terms of many additional open source licenses. For exmaple, some licenses require that we cite them in all of our end-user documentation. Do we do that?

But I’m just taking wild guesses, here. Can anybody shed more light on this?

Jim


From: Eliza Chan [mailto:elc2013@...]
Sent: Thursday, July 07, 2011 11:59 AM
To: VIVO Development List
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Hi Nicholas and Brian,

I can definitely commit D2RMap to sourceforge.
Just a policy question before doing that.
As you are probably aware, D2RMap is a sourceforge project: https://sourceforge.net/projects/d2r-map/
Do we need to do anything extra (whatever that is) before we post their modified copy under our project?

Thanks,
Eliza

From: Nicholas Skaggs <nskaggs@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Thu, 7 Jul 2011 11:10:13 -0400
To: "vivo-dev-all@...>" <vivo-dev-all@...>>
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Brian and Eliza,
    Let's get a merged copy of your code into the harvester repo. Brian, it seems your new jar is missing some of Eliza's code -- specifically the CSV Map:

The method processCsvMap(String, String, String) is undefined for the type D2rProcessor    D2RMapFetch.java    /Harvester-hg/src/main/java/org/vivoweb/harvester/fetch    line 113    Java Problem

Eliza, I know you have commit rights to SVN -- Brian do you? Eliza can you check in your source for D2RMap, and then have Brian add his changes in as another commit, etc. That would be really awesome. I'm sure others who are using it would be excited to be able to commit fixes and updates as well.

Right now we're still working on restoring the SVN from the mercurial commits we did. As soon as that's done, we can get your commits in.

Thanks!

Nicholas


On 7/1/11 2:43 PM, Eliza Chan wrote:
You are right, we should have a repository for the d2rmap source code. Since now you have a modified copy, and I also have one. It's better to merge them.


From: "Keese, Brian W" <bkeese@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Fri, 1 Jul 2011 14:37:11 -0400
To: VIVO Development List <vivo-dev-all@...>>
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Sure. Let me know how I can help with the docs.

For the bugs, one of them was fixed before (the oracle semi-colon thing, Map.java around line 94).

The other bug has to do with null values for instance identifiers. In Map.java around line 126, a list of new instances is being compiled. Unique instances are defined by values in a discreet set of fields. If one those values happens to be null, this line would give a null pointer exception:

if (!currentTuple.get(fieldName).equals(lastTuple.get(fieldName)))

So, I changed the logic to this:

if (!(currentTuple.get(fieldName)==null && lastTuple.get(fieldName)==null) &&
                      (currentTuple.get(fieldName)==null || !currentTuple.get(fieldName).equals(lastTuple.get(fieldName))))

I also added some logging info to help with diagnosing problems with the harvest. And I removed some logging info which seemed to me to be not useful.

We don’t have an svn repository for the d2rmap source code do we? Maybe we should set that up.

Brian

Brian Keese
Indiana University Digital Library Program
Herman B Wells Library W501
812-856-3865

From: Eliza Chan [mailto:elc2013@...]
Sent: Friday, July 01, 2011 2:22 PM
To: VIVO Development List
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Hi Brian,

Thanks for making the changes.
It'd be nice to have d2rmap harvests (db and csv) being supported in future releases, since there are people who use the tools on a regular basis.
I have been trying to update the documentation on d2rmap but it has not been able to keep up with so many changes with Harvester. :-(
Would you be able to document some of it? Also, would you be able to list the bugs that you fixed?

Thanks,
Eliza

From: "Keese, Brian W" <bkeese@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Fri, 1 Jul 2011 14:05:23 -0400
To: VIVO Development List <vivo-dev-all@...>>
Subject: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Nick, Thanks for the info on Oracle. Makes sense.

Here’s another thing. I have made a couple of changes to the d2rmap processor that is included with the harvester. I have uploaded the new jar file to https://www.slashtmp.iu.edu/files/download?FILE=bkeese%2F94328zp0Qy2 (the password is ‘d2r4vivo’).
It fixes a couple of bugs, and adds some status info to the logs for d2rmap fetch.

I noticed that there is no example script for d2rmap in the latest release. I hope support for d2r harvests will be continued.

Brian

Brian Keese
Indiana University Digital Library Program
Herman B Wells Library W501
812-856-3865

From: Nicholas Skaggs [mailto:nskaggs@...]
Sent: Friday, July 01, 2011 10:51 AM
To: vivo-dev-all@...>
Subject: Re: [Vivo-dev-all] Vivo development call / meeting reminder for today.

Great catch Brian. The trouble with oracle is that it requires a specific jdbc connector for each release -- including minor point release versions.

http://www.oracle.com/technetwork/database/features/jdbc/index-091264.html

We can't package every version of the connector with the harvester unfortunately. I have heard of a so called "universal" connector for oracle that connects to the databases through lowest common denominator SQL. I've never been able to find anything additional about this, nor a download for it -- so I can't verify it exists or not.  In addition, the connectors are released by Oracle under the Oracle Technology Network Development and Distribution License. This prevents us from distributing these files by packaging them directly without complying with the license terms. Specifically, the redistribution clause of the license causes trouble:

Program Distribution
We grant you a nonexclusive, nontransferable right to copy and distribute the programs to your end users provided that you do not charge your end users for use of the programs and provided your end users may only use the programs to run your applications for their business operations. Prior to distributing the programs you shall require your end users to execute an agreement binding them to terms consistent with those contained in this section and the sections of this agreement entitled "License Rights," "Ownership and Restrictions," "Export," "Disclaimer of Warranties and Exclusive Remedies," "No Technical Support," "End of Agreement," "Relationship Between the Parties," and "Open Source." You must also include a provision stating that your end users shall have no right to distribute the programs, and a provision specifying us as a third party beneficiary of the agreement. You are responsible for obtaining these agreements with your end users.

You can read the entire license here:

http://www.oracle.com/technetwork/licenses/distribution-license-152002.html

If you have an idea about how we can distribute something to help the harvester work out of the box with oracle, please let us know. Thanks,

Nicholas

On 7/1/11 10:39 AM, Keese, Brian W wrote:

The attachment didn't work. The driver can be downloaded from Oracle here:



http://www.oracle.com/technetwork/database/enterprise-edition/jdbc-10201-088211.html

Brian

________________________________________

From: Keese, Brian W [bkeese@...>]

Sent: Friday, July 01, 2011 10:36 AM

To: jrpence@...>; VIVO Development List

Subject: Re: [Vivo-dev-all] Vivo development call / meeting reminder for today.



Hey Harvester Guys,

I noticed that the oracle jdbc driver was never added (or was added, then removed) to the harvester bin/dependencies folder for the latest release. Could you add that in please?

It is ojdbc14.jar (attached to this email).



Thanks,

Brian





________________________________________

From: James Pence [jrpence@...>]

Sent: Thursday, June 30, 2011 1:23 PM

To: vivo-dev-all@...>

Subject: [Vivo-dev-all] Vivo development call / meeting reminder for today.



Hello,



    Today the vivo developers, and maybe you, are having a goto-meeting at 3:00pm EST (UTC-4 hours)



    We have the beginnings of an agenda at<http://sourceforge.net/apps/mediawiki/vivo/index.php?title=2011-06-30_Development_Leads_Call><http://sourceforge.net/apps/mediawiki/vivo/index.php?title=2011-06-30_Development_Leads_Call>



If you have any agenda items to add to the meeting please let me know (jrpence@...>   ).



   Since we only have 25 connections for the goto meeting please consolidate where possible.



   1. Please join my meeting, Thursday, June 30 at 3:00 PM Eastern DaylightTime.

   https://www1.gotomeeting.com/join/486839241



   2. Use your microphone and speakers (VoIP) - a headset is recommended.



   Or, call in using your telephone.

   Dial +1 (215) 383-1013



   Access Code: 486-839-241

   Audio PIN: Shown after joining the meeting

   Meeting ID: 486-839-241



   GoToMeeting®

   Online Meetings Made Easy™







--

James R. Pence

jrpence@...>

Software Engineer

Communicore C2-12

(352)294-5154 (ex. 45154)

Enterprise Software Engineering

University of Florida

Academic Health Center IT

Developer on the VIVO Grant





------------------------------------------------------------------------------

All of the data generated in your IT infrastructure is seriously valuable.

Why? It contains a definitive record of application performance, security

threats, fraudulent activity, and more. Splunk takes this data and makes

sense of it. IT sense. And common sense.

http://p.sf.net/sfu/splunk-d2d-c2

_______________________________________________

Vivo-dev-all mailing list

Vivo-dev-all@...>

https://lists.sourceforge.net/lists/listinfo/vivo-dev-all



------------------------------------------------------------------------------

All of the data generated in your IT infrastructure is seriously valuable.

Why? It contains a definitive record of application performance, security

threats, fraudulent activity, and more. Splunk takes this data and makes

sense of it. IT sense. And common sense.

http://p.sf.net/sfu/splunk-d2d-c2

_______________________________________________

Vivo-dev-all mailing list

Vivo-dev-all@...>

https://lists.sourceforge.net/lists/listinfo/vivo-dev-all

Eliza, I'm not seeing your commits in etheir branch. My apologies if 
they are there and I'm missing them. Did you push back upstream? Or did 
you just commit locally? I did try and remove h2 from the equation and 
just use the mysql database directly, but all i got out of the translate 
is the following:

<?xml version="1.0" encoding="UTF-8" ?><rdf:RDF
     xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
     xmlns:bibo="http://purl.org/ontology/bibo/"
     xmlns:foaf="http://xmlns.com/foaf/0.1/"
     xmlns:localVivo="http://vivo.sample.edu/ontology/"
     xmlns:owl="http://www.w3.org/2002/07/owl#"
     xmlns:xsd="http://www.w3.org/2001/XMLSchema#"
     xmlns:core="http://vivoweb.org/ontology/core#"
     xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#" >
<rdf:Description 
rdf:about="http://vivoweb.org/harvest/example/d2r/person/personnull">
<rdf:type rdf:resource="http://xmlns.com/foaf/0.1/Person"/>
</rdf:Description>
<rdf:Description 
rdf:about="http://vivoweb.org/harvest/example/d2r/org/deptnull">
<rdf:type rdf:resource="http://xmlns.com/foaf/0.1/Organization"/>
</rdf:Description>
</rdf:RDF>

Thanks for your help!

Nicholas

On 07/21/2011 11:08 AM, Eliza Chan wrote:
> Hi Nicholas and Brian,
>
> Changes have been committed to the d2rmap branch. A GPL/LGPL statement 
> was added to D2rProcessor. Please make changes to the statement if 
> needed, and copy to the other classes that have been modified. The 
> revised classes are: D2rProcessor, Map, Instance, SourceControllerD2R.
>
> Also, the d2rmap example in the master branch should work now. It's 
> related to a bug in D2RMap that returned the column name (e.g. 
> user_id) rather that the column label (e.g. UID) in a query with a 
> "AS" clause (e.g. user_id AS UID). It seemed to affect H2 and not 
> Oracle. Please give it a try and confirm.
>
> Thanks,
> Eliza
>
> From: Nicholas Skaggs <nskaggs@... <mailto:nskaggs@...>>
> Reply-To: VIVO Development List <vivo-dev-all@... 
> <mailto:vivo-dev-all@...>>
> Date: Tue, 19 Jul 2011 11:14:24 -0400
> To: "vivo-dev-all@... 
> <mailto:vivo-dev-all@...>" 
> <vivo-dev-all@... 
> <mailto:vivo-dev-all@...>>
> Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster
>
> Eliza and Brian,
>      we've created a d2rmap branch in the git repository for the 
> harvester -- origin/D2rMapIntegration. It contains the latest code 
> from the d2rmap repository. Can you two add your patchsets to this 
> code to bring it up to date with what we want to ship with the 
> harvester? We'll then need to update maven to build a jar for the 
> harvester to use as part of the build for the harvester.
>
> Secondly, we have the d2rmap example in the master branch on git. 
> While the example doesn't give an error, no data is being added to 
> VIVO. Can you take a look and figure out what's wrong? The team is 
> ready to push out a 1.2 final of the harvester once this example is 
> complete :-)
>
> Now, onto your questions below:
>
> 1. I mean you should license your modifications to d2rmap as GPL/LGPL 
> so that there project can accept them under than same license terms as 
> d2rmap is licensed.
>
> 2. We are NOT required to followup with them, but I think it would be 
> a good idea to let them know of our work. Once we have your code 
> integrated, I would suggest sending the patchset to the d2rmap mailing 
> list and/or letting them now we have changes in our git which are 
> publically available.
>
> 3. The team seemed to favor including the code inline with the 
> harvester, so no sub-project has been created.
>
> 4. Licensing file has been updated appropriately for this, so we're good.
>
> 5. I'm not sure where this is coming from, but making available the 
> source, and making our changes available, as well as noting the 
> license of the code as being GPL/LGPL meets the terms of the license.
>
> Nicholas
>
>
> On 7/8/11 11:26 AM, Eliza Chan wrote:
>> Hi Nicholas,
>>
>> Please see my questions regarding your action items:
>>
>> 1. License modifications as GPL/LGPL
>>
>> Did you mean adding GPL/LGPL to Licensing & Acknowledgements?
>> Or did you mean adding a license statement to the source code?
>>
>> 2. Push patchsets to D2RMap community
>>
>> Under the license agreement, I don't see anywhere that we are 
>> required to push the patchsets back to the D2RMap community. Could 
>> someone confirm this please?
>> However, if we really want to check in the changes, Brian's patch 
>> seems to be more relevant because it's related to a bug fix. But my 
>> part was mainly for integrating D2RMap to VIVO, so I'm not sure if 
>> it's relevant to the D2RMap project.
>>
>> 3. Host our modified version of the code on SF (seperate sub-project?)
>>
>> How can one create a sub-project in SF?
>>
>> 4. Use the modified binary in the harvester, noting the GPL license
>>
>> Did you mean adding a LICENSE file somewhere in the Harvester source 
>> code?
>>
>> 5.
>> Under the "TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND 
>> MODIFICATION",
>> section 6, paragraph b: You must cause the files modified to carry 
>> prominent notices stating that you changed the files and the date of 
>> any change.
>>
>> Have we addressed this? I'm also not a lawyer. ;-)
>>
>> Thanks,
>> Eliza
>>
>> From: Nicholas Skaggs <nskaggs@... <mailto:nskaggs@...>>
>> Reply-To: VIVO Development List <vivo-dev-all@... 
>> <mailto:vivo-dev-all@...>>
>> Date: Thu, 7 Jul 2011 14:02:49 -0400
>> To: "vivo-dev-all@... 
>> <mailto:vivo-dev-all@...>" 
>> <vivo-dev-all@... 
>> <mailto:vivo-dev-all@...>>
>> Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster
>>
>> Given the shortcomings of email, I don't want to be mis-interpreted 
>> on my reply.
>>
>> I would like to preface this email with "In my opinion", and "I think 
>> we should' :-)
>>
>> Thanks,
>>
>> Nicholas
>>
>> On 7/7/11 1:58 PM, Nicholas Skaggs wrote:
>>> Jim, yes we must consider these. For distributing a binary, we can 
>>> simply disclose it's license as GPL, and do nothing more.  LGPL and 
>>> GPL allow for different things in terms of linking and non-open 
>>> source software, but we don't need to get into that here. BSD is 
>>> "compatible" with the GPL/LGPL. In this case, Eliza/Brian has 
>>> modified GPL/LGPL source (they appear to have it dual licensed). We 
>>> can distribute this of course, but we do need to make those 
>>> modifications available in source form upon request. This can be 
>>> done thru writing and physical media (we can even charge the cost of 
>>> distribution), etc, but the simplest and most common is to host the 
>>> source in a freely downloadable place.
>>>
>>> So, I didn't realize D2RMap was an existing SF project. We 
>>> essentially have 2 options. We can fork there project and host it 
>>> ourselves with our modifications, or we can contribute them back 
>>> upstream and see if we can get them to adopt them. I would suggest 
>>> starting with the latter. Depending on the outcome, in the meantime, 
>>> we will be maintaining a "fork" of the project with our 
>>> modifications as part of the harvester since we want to utilize it. 
>>> Now, the best practice for Eliza and Brian would be to make there 
>>> modifications back to the D2RMap project as GPL/LGPL (there current 
>>> license format). If they licensed it as BSD or something else, the 
>>> project would have part of the code under a different license which 
>>> is no good, and would make it harder for them to accept.
>>>
>>> The ramifications then for the harvester is we must ship the GPL 
>>> license with our project and state that D2RMap is under that 
>>> license. Not a big deal, it's an external library; but we should 
>>> make note of it.
>>>
>>> In summary we should:
>>>
>>> License modifications as GPL/LGPL
>>> Push patchsets to D2RMap community
>>> Host our modified version of the code on SF (seperate sub-project?)
>>> Use the modified binary in the harvester, noting the GPL license
>>>
>>> Nicholas
>>>
>>> On 7/7/11 1:50 PM, Eliza Chan wrote:
>>>> Hi Jim,
>>>>
>>>> Thanks for looking up.
>>>> In this case, committing the D2RMap source code to our project is 
>>>> not only Ok, it's required.
>>>>
>>>> Eliza
>>>>
>>>> From: Jim Blake <jeb228@... <mailto:jeb228@...>>
>>>> Reply-To: VIVO Development List <vivo-dev-all@... 
>>>> <mailto:vivo-dev-all@...>>
>>>> Date: Thu, 7 Jul 2011 13:41:20 -0400
>>>> To: VIVO Development List <vivo-dev-all@... 
>>>> <mailto:vivo-dev-all@...>>
>>>> Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for 
>>>> harverster
>>>>
>>>> Eliza;
>>>>
>>>> That’s a very interesting question, and it opens an big can of worms.
>>>>
>>>> I am not a lawyer, but it appears that D2RMap is covered under the 
>>>> GNU Lesser General Public License (LGPL), which is a modification 
>>>> of the GNU General Public License, which appears to cover this 
>>>> scenario in section 6, paragraph d.
>>>>
>>>> If I understand this correctly, if you offer the object code (i.e., 
>>>> JAR files or CLASS files) for public download as part of your 
>>>> product, you are required to offer the source code (i.e. JAVA 
>>>> files) for public download on the same server, or an equivalent 
>>>> server operated by you or by a third party.
>>>>
>>>> So if our software is available for download from SourceForge, and 
>>>> the D2RMap source code is available from SourceForge, have we 
>>>> satisfied the terms of the license?
>>>>
>>>> Have I mentioned that I have no legal training or expertise?
>>>>
>>>> I wonder whether the VIVO project is in violation of the terms of 
>>>> many additional open source licenses. For exmaple, some licenses 
>>>> require that we cite them in all of our end-user documentation. Do 
>>>> we do that?
>>>>
>>>> But I’m just taking wild guesses, here. Can anybody shed more light 
>>>> on this?
>>>>
>>>> Jim
>>>>
>>>> *From:*Eliza Chan [mailto:elc2013@...]
>>>> *Sent:* Thursday, July 07, 2011 11:59 AM
>>>> *To:* VIVO Development List
>>>> *Subject:* Re: [Vivo-dev-all] d2rmap harvesting, was oracle for 
>>>> harverster
>>>>
>>>> Hi Nicholas and Brian,
>>>>
>>>> I can definitely commit D2RMap to sourceforge.
>>>>
>>>> Just a policy question before doing that.
>>>>
>>>> As you are probably aware, D2RMap is a sourceforge project: 
>>>> https://sourceforge.net/projects/d2r-map/
>>>>
>>>> Do we need to do anything extra (whatever that is) before we post 
>>>> their modified copy under our project?
>>>>
>>>> Thanks,
>>>>
>>>> Eliza
>>>>
>>>> *From: *Nicholas Skaggs <nskaggs@... <mailto:nskaggs@...>>
>>>> *Reply-To: *VIVO Development List 
>>>> <vivo-dev-all@... 
>>>> <mailto:vivo-dev-all@...>>
>>>> *Date: *Thu, 7 Jul 2011 11:10:13 -0400
>>>> *To: *"vivo-dev-all@... 
>>>> <mailto:vivo-dev-all@...>" 
>>>> <vivo-dev-all@... 
>>>> <mailto:vivo-dev-all@...>>
>>>> *Subject: *Re: [Vivo-dev-all] d2rmap harvesting, was oracle for 
>>>> harverster
>>>>
>>>> Brian and Eliza,
>>>>     Let's get a merged copy of your code into the harvester repo. 
>>>> Brian, it seems your new jar is missing some of Eliza's code -- 
>>>> specifically the CSV Map:
>>>>
>>>> The method processCsvMap(String, String, String) is undefined for 
>>>> the type D2rProcessor    D2RMapFetch.java    
>>>> /Harvester-hg/src/main/java/org/vivoweb/harvester/fetch    line 
>>>> 113    Java Problem
>>>>
>>>> Eliza, I know you have commit rights to SVN -- Brian do you? Eliza 
>>>> can you check in your source for D2RMap, and then have Brian add 
>>>> his changes in as another commit, etc. That would be really 
>>>> awesome. I'm sure others who are using it would be excited to be 
>>>> able to commit fixes and updates as well.
>>>>
>>>> Right now we're still working on restoring the SVN from the 
>>>> mercurial commits we did. As soon as that's done, we can get your 
>>>> commits in.
>>>>
>>>> Thanks!
>>>>
>>>> Nicholas
>>>>
>>>>
>>>> On 7/1/11 2:43 PM, Eliza Chan wrote:
>>>>
>>>> You are right, we should have a repository for the d2rmap source 
>>>> code. Since now you have a modified copy, and I also have one. It's 
>>>> better to merge them.
>>>>
>>>> *From: *"Keese, Brian W" <bkeese@... 
>>>> <mailto:bkeese@...>>
>>>> *Reply-To: *VIVO Development List 
>>>> <vivo-dev-all@... 
>>>> <mailto:vivo-dev-all@...>>
>>>> *Date: *Fri, 1 Jul 2011 14:37:11 -0400
>>>> *To: *VIVO Development List <vivo-dev-all@... 
>>>> <mailto:vivo-dev-all@...>>
>>>> *Subject: *Re: [Vivo-dev-all] d2rmap harvesting, was oracle for 
>>>> harverster
>>>>
>>>> Sure. Let me know how I can help with the docs.
>>>>
>>>> For the bugs, one of them was fixed before (the oracle semi-colon 
>>>> thing, Map.java around line 94).
>>>>
>>>> The other bug has to do with null values for instance identifiers. 
>>>> In Map.java around line 126, a list of new instances is being 
>>>> compiled. Unique instances are defined by values in a discreet set 
>>>> of fields. If one those values happens to be null, this line would 
>>>> give a null pointer exception:
>>>>
>>>> if (!currentTuple.get(fieldName).equals(lastTuple.get(fieldName)))
>>>>
>>>> So, I changed the logic to this:
>>>>
>>>> if (!(currentTuple.get(fieldName)==null && 
>>>> lastTuple.get(fieldName)==null) &&
>>>>
>>>>                       (currentTuple.get(fieldName)==null || 
>>>> !currentTuple.get(fieldName).equals(lastTuple.get(fieldName))))
>>>>
>>>> I also added some logging info to help with diagnosing problems 
>>>> with the harvest. And I removed some logging info which seemed to 
>>>> me to be not useful.
>>>>
>>>> We don’t have an svn repository for the d2rmap source code do we? 
>>>> Maybe we should set that up.
>>>>
>>>> Brian
>>>>
>>>> Brian Keese
>>>>
>>>> Indiana University Digital Library Program
>>>>
>>>> Herman B Wells Library W501
>>>>
>>>> 812-856-3865
>>>>
>>>> *From:*Eliza Chan [mailto:elc2013@...]
>>>> *Sent:* Friday, July 01, 2011 2:22 PM
>>>> *To:* VIVO Development List
>>>> *Subject:* Re: [Vivo-dev-all] d2rmap harvesting, was oracle for 
>>>> harverster
>>>>
>>>> Hi Brian,
>>>>
>>>> Thanks for making the changes.
>>>>
>>>> It'd be nice to have d2rmap harvests (db and csv) being supported 
>>>> in future releases, since there are people who use the tools on a 
>>>> regular basis.
>>>>
>>>> I have been trying to update the documentation on d2rmap but it has 
>>>> not been able to keep up with so many changes with Harvester. :-(
>>>>
>>>> Would you be able to document some of it? Also, would you be able 
>>>> to list the bugs that you fixed?
>>>>
>>>> Thanks,
>>>>
>>>> Eliza
>>>>
>>>> *From: *"Keese, Brian W" <bkeese@... 
>>>> <mailto:bkeese@...>>
>>>> *Reply-To: *VIVO Development List 
>>>> <vivo-dev-all@... 
>>>> <mailto:vivo-dev-all@...>>
>>>> *Date: *Fri, 1 Jul 2011 14:05:23 -0400
>>>> *To: *VIVO Development List <vivo-dev-all@... 
>>>> <mailto:vivo-dev-all@...>>
>>>> *Subject: *[Vivo-dev-all] d2rmap harvesting, was oracle for harverster
>>>>
>>>> Nick, Thanks for the info on Oracle. Makes sense.
>>>>
>>>> Here’s another thing. I have made a couple of changes to the d2rmap 
>>>> processor that is included with the harvester. I have uploaded the 
>>>> new jar file to 
>>>> https://www.slashtmp.iu.edu/files/download?FILE=bkeese%2F94328zp0Qy2 (the 
>>>> password is ‘d2r4vivo’).
>>>>
>>>> It fixes a couple of bugs, and adds some status info to the logs 
>>>> for d2rmap fetch.
>>>>
>>>> I noticed that there is no example script for d2rmap in the latest 
>>>> release. I hope support for d2r harvests will be continued.
>>>>
>>>> Brian
>>>>
>>>> Brian Keese
>>>>
>>>> Indiana University Digital Library Program
>>>>
>>>> Herman B Wells Library W501
>>>>
>>>> 812-856-3865
>>>>
>>>> *From:*Nicholas Skaggs [mailto:nskaggs@...]
>>>> *Sent:* Friday, July 01, 2011 10:51 AM
>>>> *To:* vivo-dev-all@... 
>>>> <mailto:vivo-dev-all@...>
>>>> *Subject:* Re: [Vivo-dev-all] Vivo development call / meeting 
>>>> reminder for today.
>>>>
>>>> Great catch Brian. The trouble with oracle is that it requires a 
>>>> specific jdbc connector for each release -- including minor point 
>>>> release versions.
>>>>
>>>> http://www.oracle.com/technetwork/database/features/jdbc/index-091264.html
>>>>
>>>> We can't package every version of the connector with the harvester 
>>>> unfortunately. I have heard of a so called "universal" connector 
>>>> for oracle that connects to the databases through lowest common 
>>>> denominator SQL. I've never been able to find anything additional 
>>>> about this, nor a download for it -- so I can't verify it exists or 
>>>> not.  In addition, the connectors are released by Oracle under the 
>>>> Oracle Technology Network Development and Distribution License. 
>>>> This prevents us from distributing these files by packaging them 
>>>> directly without complying with the license terms. Specifically, 
>>>> the redistribution clause of the license causes trouble:
>>>>
>>>> /Program Distribution
>>>> We grant you a nonexclusive, nontransferable right to copy and 
>>>> distribute the programs to your end users provided that you do not 
>>>> charge your end users for use of the programs and provided your end 
>>>> users may only use the programs to run your applications for their 
>>>> business operations. Prior to distributing the programs you shall 
>>>> require your end users to execute an agreement binding them to 
>>>> terms consistent with those contained in this section and the 
>>>> sections of this agreement entitled "License Rights," "Ownership 
>>>> and Restrictions," "Export," "Disclaimer of Warranties and 
>>>> Exclusive Remedies," "No Technical Support," "End of Agreement," 
>>>> "Relationship Between the Parties," and "Open Source." You must 
>>>> also include a provision stating that your end users shall have no 
>>>> right to distribute the programs, and a provision specifying us as 
>>>> a third party beneficiary of the agreement. You are responsible for 
>>>> obtaining these agreements with your end users./
>>>>
>>>> You can read the entire license here:
>>>>
>>>> http://www.oracle.com/technetwork/licenses/distribution-license-152002.html
>>>>
>>>> If you have an idea about how we can distribute something to help 
>>>> the harvester work out of the box with oracle, please let us know. 
>>>> Thanks,
>>>>
>>>> Nicholas
>>>>
>>>> On 7/1/11 10:39 AM, Keese, Brian W wrote:
>>>>
>>>> The attachment didn't work. The driver can be downloaded from Oracle here:
>>>>   
>>>> http://www.oracle.com/technetwork/database/enterprise-edition/jdbc-10201-088211.html
>>>> Brian
>>>> ________________________________________
>>>> From: Keese, Brian W [bkeese@...  <mailto:bkeese@...>]
>>>> Sent: Friday, July 01, 2011 10:36 AM
>>>> To:jrpence@...  <mailto:jrpence@...>; VIVO Development List
>>>> Subject: Re: [Vivo-dev-all] Vivo development call / meeting reminder for today.
>>>>   
>>>> Hey Harvester Guys,
>>>> I noticed that the oracle jdbc driver was never added (or was added, then removed) to the harvester bin/dependencies folder for the latest release. Could you add that in please?
>>>> It is ojdbc14.jar (attached to this email).
>>>>   
>>>> Thanks,
>>>> Brian
>>>>   
>>>>   
>>>> ________________________________________
>>>> From: James Pence [jrpence@...  <mailto:jrpence@...>]
>>>> Sent: Thursday, June 30, 2011 1:23 PM
>>>> To:vivo-dev-all@...  <mailto:vivo-dev-all@...>
>>>> Subject: [Vivo-dev-all] Vivo development call / meeting reminder for today.
>>>>   
>>>> Hello,
>>>>   
>>>>      Today the vivo developers, and maybe you, are having a goto-meeting at 3:00pm EST (UTC-4 hours)
>>>>   
>>>>      We have the beginnings of an agenda at<http://sourceforge.net/apps/mediawiki/vivo/index.php?title=2011-06-30_Development_Leads_Call>
>>>>   
>>>> If you have any agenda items to add to the meeting please let me know (jrpence@...  <mailto:jrpence@...>    ).
>>>>   
>>>>     Since we only have 25 connections for the goto meeting please consolidate where possible.
>>>>   
>>>>     1. Please join my meeting, Thursday, June 30 at 3:00 PM Eastern DaylightTime.
>>>>     https://www1.gotomeeting.com/join/486839241
>>>>   
>>>>     2. Use your microphone and speakers (VoIP) - a headset is recommended.
>>>>   
>>>>     Or, call in using your telephone.
>>>>     Dial +1 (215) 383-1013
>>>>   
>>>>     Access Code: 486-839-241
>>>>     Audio PIN: Shown after joining the meeting
>>>>     Meeting ID: 486-839-241
>>>>   
>>>>     GoToMeeting®
>>>>     Online Meetings Made Easy™
>>>>   
>>>>   
>>>>   
>>>> --
>>>> James R. Pence
>>>> jrpence@...  <mailto:jrpence@...>
>>>> Software Engineer
>>>> Communicore C2-12
>>>> (352)294-5154 (ex. 45154)
>>>> Enterprise Software Engineering
>>>> University of Florida
>>>> Academic Health Center IT
>>>> Developer on the VIVO Grant
>>>>   
>>>>   
>>>> ------------------------------------------------------------------------------
>>>> All of the data generated in your IT infrastructure is seriously valuable.
>>>> Why? It contains a definitive record of application performance, security
>>>> threats, fraudulent activity, and more. Splunk takes this data and makes
>>>> sense of it. IT sense. And common sense.
>>>> http://p.sf.net/sfu/splunk-d2d-c2
>>>> _______________________________________________
>>>> Vivo-dev-all mailing list
>>>> Vivo-dev-all@...  <mailto:Vivo-dev-all@...>
>>>> https://lists.sourceforge.net/lists/listinfo/vivo-dev-all
>>>>   
>>>> ------------------------------------------------------------------------------
>>>> All of the data generated in your IT infrastructure is seriously valuable.
>>>> Why? It contains a definitive record of application performance, security
>>>> threats, fraudulent activity, and more. Splunk takes this data and makes
>>>> sense of it. IT sense. And common sense.
>>>> http://p.sf.net/sfu/splunk-d2d-c2
>>>> _______________________________________________
>>>> Vivo-dev-all mailing list
>>>> Vivo-dev-all@...  <mailto:Vivo-dev-all@...>
>>>> https://lists.sourceforge.net/lists/listinfo/vivo-dev-all
>>>>
>>>
>>
>

Sorry, it should be there now.
There is no need to change the d2rmap.config.xml file. It should work with the fixes.
Feel free to revert back to H2.

Thanks,
Eliza

From: Nicholas Skaggs <nskaggs@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Thu, 21 Jul 2011 11:59:27 -0400
To: "vivo-dev-all@...>" <vivo-dev-all@...>>
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Eliza, I'm not seeing your commits in etheir branch. My apologies if they are there and I'm missing them. Did you push back upstream? Or did you just commit locally? I did try and remove h2 from the equation and just use the mysql database directly, but all i got out of the translate is the following:

<?xml version="1.0" encoding="UTF-8" ?><rdf:RDF
    xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"<http://www.w3.org/1999/02/22-rdf-syntax-ns#>
    xmlns:bibo="http://purl.org/ontology/bibo/"<http://purl.org/ontology/bibo/>
    xmlns:foaf="http://xmlns.com/foaf/0.1/"<http://xmlns.com/foaf/0.1/>
    xmlns:localVivo="http://vivo.sample.edu/ontology/"<http://vivo.sample.edu/ontology/>
    xmlns:owl="http://www.w3.org/2002/07/owl#"<http://www.w3.org/2002/07/owl#>
    xmlns:xsd="http://www.w3.org/2001/XMLSchema#"<http://www.w3.org/2001/XMLSchema#>
    xmlns:core="http://vivoweb.org/ontology/core#"<http://vivoweb.org/ontology/core#>
    xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#"<http://www.w3.org/2000/01/rdf-schema#> >
  <rdf:Description rdf:about="http://vivoweb.org/harvest/example/d2r/person/personnull"<http://vivoweb.org/harvest/example/d2r/person/personnull>>
    <rdf:type rdf:resource="http://xmlns.com/foaf/0.1/Person"<http://xmlns.com/foaf/0.1/Person>/>
  </rdf:Description>
  <rdf:Description rdf:about="http://vivoweb.org/harvest/example/d2r/org/deptnull"<http://vivoweb.org/harvest/example/d2r/org/deptnull>>
    <rdf:type rdf:resource="http://xmlns.com/foaf/0.1/Organization"<http://xmlns.com/foaf/0.1/Organization>/>
  </rdf:Description>
</rdf:RDF>

Thanks for your help!

Nicholas

On 07/21/2011 11:08 AM, Eliza Chan wrote:
Hi Nicholas and Brian,

Changes have been committed to the d2rmap branch. A GPL/LGPL statement was added to D2rProcessor. Please make changes to the statement if needed, and copy to the other classes that have been modified. The revised classes are: D2rProcessor, Map, Instance, SourceControllerD2R.

Also, the d2rmap example in the master branch should work now. It's related to a bug in D2RMap that returned the column name (e.g. user_id) rather that the column label (e.g. UID) in a query with a "AS" clause (e.g. user_id AS UID). It seemed to affect H2 and not Oracle. Please give it a try and confirm.

Thanks,
Eliza

From: Nicholas Skaggs <nskaggs@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Tue, 19 Jul 2011 11:14:24 -0400
To: "vivo-dev-all@...>" <vivo-dev-all@...>>
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Eliza and Brian,
     we've created a d2rmap branch in the git repository for the harvester -- origin/D2rMapIntegration. It contains the latest code from the d2rmap repository. Can you two add your patchsets to this code to bring it up to date with what we want to ship with the harvester? We'll then need to update maven to build a jar for the harvester to use as part of the build for the harvester.

Secondly, we have the d2rmap example in the master branch on git. While the example doesn't give an error, no data is being added to VIVO. Can you take a look and figure out what's wrong? The team is ready to push out a 1.2 final of the harvester once this example is complete :-)

Now, onto your questions below:

1. I mean you should license your modifications to d2rmap as GPL/LGPL so that there project can accept them under than same license terms as d2rmap is licensed.

2. We are NOT required to followup with them, but I think it would be a good idea to let them know of our work. Once we have your code integrated, I would suggest sending the patchset to the d2rmap mailing list and/or letting them now we have changes in our git which are publically available.

3. The team seemed to favor including the code inline with the harvester, so no sub-project has been created.

4. Licensing file has been updated appropriately for this, so we're good.

5. I'm not sure where this is coming from, but making available the source, and making our changes available, as well as noting the license of the code as being GPL/LGPL meets the terms of the license.

Nicholas


On 7/8/11 11:26 AM, Eliza Chan wrote:
Hi Nicholas,

Please see my questions regarding your action items:

1. License modifications as GPL/LGPL

Did you mean adding GPL/LGPL to Licensing & Acknowledgements?
Or did you mean adding a license statement to the source code?

2. Push patchsets to D2RMap community

Under the license agreement, I don't see anywhere that we are required to push the patchsets back to the D2RMap community. Could someone confirm this please?
However, if we really want to check in the changes, Brian's patch seems to be more relevant because it's related to a bug fix. But my part was mainly for integrating D2RMap to VIVO, so I'm not sure if it's relevant to the D2RMap project.

3. Host our modified version of the code on SF (seperate sub-project?)

How can one create a sub-project in SF?

4. Use the modified binary in the harvester, noting the GPL license

Did you mean adding a LICENSE file somewhere in the Harvester source code?

5.
Under the "TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION",
section 6, paragraph b: You must cause the files modified to carry prominent notices stating that you changed the files and the date of any change.

Have we addressed this? I'm also not a lawyer. ;-)

Thanks,
Eliza

From: Nicholas Skaggs <nskaggs@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Thu, 7 Jul 2011 14:02:49 -0400
To: "vivo-dev-all@...>" <vivo-dev-all@...>>
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Given the shortcomings of email, I don't want to be mis-interpreted on my reply.

I would like to preface this email with "In my opinion", and "I think we should' :-)

Thanks,

Nicholas

On 7/7/11 1:58 PM, Nicholas Skaggs wrote:
Jim, yes we must consider these. For distributing a binary, we can simply disclose it's license as GPL, and do nothing more.  LGPL and GPL allow for different things in terms of linking and non-open source software, but we don't need to get into that here. BSD is "compatible" with the GPL/LGPL. In this case, Eliza/Brian has modified GPL/LGPL source (they appear to have it dual licensed). We can distribute this of course, but we do need to make those modifications available in source form upon request. This can be done thru writing and physical media (we can even charge the cost of distribution), etc, but the simplest and most common is to host the source in a freely downloadable place.

So, I didn't realize D2RMap was an existing SF project. We essentially have 2 options. We can fork there project and host it ourselves with our modifications, or we can contribute them back upstream and see if we can get them to adopt them. I would suggest starting with the latter. Depending on the outcome, in the meantime, we will be maintaining a "fork" of the project with our modifications as part of the harvester since we want to utilize it. Now, the best practice for Eliza and Brian would be to make there modifications back to the D2RMap project as GPL/LGPL (there current license format). If they licensed it as BSD or something else, the project would have part of the code under a different license which is no good, and would make it harder for them to accept.

The ramifications then for the harvester is we must ship the GPL license with our project and state that D2RMap is under that license. Not a big deal, it's an external library; but we should make note of it.

In summary we should:

License modifications as GPL/LGPL
Push patchsets to D2RMap community
Host our modified version of the code on SF (seperate sub-project?)
Use the modified binary in the harvester, noting the GPL license

Nicholas

On 7/7/11 1:50 PM, Eliza Chan wrote:
Hi Jim,

Thanks for looking up.
In this case, committing the D2RMap source code to our project is not only Ok, it's required.

Eliza

From: Jim Blake <jeb228@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Thu, 7 Jul 2011 13:41:20 -0400
To: VIVO Development List <vivo-dev-all@...>>
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Eliza;

That’s a very interesting question, and it opens an big can of worms.

I am not a lawyer, but it appears that D2RMap is covered under the GNU Lesser General Public License (LGPL), which is a modification of the GNU General Public License, which appears to cover this scenario in section 6, paragraph d.

If I understand this correctly, if you offer the object code (i.e., JAR files or CLASS files) for public download as part of your product, you are required to offer the source code (i.e. JAVA files) for public download on the same server, or an equivalent server operated by you or by a third party.

So if our software is available for download from SourceForge, and the D2RMap source code is available from SourceForge, have we satisfied the terms of the license?

Have I mentioned that I have no legal training or expertise?

I wonder whether the VIVO project is in violation of the terms of many additional open source licenses. For exmaple, some licenses require that we cite them in all of our end-user documentation. Do we do that?

But I’m just taking wild guesses, here. Can anybody shed more light on this?

Jim


From: Eliza Chan [mailto:elc2013@...]
Sent: Thursday, July 07, 2011 11:59 AM
To: VIVO Development List
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Hi Nicholas and Brian,

I can definitely commit D2RMap to sourceforge.
Just a policy question before doing that.
As you are probably aware, D2RMap is a sourceforge project: https://sourceforge.net/projects/d2r-map/
Do we need to do anything extra (whatever that is) before we post their modified copy under our project?

Thanks,
Eliza

From: Nicholas Skaggs <nskaggs@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Thu, 7 Jul 2011 11:10:13 -0400
To: "vivo-dev-all@...>" <vivo-dev-all@...>>
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Brian and Eliza,
    Let's get a merged copy of your code into the harvester repo. Brian, it seems your new jar is missing some of Eliza's code -- specifically the CSV Map:

The method processCsvMap(String, String, String) is undefined for the type D2rProcessor    D2RMapFetch.java    /Harvester-hg/src/main/java/org/vivoweb/harvester/fetch    line 113    Java Problem

Eliza, I know you have commit rights to SVN -- Brian do you? Eliza can you check in your source for D2RMap, and then have Brian add his changes in as another commit, etc. That would be really awesome. I'm sure others who are using it would be excited to be able to commit fixes and updates as well.

Right now we're still working on restoring the SVN from the mercurial commits we did. As soon as that's done, we can get your commits in.

Thanks!

Nicholas


On 7/1/11 2:43 PM, Eliza Chan wrote:
You are right, we should have a repository for the d2rmap source code. Since now you have a modified copy, and I also have one. It's better to merge them.


From: "Keese, Brian W" <bkeese@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Fri, 1 Jul 2011 14:37:11 -0400
To: VIVO Development List <vivo-dev-all@...>>
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Sure. Let me know how I can help with the docs.

For the bugs, one of them was fixed before (the oracle semi-colon thing, Map.java around line 94).

The other bug has to do with null values for instance identifiers. In Map.java around line 126, a list of new instances is being compiled. Unique instances are defined by values in a discreet set of fields. If one those values happens to be null, this line would give a null pointer exception:

if (!currentTuple.get(fieldName).equals(lastTuple.get(fieldName)))

So, I changed the logic to this:

if (!(currentTuple.get(fieldName)==null && lastTuple.get(fieldName)==null) &&
                      (currentTuple.get(fieldName)==null || !currentTuple.get(fieldName).equals(lastTuple.get(fieldName))))

I also added some logging info to help with diagnosing problems with the harvest. And I removed some logging info which seemed to me to be not useful.

We don’t have an svn repository for the d2rmap source code do we? Maybe we should set that up.

Brian

Brian Keese
Indiana University Digital Library Program
Herman B Wells Library W501
812-856-3865

From: Eliza Chan [mailto:elc2013@...]
Sent: Friday, July 01, 2011 2:22 PM
To: VIVO Development List
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Hi Brian,

Thanks for making the changes.
It'd be nice to have d2rmap harvests (db and csv) being supported in future releases, since there are people who use the tools on a regular basis.
I have been trying to update the documentation on d2rmap but it has not been able to keep up with so many changes with Harvester. :-(
Would you be able to document some of it? Also, would you be able to list the bugs that you fixed?

Thanks,
Eliza

From: "Keese, Brian W" <bkeese@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Fri, 1 Jul 2011 14:05:23 -0400
To: VIVO Development List <vivo-dev-all@...>>
Subject: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Nick, Thanks for the info on Oracle. Makes sense.

Here’s another thing. I have made a couple of changes to the d2rmap processor that is included with the harvester. I have uploaded the new jar file to https://www.slashtmp.iu.edu/files/download?FILE=bkeese%2F94328zp0Qy2 (the password is ‘d2r4vivo’).
It fixes a couple of bugs, and adds some status info to the logs for d2rmap fetch.

I noticed that there is no example script for d2rmap in the latest release. I hope support for d2r harvests will be continued.

Brian

Brian Keese
Indiana University Digital Library Program
Herman B Wells Library W501
812-856-3865

From: Nicholas Skaggs [mailto:nskaggs@...]
Sent: Friday, July 01, 2011 10:51 AM
To: vivo-dev-all@...>
Subject: Re: [Vivo-dev-all] Vivo development call / meeting reminder for today.

Great catch Brian. The trouble with oracle is that it requires a specific jdbc connector for each release -- including minor point release versions.

http://www.oracle.com/technetwork/database/features/jdbc/index-091264.html

We can't package every version of the connector with the harvester unfortunately. I have heard of a so called "universal" connector for oracle that connects to the databases through lowest common denominator SQL. I've never been able to find anything additional about this, nor a download for it -- so I can't verify it exists or not.  In addition, the connectors are released by Oracle under the Oracle Technology Network Development and Distribution License. This prevents us from distributing these files by packaging them directly without complying with the license terms. Specifically, the redistribution clause of the license causes trouble:

Program Distribution
We grant you a nonexclusive, nontransferable right to copy and distribute the programs to your end users provided that you do not charge your end users for use of the programs and provided your end users may only use the programs to run your applications for their business operations. Prior to distributing the programs you shall require your end users to execute an agreement binding them to terms consistent with those contained in this section and the sections of this agreement entitled "License Rights," "Ownership and Restrictions," "Export," "Disclaimer of Warranties and Exclusive Remedies," "No Technical Support," "End of Agreement," "Relationship Between the Parties," and "Open Source." You must also include a provision stating that your end users shall have no right to distribute the programs, and a provision specifying us as a third party beneficiary of the agreement. You are responsible for obtaining these agreements with your end users.

You can read the entire license here:

http://www.oracle.com/technetwork/licenses/distribution-license-152002.html

If you have an idea about how we can distribute something to help the harvester work out of the box with oracle, please let us know. Thanks,

Nicholas

On 7/1/11 10:39 AM, Keese, Brian W wrote:

The attachment didn't work. The driver can be downloaded from Oracle here:



http://www.oracle.com/technetwork/database/enterprise-edition/jdbc-10201-088211.html

Brian

________________________________________

From: Keese, Brian W [bkeese@...>]

Sent: Friday, July 01, 2011 10:36 AM

To: jrpence@...>; VIVO Development List

Subject: Re: [Vivo-dev-all] Vivo development call / meeting reminder for today.



Hey Harvester Guys,

I noticed that the oracle jdbc driver was never added (or was added, then removed) to the harvester bin/dependencies folder for the latest release. Could you add that in please?

It is ojdbc14.jar (attached to this email).



Thanks,

Brian





________________________________________

From: James Pence [jrpence@...>]

Sent: Thursday, June 30, 2011 1:23 PM

To: vivo-dev-all@...>

Subject: [Vivo-dev-all] Vivo development call / meeting reminder for today.



Hello,



    Today the vivo developers, and maybe you, are having a goto-meeting at 3:00pm EST (UTC-4 hours)



    We have the beginnings of an agenda at<http://sourceforge.net/apps/mediawiki/vivo/index.php?title=2011-06-30_Development_Leads_Call><http://sourceforge.net/apps/mediawiki/vivo/index.php?title=2011-06-30_Development_Leads_Call>



If you have any agenda items to add to the meeting please let me know (jrpence@...>   ).



   Since we only have 25 connections for the goto meeting please consolidate where possible.



   1. Please join my meeting, Thursday, June 30 at 3:00 PM Eastern DaylightTime.

   https://www1.gotomeeting.com/join/486839241



   2. Use your microphone and speakers (VoIP) - a headset is recommended.



   Or, call in using your telephone.

   Dial +1 (215) 383-1013



   Access Code: 486-839-241

   Audio PIN: Shown after joining the meeting

   Meeting ID: 486-839-241



   GoToMeeting®

   Online Meetings Made Easy™







--

James R. Pence

jrpence@...>

Software Engineer

Communicore C2-12

(352)294-5154 (ex. 45154)

Enterprise Software Engineering

University of Florida

Academic Health Center IT

Developer on the VIVO Grant





------------------------------------------------------------------------------

All of the data generated in your IT infrastructure is seriously valuable.

Why? It contains a definitive record of application performance, security

threats, fraudulent activity, and more. Splunk takes this data and makes

sense of it. IT sense. And common sense.

http://p.sf.net/sfu/splunk-d2d-c2

_______________________________________________

Vivo-dev-all mailing list

Vivo-dev-all@...>

https://lists.sourceforge.net/lists/listinfo/vivo-dev-all



------------------------------------------------------------------------------

All of the data generated in your IT infrastructure is seriously valuable.

Why? It contains a definitive record of application performance, security

threats, fraudulent activity, and more. Splunk takes this data and makes

sense of it. IT sense. And common sense.

http://p.sf.net/sfu/splunk-d2d-c2

_______________________________________________

Vivo-dev-all mailing list

Vivo-dev-all@...>

https://lists.sourceforge.net/lists/listinfo/vivo-dev-all

I have committed my bug fixes to the D2r code on git. Should be good to go.

Brian

Brian Keese
Indiana University Digital Library Program
Herman B Wells Library W501
812-856-3865

From: Eliza Chan [mailto:elc2013@...]
Sent: Thursday, July 21, 2011 12:14 PM
To: VIVO Development List
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster


Sorry, it should be there now.
There is no need to change the d2rmap.config.xml file. It should work with the fixes.
Feel free to revert back to H2.

Thanks,
Eliza

From: Nicholas Skaggs <nskaggs@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Thu, 21 Jul 2011 11:59:27 -0400
To: "vivo-dev-all@...>" <vivo-dev-all@...>>
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Eliza, I'm not seeing your commits in etheir branch. My apologies if they are there and I'm missing them. Did you push back upstream? Or did you just commit locally? I did try and remove h2 from the equation and just use the mysql database directly, but all i got out of the translate is the following:

<?xml version="1.0" encoding="UTF-8" ?><rdf:RDF
    xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"<http://www.w3.org/1999/02/22-rdf-syntax-ns>
    xmlns:bibo="http://purl.org/ontology/bibo/"<http://purl.org/ontology/bibo/>
    xmlns:foaf="http://xmlns.com/foaf/0.1/"<http://xmlns.com/foaf/0.1/>
    xmlns:localVivo="http://vivo.sample.edu/ontology/"<http://vivo.sample.edu/ontology/>
    xmlns:owl="http://www.w3.org/2002/07/owl#"<http://www.w3.org/2002/07/owl>
    xmlns:xsd="http://www.w3.org/2001/XMLSchema#"<http://www.w3.org/2001/XMLSchema>
    xmlns:core="http://vivoweb.org/ontology/core#"<http://vivoweb.org/ontology/core>
    xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#"<http://www.w3.org/2000/01/rdf-schema> >
  <rdf:Description rdf:about="http://vivoweb.org/harvest/example/d2r/person/personnull"<http://vivoweb.org/harvest/example/d2r/person/personnull>>
    <rdf:type rdf:resource="http://xmlns.com/foaf/0.1/Person"<http://xmlns.com/foaf/0.1/Person>/>
  </rdf:Description>
  <rdf:Description rdf:about="http://vivoweb.org/harvest/example/d2r/org/deptnull"<http://vivoweb.org/harvest/example/d2r/org/deptnull>>
    <rdf:type rdf:resource="http://xmlns.com/foaf/0.1/Organization"<http://xmlns.com/foaf/0.1/Organization>/>
  </rdf:Description>
</rdf:RDF>

Thanks for your help!

Nicholas

On 07/21/2011 11:08 AM, Eliza Chan wrote:
Hi Nicholas and Brian,

Changes have been committed to the d2rmap branch. A GPL/LGPL statement was added to D2rProcessor. Please make changes to the statement if needed, and copy to the other classes that have been modified. The revised classes are: D2rProcessor, Map, Instance, SourceControllerD2R.

Also, the d2rmap example in the master branch should work now. It's related to a bug in D2RMap that returned the column name (e.g. user_id) rather that the column label (e.g. UID) in a query with a "AS" clause (e.g. user_id AS UID). It seemed to affect H2 and not Oracle. Please give it a try and confirm.

Thanks,
Eliza

From: Nicholas Skaggs <nskaggs@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Tue, 19 Jul 2011 11:14:24 -0400
To: "vivo-dev-all@...>" <vivo-dev-all@...>>
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Eliza and Brian,
     we've created a d2rmap branch in the git repository for the harvester -- origin/D2rMapIntegration. It contains the latest code from the d2rmap repository. Can you two add your patchsets to this code to bring it up to date with what we want to ship with the harvester? We'll then need to update maven to build a jar for the harvester to use as part of the build for the harvester.

Secondly, we have the d2rmap example in the master branch on git. While the example doesn't give an error, no data is being added to VIVO. Can you take a look and figure out what's wrong? The team is ready to push out a 1.2 final of the harvester once this example is complete :-)

Now, onto your questions below:

1. I mean you should license your modifications to d2rmap as GPL/LGPL so that there project can accept them under than same license terms as d2rmap is licensed.

2. We are NOT required to followup with them, but I think it would be a good idea to let them know of our work. Once we have your code integrated, I would suggest sending the patchset to the d2rmap mailing list and/or letting them now we have changes in our git which are publically available.

3. The team seemed to favor including the code inline with the harvester, so no sub-project has been created.

4. Licensing file has been updated appropriately for this, so we're good.

5. I'm not sure where this is coming from, but making available the source, and making our changes available, as well as noting the license of the code as being GPL/LGPL meets the terms of the license.

Nicholas


On 7/8/11 11:26 AM, Eliza Chan wrote:
Hi Nicholas,

Please see my questions regarding your action items:

1. License modifications as GPL/LGPL

Did you mean adding GPL/LGPL to Licensing & Acknowledgements?
Or did you mean adding a license statement to the source code?

2. Push patchsets to D2RMap community

Under the license agreement, I don't see anywhere that we are required to push the patchsets back to the D2RMap community. Could someone confirm this please?
However, if we really want to check in the changes, Brian's patch seems to be more relevant because it's related to a bug fix. But my part was mainly for integrating D2RMap to VIVO, so I'm not sure if it's relevant to the D2RMap project.

3. Host our modified version of the code on SF (seperate sub-project?)

How can one create a sub-project in SF?

4. Use the modified binary in the harvester, noting the GPL license

Did you mean adding a LICENSE file somewhere in the Harvester source code?

5.
Under the "TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION",
section 6, paragraph b: You must cause the files modified to carry prominent notices stating that you changed the files and the date of any change.

Have we addressed this? I'm also not a lawyer. ;-)

Thanks,
Eliza

From: Nicholas Skaggs <nskaggs@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Thu, 7 Jul 2011 14:02:49 -0400
To: "vivo-dev-all@...>" <vivo-dev-all@...>>
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Given the shortcomings of email, I don't want to be mis-interpreted on my reply.

I would like to preface this email with "In my opinion", and "I think we should' :-)

Thanks,

Nicholas

On 7/7/11 1:58 PM, Nicholas Skaggs wrote:
Jim, yes we must consider these. For distributing a binary, we can simply disclose it's license as GPL, and do nothing more.  LGPL and GPL allow for different things in terms of linking and non-open source software, but we don't need to get into that here. BSD is "compatible" with the GPL/LGPL. In this case, Eliza/Brian has modified GPL/LGPL source (they appear to have it dual licensed). We can distribute this of course, but we do need to make those modifications available in source form upon request. This can be done thru writing and physical media (we can even charge the cost of distribution), etc, but the simplest and most common is to host the source in a freely downloadable place.

So, I didn't realize D2RMap was an existing SF project. We essentially have 2 options. We can fork there project and host it ourselves with our modifications, or we can contribute them back upstream and see if we can get them to adopt them. I would suggest starting with the latter. Depending on the outcome, in the meantime, we will be maintaining a "fork" of the project with our modifications as part of the harvester since we want to utilize it. Now, the best practice for Eliza and Brian would be to make there modifications back to the D2RMap project as GPL/LGPL (there current license format). If they licensed it as BSD or something else, the project would have part of the code under a different license which is no good, and would make it harder for them to accept.

The ramifications then for the harvester is we must ship the GPL license with our project and state that D2RMap is under that license. Not a big deal, it's an external library; but we should make note of it.

In summary we should:

License modifications as GPL/LGPL
Push patchsets to D2RMap community
Host our modified version of the code on SF (seperate sub-project?)
Use the modified binary in the harvester, noting the GPL license

Nicholas

On 7/7/11 1:50 PM, Eliza Chan wrote:
Hi Jim,

Thanks for looking up.
In this case, committing the D2RMap source code to our project is not only Ok, it's required.

Eliza

From: Jim Blake <jeb228@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Thu, 7 Jul 2011 13:41:20 -0400
To: VIVO Development List <vivo-dev-all@...>>
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Eliza;

That's a very interesting question, and it opens an big can of worms.

I am not a lawyer, but it appears that D2RMap is covered under the GNU Lesser General Public License (LGPL), which is a modification of the GNU General Public License, which appears to cover this scenario in section 6, paragraph d.

If I understand this correctly, if you offer the object code (i.e., JAR files or CLASS files) for public download as part of your product, you are required to offer the source code (i.e. JAVA files) for public download on the same server, or an equivalent server operated by you or by a third party.

So if our software is available for download from SourceForge, and the D2RMap source code is available from SourceForge, have we satisfied the terms of the license?

Have I mentioned that I have no legal training or expertise?

I wonder whether the VIVO project is in violation of the terms of many additional open source licenses. For exmaple, some licenses require that we cite them in all of our end-user documentation. Do we do that?

But I'm just taking wild guesses, here. Can anybody shed more light on this?

Jim


From: Eliza Chan [mailto:elc2013@...]
Sent: Thursday, July 07, 2011 11:59 AM
To: VIVO Development List
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Hi Nicholas and Brian,

I can definitely commit D2RMap to sourceforge.
Just a policy question before doing that.
As you are probably aware, D2RMap is a sourceforge project: https://sourceforge.net/projects/d2r-map/
Do we need to do anything extra (whatever that is) before we post their modified copy under our project?

Thanks,
Eliza

From: Nicholas Skaggs <nskaggs@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Thu, 7 Jul 2011 11:10:13 -0400
To: "vivo-dev-all@...>" <vivo-dev-all@...>>
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Brian and Eliza,
    Let's get a merged copy of your code into the harvester repo. Brian, it seems your new jar is missing some of Eliza's code -- specifically the CSV Map:

The method processCsvMap(String, String, String) is undefined for the type D2rProcessor    D2RMapFetch.java    /Harvester-hg/src/main/java/org/vivoweb/harvester/fetch    line 113    Java Problem

Eliza, I know you have commit rights to SVN -- Brian do you? Eliza can you check in your source for D2RMap, and then have Brian add his changes in as another commit, etc. That would be really awesome. I'm sure others who are using it would be excited to be able to commit fixes and updates as well.

Right now we're still working on restoring the SVN from the mercurial commits we did. As soon as that's done, we can get your commits in.

Thanks!

Nicholas


On 7/1/11 2:43 PM, Eliza Chan wrote:
You are right, we should have a repository for the d2rmap source code. Since now you have a modified copy, and I also have one. It's better to merge them.


From: "Keese, Brian W" <bkeese@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Fri, 1 Jul 2011 14:37:11 -0400
To: VIVO Development List <vivo-dev-all@...>>
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Sure. Let me know how I can help with the docs.

For the bugs, one of them was fixed before (the oracle semi-colon thing, Map.java around line 94).

The other bug has to do with null values for instance identifiers. In Map.java around line 126, a list of new instances is being compiled. Unique instances are defined by values in a discreet set of fields. If one those values happens to be null, this line would give a null pointer exception:

if (!currentTuple.get(fieldName).equals(lastTuple.get(fieldName)))

So, I changed the logic to this:

if (!(currentTuple.get(fieldName)==null && lastTuple.get(fieldName)==null) &&
                      (currentTuple.get(fieldName)==null || !currentTuple.get(fieldName).equals(lastTuple.get(fieldName))))

I also added some logging info to help with diagnosing problems with the harvest. And I removed some logging info which seemed to me to be not useful.

We don't have an svn repository for the d2rmap source code do we? Maybe we should set that up.

Brian

Brian Keese
Indiana University Digital Library Program
Herman B Wells Library W501
812-856-3865

From: Eliza Chan [mailto:elc2013@...]
Sent: Friday, July 01, 2011 2:22 PM
To: VIVO Development List
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Hi Brian,

Thanks for making the changes.
It'd be nice to have d2rmap harvests (db and csv) being supported in future releases, since there are people who use the tools on a regular basis.
I have been trying to update the documentation on d2rmap but it has not been able to keep up with so many changes with Harvester. :-(
Would you be able to document some of it? Also, would you be able to list the bugs that you fixed?

Thanks,
Eliza

From: "Keese, Brian W" <bkeese@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Fri, 1 Jul 2011 14:05:23 -0400
To: VIVO Development List <vivo-dev-all@...>>
Subject: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Nick, Thanks for the info on Oracle. Makes sense.

Here's another thing. I have made a couple of changes to the d2rmap processor that is included with the harvester. I have uploaded the new jar file to https://www.slashtmp.iu.edu/files/download?FILE=bkeese%2F94328zp0Qy2 (the password is 'd2r4vivo').
It fixes a couple of bugs, and adds some status info to the logs for d2rmap fetch.

I noticed that there is no example script for d2rmap in the latest release. I hope support for d2r harvests will be continued.

Brian

Brian Keese
Indiana University Digital Library Program
Herman B Wells Library W501
812-856-3865

From: Nicholas Skaggs [mailto:nskaggs@...]
Sent: Friday, July 01, 2011 10:51 AM
To: vivo-dev-all@...>
Subject: Re: [Vivo-dev-all] Vivo development call / meeting reminder for today.

Great catch Brian. The trouble with oracle is that it requires a specific jdbc connector for each release -- including minor point release versions.

http://www.oracle.com/technetwork/database/features/jdbc/index-091264.html

We can't package every version of the connector with the harvester unfortunately. I have heard of a so called "universal" connector for oracle that connects to the databases through lowest common denominator SQL. I've never been able to find anything additional about this, nor a download for it -- so I can't verify it exists or not.  In addition, the connectors are released by Oracle under the Oracle Technology Network Development and Distribution License. This prevents us from distributing these files by packaging them directly without complying with the license terms. Specifically, the redistribution clause of the license causes trouble:

Program Distribution
We grant you a nonexclusive, nontransferable right to copy and distribute the programs to your end users provided that you do not charge your end users for use of the programs and provided your end users may only use the programs to run your applications for their business operations. Prior to distributing the programs you shall require your end users to execute an agreement binding them to terms consistent with those contained in this section and the sections of this agreement entitled "License Rights," "Ownership and Restrictions," "Export," "Disclaimer of Warranties and Exclusive Remedies," "No Technical Support," "End of Agreement," "Relationship Between the Parties," and "Open Source." You must also include a provision stating that your end users shall have no right to distribute the programs, and a provision specifying us as a third party beneficiary of the agreement. You are responsible for obtaining these agreements with your end users.

You can read the entire license here:

http://www.oracle.com/technetwork/licenses/distribution-license-152002.html

If you have an idea about how we can distribute something to help the harvester work out of the box with oracle, please let us know. Thanks,

Nicholas

On 7/1/11 10:39 AM, Keese, Brian W wrote:

The attachment didn't work. The driver can be downloaded from Oracle here:



http://www.oracle.com/technetwork/database/enterprise-edition/jdbc-10201-088211.html

Brian

________________________________________

From: Keese, Brian W [bkeese@...>]

Sent: Friday, July 01, 2011 10:36 AM

To: jrpence@...>; VIVO Development List

Subject: Re: [Vivo-dev-all] Vivo development call / meeting reminder for today.



Hey Harvester Guys,

I noticed that the oracle jdbc driver was never added (or was added, then removed) to the harvester bin/dependencies folder for the latest release. Could you add that in please?

It is ojdbc14.jar (attached to this email).



Thanks,

Brian





________________________________________

From: James Pence [jrpence@...>]

Sent: Thursday, June 30, 2011 1:23 PM

To: vivo-dev-all@...>

Subject: [Vivo-dev-all] Vivo development call / meeting reminder for today.



Hello,



    Today the vivo developers, and maybe you, are having a goto-meeting at 3:00pm EST (UTC-4 hours)



    We have the beginnings of an agenda at<http://sourceforge.net/apps/mediawiki/vivo/index.php?title=2011-06-30_Development_Leads_Call><http://sourceforge.net/apps/mediawiki/vivo/index.php?title=2011-06-30_Development_Leads_Call>



If you have any agenda items to add to the meeting please let me know (jrpence@...>   ).



   Since we only have 25 connections for the goto meeting please consolidate where possible.



   1. Please join my meeting, Thursday, June 30 at 3:00 PM Eastern DaylightTime.

   https://www1.gotomeeting.com/join/486839241



   2. Use your microphone and speakers (VoIP) - a headset is recommended.



   Or, call in using your telephone.

   Dial +1 (215) 383-1013



   Access Code: 486-839-241

   Audio PIN: Shown after joining the meeting

   Meeting ID: 486-839-241



   GoToMeeting(r)

   Online Meetings Made Easy(tm)







--

James R. Pence

jrpence@...>

Software Engineer

Communicore C2-12

(352)294-5154 (ex. 45154)

Enterprise Software Engineering

University of Florida

Academic Health Center IT

Developer on the VIVO Grant





------------------------------------------------------------------------------

All of the data generated in your IT infrastructure is seriously valuable.

Why? It contains a definitive record of application performance, security

threats, fraudulent activity, and more. Splunk takes this data and makes

sense of it. IT sense. And common sense.

http://p.sf.net/sfu/splunk-d2d-c2

_______________________________________________

Vivo-dev-all mailing list

Vivo-dev-all@...>

https://lists.sourceforge.net/lists/listinfo/vivo-dev-all



------------------------------------------------------------------------------

All of the data generated in your IT infrastructure is seriously valuable.

Why? It contains a definitive record of application performance, security

threats, fraudulent activity, and more. Splunk takes this data and makes

sense of it. IT sense. And common sense.

http://p.sf.net/sfu/splunk-d2d-c2

_______________________________________________

Vivo-dev-all mailing list

Vivo-dev-all@...>

https://lists.sourceforge.net/lists/listinfo/vivo-dev-all

Hi Nicholas,

Just waiting for your feedback regarding the license statement in D2rProcessor.java.

Thanks,
Eliza

From: "Keese, Brian W" <bkeese@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Thu, 21 Jul 2011 15:54:50 -0400
To: VIVO Development List <vivo-dev-all@...>>
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

I have committed my bug fixes to the D2r code on git. Should be good to go.

Brian

Brian Keese
Indiana University Digital Library Program
Herman B Wells Library W501
812-856-3865

From: Eliza Chan [mailto:elc2013@...]
Sent: Thursday, July 21, 2011 12:14 PM
To: VIVO Development List
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster


Sorry, it should be there now.
There is no need to change the d2rmap.config.xml file. It should work with the fixes.
Feel free to revert back to H2.

Thanks,
Eliza

From: Nicholas Skaggs <nskaggs@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Thu, 21 Jul 2011 11:59:27 -0400
To: "vivo-dev-all@...>" <vivo-dev-all@...>>
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Eliza, I'm not seeing your commits in etheir branch. My apologies if they are there and I'm missing them. Did you push back upstream? Or did you just commit locally? I did try and remove h2 from the equation and just use the mysql database directly, but all i got out of the translate is the following:

<?xml version="1.0" encoding="UTF-8" ?><rdf:RDF
    xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"<http://www.w3.org/1999/02/22-rdf-syntax-ns>
    xmlns:bibo="http://purl.org/ontology/bibo/"<http://purl.org/ontology/bibo/>
    xmlns:foaf="http://xmlns.com/foaf/0.1/"<http://xmlns.com/foaf/0.1/>
    xmlns:localVivo="http://vivo.sample.edu/ontology/"<http://vivo.sample.edu/ontology/>
    xmlns:owl="http://www.w3.org/2002/07/owl#"<http://www.w3.org/2002/07/owl>
    xmlns:xsd="http://www.w3.org/2001/XMLSchema#"<http://www.w3.org/2001/XMLSchema>
    xmlns:core="http://vivoweb.org/ontology/core#"<http://vivoweb.org/ontology/core>
    xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#"<http://www.w3.org/2000/01/rdf-schema> >
  <rdf:Description rdf:about="http://vivoweb.org/harvest/example/d2r/person/personnull"<http://vivoweb.org/harvest/example/d2r/person/personnull>>
    <rdf:type rdf:resource="http://xmlns.com/foaf/0.1/Person"<http://xmlns.com/foaf/0.1/Person>/>
  </rdf:Description>
  <rdf:Description rdf:about="http://vivoweb.org/harvest/example/d2r/org/deptnull"<http://vivoweb.org/harvest/example/d2r/org/deptnull>>
    <rdf:type rdf:resource="http://xmlns.com/foaf/0.1/Organization"<http://xmlns.com/foaf/0.1/Organization>/>
  </rdf:Description>
</rdf:RDF>

Thanks for your help!

Nicholas

On 07/21/2011 11:08 AM, Eliza Chan wrote:
Hi Nicholas and Brian,

Changes have been committed to the d2rmap branch. A GPL/LGPL statement was added to D2rProcessor. Please make changes to the statement if needed, and copy to the other classes that have been modified. The revised classes are: D2rProcessor, Map, Instance, SourceControllerD2R.

Also, the d2rmap example in the master branch should work now. It's related to a bug in D2RMap that returned the column name (e.g. user_id) rather that the column label (e.g. UID) in a query with a "AS" clause (e.g. user_id AS UID). It seemed to affect H2 and not Oracle. Please give it a try and confirm.

Thanks,
Eliza

From: Nicholas Skaggs <nskaggs@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Tue, 19 Jul 2011 11:14:24 -0400
To: "vivo-dev-all@...>" <vivo-dev-all@...>>
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Eliza and Brian,
     we've created a d2rmap branch in the git repository for the harvester -- origin/D2rMapIntegration. It contains the latest code from the d2rmap repository. Can you two add your patchsets to this code to bring it up to date with what we want to ship with the harvester? We'll then need to update maven to build a jar for the harvester to use as part of the build for the harvester.

Secondly, we have the d2rmap example in the master branch on git. While the example doesn't give an error, no data is being added to VIVO. Can you take a look and figure out what's wrong? The team is ready to push out a 1.2 final of the harvester once this example is complete :-)

Now, onto your questions below:

1. I mean you should license your modifications to d2rmap as GPL/LGPL so that there project can accept them under than same license terms as d2rmap is licensed.

2. We are NOT required to followup with them, but I think it would be a good idea to let them know of our work. Once we have your code integrated, I would suggest sending the patchset to the d2rmap mailing list and/or letting them now we have changes in our git which are publically available.

3. The team seemed to favor including the code inline with the harvester, so no sub-project has been created.

4. Licensing file has been updated appropriately for this, so we're good.

5. I'm not sure where this is coming from, but making available the source, and making our changes available, as well as noting the license of the code as being GPL/LGPL meets the terms of the license.

Nicholas


On 7/8/11 11:26 AM, Eliza Chan wrote:
Hi Nicholas,

Please see my questions regarding your action items:

1. License modifications as GPL/LGPL

Did you mean adding GPL/LGPL to Licensing & Acknowledgements?
Or did you mean adding a license statement to the source code?

2. Push patchsets to D2RMap community

Under the license agreement, I don't see anywhere that we are required to push the patchsets back to the D2RMap community. Could someone confirm this please?
However, if we really want to check in the changes, Brian's patch seems to be more relevant because it's related to a bug fix. But my part was mainly for integrating D2RMap to VIVO, so I'm not sure if it's relevant to the D2RMap project.

3. Host our modified version of the code on SF (seperate sub-project?)

How can one create a sub-project in SF?

4. Use the modified binary in the harvester, noting the GPL license

Did you mean adding a LICENSE file somewhere in the Harvester source code?

5.
Under the "TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION",
section 6, paragraph b: You must cause the files modified to carry prominent notices stating that you changed the files and the date of any change.

Have we addressed this? I'm also not a lawyer. ;-)

Thanks,
Eliza

From: Nicholas Skaggs <nskaggs@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Thu, 7 Jul 2011 14:02:49 -0400
To: "vivo-dev-all@...>" <vivo-dev-all@...>>
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Given the shortcomings of email, I don't want to be mis-interpreted on my reply.

I would like to preface this email with "In my opinion", and "I think we should' :-)

Thanks,

Nicholas

On 7/7/11 1:58 PM, Nicholas Skaggs wrote:
Jim, yes we must consider these. For distributing a binary, we can simply disclose it's license as GPL, and do nothing more.  LGPL and GPL allow for different things in terms of linking and non-open source software, but we don't need to get into that here. BSD is "compatible" with the GPL/LGPL. In this case, Eliza/Brian has modified GPL/LGPL source (they appear to have it dual licensed). We can distribute this of course, but we do need to make those modifications available in source form upon request. This can be done thru writing and physical media (we can even charge the cost of distribution), etc, but the simplest and most common is to host the source in a freely downloadable place.

So, I didn't realize D2RMap was an existing SF project. We essentially have 2 options. We can fork there project and host it ourselves with our modifications, or we can contribute them back upstream and see if we can get them to adopt them. I would suggest starting with the latter. Depending on the outcome, in the meantime, we will be maintaining a "fork" of the project with our modifications as part of the harvester since we want to utilize it. Now, the best practice for Eliza and Brian would be to make there modifications back to the D2RMap project as GPL/LGPL (there current license format). If they licensed it as BSD or something else, the project would have part of the code under a different license which is no good, and would make it harder for them to accept.

The ramifications then for the harvester is we must ship the GPL license with our project and state that D2RMap is under that license. Not a big deal, it's an external library; but we should make note of it.

In summary we should:

License modifications as GPL/LGPL
Push patchsets to D2RMap community
Host our modified version of the code on SF (seperate sub-project?)
Use the modified binary in the harvester, noting the GPL license

Nicholas

On 7/7/11 1:50 PM, Eliza Chan wrote:
Hi Jim,

Thanks for looking up.
In this case, committing the D2RMap source code to our project is not only Ok, it's required.

Eliza

From: Jim Blake <jeb228@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Thu, 7 Jul 2011 13:41:20 -0400
To: VIVO Development List <vivo-dev-all@...>>
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Eliza;

That’s a very interesting question, and it opens an big can of worms.

I am not a lawyer, but it appears that D2RMap is covered under the GNU Lesser General Public License (LGPL), which is a modification of the GNU General Public License, which appears to cover this scenario in section 6, paragraph d.

If I understand this correctly, if you offer the object code (i.e., JAR files or CLASS files) for public download as part of your product, you are required to offer the source code (i.e. JAVA files) for public download on the same server, or an equivalent server operated by you or by a third party.

So if our software is available for download from SourceForge, and the D2RMap source code is available from SourceForge, have we satisfied the terms of the license?

Have I mentioned that I have no legal training or expertise?

I wonder whether the VIVO project is in violation of the terms of many additional open source licenses. For exmaple, some licenses require that we cite them in all of our end-user documentation. Do we do that?

But I’m just taking wild guesses, here. Can anybody shed more light on this?

Jim


From: Eliza Chan [mailto:elc2013@...]
Sent: Thursday, July 07, 2011 11:59 AM
To: VIVO Development List
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Hi Nicholas and Brian,

I can definitely commit D2RMap to sourceforge.
Just a policy question before doing that.
As you are probably aware, D2RMap is a sourceforge project: https://sourceforge.net/projects/d2r-map/
Do we need to do anything extra (whatever that is) before we post their modified copy under our project?

Thanks,
Eliza

From: Nicholas Skaggs <nskaggs@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Thu, 7 Jul 2011 11:10:13 -0400
To: "vivo-dev-all@...>" <vivo-dev-all@...>>
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Brian and Eliza,
    Let's get a merged copy of your code into the harvester repo. Brian, it seems your new jar is missing some of Eliza's code -- specifically the CSV Map:

The method processCsvMap(String, String, String) is undefined for the type D2rProcessor    D2RMapFetch.java    /Harvester-hg/src/main/java/org/vivoweb/harvester/fetch    line 113    Java Problem

Eliza, I know you have commit rights to SVN -- Brian do you? Eliza can you check in your source for D2RMap, and then have Brian add his changes in as another commit, etc. That would be really awesome. I'm sure others who are using it would be excited to be able to commit fixes and updates as well.

Right now we're still working on restoring the SVN from the mercurial commits we did. As soon as that's done, we can get your commits in.

Thanks!

Nicholas


On 7/1/11 2:43 PM, Eliza Chan wrote:
You are right, we should have a repository for the d2rmap source code. Since now you have a modified copy, and I also have one. It's better to merge them.


From: "Keese, Brian W" <bkeese@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Fri, 1 Jul 2011 14:37:11 -0400
To: VIVO Development List <vivo-dev-all@...>>
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Sure. Let me know how I can help with the docs.

For the bugs, one of them was fixed before (the oracle semi-colon thing, Map.java around line 94).

The other bug has to do with null values for instance identifiers. In Map.java around line 126, a list of new instances is being compiled. Unique instances are defined by values in a discreet set of fields. If one those values happens to be null, this line would give a null pointer exception:

if (!currentTuple.get(fieldName).equals(lastTuple.get(fieldName)))

So, I changed the logic to this:

if (!(currentTuple.get(fieldName)==null && lastTuple.get(fieldName)==null) &&
                      (currentTuple.get(fieldName)==null || !currentTuple.get(fieldName).equals(lastTuple.get(fieldName))))

I also added some logging info to help with diagnosing problems with the harvest. And I removed some logging info which seemed to me to be not useful.

We don’t have an svn repository for the d2rmap source code do we? Maybe we should set that up.

Brian

Brian Keese
Indiana University Digital Library Program
Herman B Wells Library W501
812-856-3865

From: Eliza Chan [mailto:elc2013@...]
Sent: Friday, July 01, 2011 2:22 PM
To: VIVO Development List
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Hi Brian,

Thanks for making the changes.
It'd be nice to have d2rmap harvests (db and csv) being supported in future releases, since there are people who use the tools on a regular basis.
I have been trying to update the documentation on d2rmap but it has not been able to keep up with so many changes with Harvester. :-(
Would you be able to document some of it? Also, would you be able to list the bugs that you fixed?

Thanks,
Eliza

From: "Keese, Brian W" <bkeese@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Fri, 1 Jul 2011 14:05:23 -0400
To: VIVO Development List <vivo-dev-all@...>>
Subject: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Nick, Thanks for the info on Oracle. Makes sense.

Here’s another thing. I have made a couple of changes to the d2rmap processor that is included with the harvester. I have uploaded the new jar file to https://www.slashtmp.iu.edu/files/download?FILE=bkeese%2F94328zp0Qy2 (the password is ‘d2r4vivo’).
It fixes a couple of bugs, and adds some status info to the logs for d2rmap fetch.

I noticed that there is no example script for d2rmap in the latest release. I hope support for d2r harvests will be continued.

Brian

Brian Keese
Indiana University Digital Library Program
Herman B Wells Library W501
812-856-3865

From: Nicholas Skaggs [mailto:nskaggs@...]
Sent: Friday, July 01, 2011 10:51 AM
To: vivo-dev-all@...>
Subject: Re: [Vivo-dev-all] Vivo development call / meeting reminder for today.

Great catch Brian. The trouble with oracle is that it requires a specific jdbc connector for each release -- including minor point release versions.

http://www.oracle.com/technetwork/database/features/jdbc/index-091264.html

We can't package every version of the connector with the harvester unfortunately. I have heard of a so called "universal" connector for oracle that connects to the databases through lowest common denominator SQL. I've never been able to find anything additional about this, nor a download for it -- so I can't verify it exists or not.  In addition, the connectors are released by Oracle under the Oracle Technology Network Development and Distribution License. This prevents us from distributing these files by packaging them directly without complying with the license terms. Specifically, the redistribution clause of the license causes trouble:

Program Distribution
We grant you a nonexclusive, nontransferable right to copy and distribute the programs to your end users provided that you do not charge your end users for use of the programs and provided your end users may only use the programs to run your applications for their business operations. Prior to distributing the programs you shall require your end users to execute an agreement binding them to terms consistent with those contained in this section and the sections of this agreement entitled "License Rights," "Ownership and Restrictions," "Export," "Disclaimer of Warranties and Exclusive Remedies," "No Technical Support," "End of Agreement," "Relationship Between the Parties," and "Open Source." You must also include a provision stating that your end users shall have no right to distribute the programs, and a provision specifying us as a third party beneficiary of the agreement. You are responsible for obtaining these agreements with your end users.

You can read the entire license here:

http://www.oracle.com/technetwork/licenses/distribution-license-152002.html

If you have an idea about how we can distribute something to help the harvester work out of the box with oracle, please let us know. Thanks,

Nicholas

On 7/1/11 10:39 AM, Keese, Brian W wrote:

The attachment didn't work. The driver can be downloaded from Oracle here:



http://www.oracle.com/technetwork/database/enterprise-edition/jdbc-10201-088211.html

Brian

________________________________________

From: Keese, Brian W [bkeese@...>]

Sent: Friday, July 01, 2011 10:36 AM

To: jrpence@...>; VIVO Development List

Subject: Re: [Vivo-dev-all] Vivo development call / meeting reminder for today.



Hey Harvester Guys,

I noticed that the oracle jdbc driver was never added (or was added, then removed) to the harvester bin/dependencies folder for the latest release. Could you add that in please?

It is ojdbc14.jar (attached to this email).



Thanks,

Brian





________________________________________

From: James Pence [jrpence@...>]

Sent: Thursday, June 30, 2011 1:23 PM

To: vivo-dev-all@...>

Subject: [Vivo-dev-all] Vivo development call / meeting reminder for today.



Hello,



    Today the vivo developers, and maybe you, are having a goto-meeting at 3:00pm EST (UTC-4 hours)



    We have the beginnings of an agenda at<http://sourceforge.net/apps/mediawiki/vivo/index.php?title=2011-06-30_Development_Leads_Call><http://sourceforge.net/apps/mediawiki/vivo/index.php?title=2011-06-30_Development_Leads_Call>



If you have any agenda items to add to the meeting please let me know (jrpence@...>   ).



   Since we only have 25 connections for the goto meeting please consolidate where possible.



   1. Please join my meeting, Thursday, June 30 at 3:00 PM Eastern DaylightTime.

   https://www1.gotomeeting.com/join/486839241



   2. Use your microphone and speakers (VoIP) - a headset is recommended.



   Or, call in using your telephone.

   Dial +1 (215) 383-1013



   Access Code: 486-839-241

   Audio PIN: Shown after joining the meeting

   Meeting ID: 486-839-241



   GoToMeeting®

   Online Meetings Made Easy™







--

James R. Pence

jrpence@...>

Software Engineer

Communicore C2-12

(352)294-5154 (ex. 45154)

Enterprise Software Engineering

University of Florida

Academic Health Center IT

Developer on the VIVO Grant





------------------------------------------------------------------------------

All of the data generated in your IT infrastructure is seriously valuable.

Why? It contains a definitive record of application performance, security

threats, fraudulent activity, and more. Splunk takes this data and makes

sense of it. IT sense. And common sense.

http://p.sf.net/sfu/splunk-d2d-c2

_______________________________________________

Vivo-dev-all mailing list

Vivo-dev-all@...>

https://lists.sourceforge.net/lists/listinfo/vivo-dev-all



------------------------------------------------------------------------------

All of the data generated in your IT infrastructure is seriously valuable.

Why? It contains a definitive record of application performance, security

threats, fraudulent activity, and more. Splunk takes this data and makes

sense of it. IT sense. And common sense.

http://p.sf.net/sfu/splunk-d2d-c2

_______________________________________________

Vivo-dev-all mailing list

Vivo-dev-all@...>

https://lists.sourceforge.net/lists/listinfo/vivo-dev-all

Eliza -- just looking at the code now. Umm, yes odd that the D2R team 
didn't include the license in the file headers. The only issue incorrect 
with what you have is the version of the GPL included -- since the code 
was written before gpl v3, it's gpl v2 :-) I'll update accordingly. Thanks,

Nicholas


On 07/21/2011 04:37 PM, Eliza Chan wrote:
> Hi Nicholas,
>
> Just waiting for your feedback regarding the license statement 
> in D2rProcessor.java.
>
> Thanks,
> Eliza
>
> From: "Keese, Brian W" <bkeese@... <mailto:bkeese@...>>
> Reply-To: VIVO Development List <vivo-dev-all@... 
> <mailto:vivo-dev-all@...>>
> Date: Thu, 21 Jul 2011 15:54:50 -0400
> To: VIVO Development List <vivo-dev-all@... 
> <mailto:vivo-dev-all@...>>
> Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster
>
> I have committed my bug fixes to the D2r code on git. Should be good 
> to go.
>
> Brian
>
> Brian Keese
>
> Indiana University Digital Library Program
>
> Herman B Wells Library W501
>
> 812-856-3865
>
> *From:*Eliza Chan [mailto:elc2013@...]
> *Sent:* Thursday, July 21, 2011 12:14 PM
> *To:* VIVO Development List
> *Subject:* Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster
>
> Sorry, it should be there now.
>
> There is no need to change the d2rmap.config.xml file. It should work 
> with the fixes.
>
> Feel free to revert back to H2.
>
> Thanks,
>
> Eliza
>
> *From: *Nicholas Skaggs <nskaggs@... <mailto:nskaggs@...>>
> *Reply-To: *VIVO Development List <vivo-dev-all@... 
> <mailto:vivo-dev-all@...>>
> *Date: *Thu, 21 Jul 2011 11:59:27 -0400
> *To: *"vivo-dev-all@... 
> <mailto:vivo-dev-all@...>" 
> <vivo-dev-all@... 
> <mailto:vivo-dev-all@...>>
> *Subject: *Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster
>
> Eliza, I'm not seeing your commits in etheir branch. My apologies if 
> they are there and I'm missing them. Did you push back upstream? Or 
> did you just commit locally? I did try and remove h2 from the equation 
> and just use the mysql database directly, but all i got out of the 
> translate is the following:
>
> <?xml version="1.0" encoding="UTF-8" ?><rdf:RDF
>     xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" 
> <http://www.w3.org/1999/02/22-rdf-syntax-ns>
>     xmlns:bibo="http://purl.org/ontology/bibo/" 
> <http://purl.org/ontology/bibo/>
>     xmlns:foaf="http://xmlns.com/foaf/0.1/" <http://xmlns.com/foaf/0.1/>
>     xmlns:localVivo="http://vivo.sample.edu/ontology/" 
> <http://vivo.sample.edu/ontology/>
>     xmlns:owl="http://www.w3.org/2002/07/owl#" 
> <http://www.w3.org/2002/07/owl>
>     xmlns:xsd="http://www.w3.org/2001/XMLSchema#" 
> <http://www.w3.org/2001/XMLSchema>
>     xmlns:core="http://vivoweb.org/ontology/core#" 
> <http://vivoweb.org/ontology/core>
>     xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#" 
> <http://www.w3.org/2000/01/rdf-schema> >
> <rdf:Description 
> rdf:about="http://vivoweb.org/harvest/example/d2r/person/personnull" 
> <http://vivoweb.org/harvest/example/d2r/person/personnull>>
> <rdf:type rdf:resource="http://xmlns.com/foaf/0.1/Person" 
> <http://xmlns.com/foaf/0.1/Person>/>
> </rdf:Description>
> <rdf:Description 
> rdf:about="http://vivoweb.org/harvest/example/d2r/org/deptnull" 
> <http://vivoweb.org/harvest/example/d2r/org/deptnull>>
> <rdf:type rdf:resource="http://xmlns.com/foaf/0.1/Organization" 
> <http://xmlns.com/foaf/0.1/Organization>/>
> </rdf:Description>
> </rdf:RDF>
>
> Thanks for your help!
>
> Nicholas
>
> On 07/21/2011 11:08 AM, Eliza Chan wrote:
>
> Hi Nicholas and Brian,
>
> Changes have been committed to the d2rmap branch. A GPL/LGPL statement 
> was added to D2rProcessor. Please make changes to the statement if 
> needed, and copy to the other classes that have been modified. The 
> revised classes are: D2rProcessor, Map, Instance, SourceControllerD2R.
>
> Also, the d2rmap example in the master branch should work now. It's 
> related to a bug in D2RMap that returned the column name (e.g. 
> user_id) rather that the column label (e.g. UID) in a query with a 
> "AS" clause (e.g. user_id AS UID). It seemed to affect H2 and not 
> Oracle. Please give it a try and confirm.
>
> Thanks,
>
> Eliza
>
> *From: *Nicholas Skaggs <nskaggs@... <mailto:nskaggs@...>>
> *Reply-To: *VIVO Development List <vivo-dev-all@... 
> <mailto:vivo-dev-all@...>>
> *Date: *Tue, 19 Jul 2011 11:14:24 -0400
> *To: *"vivo-dev-all@... 
> <mailto:vivo-dev-all@...>" 
> <vivo-dev-all@... 
> <mailto:vivo-dev-all@...>>
> *Subject: *Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster
>
> Eliza and Brian,
>      we've created a d2rmap branch in the git repository for the 
> harvester -- origin/D2rMapIntegration. It contains the latest code 
> from the d2rmap repository. Can you two add your patchsets to this 
> code to bring it up to date with what we want to ship with the 
> harvester? We'll then need to update maven to build a jar for the 
> harvester to use as part of the build for the harvester.
>
> Secondly, we have the d2rmap example in the master branch on git. 
> While the example doesn't give an error, no data is being added to 
> VIVO. Can you take a look and figure out what's wrong? The team is 
> ready to push out a 1.2 final of the harvester once this example is 
> complete :-)
>
> Now, onto your questions below:
>
> 1. I mean you should license your modifications to d2rmap as GPL/LGPL 
> so that there project can accept them under than same license terms as 
> d2rmap is licensed.
>
> 2. We are NOT required to followup with them, but I think it would be 
> a good idea to let them know of our work. Once we have your code 
> integrated, I would suggest sending the patchset to the d2rmap mailing 
> list and/or letting them now we have changes in our git which are 
> publically available.
>
> 3. The team seemed to favor including the code inline with the 
> harvester, so no sub-project has been created.
>
> 4. Licensing file has been updated appropriately for this, so we're good.
>
> 5. I'm not sure where this is coming from, but making available the 
> source, and making our changes available, as well as noting the 
> license of the code as being GPL/LGPL meets the terms of the license.
>
> Nicholas
>
>
> On 7/8/11 11:26 AM, Eliza Chan wrote:
>
> Hi Nicholas,
>
> Please see my questions regarding your action items:
>
> 1. License modifications as GPL/LGPL
>
> Did you mean adding GPL/LGPL to Licensing & Acknowledgements?
>
> Or did you mean adding a license statement to the source code?
>
> 2. Push patchsets to D2RMap community
>
> Under the license agreement, I don't see anywhere that we are required 
> to push the patchsets back to the D2RMap community. Could someone 
> confirm this please?
>
> However, if we really want to check in the changes, Brian's patch 
> seems to be more relevant because it's related to a bug fix. But my 
> part was mainly for integrating D2RMap to VIVO, so I'm not sure if 
> it's relevant to the D2RMap project.
>
> 3. Host our modified version of the code on SF (seperate sub-project?)
>
> How can one create a sub-project in SF?
>
> 4. Use the modified binary in the harvester, noting the GPL license
>
> Did you mean adding a LICENSE file somewhere in the Harvester source code?
>
> 5.
>
> Under the "TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND 
> MODIFICATION",
>
> section 6, paragraph b: You must cause the files modified to carry 
> prominent notices stating that you changed the files and the date of 
> any change.
>
> Have we addressed this? I'm also not a lawyer. ;-)
>
> Thanks,
>
> Eliza
>
> *From: *Nicholas Skaggs <nskaggs@... <mailto:nskaggs@...>>
> *Reply-To: *VIVO Development List <vivo-dev-all@... 
> <mailto:vivo-dev-all@...>>
> *Date: *Thu, 7 Jul 2011 14:02:49 -0400
> *To: *"vivo-dev-all@... 
> <mailto:vivo-dev-all@...>" 
> <vivo-dev-all@... 
> <mailto:vivo-dev-all@...>>
> *Subject: *Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster
>
> Given the shortcomings of email, I don't want to be mis-interpreted on 
> my reply.
>
> I would like to preface this email with "In my opinion", and "I think 
> we should' :-)
>
> Thanks,
>
> Nicholas
>
> On 7/7/11 1:58 PM, Nicholas Skaggs wrote:
>
> Jim, yes we must consider these. For distributing a binary, we can 
> simply disclose it's license as GPL, and do nothing more.  LGPL and 
> GPL allow for different things in terms of linking and non-open source 
> software, but we don't need to get into that here. BSD is "compatible" 
> with the GPL/LGPL. In this case, Eliza/Brian has modified GPL/LGPL 
> source (they appear to have it dual licensed). We can distribute this 
> of course, but we do need to make those modifications available in 
> source form upon request. This can be done thru writing and physical 
> media (we can even charge the cost of distribution), etc, but the 
> simplest and most common is to host the source in a freely 
> downloadable place.
>
> So, I didn't realize D2RMap was an existing SF project. We essentially 
> have 2 options. We can fork there project and host it ourselves with 
> our modifications, or we can contribute them back upstream and see if 
> we can get them to adopt them. I would suggest starting with the 
> latter. Depending on the outcome, in the meantime, we will be 
> maintaining a "fork" of the project with our modifications as part of 
> the harvester since we want to utilize it. Now, the best practice for 
> Eliza and Brian would be to make there modifications back to the 
> D2RMap project as GPL/LGPL (there current license format). If they 
> licensed it as BSD or something else, the project would have part of 
> the code under a different license which is no good, and would make it 
> harder for them to accept.
>
> The ramifications then for the harvester is we must ship the GPL 
> license with our project and state that D2RMap is under that license. 
> Not a big deal, it's an external library; but we should make note of it.
>
> In summary we should:
>
> License modifications as GPL/LGPL
> Push patchsets to D2RMap community
> Host our modified version of the code on SF (seperate sub-project?)
> Use the modified binary in the harvester, noting the GPL license
>
> Nicholas
>
> On 7/7/11 1:50 PM, Eliza Chan wrote:
>
> Hi Jim,
>
> Thanks for looking up.
>
> In this case, committing the D2RMap source code to our project is not 
> only Ok, it's required.
>
> Eliza
>
> *From: *Jim Blake <jeb228@... <mailto:jeb228@...>>
> *Reply-To: *VIVO Development List <vivo-dev-all@... 
> <mailto:vivo-dev-all@...>>
> *Date: *Thu, 7 Jul 2011 13:41:20 -0400
> *To: *VIVO Development List <vivo-dev-all@... 
> <mailto:vivo-dev-all@...>>
> *Subject: *Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster
>
> Eliza;
>
> That’s a very interesting question, and it opens an big can of worms.
>
> I am not a lawyer, but it appears that D2RMap is covered under the GNU 
> Lesser General Public License (LGPL), which is a modification of the 
> GNU General Public License, which appears to cover this scenario in 
> section 6, paragraph d.
>
> If I understand this correctly, if you offer the object code (i.e., 
> JAR files or CLASS files) for public download as part of your product, 
> you are required to offer the source code (i.e. JAVA files) for public 
> download on the same server, or an equivalent server operated by you 
> or by a third party.
>
> So if our software is available for download from SourceForge, and the 
> D2RMap source code is available from SourceForge, have we satisfied 
> the terms of the license?
>
> Have I mentioned that I have no legal training or expertise?
>
> I wonder whether the VIVO project is in violation of the terms of many 
> additional open source licenses. For exmaple, some licenses require 
> that we cite them in all of our end-user documentation. Do we do that?
>
> But I’m just taking wild guesses, here. Can anybody shed more light on 
> this?
>
> Jim
>
> *From:*Eliza Chan [mailto:elc2013@...]
> *Sent:* Thursday, July 07, 2011 11:59 AM
> *To:* VIVO Development List
> *Subject:* Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster
>
> Hi Nicholas and Brian,
>
> I can definitely commit D2RMap to sourceforge.
>
> Just a policy question before doing that.
>
> As you are probably aware, D2RMap is a sourceforge project: 
> https://sourceforge.net/projects/d2r-map/
>
> Do we need to do anything extra (whatever that is) before we post 
> their modified copy under our project?
>
> Thanks,
>
> Eliza
>
> *From: *Nicholas Skaggs <nskaggs@... <mailto:nskaggs@...>>
> *Reply-To: *VIVO Development List <vivo-dev-all@... 
> <mailto:vivo-dev-all@...>>
> *Date: *Thu, 7 Jul 2011 11:10:13 -0400
> *To: *"vivo-dev-all@... 
> <mailto:vivo-dev-all@...>" 
> <vivo-dev-all@... 
> <mailto:vivo-dev-all@...>>
> *Subject: *Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster
>
> Brian and Eliza,
>     Let's get a merged copy of your code into the harvester repo. 
> Brian, it seems your new jar is missing some of Eliza's code -- 
> specifically the CSV Map:
>
> The method processCsvMap(String, String, String) is undefined for the 
> type D2rProcessor    D2RMapFetch.java    
> /Harvester-hg/src/main/java/org/vivoweb/harvester/fetch    line 113    
> Java Problem
>
> Eliza, I know you have commit rights to SVN -- Brian do you? Eliza can 
> you check in your source for D2RMap, and then have Brian add his 
> changes in as another commit, etc. That would be really awesome. I'm 
> sure others who are using it would be excited to be able to commit 
> fixes and updates as well.
>
> Right now we're still working on restoring the SVN from the mercurial 
> commits we did. As soon as that's done, we can get your commits in.
>
> Thanks!
>
> Nicholas
>
>
> On 7/1/11 2:43 PM, Eliza Chan wrote:
>
> You are right, we should have a repository for the d2rmap source code. 
> Since now you have a modified copy, and I also have one. It's better 
> to merge them.
>
> *From: *"Keese, Brian W" <bkeese@... <mailto:bkeese@...>>
> *Reply-To: *VIVO Development List <vivo-dev-all@... 
> <mailto:vivo-dev-all@...>>
> *Date: *Fri, 1 Jul 2011 14:37:11 -0400
> *To: *VIVO Development List <vivo-dev-all@... 
> <mailto:vivo-dev-all@...>>
> *Subject: *Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster
>
> Sure. Let me know how I can help with the docs.
>
> For the bugs, one of them was fixed before (the oracle semi-colon 
> thing, Map.java around line 94).
>
> The other bug has to do with null values for instance identifiers. In 
> Map.java around line 126, a list of new instances is being compiled. 
> Unique instances are defined by values in a discreet set of fields. If 
> one those values happens to be null, this line would give a null 
> pointer exception:
>
> if (!currentTuple.get(fieldName).equals(lastTuple.get(fieldName)))
>
> So, I changed the logic to this:
>
> if (!(currentTuple.get(fieldName)==null && 
> lastTuple.get(fieldName)==null) &&
>
>                       (currentTuple.get(fieldName)==null || 
> !currentTuple.get(fieldName).equals(lastTuple.get(fieldName))))
>
> I also added some logging info to help with diagnosing problems with 
> the harvest. And I removed some logging info which seemed to me to be 
> not useful.
>
> We don’t have an svn repository for the d2rmap source code do we? 
> Maybe we should set that up.
>
> Brian
>
> Brian Keese
>
> Indiana University Digital Library Program
>
> Herman B Wells Library W501
>
> 812-856-3865
>
> *From:*Eliza Chan [mailto:elc2013@...]
> *Sent:* Friday, July 01, 2011 2:22 PM
> *To:* VIVO Development List
> *Subject:* Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster
>
> Hi Brian,
>
> Thanks for making the changes.
>
> It'd be nice to have d2rmap harvests (db and csv) being supported in 
> future releases, since there are people who use the tools on a regular 
> basis.
>
> I have been trying to update the documentation on d2rmap but it has 
> not been able to keep up with so many changes with Harvester. :-(
>
> Would you be able to document some of it? Also, would you be able to 
> list the bugs that you fixed?
>
> Thanks,
>
> Eliza
>
> *From: *"Keese, Brian W" <bkeese@... <mailto:bkeese@...>>
> *Reply-To: *VIVO Development List <vivo-dev-all@... 
> <mailto:vivo-dev-all@...>>
> *Date: *Fri, 1 Jul 2011 14:05:23 -0400
> *To: *VIVO Development List <vivo-dev-all@... 
> <mailto:vivo-dev-all@...>>
> *Subject: *[Vivo-dev-all] d2rmap harvesting, was oracle for harverster
>
> Nick, Thanks for the info on Oracle. Makes sense.
>
> Here’s another thing. I have made a couple of changes to the d2rmap 
> processor that is included with the harvester. I have uploaded the new 
> jar file to 
> https://www.slashtmp.iu.edu/files/download?FILE=bkeese%2F94328zp0Qy2 
> (the password is ‘d2r4vivo’).
>
> It fixes a couple of bugs, and adds some status info to the logs for 
> d2rmap fetch.
>
> I noticed that there is no example script for d2rmap in the latest 
> release. I hope support for d2r harvests will be continued.
>
> Brian
>
> Brian Keese
>
> Indiana University Digital Library Program
>
> Herman B Wells Library W501
>
> 812-856-3865
>
> *From:*Nicholas Skaggs [mailto:nskaggs@...]
> *Sent:* Friday, July 01, 2011 10:51 AM
> *To:* vivo-dev-all@... 
> <mailto:vivo-dev-all@...>
> *Subject:* Re: [Vivo-dev-all] Vivo development call / meeting reminder 
> for today.
>
> Great catch Brian. The trouble with oracle is that it requires a 
> specific jdbc connector for each release -- including minor point 
> release versions.
>
> http://www.oracle.com/technetwork/database/features/jdbc/index-091264.html
>
> We can't package every version of the connector with the harvester 
> unfortunately. I have heard of a so called "universal" connector for 
> oracle that connects to the databases through lowest common 
> denominator SQL. I've never been able to find anything additional 
> about this, nor a download for it -- so I can't verify it exists or 
> not.  In addition, the connectors are released by Oracle under the 
> Oracle Technology Network Development and Distribution License. This 
> prevents us from distributing these files by packaging them directly 
> without complying with the license terms. Specifically, the 
> redistribution clause of the license causes trouble:
>
> /Program Distribution
> We grant you a nonexclusive, nontransferable right to copy and 
> distribute the programs to your end users provided that you do not 
> charge your end users for use of the programs and provided your end 
> users may only use the programs to run your applications for their 
> business operations. Prior to distributing the programs you shall 
> require your end users to execute an agreement binding them to terms 
> consistent with those contained in this section and the sections of 
> this agreement entitled "License Rights," "Ownership and 
> Restrictions," "Export," "Disclaimer of Warranties and Exclusive 
> Remedies," "No Technical Support," "End of Agreement," "Relationship 
> Between the Parties," and "Open Source." You must also include a 
> provision stating that your end users shall have no right to 
> distribute the programs, and a provision specifying us as a third 
> party beneficiary of the agreement. You are responsible for obtaining 
> these agreements with your end users./
>
> You can read the entire license here:
>
> http://www.oracle.com/technetwork/licenses/distribution-license-152002.html
>
> If you have an idea about how we can distribute something to help the 
> harvester work out of the box with oracle, please let us know. Thanks,
>
> Nicholas
>
> On 7/1/11 10:39 AM, Keese, Brian W wrote:
>
> The attachment didn't work. The driver can be downloaded from Oracle here:
>   
> http://www.oracle.com/technetwork/database/enterprise-edition/jdbc-10201-088211.html
> Brian
> ________________________________________
> From: Keese, Brian W [bkeese@...  <mailto:bkeese@...>]
> Sent: Friday, July 01, 2011 10:36 AM
> To:jrpence@...  <mailto:jrpence@...>; VIVO Development List
> Subject: Re: [Vivo-dev-all] Vivo development call / meeting reminder for today.
>   
> Hey Harvester Guys,
> I noticed that the oracle jdbc driver was never added (or was added, then removed) to the harvester bin/dependencies folder for the latest release. Could you add that in please?
> It is ojdbc14.jar (attached to this email).
>   
> Thanks,
> Brian
>   
>   
> ________________________________________
> From: James Pence [jrpence@...  <mailto:jrpence@...>]
> Sent: Thursday, June 30, 2011 1:23 PM
> To:vivo-dev-all@...  <mailto:vivo-dev-all@...>
> Subject: [Vivo-dev-all] Vivo development call / meeting reminder for today.
>   
> Hello,
>   
>      Today the vivo developers, and maybe you, are having a goto-meeting at 3:00pm EST (UTC-4 hours)
>   
>      We have the beginnings of an agenda at<http://sourceforge.net/apps/mediawiki/vivo/index.php?title=2011-06-30_Development_Leads_Call>
>   
> If you have any agenda items to add to the meeting please let me know (jrpence@...  <mailto:jrpence@...>    ).
>   
>     Since we only have 25 connections for the goto meeting please consolidate where possible.
>   
>     1. Please join my meeting, Thursday, June 30 at 3:00 PM Eastern DaylightTime.
>     https://www1.gotomeeting.com/join/486839241
>   
>     2. Use your microphone and speakers (VoIP) - a headset is recommended.
>   
>     Or, call in using your telephone.
>     Dial +1 (215) 383-1013
>   
>     Access Code: 486-839-241
>     Audio PIN: Shown after joining the meeting
>     Meeting ID: 486-839-241
>   
>     GoToMeeting®
>     Online Meetings Made Easy™
>   
>   
>   
> --
> James R. Pence
> jrpence@...  <mailto:jrpence@...>
> Software Engineer
> Communicore C2-12
> (352)294-5154 (ex. 45154)
> Enterprise Software Engineering
> University of Florida
> Academic Health Center IT
> Developer on the VIVO Grant
>   
>   
> ------------------------------------------------------------------------------
> All of the data generated in your IT infrastructure is seriously valuable.
> Why? It contains a definitive record of application performance, security
> threats, fraudulent activity, and more. Splunk takes this data and makes
> sense of it. IT sense. And common sense.
> http://p.sf.net/sfu/splunk-d2d-c2
> _______________________________________________
> Vivo-dev-all mailing list
> Vivo-dev-all@...  <mailto:Vivo-dev-all@...>
> https://lists.sourceforge.net/lists/listinfo/vivo-dev-all
>   
> ------------------------------------------------------------------------------
> All of the data generated in your IT infrastructure is seriously valuable.
> Why? It contains a definitive record of application performance, security
> threats, fraudulent activity, and more. Splunk takes this data and makes
> sense of it. IT sense. And common sense.
> http://p.sf.net/sfu/splunk-d2d-c2
> _______________________________________________
> Vivo-dev-all mailing list
> Vivo-dev-all@...  <mailto:Vivo-dev-all@...>
> https://lists.sourceforge.net/lists/listinfo/vivo-dev-all
>

Hi Nicholas,

Yeah, it's odd that there were no license statements at all. If there is no objection from Brian, would you mind adding his name to the author list? Thanks for updating.

Eliza

From: Nicholas Skaggs <nskaggs@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Thu, 21 Jul 2011 16:51:54 -0400
To: "vivo-dev-all@...>" <vivo-dev-all@...>>
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Eliza -- just looking at the code now. Umm, yes odd that the D2R team didn't include the license in the file headers. The only issue incorrect with what you have is the version of the GPL included -- since the code was written before gpl v3, it's gpl v2 :-) I'll update accordingly. Thanks,

Nicholas


On 07/21/2011 04:37 PM, Eliza Chan wrote:
Hi Nicholas,

Just waiting for your feedback regarding the license statement in D2rProcessor.java.

Thanks,
Eliza

From: "Keese, Brian W" <bkeese@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Thu, 21 Jul 2011 15:54:50 -0400
To: VIVO Development List <vivo-dev-all@...>>
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

I have committed my bug fixes to the D2r code on git. Should be good to go.

Brian

Brian Keese
Indiana University Digital Library Program
Herman B Wells Library W501
812-856-3865

From: Eliza Chan [mailto:elc2013@...]
Sent: Thursday, July 21, 2011 12:14 PM
To: VIVO Development List
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster


Sorry, it should be there now.
There is no need to change the d2rmap.config.xml file. It should work with the fixes.
Feel free to revert back to H2.

Thanks,
Eliza

From: Nicholas Skaggs <nskaggs@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Thu, 21 Jul 2011 11:59:27 -0400
To: "vivo-dev-all@...>" <vivo-dev-all@...>>
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Eliza, I'm not seeing your commits in etheir branch. My apologies if they are there and I'm missing them. Did you push back upstream? Or did you just commit locally? I did try and remove h2 from the equation and just use the mysql database directly, but all i got out of the translate is the following:

<?xml version="1.0" encoding="UTF-8" ?><rdf:RDF
    xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"<http://www.w3.org/1999/02/22-rdf-syntax-ns>
    xmlns:bibo="http://purl.org/ontology/bibo/"<http://purl.org/ontology/bibo/>
    xmlns:foaf="http://xmlns.com/foaf/0.1/"<http://xmlns.com/foaf/0.1/>
    xmlns:localVivo="http://vivo.sample.edu/ontology/"<http://vivo.sample.edu/ontology/>
    xmlns:owl="http://www.w3.org/2002/07/owl#"<http://www.w3.org/2002/07/owl>
    xmlns:xsd="http://www.w3.org/2001/XMLSchema#"<http://www.w3.org/2001/XMLSchema>
    xmlns:core="http://vivoweb.org/ontology/core#"<http://vivoweb.org/ontology/core>
    xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#"<http://www.w3.org/2000/01/rdf-schema> >
  <rdf:Description rdf:about="http://vivoweb.org/harvest/example/d2r/person/personnull"<http://vivoweb.org/harvest/example/d2r/person/personnull>>
    <rdf:type rdf:resource="http://xmlns.com/foaf/0.1/Person"<http://xmlns.com/foaf/0.1/Person>/>
  </rdf:Description>
  <rdf:Description rdf:about="http://vivoweb.org/harvest/example/d2r/org/deptnull"<http://vivoweb.org/harvest/example/d2r/org/deptnull>>
    <rdf:type rdf:resource="http://xmlns.com/foaf/0.1/Organization"<http://xmlns.com/foaf/0.1/Organization>/>
  </rdf:Description>
</rdf:RDF>

Thanks for your help!

Nicholas

On 07/21/2011 11:08 AM, Eliza Chan wrote:
Hi Nicholas and Brian,

Changes have been committed to the d2rmap branch. A GPL/LGPL statement was added to D2rProcessor. Please make changes to the statement if needed, and copy to the other classes that have been modified. The revised classes are: D2rProcessor, Map, Instance, SourceControllerD2R.

Also, the d2rmap example in the master branch should work now. It's related to a bug in D2RMap that returned the column name (e.g. user_id) rather that the column label (e.g. UID) in a query with a "AS" clause (e.g. user_id AS UID). It seemed to affect H2 and not Oracle. Please give it a try and confirm.

Thanks,
Eliza

From: Nicholas Skaggs <nskaggs@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Tue, 19 Jul 2011 11:14:24 -0400
To: "vivo-dev-all@...>" <vivo-dev-all@...>>
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Eliza and Brian,
     we've created a d2rmap branch in the git repository for the harvester -- origin/D2rMapIntegration. It contains the latest code from the d2rmap repository. Can you two add your patchsets to this code to bring it up to date with what we want to ship with the harvester? We'll then need to update maven to build a jar for the harvester to use as part of the build for the harvester.

Secondly, we have the d2rmap example in the master branch on git. While the example doesn't give an error, no data is being added to VIVO. Can you take a look and figure out what's wrong? The team is ready to push out a 1.2 final of the harvester once this example is complete :-)

Now, onto your questions below:

1. I mean you should license your modifications to d2rmap as GPL/LGPL so that there project can accept them under than same license terms as d2rmap is licensed.

2. We are NOT required to followup with them, but I think it would be a good idea to let them know of our work. Once we have your code integrated, I would suggest sending the patchset to the d2rmap mailing list and/or letting them now we have changes in our git which are publically available.

3. The team seemed to favor including the code inline with the harvester, so no sub-project has been created.

4. Licensing file has been updated appropriately for this, so we're good.

5. I'm not sure where this is coming from, but making available the source, and making our changes available, as well as noting the license of the code as being GPL/LGPL meets the terms of the license.

Nicholas


On 7/8/11 11:26 AM, Eliza Chan wrote:
Hi Nicholas,

Please see my questions regarding your action items:

1. License modifications as GPL/LGPL

Did you mean adding GPL/LGPL to Licensing & Acknowledgements?
Or did you mean adding a license statement to the source code?

2. Push patchsets to D2RMap community

Under the license agreement, I don't see anywhere that we are required to push the patchsets back to the D2RMap community. Could someone confirm this please?
However, if we really want to check in the changes, Brian's patch seems to be more relevant because it's related to a bug fix. But my part was mainly for integrating D2RMap to VIVO, so I'm not sure if it's relevant to the D2RMap project.

3. Host our modified version of the code on SF (seperate sub-project?)

How can one create a sub-project in SF?

4. Use the modified binary in the harvester, noting the GPL license

Did you mean adding a LICENSE file somewhere in the Harvester source code?

5.
Under the "TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION",
section 6, paragraph b: You must cause the files modified to carry prominent notices stating that you changed the files and the date of any change.

Have we addressed this? I'm also not a lawyer. ;-)

Thanks,
Eliza

From: Nicholas Skaggs <nskaggs@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Thu, 7 Jul 2011 14:02:49 -0400
To: "vivo-dev-all@...>" <vivo-dev-all@...>>
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Given the shortcomings of email, I don't want to be mis-interpreted on my reply.

I would like to preface this email with "In my opinion", and "I think we should' :-)

Thanks,

Nicholas

On 7/7/11 1:58 PM, Nicholas Skaggs wrote:
Jim, yes we must consider these. For distributing a binary, we can simply disclose it's license as GPL, and do nothing more.  LGPL and GPL allow for different things in terms of linking and non-open source software, but we don't need to get into that here. BSD is "compatible" with the GPL/LGPL. In this case, Eliza/Brian has modified GPL/LGPL source (they appear to have it dual licensed). We can distribute this of course, but we do need to make those modifications available in source form upon request. This can be done thru writing and physical media (we can even charge the cost of distribution), etc, but the simplest and most common is to host the source in a freely downloadable place.

So, I didn't realize D2RMap was an existing SF project. We essentially have 2 options. We can fork there project and host it ourselves with our modifications, or we can contribute them back upstream and see if we can get them to adopt them. I would suggest starting with the latter. Depending on the outcome, in the meantime, we will be maintaining a "fork" of the project with our modifications as part of the harvester since we want to utilize it. Now, the best practice for Eliza and Brian would be to make there modifications back to the D2RMap project as GPL/LGPL (there current license format). If they licensed it as BSD or something else, the project would have part of the code under a different license which is no good, and would make it harder for them to accept.

The ramifications then for the harvester is we must ship the GPL license with our project and state that D2RMap is under that license. Not a big deal, it's an external library; but we should make note of it.

In summary we should:

License modifications as GPL/LGPL
Push patchsets to D2RMap community
Host our modified version of the code on SF (seperate sub-project?)
Use the modified binary in the harvester, noting the GPL license

Nicholas

On 7/7/11 1:50 PM, Eliza Chan wrote:
Hi Jim,

Thanks for looking up.
In this case, committing the D2RMap source code to our project is not only Ok, it's required.

Eliza

From: Jim Blake <jeb228@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Thu, 7 Jul 2011 13:41:20 -0400
To: VIVO Development List <vivo-dev-all@...>>
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Eliza;

That’s a very interesting question, and it opens an big can of worms.

I am not a lawyer, but it appears that D2RMap is covered under the GNU Lesser General Public License (LGPL), which is a modification of the GNU General Public License, which appears to cover this scenario in section 6, paragraph d.

If I understand this correctly, if you offer the object code (i.e., JAR files or CLASS files) for public download as part of your product, you are required to offer the source code (i.e. JAVA files) for public download on the same server, or an equivalent server operated by you or by a third party.

So if our software is available for download from SourceForge, and the D2RMap source code is available from SourceForge, have we satisfied the terms of the license?

Have I mentioned that I have no legal training or expertise?

I wonder whether the VIVO project is in violation of the terms of many additional open source licenses. For exmaple, some licenses require that we cite them in all of our end-user documentation. Do we do that?

But I’m just taking wild guesses, here. Can anybody shed more light on this?

Jim


From: Eliza Chan [mailto:elc2013@...]
Sent: Thursday, July 07, 2011 11:59 AM
To: VIVO Development List
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Hi Nicholas and Brian,

I can definitely commit D2RMap to sourceforge.
Just a policy question before doing that.
As you are probably aware, D2RMap is a sourceforge project: https://sourceforge.net/projects/d2r-map/
Do we need to do anything extra (whatever that is) before we post their modified copy under our project?

Thanks,
Eliza

From: Nicholas Skaggs <nskaggs@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Thu, 7 Jul 2011 11:10:13 -0400
To: "vivo-dev-all@...>" <vivo-dev-all@...>>
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Brian and Eliza,
    Let's get a merged copy of your code into the harvester repo. Brian, it seems your new jar is missing some of Eliza's code -- specifically the CSV Map:

The method processCsvMap(String, String, String) is undefined for the type D2rProcessor    D2RMapFetch.java    /Harvester-hg/src/main/java/org/vivoweb/harvester/fetch    line 113    Java Problem

Eliza, I know you have commit rights to SVN -- Brian do you? Eliza can you check in your source for D2RMap, and then have Brian add his changes in as another commit, etc. That would be really awesome. I'm sure others who are using it would be excited to be able to commit fixes and updates as well.

Right now we're still working on restoring the SVN from the mercurial commits we did. As soon as that's done, we can get your commits in.

Thanks!

Nicholas


On 7/1/11 2:43 PM, Eliza Chan wrote:
You are right, we should have a repository for the d2rmap source code. Since now you have a modified copy, and I also have one. It's better to merge them.


From: "Keese, Brian W" <bkeese@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Fri, 1 Jul 2011 14:37:11 -0400
To: VIVO Development List <vivo-dev-all@...>>
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Sure. Let me know how I can help with the docs.

For the bugs, one of them was fixed before (the oracle semi-colon thing, Map.java around line 94).

The other bug has to do with null values for instance identifiers. In Map.java around line 126, a list of new instances is being compiled. Unique instances are defined by values in a discreet set of fields. If one those values happens to be null, this line would give a null pointer exception:

if (!currentTuple.get(fieldName).equals(lastTuple.get(fieldName)))

So, I changed the logic to this:

if (!(currentTuple.get(fieldName)==null && lastTuple.get(fieldName)==null) &&
                      (currentTuple.get(fieldName)==null || !currentTuple.get(fieldName).equals(lastTuple.get(fieldName))))

I also added some logging info to help with diagnosing problems with the harvest. And I removed some logging info which seemed to me to be not useful.

We don’t have an svn repository for the d2rmap source code do we? Maybe we should set that up.

Brian

Brian Keese
Indiana University Digital Library Program
Herman B Wells Library W501
812-856-3865

From: Eliza Chan [mailto:elc2013@...]
Sent: Friday, July 01, 2011 2:22 PM
To: VIVO Development List
Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Hi Brian,

Thanks for making the changes.
It'd be nice to have d2rmap harvests (db and csv) being supported in future releases, since there are people who use the tools on a regular basis.
I have been trying to update the documentation on d2rmap but it has not been able to keep up with so many changes with Harvester. :-(
Would you be able to document some of it? Also, would you be able to list the bugs that you fixed?

Thanks,
Eliza

From: "Keese, Brian W" <bkeese@...>>
Reply-To: VIVO Development List <vivo-dev-all@...>>
Date: Fri, 1 Jul 2011 14:05:23 -0400
To: VIVO Development List <vivo-dev-all@...>>
Subject: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster

Nick, Thanks for the info on Oracle. Makes sense.

Here’s another thing. I have made a couple of changes to the d2rmap processor that is included with the harvester. I have uploaded the new jar file to https://www.slashtmp.iu.edu/files/download?FILE=bkeese%2F94328zp0Qy2 (the password is ‘d2r4vivo’).
It fixes a couple of bugs, and adds some status info to the logs for d2rmap fetch.

I noticed that there is no example script for d2rmap in the latest release. I hope support for d2r harvests will be continued.

Brian

Brian Keese
Indiana University Digital Library Program
Herman B Wells Library W501
812-856-3865

From: Nicholas Skaggs [mailto:nskaggs@...]
Sent: Friday, July 01, 2011 10:51 AM
To: vivo-dev-all@...>
Subject: Re: [Vivo-dev-all] Vivo development call / meeting reminder for today.

Great catch Brian. The trouble with oracle is that it requires a specific jdbc connector for each release -- including minor point release versions.

http://www.oracle.com/technetwork/database/features/jdbc/index-091264.html

We can't package every version of the connector with the harvester unfortunately. I have heard of a so called "universal" connector for oracle that connects to the databases through lowest common denominator SQL. I've never been able to find anything additional about this, nor a download for it -- so I can't verify it exists or not.  In addition, the connectors are released by Oracle under the Oracle Technology Network Development and Distribution License. This prevents us from distributing these files by packaging them directly without complying with the license terms. Specifically, the redistribution clause of the license causes trouble:

Program Distribution
We grant you a nonexclusive, nontransferable right to copy and distribute the programs to your end users provided that you do not charge your end users for use of the programs and provided your end users may only use the programs to run your applications for their business operations. Prior to distributing the programs you shall require your end users to execute an agreement binding them to terms consistent with those contained in this section and the sections of this agreement entitled "License Rights," "Ownership and Restrictions," "Export," "Disclaimer of Warranties and Exclusive Remedies," "No Technical Support," "End of Agreement," "Relationship Between the Parties," and "Open Source." You must also include a provision stating that your end users shall have no right to distribute the programs, and a provision specifying us as a third party beneficiary of the agreement. You are responsible for obtaining these agreements with your end users.

You can read the entire license here:

http://www.oracle.com/technetwork/licenses/distribution-license-152002.html

If you have an idea about how we can distribute something to help the harvester work out of the box with oracle, please let us know. Thanks,

Nicholas

On 7/1/11 10:39 AM, Keese, Brian W wrote:

The attachment didn't work. The driver can be downloaded from Oracle here:



http://www.oracle.com/technetwork/database/enterprise-edition/jdbc-10201-088211.html

Brian

________________________________________

From: Keese, Brian W [bkeese@...>]

Sent: Friday, July 01, 2011 10:36 AM

To: jrpence@...>; VIVO Development List

Subject: Re: [Vivo-dev-all] Vivo development call / meeting reminder for today.



Hey Harvester Guys,

I noticed that the oracle jdbc driver was never added (or was added, then removed) to the harvester bin/dependencies folder for the latest release. Could you add that in please?

It is ojdbc14.jar (attached to this email).



Thanks,

Brian





________________________________________

From: James Pence [jrpence@...>]

Sent: Thursday, June 30, 2011 1:23 PM

To: vivo-dev-all@...>

Subject: [Vivo-dev-all] Vivo development call / meeting reminder for today.



Hello,



    Today the vivo developers, and maybe you, are having a goto-meeting at 3:00pm EST (UTC-4 hours)



    We have the beginnings of an agenda at<http://sourceforge.net/apps/mediawiki/vivo/index.php?title=2011-06-30_Development_Leads_Call><http://sourceforge.net/apps/mediawiki/vivo/index.php?title=2011-06-30_Development_Leads_Call>



If you have any agenda items to add to the meeting please let me know (jrpence@...>   ).



   Since we only have 25 connections for the goto meeting please consolidate where possible.



   1. Please join my meeting, Thursday, June 30 at 3:00 PM Eastern DaylightTime.

   https://www1.gotomeeting.com/join/486839241



   2. Use your microphone and speakers (VoIP) - a headset is recommended.



   Or, call in using your telephone.

   Dial +1 (215) 383-1013



   Access Code: 486-839-241

   Audio PIN: Shown after joining the meeting

   Meeting ID: 486-839-241



   GoToMeeting®

   Online Meetings Made Easy™







--

James R. Pence

jrpence@...>

Software Engineer

Communicore C2-12

(352)294-5154 (ex. 45154)

Enterprise Software Engineering

University of Florida

Academic Health Center IT

Developer on the VIVO Grant





------------------------------------------------------------------------------

All of the data generated in your IT infrastructure is seriously valuable.

Why? It contains a definitive record of application performance, security

threats, fraudulent activity, and more. Splunk takes this data and makes

sense of it. IT sense. And common sense.

http://p.sf.net/sfu/splunk-d2d-c2

_______________________________________________

Vivo-dev-all mailing list

Vivo-dev-all@...>

https://lists.sourceforge.net/lists/listinfo/vivo-dev-all



------------------------------------------------------------------------------

All of the data generated in your IT infrastructure is seriously valuable.

Why? It contains a definitive record of application performance, security

threats, fraudulent activity, and more. Splunk takes this data and makes

sense of it. IT sense. And common sense.

http://p.sf.net/sfu/splunk-d2d-c2

_______________________________________________

Vivo-dev-all mailing list

Vivo-dev-all@...>

https://lists.sourceforge.net/lists/listinfo/vivo-dev-all

Brian, I did add you to the author list. No worries :-) Thanks for the 
patches.

Nicholas


On 7/21/11 5:03 PM, Eliza Chan wrote:
> Hi Nicholas,
>
> Yeah, it's odd that there were no license statements at all. If there 
> is no objection from Brian, would you mind adding his name to the 
> author list? Thanks for updating.
>
> Eliza
>
> From: Nicholas Skaggs <nskaggs@... <mailto:nskaggs@...>>
> Reply-To: VIVO Development List <vivo-dev-all@... 
> <mailto:vivo-dev-all@...>>
> Date: Thu, 21 Jul 2011 16:51:54 -0400
> To: "vivo-dev-all@... 
> <mailto:vivo-dev-all@...>" 
> <vivo-dev-all@... 
> <mailto:vivo-dev-all@...>>
> Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster
>
> Eliza -- just looking at the code now. Umm, yes odd that the D2R team 
> didn't include the license in the file headers. The only issue 
> incorrect with what you have is the version of the GPL included -- 
> since the code was written before gpl v3, it's gpl v2 :-) I'll update 
> accordingly. Thanks,
>
> Nicholas
>
>
> On 07/21/2011 04:37 PM, Eliza Chan wrote:
>> Hi Nicholas,
>>
>> Just waiting for your feedback regarding the license statement 
>> in D2rProcessor.java.
>>
>> Thanks,
>> Eliza
>>
>> From: "Keese, Brian W" <bkeese@... <mailto:bkeese@...>>
>> Reply-To: VIVO Development List <vivo-dev-all@... 
>> <mailto:vivo-dev-all@...>>
>> Date: Thu, 21 Jul 2011 15:54:50 -0400
>> To: VIVO Development List <vivo-dev-all@... 
>> <mailto:vivo-dev-all@...>>
>> Subject: Re: [Vivo-dev-all] d2rmap harvesting, was oracle for harverster
>>
>> I have committed my bug fixes to the D2r code on git. Should be good 
>> to go.
>>
>> Brian
>>
>> Brian Keese
>>
>> Indiana University Digital Library Program
>>
>> Herman B Wells Library W501
>>
>> 812-856-3865
>>
>> *From:*Eliza Chan [mailto:elc2013@...]
>> *Sent:* Thursday, July 21, 2011 12:14 PM
>> *To:* VIVO Development List
>> *Subject:* Re: [Vivo-dev-all] d2rmap harvesting, was oracle for 
>> harverster
>>
>> Sorry, it should be there now.
>>
>> There is no need to change the d2rmap.config.xml file. It should work 
>> with the fixes.
>>
>> Feel free to revert back to H2.
>>
>> Thanks,
>>
>> Eliza
>>
>> *From: *Nicholas Skaggs <nskaggs@... <mailto:nskaggs@...>>
>> *Reply-To: *VIVO Development List <vivo-dev-all@... 
>> <mailto:vivo-dev-all@...>>
>> *Date: *Thu, 21 Jul 2011 11:59:27 -0400
>> *To: *"vivo-dev-all@... 
>> <mailto:vivo-dev-all@...>" 
>> <vivo-dev-all@... 
>> <mailto:vivo-dev-all@...>>
>> *Subject: *Re: [Vivo-dev-all] d2rmap harvesting, was oracle for 
>> harverster
>>
>> Eliza, I'm not seeing your commits in etheir branch. My apologies if 
>> they are there and I'm missing them. Did you push back upstream? Or 
>> did you just commit locally? I did try and remove h2 from the 
>> equation and just use the mysql database directly, but all i got out 
>> of the translate is the following:
>>
>> <?xml version="1.0" encoding="UTF-8" ?><rdf:RDF
>>     xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" 
>> <http://www.w3.org/1999/02/22-rdf-syntax-ns>
>>     xmlns:bibo="http://purl.org/ontology/bibo/" 
>> <http://purl.org/ontology/bibo/>
>>     xmlns:foaf="http://xmlns.com/foaf/0.1/" <http://xmlns.com/foaf/0.1/>
>>     xmlns:localVivo="http://vivo.sample.edu/ontology/" 
>> <http://vivo.sample.edu/ontology/>
>>     xmlns:owl="http://www.w3.org/2002/07/owl#" 
>> <http://www.w3.org/2002/07/owl>
>>     xmlns:xsd="http://www.w3.org/2001/XMLSchema#" 
>> <http://www.w3.org/2001/XMLSchema>
>>     xmlns:core="http://vivoweb.org/ontology/core#" 
>> <http://vivoweb.org/ontology/core>
>>     xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#" 
>> <http://www.w3.org/2000/01/rdf-schema> >
>> <rdf:Description 
>> rdf:about="http://vivoweb.org/harvest/example/d2r/person/personnull" 
>> <http://vivoweb.org/harvest/example/d2r/person/personnull>>
>> <rdf:type rdf:resource="http://xmlns.com/foaf/0.1/Person" 
>> <http://xmlns.com/foaf/0.1/Person>/>
>> </rdf:Description>
>> <rdf:Description 
>> rdf:about="http://vivoweb.org/harvest/example/d2r/org/deptnull" 
>> <http://vivoweb.org/harvest/example/d2r/org/deptnull>>
>> <rdf:type rdf:resource="http://xmlns.com/foaf/0.1/Organization" 
>> <http://xmlns.com/foaf/0.1/Organization>/>
>> </rdf:Description>
>> </rdf:RDF>
>>
>> Thanks for your help!
>>
>> Nicholas
>>
>> On 07/21/2011 11:08 AM, Eliza Chan wrote:
>>
>> Hi Nicholas and Brian,
>>
>> Changes have been committed to the d2rmap branch. A GPL/LGPL 
>> statement was added to D2rProcessor. Please make changes to the 
>> statement if needed, and copy to the other classes that have been 
>> modified. The revised classes 
>> are: D2rProcessor, Map, Instance, SourceControllerD2R.
>>
>> Also, the d2rmap example in the master branch should work now. It's 
>> related to a bug in D2RMap that returned the column name (e.g. 
>> user_id) rather that the column label (e.g. UID) in a query with a 
>> "AS" clause (e.g. user_id AS UID). It seemed to affect H2 and not 
>> Oracle. Please give it a try and confirm.
>>
>> Thanks,
>>
>> Eliza
>>
>> *From: *Nicholas Skaggs <nskaggs@... <mailto:nskaggs@...>>
>> *Reply-To: *VIVO Development List <vivo-dev-all@... 
>> <mailto:vivo-dev-all@...>>
>> *Date: *Tue, 19 Jul 2011 11:14:24 -0400
>> *To: *"vivo-dev-all@... 
>> <mailto:vivo-dev-all@...>" 
>> <vivo-dev-all@... 
>> <mailto:vivo-dev-all@...>>
>> *Subject: *Re: [Vivo-dev-all] d2rmap harvesting, was oracle for 
>> harverster
>>
>> Eliza and Brian,
>>      we've created a d2rmap branch in the git repository for the 
>> harvester -- origin/D2rMapIntegration. It contains the latest code 
>> from the d2rmap repository. Can you two add your patchsets to this 
>> code to bring it up to date with what we want to ship with the 
>> harvester? We'll then need to update maven to build a jar for the 
>> harvester to use as part of the build for the harvester.
>>
>> Secondly, we have the d2rmap example in the master branch on git. 
>> While the example doesn't give an error, no data is being added to 
>> VIVO. Can you take a look and figure out what's wrong? The team is 
>> ready to push out a 1.2 final of the harvester once this example is 
>> complete :-)
>>
>> Now, onto your questions below:
>>
>> 1. I mean you should license your modifications to d2rmap as GPL/LGPL 
>> so that there project can accept them under than same license terms 
>> as d2rmap is licensed.
>>
>> 2. We are NOT required to followup with them, but I think it would be 
>> a good idea to let them know of our work. Once we have your code 
>> integrated, I would suggest sending the patchset to the d2rmap 
>> mailing list and/or letting them now we have changes in our git which 
>> are publically available.
>>
>> 3. The team seemed to favor including the code inline with the 
>> harvester, so no sub-project has been created.
>>
>> 4. Licensing file has been updated appropriately for this, so we're good.
>>
>> 5. I'm not sure where this is coming from, but making available the 
>> source, and making our changes available, as well as noting the 
>> license of the code as being GPL/LGPL meets the terms of the license.
>>
>> Nicholas
>>
>>
>> On 7/8/11 11:26 AM, Eliza Chan wrote:
>>
>> Hi Nicholas,
>>
>> Please see my questions regarding your action items:
>>
>> 1. License modifications as GPL/LGPL
>>
>> Did you mean adding GPL/LGPL to Licensing & Acknowledgements?
>>
>> Or did you mean adding a license statement to the source code?
>>
>> 2. Push patchsets to D2RMap community
>>
>> Under the license agreement, I don't see anywhere that we are 
>> required to push the patchsets back to the D2RMap community. Could 
>> someone confirm this please?
>>
>> However, if we really want to check in the changes, Brian's patch 
>> seems to be more relevant because it's related to a bug fix. But my 
>> part was mainly for integrating D2RMap to VIVO, so I'm not sure if 
>> it's relevant to the D2RMap project.
>>
>> 3. Host our modified version of the code on SF (seperate sub-project?)
>>
>> How can one create a sub-project in SF?
>>
>> 4. Use the modified binary in the harvester, noting the GPL license
>>
>> Did you mean adding a LICENSE file somewhere in the Harvester source 
>> code?
>>
>> 5.
>>
>> Under the "TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND 
>> MODIFICATION",
>>
>> section 6, paragraph b: You must cause the files modified to carry 
>> prominent notices stating that you changed the files and the date of 
>> any change.
>>
>> Have we addressed this? I'm also not a lawyer. ;-)
>>
>> Thanks,
>>
>> Eliza
>>
>> *From: *Nicholas Skaggs <nskaggs@... <mailto:nskaggs@...>>
>> *Reply-To: *VIVO Development List <vivo-dev-all@... 
>> <mailto:vivo-dev-all@...>>
>> *Date: *Thu, 7 Jul 2011 14:02:49 -0400
>> *To: *"vivo-dev-all@... 
>> <mailto:vivo-dev-all@...>" 
>> <vivo-dev-all@... 
>> <mailto:vivo-dev-all@...>>
>> *Subject: *Re: [Vivo-dev-all] d2rmap harvesting, was oracle for 
>> harverster
>>
>> Given the shortcomings of email, I don't want to be mis-interpreted 
>> on my reply.
>>
>> I would like to preface this email with "In my opinion", and "I think 
>> we should' :-)
>>
>> Thanks,
>>
>> Nicholas
>>
>> On 7/7/11 1:58 PM, Nicholas Skaggs wrote:
>>
>> Jim, yes we must consider these. For distributing a binary, we can 
>> simply disclose it's license as GPL, and do nothing more.  LGPL and 
>> GPL allow for different things in terms of linking and non-open 
>> source software, but we don't need to get into that here. BSD is 
>> "compatible" with the GPL/LGPL. In this case, Eliza/Brian has 
>> modified GPL/LGPL source (they appear to have it dual licensed). We 
>> can distribute this of course, but we do need to make those 
>> modifications available in source form upon request. This can be done 
>> thru writing and physical media (we can even charge the cost of 
>> distribution), etc, but the simplest and most common is to host the 
>> source in a freely downloadable place.
>>
>> So, I didn't realize D2RMap was an existing SF project. We 
>> essentially have 2 options. We can fork there project and host it 
>> ourselves with our modifications, or we can contribute them back 
>> upstream and see if we can get them to adopt them. I would suggest 
>> starting with the latter. Depending on the outcome, in the meantime, 
>> we will be maintaining a "fork" of the project with our modifications 
>> as part of the harvester since we want to utilize it. Now, the best 
>> practice for Eliza and Brian would be to make there modifications 
>> back to the D2RMap project as GPL/LGPL (there current license 
>> format). If they licensed it as BSD or something else, the project 
>> would have part of the code under a different license which is no 
>> good, and would make it harder for them to accept.
>>
>> The ramifications then for the harvester is we must ship the GPL 
>> license with our project and state that D2RMap is under that license. 
>> Not a big deal, it's an external library; but we should make note of it.
>>
>> In summary we should:
>>
>> License modifications as GPL/LGPL
>> Push patchsets to D2RMap community
>> Host our modified version of the code on SF (seperate sub-project?)
>> Use the modified binary in the harvester, noting the GPL license
>>
>> Nicholas
>>
>> On 7/7/11 1:50 PM, Eliza Chan wrote:
>>
>> Hi Jim,
>>
>> Thanks for looking up.
>>
>> In this case, committing the D2RMap source code to our project is not 
>> only Ok, it's required.
>>
>> Eliza
>>
>> *From: *Jim Blake <jeb228@... <mailto:jeb228@...>>
>> *Reply-To: *VIVO Development List <vivo-dev-all@... 
>> <mailto:vivo-dev-all@...>>
>> *Date: *Thu, 7 Jul 2011 13:41:20 -0400
>> *To: *VIVO Development List <vivo-dev-all@... 
>> <mailto:vivo-dev-all@...>>
>> *Subject: *Re: [Vivo-dev-all] d2rmap harvesting, was oracle for 
>> harverster
>>
>> Eliza;
>>
>> That’s a very interesting question, and it opens an big can of worms.
>>
>> I am not a lawyer, but it appears that D2RMap is covered under the 
>> GNU Lesser General Public License (LGPL), which is a modification of 
>> the GNU General Public License, which appears to cover this scenario 
>> in section 6, paragraph d.
>>
>> If I understand this correctly, if you offer the object code (i.e., 
>> JAR files or CLASS files) for public download as part of your 
>> product, you are required to offer the source code (i.e. JAVA files) 
>> for public download on the same server, or an equivalent server 
>> operated by you or by a third party.
>>
>> So if our software is available for download from SourceForge, and 
>> the D2RMap source code is available from SourceForge, have we 
>> satisfied the terms of the license?
>>
>> Have I mentioned that I have no legal training or expertise?
>>
>> I wonder whether the VIVO project is in violation of the terms of 
>> many additional open source licenses. For exmaple, some licenses 
>> require that we cite them in all of our end-user documentation. Do we 
>> do that?
>>
>> But I’m just taking wild guesses, here. Can anybody shed more light 
>> on this?
>>
>> Jim
>>
>> *From:*Eliza Chan [mailto:elc2013@...]
>> *Sent:* Thursday, July 07, 2011 11:59 AM
>> *To:* VIVO Development List
>> *Subject:* Re: [Vivo-dev-all] d2rmap harvesting, was oracle for 
>> harverster
>>
>> Hi Nicholas and Brian,
>>
>> I can definitely commit D2RMap to sourceforge.
>>
>> Just a policy question before doing that.
>>
>> As you are probably aware, D2RMap is a sourceforge project: 
>> https://sourceforge.net/projects/d2r-map/
>>
>> Do we need to do anything extra (whatever that is) before we post 
>> their modified copy under our project?
>>
>> Thanks,
>>
>> Eliza
>>
>> *From: *Nicholas Skaggs <nskaggs@... <mailto:nskaggs@...>>
>> *Reply-To: *VIVO Development List <vivo-dev-all@... 
>> <mailto:vivo-dev-all@...>>
>> *Date: *Thu, 7 Jul 2011 11:10:13 -0400
>> *To: *"vivo-dev-all@... 
>> <mailto:vivo-dev-all@...>" 
>> <vivo-dev-all@... 
>> <mailto:vivo-dev-all@...>>
>> *Subject: *Re: [Vivo-dev-all] d2rmap harvesting, was oracle for 
>> harverster
>>
>> Brian and Eliza,
>>     Let's get a merged copy of your code into the harvester repo. 
>> Brian, it seems your new jar is missing some of Eliza's code -- 
>> specifically the CSV Map:
>>
>> The method processCsvMap(String, String, String) is undefined for the 
>> type D2rProcessor    D2RMapFetch.java    
>> /Harvester-hg/src/main/java/org/vivoweb/harvester/fetch    line 
>> 113    Java Problem
>>
>> Eliza, I know you have commit rights to SVN -- Brian do you? Eliza 
>> can you check in your source for D2RMap, and then have Brian add his 
>> changes in as another commit, etc. That would be really awesome. I'm 
>> sure others who are using it would be excited to be able to commit 
>> fixes and updates as well.
>>
>> Right now we're still working on restoring the SVN from the mercurial 
>> commits we did. As soon as that's done, we can get your commits in.
>>
>> Thanks!
>>
>> Nicholas
>>
>>
>> On 7/1/11 2:43 PM, Eliza Chan wrote:
>>
>> You are right, we should have a repository for the d2rmap source 
>> code. Since now you have a modified copy, and I also have one. It's 
>> better to merge them.
>>
>> *From: *"Keese, Brian W" <bkeese@... <mailto:bkeese@...>>
>> *Reply-To: *VIVO Development List <vivo-dev-all@... 
>> <mailto:vivo-dev-all@...>>
>> *Date: *Fri, 1 Jul 2011 14:37:11 -0400
>> *To: *VIVO Development List <vivo-dev-all@... 
>> <mailto:vivo-dev-all@...>>
>> *Subject: *Re: [Vivo-dev-all] d2rmap harvesting, was oracle for 
>> harverster
>>
>> Sure. Let me know how I can help with the docs.
>>
>> For the bugs, one of them was fixed before (the oracle semi-colon 
>> thing, Map.java around line 94).
>>
>> The other bug has to do with null values for instance identifiers. In 
>> Map.java around line 126, a list of new instances is being compiled. 
>> Unique instances are defined by values in a discreet set of fields. 
>> If one those values happens to be null, this line would give a null 
>> pointer exception:
>>
>> if (!currentTuple.get(fieldName).equals(lastTuple.get(fieldName)))
>>
>> So, I changed the logic to this:
>>
>> if (!(currentTuple.get(fieldName)==null && 
>> lastTuple.get(fieldName)==null) &&
>>
>>                       (currentTuple.get(fieldName)==null || 
>> !currentTuple.get(fieldName).equals(lastTuple.get(fieldName))))
>>
>> I also added some logging info to help with diagnosing problems with 
>> the harvest. And I removed some logging info which seemed to me to be 
>> not useful.
>>
>> We don’t have an svn repository for the d2rmap source code do we? 
>> Maybe we should set that up.
>>
>> Brian
>>
>> Brian Keese
>>
>> Indiana University Digital Library Program
>>
>> Herman B Wells Library W501
>>
>> 812-856-3865
>>
>> *From:*Eliza Chan [mailto:elc2013@...]
>> *Sent:* Friday, July 01, 2011 2:22 PM
>> *To:* VIVO Development List
>> *Subject:* Re: [Vivo-dev-all] d2rmap harvesting, was oracle for 
>> harverster
>>
>> Hi Brian,
>>
>> Thanks for making the changes.
>>
>> It'd be nice to have d2rmap harvests (db and csv) being supported in 
>> future releases, since there are people who use the tools on a 
>> regular basis.
>>
>> I have been trying to update the documentation on d2rmap but it has 
>> not been able to keep up with so many changes with Harvester. :-(
>>
>> Would you be able to document some of it? Also, would you be able to 
>> list the bugs that you fixed?
>>
>> Thanks,
>>
>> Eliza
>>
>> *From: *"Keese, Brian W" <bkeese@... <mailto:bkeese@...>>
>> *Reply-To: *VIVO Development List <vivo-dev-all@... 
>> <mailto:vivo-dev-all@...>>
>> *Date: *Fri, 1 Jul 2011 14:05:23 -0400
>> *To: *VIVO Development List <vivo-dev-all@... 
>> <mailto:vivo-dev-all@...>>
>> *Subject: *[Vivo-dev-all] d2rmap harvesting, was oracle for harverster
>>
>> Nick, Thanks for the info on Oracle. Makes sense.
>>
>> Here’s another thing. I have made a couple of changes to the d2rmap 
>> processor that is included with the harvester. I have uploaded the 
>> new jar file to 
>> https://www.slashtmp.iu.edu/files/download?FILE=bkeese%2F94328zp0Qy2 
>> (the password is ‘d2r4vivo’).
>>
>> It fixes a couple of bugs, and adds some status info to the logs for 
>> d2rmap fetch.
>>
>> I noticed that there is no example script for d2rmap in the latest 
>> release. I hope support for d2r harvests will be continued.
>>
>> Brian
>>
>> Brian Keese
>>
>> Indiana University Digital Library Program
>>
>> Herman B Wells Library W501
>>
>> 812-856-3865
>>
>> *From:*Nicholas Skaggs [mailto:nskaggs@...]
>> *Sent:* Friday, July 01, 2011 10:51 AM
>> *To:* vivo-dev-all@... 
>> <mailto:vivo-dev-all@...>
>> *Subject:* Re: [Vivo-dev-all] Vivo development call / meeting 
>> reminder for today.
>>
>> Great catch Brian. The trouble with oracle is that it requires a 
>> specific jdbc connector for each release -- including minor point 
>> release versions.
>>
>> http://www.oracle.com/technetwork/database/features/jdbc/index-091264.html
>>
>> We can't package every version of the connector with the harvester 
>> unfortunately. I have heard of a so called "universal" connector for 
>> oracle that connects to the databases through lowest common 
>> denominator SQL. I've never been able to find anything additional 
>> about this, nor a download for it -- so I can't verify it exists or 
>> not.  In addition, the connectors are released by Oracle under the 
>> Oracle Technology Network Development and Distribution License. This 
>> prevents us from distributing these files by packaging them directly 
>> without complying with the license terms. Specifically, the 
>> redistribution clause of the license causes trouble:
>>
>> /Program Distribution
>> We grant you a nonexclusive, nontransferable right to copy and 
>> distribute the programs to your end users provided that you do not 
>> charge your end users for use of the programs and provided your end 
>> users may only use the programs to run your applications for their 
>> business operations. Prior to distributing the programs you shall 
>> require your end users to execute an agreement binding them to terms 
>> consistent with those contained in this section and the sections of 
>> this agreement entitled "License Rights," "Ownership and 
>> Restrictions," "Export," "Disclaimer of Warranties and Exclusive 
>> Remedies," "No Technical Support," "End of Agreement," "Relationship 
>> Between the Parties," and "Open Source." You must also include a 
>> provision stating that your end users shall have no right to 
>> distribute the programs, and a provision specifying us as a third 
>> party beneficiary of the agreement. You are responsible for obtaining 
>> these agreements with your end users./
>>
>> You can read the entire license here:
>>
>> http://www.oracle.com/technetwork/licenses/distribution-license-152002.html
>>
>> If you have an idea about how we can distribute something to help the 
>> harvester work out of the box with oracle, please let us know. Thanks,
>>
>> Nicholas
>>
>> On 7/1/11 10:39 AM, Keese, Brian W wrote:
>>
>> The attachment didn't work. The driver can be downloaded from Oracle here:
>>   
>> http://www.oracle.com/technetwork/database/enterprise-edition/jdbc-10201-088211.html
>> Brian
>> ________________________________________
>> From: Keese, Brian W [bkeese@...  <mailto:bkeese@...>]
>> Sent: Friday, July 01, 2011 10:36 AM
>> To:jrpence@...  <mailto:jrpence@...>; VIVO Development List
>> Subject: Re: [Vivo-dev-all] Vivo development call / meeting reminder for today.
>>   
>> Hey Harvester Guys,
>> I noticed that the oracle jdbc driver was never added (or was added, then removed) to the harvester bin/dependencies folder for the latest release. Could you add that in please?
>> It is ojdbc14.jar (attached to this email).
>>   
>> Thanks,
>> Brian
>>   
>>   
>> ________________________________________
>> From: James Pence [jrpence@...  <mailto:jrpence@...>]
>> Sent: Thursday, June 30, 2011 1:23 PM
>> To:vivo-dev-all@...  <mailto:vivo-dev-all@...>
>> Subject: [Vivo-dev-all] Vivo development call / meeting reminder for today.
>>   
>> Hello,
>>   
>>      Today the vivo developers, and maybe you, are having a goto-meeting at 3:00pm EST (UTC-4 hours)
>>   
>>      We have the beginnings of an agenda at<http://sourceforge.net/apps/mediawiki/vivo/index.php?title=2011-06-30_Development_Leads_Call>
>>   
>> If you have any agenda items to add to the meeting please let me know (jrpence@...  <mailto:jrpence@...>    ).
>>   
>>     Since we only have 25 connections for the goto meeting please consolidate where possible.
>>   
>>     1. Please join my meeting, Thursday, June 30 at 3:00 PM Eastern DaylightTime.
>>     https://www1.gotomeeting.com/join/486839241
>>   
>>     2. Use your microphone and speakers (VoIP) - a headset is recommended.
>>   
>>     Or, call in using your telephone.
>>     Dial +1 (215) 383-1013
>>   
>>     Access Code: 486-839-241
>>     Audio PIN: Shown after joining the meeting
>>     Meeting ID: 486-839-241
>>   
>>     GoToMeeting®
>>     Online Meetings Made Easy™
>>   
>>   
>>   
>> --
>> James R. Pence
>> jrpence@...  <mailto:jrpence@...>
>> Software Engineer
>> Communicore C2-12
>> (352)294-5154 (ex. 45154)
>> Enterprise Software Engineering
>> University of Florida
>> Academic Health Center IT
>> Developer on the VIVO Grant
>>   
>>   
>> ------------------------------------------------------------------------------
>> All of the data generated in your IT infrastructure is seriously valuable.
>> Why? It contains a definitive record of application performance, security
>> threats, fraudulent activity, and more. Splunk takes this data and makes
>> sense of it. IT sense. And common sense.
>> http://p.sf.net/sfu/splunk-d2d-c2
>> _______________________________________________
>> Vivo-dev-all mailing list
>> Vivo-dev-all@...  <mailto:Vivo-dev-all@...>
>> https://lists.sourceforge.net/lists/listinfo/vivo-dev-all
>>   
>> ------------------------------------------------------------------------------
>> All of the data generated in your IT infrastructure is seriously valuable.
>> Why? It contains a definitive record of application performance, security
>> threats, fraudulent activity, and more. Splunk takes this data and makes
>> sense of it. IT sense. And common sense.
>> http://p.sf.net/sfu/splunk-d2d-c2
>> _______________________________________________
>> Vivo-dev-all mailing list
>> Vivo-dev-all@...  <mailto:Vivo-dev-all@...>
>> https://lists.sourceforge.net/lists/listinfo/vivo-dev-all
>>
>