basedb-devel — BASE developers mailing list

Re: [basedb-devel] Question about (To)Transferables

[Nicklas N. <ni...@th...>]

Olivier Lefevre wrote:
> Why don't *all* the BasicItem classes for which an info class exists
> implement ToTransferable? E.g., Annotation (doesn't) vs. Experiment
> (does). Just trying to understand the API.

With annotations there is no one-to-one relationship between Annotation and
AnnotationInfo. A single AnnotationInfo object can collect values from
multiple Annotations. This is because annotations can be inherited and an
item can have multiple parents with different annotation values.

> Also, is there a method somewhere connecting FooInfo classes to their
> Foo and vice-versa, maybe along the lines of Item.getItemClass()? 
> For ToTransferable classes the toTransferable method or the type
> parameter provide the link (although not quite in the desired way) 
> but for the others I draw a blank.

No.

/Nicklas

[basedb-devel] Question about (To)Transferables

[Olivier L. <lef...@ya...>]

Why don't *all* the BasicItem classes for which an info class exists
implement ToTransferable? E.g., Annotation (doesn't) vs. Experiment
(does). Just trying to understand the API.

Also, is there a method somewhere connecting FooInfo classes to their
Foo and vice-versa, maybe along the lines of Item.getItemClass()? 
For ToTransferable classes the toTransferable method or the type
parameter provide the link (although not quite in the desired way) 
but for the others I draw a blank.

Thanks

Re: [basedb-devel] ServicesUtil not found

[Olivier L. <lef...@ya...>]

>> So I am reasonably happy with 4)

With 3) I meant. Typo.

> I have heard the name JSON but didn't know that it had to
> do with web services. 

Indeed it isn't closely "affiliated" with web services: it is
just another string encoding for tree-like semi-structured data. 
My understanding is that it is close to ASN.1 and that in terms 
of complexity ASN.1 >> XML ~ YAML > JSON > SEXP (S-Expressions)
JSON seems very popular with the Javascript crowd.

> Just because it is Java objects on the server side doesn't 
> mean that it has to be the same objects on the client side 

Yes but interop is one of those here-be-dragons things and 
Axis gives you the Java for free whereas you have to roll out
anything else by yourself, which not everyone has the skills
or the inclination to do (right now I'd be challenged to it
myself, although I fear I'm going to have to learn).

-- O.L.

Re: [basedb-devel] ServicesUtil not found

[Nicklas N. <ni...@th...>]

Olivier Lefevre wrote:
> Re. the earlier problem, here is what I found:
> 1) custom classes in their own package and in their own jar 
>    under WEB-INF/lib: ClassNotFoundException 
> 2) custom classes in their own package and in their own jar or
>    aar under WEB-INF/services: ClassNotFoundException; note that
>    in this case you must include a services.xml files (I used
>    an empty one, i.e., just <serviceGroup/>) in the jar or aar 
>    file, otherwise Axis will complain about that as well
> 3) custom classes in their own package and in core.aar: works!
> 4) custom classes in net/sf/basedb/ws/server and therefore in
>    core.aar: works but it is a last-resort option
> So I am reasonably happy with 4): the worst scenario is that
> you have to unpack and then repack third-party jars.
> 

I made some quick investigations as well and it seems like each .aar
file has it's own class loader (created by Axis). The parent class 
loader is the regular Tomcat class loader which means that classes in an 
.aar file can only use classes that are in the same .aar file or classes 
in the WEB-INF/lib/ directory. Classes in the WEB-INF/lib/ directory 
can't use classes from any .aar file because those files are not known 
by the Tomcat class loader.

This is described as a feature since it means that an .aar file can be 
replaced with a new version and deployed without having to restart the 
web application. So, since the ServicesUtil class is found in the 
core.aar file it means that it can only be used by other classes in 
core.aar.

>> I don't see how it would be possible to return any
>> structured information as simple strings. 
> 
> One could use JSON or even XML.

I have heard the name JSON but didn't know that it had to do with web 
services. Regarding XML, everything is already sent as XML which is 
unwrapped and converted to Java objects by Axis. I have seen code 
examples (from the Axis tutorial) were the response is accessed as XML 
instead so I think that option is already available. Just because it is 
Java objects on the server side doesn't mean that it has to be the same 
objects on the client side (but it is convenient).

/Nicklas

Re: [basedb-devel] ServicesUtil not found

[Olivier L. <lef...@ya...>]

Re. the earlier problem, here is what I found:
1) custom classes in their own package and in their own jar 
   under WEB-INF/lib: ClassNotFoundException 
2) custom classes in their own package and in their own jar or
   aar under WEB-INF/services: ClassNotFoundException; note that
   in this case you must include a services.xml files (I used
   an empty one, i.e., just <serviceGroup/>) in the jar or aar 
   file, otherwise Axis will complain about that as well
3) custom classes in their own package and in core.aar: works!
4) custom classes in net/sf/basedb/ws/server and therefore in
   core.aar: works but it is a last-resort option
So I am reasonably happy with 4): the worst scenario is that
you have to unpack and then repack third-party jars.

> I don't see how it would be possible to return any
> structured information as simple strings. 

One could use JSON or even XML.

> As far as I know it was no problem with perl.

OK.

-- O.L.

Re: [basedb-devel] ServicesUtil not found

[Nicklas N. <ni...@th...>]

Olivier Lefevre wrote:
> So it's a class loader issue. Thanks for the explanation.
> I'll find some arrangement that works; worst case is I just 
> shove everything into net.sf.basedb.ws.server: it's my own 
> code, so I can repackage it.
> 
> While we are talking about design, I have another quick
> question: from the interop point of view, i.e., with non-Java
> clients in mind, isn't it asking for trouble to use complex
> data types from net.sf.basedb.info: shouldn't one stick to 
> strings and other basic universal types instead?

I don't see how it would be possible to return any structured
information as simple strings. As far as I know it was no problem
with perl.

/Nicklas

Re: [basedb-devel] ServicesUtil not found

[Olivier L. <lef...@ya...>]

So it's a class loader issue. Thanks for the explanation.
I'll find some arrangement that works; worst case is I just 
shove everything into net.sf.basedb.ws.server: it's my own 
code, so I can repackage it.

While we are talking about design, I have another quick
question: from the interop point of view, i.e., with non-Java
clients in mind, isn't it asking for trouble to use complex
data types from net.sf.basedb.info: shouldn't one stick to 
strings and other basic universal types instead?

-- O.L.

Re: [basedb-devel] ServicesUtil not found

[Nicklas N. <ni...@th...>]

The ServicesUtil and other classes in the net.sf.basedb.ws.server package
are not found in the WEB-INF/lib/ directory and thus can't be referenced from
classes that are there.

Axis has it's own class loader implementation and expects to find classes
in the WEB-INF/services/ directory in the form of *.aar files. There should
currently be only one file: core.aar. Classes in the .aar files can
reference/use any class that is found in WEB-INF/lib/ but not the other way
around.

I don't know if .aar files are visible to each other, but you can try to
put your custom file in WEB-INF/services/ directory and rename it to
end with .aar.

/Nicklas



lef...@ya... wrote:
> Sorry, why did I write "client"? It was late, I should have slept
> before posting. This happens in the service class. I have my own
> utility class, which refers to ServicesUtil, and the service class
> loads that utility class. The utility class can be loaded but not
> ServicesUtil, which is odd since it *is* loaded in AbstractRPCService.
> I have attached a minimal yet full example that can be deployed to
> reproduce the problem (I have not attached the modified services.xml 
> because the mods there are obvious); the service and client classes
> are in the appropriate jars (I just rebuild using ant), TestUtil in 
> its own jar under WEB-INF/lib. The bottom of the stack trace is
> 
> Caused by: java.lang.NoClassDefFoundError: net/sf/basedb/ws/server/ServicesUtil
>     at com.acme.TestUtil.<clinit>(TestUtil.java:8)
>     at net.sf.basedb.ws.server.TestService.getResult(TestService.java:14)
>     ... 25 more
> Caused by: java.lang.ClassNotFoundException: net.sf.basedb.ws.server.ServicesUtil
>     at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1387)
>     at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1233)
>     at java.lang.ClassLoader.loadClassInternal(Unknown Source)
>     ... 27 more
> 
> There is nothing special about ServicesUtil, btw: attempting to
> instantiate, say, TempFileDataSource, causes the same error. The 
> chain of references is net.sf.basedb.ws.server -> com.acme -> 
> net.sf.basedb.ws.server. Does the detour through another package 
> trigger some classloader complication?
> 
>> A good starting point is the web services example that you can 
>> download from [...]
> 
> Of course I studied the built-in examples first; I don't think I
> could have so much as started otherwise. Still, I want to be able
> to use custom classes on the server side. Why does (apparently)
> everything have to be in the net.sf.basedb.ws.server package?
> 
> Thanks,
> 
> -- O.L.
> 
> =====================================================================
> package com.acme;
> 
> import net.sf.basedb.ws.server.ServicesUtil;
> 
> public class TestUtil {
> 
>     // just to test whether ServicesUtil can be loaded
>     private static ServicesUtil util = new ServicesUtil();
> 
>     public static String process(String x) {
>         return x.toUpperCase();
>     }
> }
> 
> package net.sf.basedb.ws.server;
> 
> import net.sf.basedb.core.SessionControl;
> import net.sf.basedb.ws.server.ServicesUtil;
> 
> import com.acme.TestUtil;
> 
> public class TestService extends AbstractRPCService {
> 
>     public TestService() { }
> 
>     public String getResult(String sid, String q) {
>         SessionControl sc = getSessionControl(sid);
>         return TestUtil.process(q);
>     }
> }
> 
> package net.sf.basedb.ws.client;
> 
> import org.apache.axis2.AxisFault;
> 
> import com.acme.TestUtil;
> 
> public class TestClient extends AbstractRPCClient {
> 
>     private final SessionClient session;
> 
>     public TestClient(SessionClient session) {
>         super(session.getUrl(), "Test", session.getServiceFactory());
>         this.session = session;
>     }
> 
>     public String getResult(String q) throws AxisFault {
>         String sid = session.getId();
>         return invokeBlocking("getResult", String.class, sid, q);
>     }
> }
> 
> 
> 
> 
> 
> 
> ------------------------------------------------------------------------------
> Crystal Reports - New Free Runtime and 30 Day Trial
> Check out the new simplified licensing option that enables unlimited
> royalty-free distribution of the report engine for externally facing 
> server and web deployment.
> http://p.sf.net/sfu/businessobjects
> _______________________________________________
> basedb-devel mailing list
> bas...@li...
> https://lists.sourceforge.net/lists/listinfo/basedb-devel

Re: [basedb-devel] ServicesUtil not found

[Olivier L. <lef...@ya...>]

PS: There are a couple of superfluous imports in my example files
because of cut-and-paste but that does not affect the argument.

Re: [basedb-devel] ServicesUtil not found

[<lef...@ya...>]

Sorry, why did I write "client"? It was late, I should have slept
before posting. This happens in the service class. I have my own
utility class, which refers to ServicesUtil, and the service class
loads that utility class. The utility class can be loaded but not
ServicesUtil, which is odd since it *is* loaded in AbstractRPCService.
I have attached a minimal yet full example that can be deployed to
reproduce the problem (I have not attached the modified services.xml 
because the mods there are obvious); the service and client classes
are in the appropriate jars (I just rebuild using ant), TestUtil in 
its own jar under WEB-INF/lib. The bottom of the stack trace is

Caused by: java.lang.NoClassDefFoundError: net/sf/basedb/ws/server/ServicesUtil
    at com.acme.TestUtil.<clinit>(TestUtil.java:8)
    at net.sf.basedb.ws.server.TestService.getResult(TestService.java:14)
    ... 25 more
Caused by: java.lang.ClassNotFoundException: net.sf.basedb.ws.server.ServicesUtil
    at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1387)
    at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1233)
    at java.lang.ClassLoader.loadClassInternal(Unknown Source)
    ... 27 more

There is nothing special about ServicesUtil, btw: attempting to
instantiate, say, TempFileDataSource, causes the same error. The 
chain of references is net.sf.basedb.ws.server -> com.acme -> 
net.sf.basedb.ws.server. Does the detour through another package 
trigger some classloader complication?

> A good starting point is the web services example that you can 
> download from [...]

Of course I studied the built-in examples first; I don't think I
could have so much as started otherwise. Still, I want to be able
to use custom classes on the server side. Why does (apparently)
everything have to be in the net.sf.basedb.ws.server package?

Thanks,

-- O.L.

=====================================================================
package com.acme;

import net.sf.basedb.ws.server.ServicesUtil;

public class TestUtil {

    // just to test whether ServicesUtil can be loaded
    private static ServicesUtil util = new ServicesUtil();

    public static String process(String x) {
        return x.toUpperCase();
    }
}

package net.sf.basedb.ws.server;

import net.sf.basedb.core.SessionControl;
import net.sf.basedb.ws.server.ServicesUtil;

import com.acme.TestUtil;

public class TestService extends AbstractRPCService {

    public TestService() { }

    public String getResult(String sid, String q) {
        SessionControl sc = getSessionControl(sid);
        return TestUtil.process(q);
    }
}

package net.sf.basedb.ws.client;

import org.apache.axis2.AxisFault;

import com.acme.TestUtil;

public class TestClient extends AbstractRPCClient {

    private final SessionClient session;

    public TestClient(SessionClient session) {
        super(session.getUrl(), "Test", session.getServiceFactory());
        this.session = session;
    }

    public String getResult(String q) throws AxisFault {
        String sid = session.getId();
        return invokeBlocking("getResult", String.class, sid, q);
    }
}

Re: [basedb-devel] ServicesUtil not found

[Nicklas N. <ni...@th...>]

Olivier Lefevre wrote:
> My web service client (i.e., a child of AbstractRPCClient)
> references a custom class in a completely different package,
> which is is in its own jar under webapps/base2/WEB-INF/lib
> That class in turn references net.sf.basedb.ws.server.ServicesUtil
> Upon trying to run the service I got: 
> 
>    java.lang.NoClassDefFoundError: net/sf/basedb/ws/server/ServicesUtil
> 
> and I don't have enough experience with Axis or Tomcat to
> understand what is going on. Any suggestion?

That is a server-side class and I don't know what you need to do with
it on the client side. This class can't be used on the client-side since
it would require a full BASE installation locally.

The base2/WEB-INF/lib directory is only available on the server-side as well.
So it is not a good idea to put custom client-side JAR files in there. It
seems like you need to redesign your client side code if it depends on server
side classes.

A good starting point is the web services example that you can download from
http://baseplugins.thep.lu.se/wiki/net.sf.basedb.examples.webservices
It should contain everything you need for developing client side web services.

/Nicklas

[basedb-devel] ServicesUtil not found

[Olivier L. <lef...@ya...>]

My web service client (i.e., a child of AbstractRPCClient)
references a custom class in a completely different package,
which is is in its own jar under webapps/base2/WEB-INF/lib
That class in turn references net.sf.basedb.ws.server.ServicesUtil
Upon trying to run the service I got: 

   java.lang.NoClassDefFoundError: net/sf/basedb/ws/server/ServicesUtil

and I don't have enough experience with Axis or Tomcat to
understand what is going on. Any suggestion?

[basedb-devel] BASE 2.12.1 has been released

[Nicklas N. <ni...@th...>]

Hello all,

We are happy to announce the release of BASE 2.12.1. This is a bugfix
release that among other things fixes some critical bugs related
to running BASE 1 plug-ins.

For a complete list of changes, see:
http://base.thep.lu.se/query?status=closed&milestone=BASE+2.12.1

Installation/upgrade instructions:
http://base.thep.lu.se/chrome/site/latest/html/admindoc/installation_upgrade/installation_upgrade.html

As usual, the program can be downloaded from:
http://base.thep.lu.se/wiki/DownloadPage

/Nicklas

Re: [basedb-devel] Minor API questions

[Nicklas N. <ni...@th...>]

Olivier Lefevre wrote:
> Why aren't all the methods of ServiceUtil static since it has no
> state and why does that class sport a method with the signature
> ItemQuery<T> getConfiguredItemQuery(ItemQuery<T> query, QueryOptions qOpt)
> instead of 
> void configuredItemQuery(ItemQuery<T> query, QueryOptions qOpt)
> since it does not return a new query but the original one? Just
> making sure I am not missing something.

There is no special reason for it.

/Nicklas

[basedb-devel] Minor API questions

[Olivier L. <lef...@ya...>]

Why aren't all the methods of ServiceUtil static since it has no
state and why does that class sport a method with the signature
ItemQuery<T> getConfiguredItemQuery(ItemQuery<T> query, QueryOptions qOpt)
instead of 
void configuredItemQuery(ItemQuery<T> query, QueryOptions qOpt)
since it does not return a new query but the original one? Just
making sure I am not missing something.

Re: [basedb-devel] Query tracing

[Olivier L. <lef...@ya...>]

OK, so I enabled that and it did not show me all I wanted at
first. In case someone should hit this thread using the search 
function in the future, I think I'd finish the story.

If a WHERE clauses involves a constant string, the HQL log will
shows some kind of hash instead of the string because of how 
Expressions.string works. Then, too, apparently the SQL is a stored
proc and it is shown with '?' in lieu of actual parameters, so again
you don't see your string. In order to get the full scoop you need 
log4j.logger.net.sf.basedb.core.query.bind=debug

-- O.L.

Re: [basedb-devel] Query tracing

[Nicklas N. <ni...@th...>]

Olivier Lefevre wrote:
> Thanks. By the way, why were WHERE SQL clauses mapped to the
> unintuitive Restriction rather than Where, like ORDER was
> mapped to Order?

Restrictions can appear in other places also. For example in
queries with GROUP BY <expression> HAVING <restriction>

/Nicklas

Re: [basedb-devel] Query tracing

[Olivier L. <lef...@ya...>]

Thanks. By the way, why were WHERE SQL clauses mapped to the
unintuitive Restriction rather than Where, like ORDER was
mapped to Order?

-- O.L.

Re: [basedb-devel] Query tracing

[Nicklas N. <ni...@th...>]

Olivier Lefevre wrote:
> I am developing a service. How can I enable query tracing so that
> Query.list() appends the SQL or HQL form of the query to one of the
> Tomcat log files? My 'or' queries (Restrictions.or) seem to work 
> ok but not my 'and' queries and I'd like to see what's going on.


This can be enabled in the log4j.properties file which you find in the
WEB-INF/classes folder. There are a lot of options but the one you are
looking for is: log4j.logger.net.sf.basedb.core.query.ql=debug

Just uncomment this and the HQL/SQL should be logged. If your query is
a HQL query you may want the SQL Hibernate generates as well. This
is controlled by log4j.logger.org.hibernate.SQL=debug.

/Nicklas

[basedb-devel] Query tracing

[Olivier L. <lef...@ya...>]

I am developing a service. How can I enable query tracing so that
Query.list() appends the SQL or HQL form of the query to one of the
Tomcat log files? My 'or' queries (Restrictions.or) seem to work 
ok but not my 'and' queries and I'd like to see what's going on.

[basedb-devel] BASE 2.11.1 and BASE 2.12 released

[Nicklas N. <ni...@th...>]

Hello all,

We are happy to announce the release of BASE 2.11.1 and of BASE 2.12.
The first release is fixing some minor bugs in the 2.11 branch. BASE 2.12
includes the same (and more) bug fixes and a lot of new functionality as
well.

For a complete list of changes, see:
http://base.thep.lu.se/query?status=closed&milestone=BASE+2.11.1
http://base.thep.lu.se/query?status=closed&milestone=BASE+2.12

Installation/upgrade instructions:
http://base.thep.lu.se/chrome/site/latest/html/admindoc/installation_upgrade/installation_upgrade.html

As usual, the program can be downloaded from:
http://base.thep.lu.se/wiki/DownloadPage

/Nicklas

Re: [basedb-devel] External id for protocol

[Nicklas N. <ni...@th...>]

Pawel Sztromwasser wrote:
> Nicklas Nordborg wrote:
>> Pawel Sztromwasser wrote:
>>> Hi Nicklas,
>>>
>>> That was much quicker then I expected. Patch is attached here: 
>>> http://base.thep.lu.se/ticket/1310
>>>
>>> I just hope that I haven't missed too much lines while copy-pasting your 
>>> code;)
>> The code is looking good. I have not yet tested it but will do that as soon
>> as I have some time left. Thanks a lot for fixing this. Is it ok for us to include
>> it in the 2.12 release? We'll take care of fixing the database schema versions and
>> other stuff related to upgrading.
>>
> 
> Of course it is OK. I'll be happy if it is included.

I have now tested it and it worked like a charm. I have committed the changes to
the trunk. Thanks for the good work.

/Nicklas

Re: [basedb-devel] External id for protocol

[Pawel S. <Paw...@bc...>]

Nicklas Nordborg wrote:
> Pawel Sztromwasser wrote:
>> Hi Nicklas,
>>
>> That was much quicker then I expected. Patch is attached here: 
>> http://base.thep.lu.se/ticket/1310
>>
>> I just hope that I haven't missed too much lines while copy-pasting your 
>> code;)
> 
> The code is looking good. I have not yet tested it but will do that as soon
> as I have some time left. Thanks a lot for fixing this. Is it ok for us to include
> it in the 2.12 release? We'll take care of fixing the database schema versions and
> other stuff related to upgrading.
> 

Of course it is OK. I'll be happy if it is included.

Pawel

> /Nicklas
> 
> 
> ------------------------------------------------------------------------------
> Crystal Reports - New Free Runtime and 30 Day Trial
> Check out the new simplified licensing option that enables 
> unlimited royalty-free distribution of the report engine 
> for externally facing server and web deployment. 
> http://p.sf.net/sfu/businessobjects
> _______________________________________________
> basedb-devel mailing list
> bas...@li...
> https://lists.sourceforge.net/lists/listinfo/basedb-devel

Re: [basedb-devel] External id for protocol

[Nicklas N. <ni...@th...>]

Pawel Sztromwasser wrote:
> Hi Nicklas,
> 
> That was much quicker then I expected. Patch is attached here: 
> http://base.thep.lu.se/ticket/1310
> 
> I just hope that I haven't missed too much lines while copy-pasting your 
> code;)

The code is looking good. I have not yet tested it but will do that as soon
as I have some time left. Thanks a lot for fixing this. Is it ok for us to include
it in the 2.12 release? We'll take care of fixing the database schema versions and
other stuff related to upgrading.

/Nicklas

Re: [basedb-devel] External id for protocol

[Pawel S. <Paw...@bc...>]

Hi Nicklas,

That was much quicker then I expected. Patch is attached here: 
http://base.thep.lu.se/ticket/1310

I just hope that I haven't missed too much lines while copy-pasting your 
code;)

All the best,
Pawel


Nicklas Nordborg wrote:
> Pawel Sztromwasser wrote:
> 
>>> I don't see any reason not to have the functionality. We will appreciate any
>>> contributions. Since both functions require changes to the database you must
>>> work against the trunk source code and there are a few "bookkeeping" things to
>>> consider, but we can help you with that. Can you create one ticket for the
>>> externalId approach and one ticket for the annotation approach to the BASE website?
>>>
>> done!
> 
> Great.
> 
>>> When you have code to submit it is simple to generate at patch with the command
>>> 'ant svn.diff' and upload/attach the generated file to the ticket.
>>>
>>> What timeframe do you have on this? We have not set a date for BASE 2.12 yet, but
>>> expect to have most things completed within a few weeks. We are also doing some more
>>> work on plug-ins and extensions that are related so it may be up to a month before
>>> everything is finished.
>>>
>> Well, it could be difficult with time before July. I am off work for the 
>> whole of June and only one day in a week dedicated to BASE. I will 
>> probably start next week, but don't expect anything working before my 
>> holidays. I guess this makes it patch for 2.13 rather then 2.12...
> 
> Thats fine with us. I was just worried that you needed this sooner.
> 
> /Nicklas
> 
> ------------------------------------------------------------------------------
> The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your
> production scanning environment may not be a perfect world - but thanks to
> Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700
> Series Scanner you'll get full speed at 300 dpi even with all image 
> processing features enabled. http://p.sf.net/sfu/kodak-com
> _______________________________________________
> basedb-devel mailing list
> bas...@li...
> https://lists.sourceforge.net/lists/listinfo/basedb-devel