Listen to Tablet events, even on other window

[Tom]

Hi,

I have used JPen in Java applications before, on a JPanel. I am wondering if the following scenario is possible.
1. write a Java application with JPen
2. in that application, listen for *any* tablet event that occurs on the operating system (thus also on other windows, such as a third party text editor).

I basically want to send these events over a (local area) network.

Is this possibly? If so, can you maybe direct me to specific sections in the javadoc or give some tips? Thanks for your help.

Tom

[Nicolas Carranza]

Hey Tom,

Nice to know you have been using jpen.

I think that the scenario you described can be realized but the following conditions must be taken into account:
* The PenManager must be unpaused everytime the third party application window is activated. This is necessary because the Wintab context used by the jpen Windows OS provider is automatically deactivated (by Wintab) everytime "another" window is activated.
* The PenManager must be paused everytime the third party application window is deactivated. This is necessary because the jpen Linux provider grabs the tablet preventing any other application to get tablet data (through xinput) when the penManager is unpaused.
* You won't receive any button events. Button events are currently fired only by the jpen java awt provider and this provider wouldn't be used on this scenario.

Here goes an example. The Test class creates a PenManager using the ActiveScreenPenOwner instance (instead of the commonly used jpen.owner.awt.AwtPenOwner created when calling PenManager(Component)). It shows two buttons: one to reactivate the PenOwner by calling ActiveScreenPenOwner.reactivate() and the other to deactivate it by calling deactivate(). The Test uses two buttons but on the described scenario this methods must be called somehow by the client application, when its windows are activated/deactivated, satisfying the conditions exposed above.

======
import java.awt.event.ActionEvent;
import java.awt.event.ActionListener;
import javax.swing.JButton;
import javax.swing.JFrame;
import javax.swing.JPanel;
import jpen.event.PenListener;
import jpen.PButtonEvent;
import jpen.PenManager;
import jpen.PKindEvent;
import jpen.PLevelEvent;
import jpen.PScrollEvent;
public class Test
    implements PenListener{
    public static void main(String... args) throws Throwable{
        new Test();
    }
    Test(){
        PenManager pm=new PenManager(ActivableScreenPenOwner.getInstance());
        pm.pen.addListener(this);
        JPanel panel=new JPanel();
        JButton button=new JButton("Reactivate");
        button.addActionListener(new ActionListener(){
                    @Override
                    public void actionPerformed(ActionEvent ev){
                        ActivableScreenPenOwner.getInstance().reactivate();
                        System.out.println("--- reactivated");
                    }
                });
        panel.add(button);
        button=new JButton("Deactivate");
        button.addActionListener(new ActionListener(){
                    @Override
                    public void actionPerformed(ActionEvent ev){
                        ActivableScreenPenOwner.getInstance().deactivate();
                        System.out.println("--- deactivated");
                    }
                });
        panel.add(button);
        JFrame frame=new JFrame("ActivableScreenPenOwner Test");
        frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
        frame.add(panel);
        frame.pack();
        frame.setVisible(true);
    }
    //@Override
    public void penButtonEvent(PButtonEvent ev) {
        System.out.println(ev);
    }
    //@Override
    public void penKindEvent(PKindEvent ev) {
        System.out.println(ev);
    }
    //@Override
    public void penLevelEvent(PLevelEvent ev) {
        System.out.println(ev);
    }
    //@Override
    public void penScrollEvent(PScrollEvent ev) {
        System.out.println(ev);
    }
    //@Override
    public void penTock(long availableMillis) {
        System.out.println("TOCK - available period fraction: "+availableMillis);
    }
}
======

The ActivableScreenPenOwner is very similar to the jpen.owner.ScreenPenOwner (http://jpen.sourceforge.net/api/current/src-html/jpen/owner/ScreenPenOwner.html) but instead of pausing/unpausing the PenManager when a given AWT window is activated/deactivated, it contains two public reactivate/deactivate methods. The pen location will be given on screen (desktop) coordinates.

======
import java.awt.geom.Point2D;
import java.awt.Point;
import java.util.Arrays;
import java.util.Collection;
import jpen.owner.PenClip;
import jpen.owner.PenOwner;
import jpen.PenProvider;
import jpen.provider.osx.CocoaProvider;
import jpen.provider.wintab.WintabProvider;
import jpen.provider.xinput.XinputProvider;
public class ActivableScreenPenOwner
    implements PenOwner {
    private static ActivableScreenPenOwner instance;
    public synchronized static ActivableScreenPenOwner getInstance(){
        return instance==null? instance=new ActivableScreenPenOwner():
                     instance;
    }

    private ActivableScreenPenOwner(){}
    //  @Override
    public Collection<PenProvider.Constructor> getPenProviderConstructors(){
        return Arrays.asList(
                         new PenProvider.Constructor[]{
                             // new SystemProvider.Constructor(), //Does not work because it needs a java.awt.Component to register the MouseListener
                             new XinputProvider.Constructor(),
                             new WintabProvider.Constructor(),
                             new CocoaProvider.Constructor()
                         }
                     );
    }
    private volatile PenManagerHandle penManagerHandle;
    private boolean active;
    //  @Override
    public void setPenManagerHandle(final PenManagerHandle penManagerHandle){
        this.penManagerHandle=penManagerHandle;
        if(active)
            reactivate();
    }
    public void reactivate(){
            setActive(false);
            setActive(true);
    }
    private void setActive(boolean active){
        synchronized(penManagerHandle.getPenSchedulerLock()){
            this.active=active;
            if(penManagerHandle==null)
                return;
            penManagerHandle.setPenManagerPaused(!active);
        }
    }

    public void deactivate(){
        setActive(false);
    }
    private final PenClip penClip = new PenClip() {
                //      @Override
                public void evalLocationOnScreen(Point locationOnScreen){
                    // The location of this PenClip is always on (0, 0) screen coordinates.
                    locationOnScreen.x=locationOnScreen.y=0;
                }
                //      @Override
                public boolean contains(Point2D.Float point){
                    // This PenClip covers all the screen.
                    return true;
                }
            };
    //  @Override
    public PenClip getPenClip() {
        return penClip;
    }
    //  @Override
    public boolean isDraggingOut() {
        return false;
    }
}
======

I hope this example helps you. Please let me know how it goes or if you have any questions.

Cheers!
Nicolas

[Tom]

Hi Nicolas,

Thank you for your reply and detailed help. I have tested the code you have provided and changed it slightly to reactivate the ActivableScreenPenOwner whenever the Java application loses its active window. That is easily done:

import java.awt.event.ActionEvent;
import java.awt.event.ActionListener;
import java.awt.event.WindowEvent;
import java.awt.event.WindowListener;
import javax.swing.JButton;
import javax.swing.JFrame;
import javax.swing.JPanel;
import jpen.PButtonEvent;
import jpen.PKindEvent;
import jpen.PLevelEvent;
import jpen.PScrollEvent;
import jpen.event.PenListener;
import jpen.PenManager;
public class JPenTest implements PenListener, WindowListener{
    
    public static void main(String... args) throws Throwable{
        new JPenTest();
    }
    
    JPenTest(){
        PenManager pm = new PenManager(ActivableScreenPenOwner.getInstance());
        pm.pen.addListener(this);
        JPanel panel = new JPanel();
        
        JButton button = new JButton("Reactivate");
        
        button.addActionListener(new ActionListener(){
            public void actionPerformed(ActionEvent ev){
                ActivableScreenPenOwner.getInstance().reactivate();
                System.out.println("--- reactivated"); }
        });
        
        panel.add(button);
        button=new JButton("Deactivate");
        
        button.addActionListener(new ActionListener(){            
            public void actionPerformed(ActionEvent ev){
                ActivableScreenPenOwner.getInstance().deactivate();
                System.out.println("--- deactivated");
            }
        });
        
        panel.add(button);
        
        JFrame frame=new JFrame("ActivableScreenPenOwner Test");
        frame.addWindowListener(this);
        frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
        frame.add(panel);
        frame.pack();
        frame.setVisible(true);
    }
    
    public void penButtonEvent(PButtonEvent ev) {
        System.out.println(ev);
    }
    
    public void penKindEvent(PKindEvent ev) {
        System.out.println(ev);
    }
    
    public void penLevelEvent(PLevelEvent ev) {
        System.out.println(ev);
    }
    
    public void penScrollEvent(PScrollEvent ev) {
        System.out.println(ev);
    }
    
    public void penTock(long availableMillis) {
        System.out.println("TOCK - available period fraction: "+availableMillis);
    }
    public void windowOpened(WindowEvent e) {
    }
    public void windowClosing(WindowEvent e) {
    }
    public void windowClosed(WindowEvent e) {
    }
    public void windowIconified(WindowEvent e) {
    }
    public void windowDeiconified(WindowEvent e) {
    }
    public void windowActivated(WindowEvent e) {
    }
    public void windowDeactivated(WindowEvent e) {
        ActivableScreenPenOwner.getInstance().reactivate();
    }
}

However, this reactivation gets harder whenever some third party application's window gets deactivated. I might use some TimerTask for this, that reactivates the ActivableScreenPenOwner at specified moments (every 200 ms for example).

Regards,
Tom

[Nicolas Carranza]

Hey Tom,

Be careful, you must fulfill the conditions explained at the top of my previous post or correct tablet access will probably be impossible to a third party tablet application. Calling reactivate when a third party tablet application is accessing the tablet may cause it to stop working right (tablet access when ActivableScreenPenOwner is active is exclusive and can interrupt or forbid access to a third party tablet application). :-S

I don't know the details of your use-case… maybe the proposed approach (ActivableScreenPenOwner) is not acceptable? Let me know what you think or if you have any question.

Cheers!
Nicolas

[Tom]

Hi Nicolas,

You are right, it might be better not to grab the tablet access with ActivableScreenPenOwner all the time since other third party applications need tablet access too.
It is however a possibility for me to design a small protocol between the application with ActivableScreenPenOwner and the other application I want to get the tablet events for. Only when the other application gets an active window I reactivate ActivableScreenPenOwner, and not for the cases when some third application gets an active window.

Regards,
Tom

[Nicolas Carranza]

Hey Tom,

Sounds nice. I think it is also necessary to deactivate the ActivableScreenPenOwner when the other application window gets inactive (to ungrab the tablet on linux).

Let me know if you find problems. Cheers!
Nicolas

[Tom]

Hi Nicolas,

Thank you for that addition. I will keep it in mind.

Regards,
Tom

[Nicolas Carranza]

Hey Tom,

Here goes a couple of warnings missing on my first reply: 

* I think that the Mac OS X jpen provider won't work for the ActivableScreenPenOwner. I can not test/develop for OS X because I don't have a mac but, as far as I understand, the osx provider needs a window on its jvm receive tablet events.

* On linux/windows, when the tablet pen is in relative mode (when the mouse pointer behaves like a mouse) jpen will give coordinates as if it were in absolute mode (See http://www.faqs.org/docs/Linux-HOWTO/Wacom-Tablet-HOWTO.html#ss5.11 for absolute/relative mode definition).

I guess jpen is showing to be too weak to be used without a window and I'm beginning to think I'm causing you to loose time. :-( Please let me know what you think or if you have any questions.

Cheers!
Nicolas

[Tom]

Hi Nicolas,

Thank you for your post.

* It is funny how every operating system can act so differently in this tablet-context. For the moment I am mainly interested in Windows and Linux, so Mac OSX is not a real problem yet.

* It might be possible to get the absolute screen coordinates of the windows of the other application. Then, the absolute screen coordinates received from JPen can be translated into relative coordinates by fitting them in the screen rectangles of these windows. But maybe that is not so accurate (quick mismatches between the mouse cursor and the calculated relative position).

I guess that at the moment one of the more safe options for me would be to use the Java Native Interface. I can use JPen in a Java swing GUI and pass the relative pen tablet events to C++ code. Some output can then be returned and rendered in the Swing GUI.

Regards,
Tom

[Nicolas Carranza]

Hey Tom,

> * It is funny how every operating system can act so differently in this
> tablet-context. For the moment I am mainly interested in Windows and Linux,
> so Mac OSX is not a real problem yet.
>
> * It might be possible to get the absolute screen coordinates of the
> windows of the other application. Then, the absolute screen coordinates
> received from JPen can be translated into relative coordinates by fitting
> them in the screen rectangles of these windows. But maybe that is not so
> accurate (quick mismatches between the mouse cursor and the calculated
> relative position).

this transformation is currently also done by jpen when using the jpen.owner.awt.AwtPenOwner (the PenOwner used when instantiating a PenManager using the PenManager(Component) constructor) and it is accurate enough.

But I was referring to how the mouse cursor (arrow on screen) moves when the tablet pen is moved. When the tablet pen cursor mode is relative, the mouse cursor moves as if the tablet pen were a mouse. When the tablet pen cursor mode is absolute, the mouse cursor moves according to a mapping defined between the tablet space and the screen space. The tablet pen cursor mode can be changed by changing the tablet driver configuration (e.g. using the "Wacom Tablet Properties" "Mapping" tab using Windows/Wacom Intuos, Wacom calls these modes "Mouse" (relative) and "Pen" (absolute)). When jpen uses the ActivableScreenPenOwner it always gives coordinates by mapping the tablet space to the screen (as if the tool were always on absolute cursor mode), and if the tablet tool is in relative mode then the cursor location won't match the one given by jpen… let me know if you need a better explaination :-S

>
> I guess that at the moment one of the more safe options for me would be to
> use the Java Native Interface. I can use JPen in a Java swing GUI and pass
> the relative pen tablet events to C++ code. Some output can then be
> returned and rendered in the Swing GUI.
>

I agree with you, the ActivableScreenPenOwner based approach needs considerable work to be a solid global solution. Probably you have good knowledge about C++ and its GUI toolkits… I don't know too much about them. I'm curious, if the application is in C++ why don't you use a C++ GUI library with tablet support?

Cheers!
Nicolas

[Tom]

Hi Nicolas,

Thank you for the explanation about absolute and relative cursor modes. I understand what you mean.

The reason why I use JPen is as follows. I firstly tried to find/use (mature) C++ Gui libraries that have tablet support. The only ones I know of are: Qt, Gtk+, FLTK 2.0.
Personally, I don't  really like the way Qt has changed the C++ language with its "meta-object compiler" and that is why I avoid using Qt. I tried GTK+ but could not get tablet support to work. And as for FLTK 2.0, on its website it appears to be a "dormant" development branch, so I was not sure whether tablet support would be available in future releases. Finally, I have read on the web about future tablet support in the SDL library - version 1.3, but there is no official release yet. Thus, I have not found a mature C++ Gui library with tablet support that I would like to use.
I already knew JPen, and that is why I choose this library for the current development. Besides, I can now keep using the very useful JDK functionalities. Only some core part of the application is written in C++. Maybe later I can port it to a pure C++ environment.

Regards,
Tom

[Nicolas Carranza]

Tom, thanks for your explanation.
Cheers! Nicolas

[pawel]

Hi,

Does anybody know how to listen to tablet events from any running window in current version of JPen? I can't find ActivableScreenPenOwner or anything similar to this in the JavaDoc.

Regards,
Pawel

[Nicolas Carranza]

Hi Pawel,

You can use jpen.owner.multiAwt.AwtPenToolkit.addPenListener(Component, PenListener) on multiple Components, each one being in any window of your application.

Or If you want to listen to level events with coordinates based on the screen, when any of the windows of the application is active, you can do something like:

static PenManager penManager=new PenManager(jpen.owner.ScreenPenOwner.getInstance());
...
penManager.addListener(...);

(BTW: ScreenPenOwner has a public contstructor, it is a bug, the getInstance() method must be used.)

Or if you want to listen to level events with coordinates based on the screen, even when the application has no active windows, you can use an ActivableScreenPenOwner (the source code is on this thread) but it can be tricky as discussed above.

Cheers,
Nicolas