Interactive4J Files

Interactive4J
Osama Mohammad Oransa
(c) 2011
http://osama-oransa.blogspot.com/2011/06/interactive4j.html
============================================================
============================================================

About Interactive4J :
====================

I have created this open source project aiming to provide simple easy APIs for Java developers to use interactive abilities in their Java Applications like speech recognition, handwriting recognition, use of web cam , sound record/play, decision trees , text to speech and many others.
Without further brave enhancements to user experience in the standalone applications, users will prefer the web application for the huge features they provide, this open source enrich the user experience in such applications.
Together with JavaFX interface and openJDK ,Java SE will be in the front of standalone applications.

Features :
==========

* Easy and simple APIs for developer with any experience to use them.
* Aim for Interactivity of Java Applications (for better user experience)
* Used for NextGen Java Application with higher user interactivity, if we didn't go for such enhancements in user experience , desktop application will fade soon in competition with web apps.
* Use some other open source projects and wrap them in a very simple way.
* Have a nice simple demo to show the supported features.
* Each functionality like text to speech , speech recongnition ,etc. have a very straightforward simple methods to use.
* Human abilites is the main purpose of the 1st release as see, hear , talk , translate, listen , recognize and think.
* Move some areas from scientific research into practical usage.
* Enhance the usage of Java in Educational puropose.
* Use the Standards whenever exist.
* Uses Sphinx-4 is a state-of-the-art speech recognition system written entirely in the JavaTM programming language
* Uses FreeTTS library for text to speech.
* Uses the library LTI-CIVIL which is part of FMJ (Freedom for Media in Java) for portability of WebCam usage.
* Uses Languages translation via open source "google-api-translate-java" for most possible interactivity with different users.
* Define a new JTextField component "JITextField" having JTextField + JButton for translation + JButton for text to speech
* Provides Genetic Algorithm and Decision Tree in a very simple and generic way for easy usage.
* Use Google Gson for JSON parsing and constructions.
* Use JMF for creating movies from images and to play audio/video.

SourceForge URL : http://sourceforge.net/projects/interactive4j/


How to Use it?
==============

You must place correct jar files, there is a zipped file lib-v1.0.zip contain all needed files for windows env, 
for other than windows only Free of Java Media libraries need to be downloaded..
Refer to the my blog to see how to make the web camera work and how to download needed libraries..

In the following post:
http://osama-oransa.blogspot.com/2010/12/capture-photo-from-web-cam-usb-cam.html


You need also to send the followin java option to the java command while running your jar file so you can load 
the needed libraries for web camera module:
-Djava.library.path="\......path......\lti-civil-20070920-1721\native\win32-x86"

Note:
In case you need the options of playing movies/create movies using JMF , you need to download it from Oracle and install it.


The demo represents some of the existing functionality in Interactive4J where you can simple initialize the feature you need and use it in a simple way..

1) Text To Speech
------------------
The demo represents 2 ways of using this feature..
-Say it: where you pass the string you want the engine to say.
-Read it: where you do further manipulation to read word by word and highlight the current word.



The APIs is very simple : static method that take the string to say..
JHSpeakUtils.speak(String);


2) Speech Recognition
---------------------
3 method control speech recognition:
- JHVoiceRecUtils speakUtils = new JHVoiceRecUtils(this,currentConfigPath);
To configure it to start (you should use the current configuration files without modification, but you can still modify them)
This = represents a class that implement the VoiceRecognitionListener where you get the recognized words in a method : setVoiceResult(String recognized)
- speakUtils.StartVoiceRecognition(commands);
Where you start the recongnition , commands here is a string[] contain all the words/letters you'd like to recognize.
- speakUtils.StopVoiceRecognition();
To stop the voice recognition.


3) Handwriting recognition
--------------------------
- handwriting = new JHHandWritingUtils(jPanel1, this,JHHandWritingUtils.MODE_NUMBERS,currentConfigPath);
You construct the handwriting utils with the panel you want the user to use it and listener of the results, plus mode (either number, capital letters or small letter), and finally the current configuration path (you shouldn't modify the given configurations.
- handwriting.startHandWritint();
To start the handwriting recognition..
- handwriting.stopHandWriting();
-To stop it

*So 3 modes is supported:
1-Numbers
2-Capital letters
3-Small letters

4) Record and play sounds/voices
--------------------------------
3 methods:
- JHRecordingUtil recordingUtil=new JHRecordingUtil(maxSeconds);
construct the recorder with max seconds allowed.
- recordingUtil.startRecording("/recordSample.wav", 22000);
To start recording in a give file with sampling rate (this is configured being depend on the needed quality and it affect the recorded file size)

To play it : use the method:
recordingUtil.play("/recordSample.wav",1);
1 = one time only.
You have also stop method to stop it.

This is typically adapted for voice chat where you record a snapshot and send it and play the received snap voice and repeat the cycle..

5) Translate Text
-----------------
One simple method provided to do the translation:
String trans = JHTranslateUtil.doTranslate(jTextArea1.getText(), Language.ENGLISH, Language.ARABIC);

Where you specify the text to translate and source/destination languages.

In the demo click on translate button translate text to Arabic and the 2nd click restore the original text.

6) Screen Capture
------------------
Only 1 simple method:
JHScreenCaptureUtil.takeScreenShot("/osama.jpg");
or
takeScreenAreaShot("/osama.jpg",x,y,width,height)

This facilitate creating desktop recording applications, if we conjugate it with voice/sound recording we can do such application in no time.

7) Web Camera Capture
---------------------
Simple single method you just need to specify the file name and the lisetenr of this operation CaptureListener ..
mode=ONE_PHOTO;
webCam=new JHCaptureUtil(this, "/myWebCamPic.jpg");
webCam.start();


8) Web Camera Video Capture
---------------------------
This is actually a stream of jpg files displayed as if a video , this is the better approach for video chatting where you can send each picture separately, you can still construct a video file from these files.
mode=MULTI_PHOTO;
webCam=new JHCaptureUtil(this, "/myWebCamPic.jpg");
webCam.start();


9) Decision Tree
----------------
Use a simple way to design a decision tree and take the response to control whatever you need..
Here we used a game with decision tree to decided the PC movements..


10) Genetic Algorithm
---------------------
Simple easy way to use the genetic algorithm in search problems in easy efficient way...prepare your problem and do search using 1 method!

// init population
Individual[] populations = new Individual[popSize];
for (int i = 0; i < popSize; i++) {
populations[i] = new Individual(numberOfGenes);
populations[i].randGenes();
}

// construct the evaluation function..
GeneticEvaluator geneticEvaluator = new GeneticEvaluator() {
public int evaluate(Individual individual) {
int fitness = 0;
for (int i = 0; i < individual.size; ++i) {
fitness += individual.getGene(i);
}
individual.setFitnessValue(fitness);
return fitness;
}
public int evaluateSecondLevel(Individual individual) {
int fitness = 0;
for (int i = 0; i < individual.size; ++i) {
fitness += individual.getGene(i);
}
individual.setFitnessValue(fitness);
return fitness;
}
};

//do genetic search now to get the best individual..by setting the configuration object with all details...
GeneticConfiguration config=new GeneticConfiguration();
config.setElitism(elitism);
config.setMaxFittness(maxFittness);
config.setMaxIterations(maxIterations);
config.setMutationRate(mutationRate);
config.setCrossoverRate(crossoverRate);
config.setPopulations(populations);
config.setGeneticEvaluator(geneticEvaluator);
config.setMulitFitness(true);
config.setChangeFitnessOnIteration(400);
Individual bestIndiv = JHGeneticUtil.executeJHGeneticUtil(config);

NOTE: this previous example uses multifitness functions where 2nd one is used after 400 iteration , this is not the typical genetic algorithm where you can switch this off by set it to false.

11) New Suggested JTextField style
----------------------------------
This composed of JTextField plus 2 buttons; one for translation and one for text to speech..


12) JSON Utilities
------------------
Here you can convert from JSON into Object and vice versa using easily APIs provided by Google Gson.
Only you call toJson() and fromJson() according to conversion type. 


13) Play Video files and Audio files
-------------------------------------
Built on JMF, play video/audio in the Swing easily by a single method:

JHPlayer createPlayer(JPanel pf, String media, ControllerListener listener)


14) Convert Images to Movies
----------------------------
Build on JMF, , single method to create .mov file:
JHMovieMaker.createMovie(String movieName, int width , int height, Vector<String> images, int frameRate)

For other features and functionality, go through the java-docs of the project.