FreeMind Voice Control Using Simon

owlbrudder
2011-01-28
2013-04-12
  • owlbrudder

    owlbrudder - 2011-01-28

    Folks,

    I am pleased to announce the release of a Simon voice control scenario for FreeMind 0.9.0 RC7. Simon http://simon-listens.org/ is a free and open-source voice recognition system for end users. A 10 minute video clip introducing Simon 0.3.0 can be found at http://www.youtube.com/watch?v=bjJCl72f-Gs. I should add that I am not associated with Simon or the Simon team in any way. I am just an end user of an interesting package.

    I have created a control scenario for Simon covering over 100 FreeMind keyboard shortcuts. I had to change a few of the default keyboard shortcuts to work with my system (Fedora 14 Linux, Gnome desktop, compiz window manager) and the release of Simon I am using (0.3.0). Keyboard and command reference sheets are provided in PDF format, as is a detailed set of installation instructions.

    I am using FreeMind to structure a novel I want to write. Using the Simon scenario, I can leave my hands in the typing position on my keyboard and speak commands such as "FreeMind Style Question", "FreeMind Edit Attributes" and over 100 others. I find it easier to remember commands than keyboard shortcuts like 'Ctrl+Shift+W'.

    The full list of voice commands covered by the scenario can be viewed in the PDF http://bedahr.org/simon/scenarios/simon_freemind090RC7_Commands_Quick_Guide_03.pdf. The modified keyboard shortcuts document is at http://bedahr.org/simon/scenarios/simon_freemind090RC7_Key_Mappings_Quick_Guide_03.pdf. Install Simon, use Simon to download the scenario " FreeMind 0.9.0RC7" and follow the instructions in http://bedahr.org/simon/scenarios/simon_freemind_shortcuts_notes.pdf to modify the default keyboard shortcuts to match the scenario.

    If you are using another version or RC of FreeMind, it would pay to check that your shortcuts match those in my PDF, as those are the keystroke sequences the scenario issues in response to the respective voice commands. You may get strange behaviour if your copy of FreeMind is not set up to match the scenario.

    Naturally, you will need a microphone that works with your computer. I use a relatively cheap Logitech headset; the microphone built into my notebook was not up to the job. Generally, the better your microphone, the better will be Simon's recognition of commands.

    I hope you find it useful. It is free and open source and, of course, comes without a guarantee. For more information, go to http://kde-files.org/content/show.php/%5BEN%2BVF%5D+FreeMind+0.9.0RC7?content=137915.

    Cheers,
    Doug

     
  • Argey

    Argey - 2011-01-28

    Hi Doug,

    I'm identifying resources to help a blind student who is required to mind map for an economics course.  I have a page about all the strategies I've found, with the help of many others, here: http://www.informationtamers.com/WikIT/index.php?title=Mind_mapping_for_people_who_are_blind

    You describe your voice control scenario as being for commands.  Is it limited to those, or could it handle dictated content for nodes as well?

    Thanks
    Roy Grubb

     
  • owlbrudder

    owlbrudder - 2011-01-28

    Hi Roy.

    What a great project. I hope you are successful.

    As at version 0.3.0 (the current version), Simon does not take dictation. I know it is in their future thinking, but how far away I don't know. The folks at VoxForge http://www.voxforge.org/home are collecting transcribed speech for use with Free and  Open Source Speech Recognition Engines (on Linux, Windows and Mac). I am keen to find a GPL tool to take dictation, for my own purposes, but have had no luck yet.

    The best I can do is offer my Simon scenario as one tool in your toolkit. It may well simplify some tasks for your blind student, but I regret I did not have that use in mind when I built it and I don't claim it will set his world on fire. Perhaps you could look at the Commands document I linked above, to get a feel for whether it would be easier to memorise my commands or the underlying keystrokes. To me, verbal commands are easier, but I am not blind.

    One interesting point about Simon is that the next release (0.4.0) is supposed to have user dialogs implemented and that would certainly provide good feedback to your student. There is a short video on this at http://www.youtube.com/watch?v=3_RTJsq9m4I.

    May I suggest you raise your question as a topic on the Simon Help forum? https://sourceforge.net/projects/speech2text/forums/forum/672427 The development team keeps a close eye on this forum and I am sure they would want to help as much as possible.

    I wish I could be more help.

    Kind regards,
    Doug

     
  • owlbrudder

    owlbrudder - 2011-01-28

    Hi Roy.

    I should have added that Dragon Naturally Speaking, which is not free, can be made to work (more or less) I understand on Linux, using Wine. The nearest thing to a Linux dictation tool seems to be Xvoice http://xvoice.sourceforge.net/, which relies on IBM's non-free ViaVoice engine. I have tried tracking down ViaVoice for Linux, but had no luck. It is possible IBM are no longer offering it. The Xvoice project seems to have had little activity since 2007. There is/was a push to get Xvoice working with the free Sphinx software, from CMU, but that also seems to have died on the vine. Xvoice-Sphinx has a page at https://lists.sourceforge.net/lists/listinfo/xvoice-sphinx, and Sphinx is at http://cmusphinx.sourceforge.net/.

    As far as I have been able to determine, there is currently no free dictation software available on any platform, so getting the computer to respond to commands is the best we can do right now. That is unless you are on Windows, or have Wine on Linux, and are prepared to pay for something like Dragon Naturally Speaking. I have never used it and can make no comment about its suitability for your application, but it seems to be the benchmark product.

    That is the limit of my knowledge on the subject. I will be delighted if someone else jumps in here with a solution I have missed.

    Kind regards,
    Doug

     
  • Argey

    Argey - 2011-01-28

    Hi Doug,

    Thanks for replying with this additional information.  I'll summarize it on the project page soon.  

    I think cost is an issue for the student, so he will probably have to find other methods for now.  But maybe one day, some FOSS will be available to help other sight-impaired users.

    Kind regards,
    Roy

     
  • owlbrudder

    owlbrudder - 2011-01-29

    Hi Roy.

    Cost is an issue for me, too, and it is likely to be so for anyone in a similar situation. I am firmly of the opinion that life-changing technology like voice transcription for your student is a social responsibility, not a luxury.

    I am kicking around an idea at the moment. It may prove to be worthless, so don't hold your breath. If it comes to anything, I will post back here with what I have found.

    Cheers,
    Doug

     
  • owlbrudder

    owlbrudder - 2011-01-29

    Hi Roy.

    Something that just occurred to me: does the mind map have to have its content as text? It seems silly to go through speech-to-text conversion for the map, then text-to-speech so the user can interpret the map again. Is it feasible for the map nodes to be voice recordings: to populate a node, he records his voice into it and to interpret, he plays the recording? That way, the map could also be interpreted by both a sighted and a blind person. If we can store images in a node, we should be able to store sounds.

    Just my $0.02

    Cheers,
    Doug

     
  • Argey

    Argey - 2011-01-29

    Hi Doug,

    I believe it needs text.  As I understand it, the student has to submit reguler mind maps as part of his course.  I'll ask how much flexibility there might be.

    Cheers,
    Roy

     
  • owlbrudder

    owlbrudder - 2011-01-30

    Roy,

    The idea I was chasing has not worked out. Open Source tools cannot yet provide a dictation tool, largely because there is not yet a good speech model available. Everyone can help that along by contributing to the open source VoxForge project at http://www.voxforge.org/home. The sooner that project reaches 'critical mass', the sooner a FOSS dictation tool might appear. CMU Sphinx and the Julius product are engines which could become dictation back-ends, given good enough speech models. Actually capturing cleaning and processing speech from hundreds of speakers for tens or hundreds of hours is a mammoth task. Dragon Naturally Speaking and IBM products are possible because those companies are willing and able to pay for professional speakers and dedicated people to process the captured speech.

    Some reading I was pointed to is:
    http://www.cs.cmu.edu/~archan/personal/whyNoOpenSourceDictationDraft4.html (a really good explanation)
    http://sourceforge.net/projects/speech2text/forums/forum/672426/topic/3685883

    So, the bad news is that, for now, I cannot point you to a FOSS dictation solution for the student you are helping. I am a Linux user and 'true believer', but the best solution in this case might be to see if you can get the student some sponsorship from Microsoft and/or Dragon, to get discounted copies of their software and tie all this in with Simon and FreeMind on the Windows platform. I believe there is a non-free speech-to-text tool available for the Mac platform as well, if the Windows idea does not pan out. I am not a Mac user and could not offer you any help or advice for that platform. Anyone out there able to help?

    I really hope you are able to get a solution working. Please keep us informed of any progress, as I am sure other FreeMind users would be keen to know how to help vision-challenged users.

    Kind regards,
    Doug

     
  • owlbrudder

    owlbrudder - 2011-02-05

    The documents linked in my original post above, have moved. The relevant paragraph should now read:

    "The full list of voice commands covered by the scenario can be viewed in the PDF here. The modified keyboard shortcuts document is here. Install Simon, use Simon to download the scenario " FreeMind 0.9.0RC7" and follow the instructions in this pdf to modify the default keyboard shortcuts to match the scenario."

    Cheers,
    Doug

     

Log in to post a comment.

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:





No, thanks