Using wait() but making the gui responsive

[James Nugent]

Hi there,

I am writing an interesting little app that uses speech recognition.
I have a number of sentences (cycled through using a for loop) that
each have a number of words. What I want to happen is for the words
to appear on screen and then disappear after the user has said them
all or after a time out.

I tried using wait() to pause the program, and a notifyAll to wake it
up when the user has said all the words. However, this causes the gui
to hang.

I don't know if this is the best idea so would appreciate all offers
of help.

Thanks,
James

Code:


import java.util.Vector;
import javax.swing.JLabel;
import javax.swing.JLayeredPane;
import javax.speech.recognition.*;
import javax.speech.*;
import java.util.Locale;
import javax.swing.JOptionPane;
import java.awt.FlowLayout;
import javax.swing.SwingUtilities;

public class Sentence extends ResultAdapter{

Recognizer rec;
RuleGrammar gram;
Vector words = new Vector();
StoryWord currentWord;
String picture;
boolean errorFound;
Helper helper;
JLabel pictureHolder;
Student currentStudent;
Vector spokenWords;
JLayeredPane pane;
boolean finished=false;
int wordPointer=0;
int helpCounter;
Thread t;

public Sentence(JLabel pictureHolder, Helper helper, Student
student, String picture) {
this.pictureHolder=pictureHolder;
this.helper=helper;
this.currentStudent=student;
this.picture=picture;

this.errorFound=false;
try{


}
catch(Exception e){
JOptionPane.showMessageDialog(null,"Error allocating recogniser
"+e.toString(),"Sentence Error",
JOptionPane.ERROR_MESSAGE);
}


}

public void setRecognizer(Recognizer r){
this.rec=r;
rec.addResultListener(this);
if (rec.getRuleGrammar("Sentence")==null)
gram = rec.newRuleGrammar("Sentence");
else gram = rec.getRuleGrammar("Sentence");
gram.setEnabled(true);
String[] rules = gram.listRuleNames();
for (int i=0; i<rules.length;i++){
gram.deleteRule(rules); //delete the existing rules
}
rec.addResultListener(this);
}

public void addWord(StoryWord theWord){
try{
words.add(theWord);
System.out.print(theWord.getText()+" ");
}
catch (Exception e){
JOptionPane.showMessageDialog(null,"Error adding word:
"+e.toString(),"Sentence Error",
JOptionPane.ERROR_MESSAGE);
}

}

public void resultAccepted(ResultEvent e) {//spoken word matches a
word in grammar
String spokenWord="";

Result r = (Result) (e.getSource());
ResultToken tokens[] = r.getBestTokens();

for (int i = 0; i < tokens.length; i++)
spokenWord =tokens.getSpokenText();

processWord(spokenWord);
}

public void resultRejected(ResultEvent e){
String spokenWord=""; //word spoken doesn't match that of grammar
processWord(spokenWord);
}

public synchronized void processWord(String word){//process the
incoming word
System.out.println(word);
StoryWord currentWord = new StoryWord("nullxxyy");
if (wordPointer<words.size()){
currentWord = (StoryWord) words.elementAt(wordPointer);
}
StoryWord previousWord = new StoryWord("nullxxyy");
if (wordPointer!=0){
previousWord = (StoryWord) words.elementAt(wordPointer - 1);
}

StoryWord nextWord = new StoryWord("nullxxyy");
if(wordPointer+1<words.size()){
nextWord = (StoryWord)words.elementAt(wordPointer+1);
}

if (word.equals(currentWord.getText())){//word is what we expect
if (wordPointer==words.size()) notifyAll(); //we have reached
the end
else wordPointer++;
}
else if(word.equals(previousWord.getText()))//deals with
repetition of single word or self correction
//wordpointer remains the same
previousWord.setMisspoken(false); //correction a success

else if(word.equals("")) //mispronunication
currentWord.setMisspoken(true); //this word is incorrect

else{ //we must search forwards (for multiple skips), check for
possible restarts, or backsteps
boolean found = false;
int i = wordPointer;
while(i<words.size()&&found==false){
StoryWord search = (StoryWord)words.elementAt(i); //find the
word
if(search.getText().equals(word)) found = true;
else i++;
}
if(found==true){ //we have skipped a long way forward
for (int j=wordPointer;j<i;i++){
StoryWord incorrect = (StoryWord)words.elementAt(j); //set
in between words incorrect
incorrect.setSkipped(true);
wordPointer=i+1; //set the pointer to the next word
}
}
else{//we didn't find the word, start search backwards
found = false;
i=wordPointer;
while(i>=0&&found==false){
StoryWord search = (StoryWord)words.elementAt(i); //find the
word
if(search.getText().equals(word)) found = true;
else i--;
}
wordPointer=i+1; //reset the word pointer
}
}
}

public synchronized double show(JLayeredPane panel){
this.pane=panel;
double score=0;
panel.removeAll();
panel.setLayout(new FlowLayout(FlowLayout.LEADING));
System.out.println("Showing sentence");
for (int i=0;i<words.size();i++){
StoryWord word = (StoryWord)words.elementAt(i);

//add the word to the pane in the right place
panel.add(word);
word.setVisible(true);
System.out.print(word.getText()+" ");
}



panel.repaint();
panel.updateUI();
panel.validate();

this.errorFound=true;
int tries=0;
int errors=0;

String[] wordRules = new String[words.size()];
for (int i=0;i<words.size();i++){
StoryWord sw = (StoryWord)words.elementAt(i);
wordRules=sw.getText();
}
Rule r = new RuleAlternatives(wordRules);
gram.setRule("Sentence",r,true);
try{
rec.commitChanges();
}
catch(Exception e){
e.printStackTrace();
}

while(tries<3&&this.errorFound==true){
wordPointer=0;
this.finished=false;
tries++;
this.errorFound=false;

try{
rec.resume();
rec.requestFocus();
rec.waitEngineState(Recognizer.LISTENING);
}
catch(Exception e){
JOptionPane.showMessageDialog(null,"Error resuming
recogniser: "+e.toString(),"Sentence Error",
JOptionPane.ERROR_MESSAGE);
}
//while(!finished);//wait here until finished

try{
wait(words.size() * 10000);
}
catch (Exception e){
e.printStackTrace();
}
rec.suspend();//suspend recognizer
for(int i=0;i<words.size();i++){
StoryWord temp = (StoryWord)words.elementAt(i);
if (temp.isIncorrect()){
this.errorFound = true; //find out if errors have been
made
errors++;
}
}
//if errors then run correction routine
if (tries==1) score = (words.size()-errors)/words.size();
//calculate score first time round
if (this.errorFound){
correctionRoutine();
helpCounter++;
}
}
//if tries != 1 then update learning preferences
if(tries!=1&&!errorFound){ //i.e. help has been made and has
corrected mistakes
currentStudent.incrementPreference(this.helpCounter % 4);
}
if(tries==3&&errorFound){ //we have had three goes and there is
still errors
for(int i=0;i<words.size();i++){
StoryWord sw = (StoryWord)words.elementAt(i);
if(sw.isIncorrect()) helper.say("This word is said
"+sw.getText());
}
helper.say("Here is the whole sentence");
for (int i=0;i<words.size();i++){
StoryWord sw = (StoryWord)words.elementAt(i);
sw.sayWord();
}
}
return score;
}

public void correctionRoutine(){
LearningPreference pref =
currentStudent.getLearningPreference(helpCounter);
for (int i=0;i<words.size();i++){
StoryWord sw = (StoryWord)words.elementAt(i);
if (sw.isIncorrect()){ //this word is incorrect
if(sw.wasMisspoken()) helper.say("I don't think you said this
word correctly");
else if (sw.wasSkipped()) helper.say("I think you missed this
word out");
sw.setHighlighted(true);
switch(pref.getType()){
case LearningPreference.PHONIC: sw.givePhonicHint(); break;
case LearningPreference.PICTURE: sw.givePictureHint();
break;
case LearningPreference.RHYME: sw.giveRhymeHint(); break;
case LearningPreference.SYNTAX: sw.giveSyntaxHint(); break;
}
sw.setHighlighted(false);
}
}
helper.say("Read the sentence again for me");
}

public void setErrorFound(boolean error){
this.errorFound=error;
}

}

 

[Matt Humphrey]

Hi there,

I am writing an interesting little app that uses speech recognition.
I have a number of sentences (cycled through using a for loop) that
each have a number of words. What I want to happen is for the words
to appear on screen and then disappear after the user has said them
all or after a time out.

I tried using wait() to pause the program, and a notifyAll to wake it
up when the user has said all the words. However, this causes the gui
to hang.

I don't know if this is the best idea so would appreciate all offers
of help.

I take it that your code called directly from the GUI thread--the one from
which you receive UI requests. If you wait or block this thread your GUI
will halt. In fact, your GUI won't even be able to update the screen until
it finishes with your code. You'll need to find some way of putting the
speech recognition into a separate thread so that the GUI will update and
continue to accept requests while the recognition work is going on. A
closer examination of your code may show how to structure it for background
use.

Cheers,
Matt Humphrey (e-mail address removed) http://www.iviz.com/

 

[Thomas Weidenfeller]

I tried using wait() to pause the program, and a notifyAll to wake it
up when the user has said all the words. However, this causes the gui
to hang.

Get the FAQ for comp.lang.java.gui. This is in the top 5 of questions.
Also you really want to work through Sun's GUI tutorial. A pointer is in
the FAQ, too.

/Thomas

 

[Roedy Green]

I tried using wait() to pause the program, and a notifyAll to wake it
up when the user has said all the words. However, this causes the gui
to hang.

I don't know if this is the best idea so would appreciate all offers
of help.

never put the GUI Swing thread to sleep. Use a timer or another
thread.

See http://mindprod.com/jgloss/sleep.html
http://mindprod.com/jgloss/timer.html

 

[scamhi]

Hi there,

I am writing an interesting little app that uses speech recognition.
I have a number of sentences (cycled through using a for loop) that
each have a number of words. What I want to happen is for the words
to appear on screen and then disappear after the user has said them
all or after a time out.

I tried using wait() to pause the program, and a notifyAll to wake it
up when the user has said all the words. However, this causes the gui
to hang.

I don't know if this is the best idea so would appreciate all offers
of help.

Thanks,
James

Take a look at SwingUtilities.invokeAndWait or SwingUtilities.invokeLater.