Choosing the Correct Dialog Flow 
Like many powerful instruments, rapid development tools can be extremely hazardous if not handled with care. This is especially the case with Angel Site Builder, which enables you to build voice applications in a matter of days versus the weeks it would take using voiceXML or some other low-level mark-up language.
The danger of such rapid development is that the lure of getting something "up and running fast" can be so strong that the crucial phase of deliberate design can be severely compromised or even ignored. The lure is strengthened by the illusion that making changes is even easier than building the application in the first place. Approaching the development of Voice Sites by thinking, "let me quickly build it, deploy it, and then make some quick changes as needed," is not a formula for yielding high quality voice applications.
On the other hand, Angel Site Builder is a perfect instrument for the deliberate voice application designer: VUI designs can be quickly implemented and tested out. Changes can be evaluated cost effectively and before committing to a deployable application. But how does one commit to a dialog flow and what should the decision process be?
Don't be the judge and the jury
Rule number one is to hand over the evaluation of your dialog flow testing to people who did not participate in the VUI design process. Ideally, choose people who have no vested interest in the design and who can render honest and harsh criticism without hesitation.
Choose a good jury
If you are a geek, chances are that you are surrounded by other geeks and are comfortable in the company of geeks. But chances are also high that the people calling your voice application will not be geeks but will instead be "regular people." Your pool of testers should reflect the diversity of people who will be calling your application once it has been deployed in "the real world."
Some important characteristics to keep in mind when choosing your testers: (1) Comfort with technology: don't assume that everyone loves talking to automated systems. (2) Age: older people may be more deliberate and slow in their interactions than younger people. (3) Gender: men and women interact differently in conversational contexts. (4) Temperament: patience plays an important role in speech interactions, so your pool of testers should include people with different levels of patience.
Listen to the verdict
The best way to assess the evidence is to listen to recordings of actual calls made to the system.
When listening to calls, try to quantify the following basic metrics (1) Effectiveness: did the callers succeed in accomplishing what they intended to accomplish? (2) Ease of use: how much effort did the callers have to put in to accomplish their objective? An easy way to assess these metrics is to measure the length of successful calls. An equally important metric is to count the number of no matches and no inputs.
You should also structure the way you listen to calls to help spot patterns of weakness in your design. For instance, listen to all the calls made by one person in a row and in the sequence they were made. This will allow you to measure how effectiveness and ease of use vary over time (i.e., are people able to learn how to use the system?). You should also take a cross section of calls for a given call condition. For example, what is the reaction pattern in noisy environments vs. quiet ones? Are older people having a more difficult time with the application than younger callers? In short, you should control your experiments the better to detect useful patterns.
