Two or three individuals doing private beta testing provided Rick with more bug reports, and more strange issues, than the public beta testing did. It sounds counter-intuitive, but that is what happened when he did do public beta releases.
I think beta testing should be kind of like a medical trial experiment. A broad, or at least a variety of people that use Esword in diverse ways. I think you have to know the people personally that are testing the software so that you have confidence in what they are doing and telling you. The design intent has to be understood by the tester.
You need power users, module developers as well as a few people that just move around in different ways browsing reading, a little editing, whatever.
I gotta say though, there isn't any flawless beta testing scheme and the only way to ensure that you get fully tested software is to wait a few months on new releases and let users shake it out. The bigger problems will be solved by the time you get there, with the case of a free software with a limited development staff there are always going to be little things that you run across.