I ordered the parts to put together a tester a while back too, but have been too focused on my dimmer system to get it all thrown together.
I'm planning on using a PIC16F722. I DID order 3 cheap rotary encoders, but I think for my first pass, I'm going to use button up and button down on each digit... total of six buttons(Seems like the fastest channel select method. Buttons could be multiplexed, but I'm not really that worried about running out of IO pins in this design). I've got a slide pot that I'm planning to adc in for the level, and a three segment green led numeric display for output.
All the functionality I was planning was select a channel and the adc level gets spit out on it, with everything else at 0.
After watching this thread... I've got 5 IO lines that aren't in use yet... so that's 5 more buttons without messing with multi-plexing...
So I think I'll add (in version 0.2) :
1. Send the codes to program one of RJ's devices to this channel
2. Everything at 25%
3. Test sequence - .5 second up, then .5 second down on each channel between 0 and the currently selected channel (so I don't have to wait for 400 empty channels to flash if I don't have that many channels in use) with a new channel starting up as the previous channel hits full (every .5 seconds) If current channel is 1, test sequence will test all 512 channels. Pressing test sequence again will cancel test.
and in version .3

4. Capture and release mode - Pressing this key means that each channel that is brought up will stay up until it is brought down... so you can bring up a channel, then dial to another channel, move the slider to a new level (or just wiggle it to 'grab' the channel if you want it at the same level as the one before) Pressing the key again will release all the channels you've 'captured' this way. In my release version, due to memory constraints you'd probably only be able to 'capture' 32 channels at one time... but this is still enough to build a reasonable static look during testing... And the 16F722 is pin compatible with some 18F series parts with enough ram to store a whole DMX packet, so you could just drop in one of those if you want the ability to capture 512 channels... and kick the firmware around a bit. (the other fix, if people thought grabbing more channels was more important than being able to set dimmed levels for them... I could use 64 bytes to do a 'bitmap' of 512 channels for off and on control only in capture and release mode...? I'd love thoughts on whether grabbing 32 channels and setting levels dynamically or being able to turn on and off 512 channels and have them latch till you hit release would be more useful to people?)
and I suppose from there, if you DID go to a chip with more storage and wanted to mutiplex the last input, you could add a couple of 'preset' buttons and a 'record' button to store the current look into the preset buttons, for quick recall later? I'll lose interest in the project before that happens, but someone else could hack it in.

I'm at least a month away from really getting into all this, and it sounds like others are working on it too, so I posted my plans in case anyone wants to pillage ideas from them. I probably will be building my tester, because I want to learn what it has to teach as a project, but if someone in the DIY community comes up with a solution first, great!

Art