Looking at the LAD page got me thinking that PyTone probally need to look at supporting JACK. JACK was designed by the LAD folk for low-latency no dropout audio and they have tons of experience gettting it to happen.
Does PyTone have any plans to try and set any of the realtime process flags for the decoder thread? Most of these would require you to suid root for pytone but it may go a long way to solving your drop out problems.
I'm thinking that the dropouts may be caused by the garbage collector in python. #1 non-deterministic problem in embedded and realtime systems is memory management. Most of the time it falls into the "Don't do that" catagory when running realtime stuff.
Considering the ammount of RAM you guys are reporting PyTone using the GC may take a good bit of time to figure out what it needs to do. I've not played with the GC tweaking stuff in python but I seem to remember that there is more than one algo you can select for GC. Perhaps a less intrusive GC + realtime priority on the decoder thread might help?