Drama 3 (35B TTS) speaking Turkish at 2.7x real time on one 16GB GPU

The project, named drama-3-tr, runs the Fish Audio Drama 3 model on a single 16 GB GPU, with a distilled 32-expert speaker. It processes Turkish text and keeps inline directions and emotion intact. The 41.5 GB model checkpoint is streamed from an NVMe drive, so it never has to sit in RAM.

Weights and full results are on Hugging Face at kasparovabi/drama-3-tr-speaker. The project is built with Fish Audio and fish-speech, but it is not affiliated with or endorsed by Fish Audio. It was tested on Windows 11, Python 3.13, torch 2.14.1+cu132, and triton-windows 3.8.

Optimizations for speed

The first release ran at 0.86x real time on short sentences in disk mode. The new version achieves about 2.7x real time for short sentences and about 3.3x for paragraphs. The speedup comes from several optimizations.

• torch.compile on the per-frame step, then CUDA graph capture of the compiled code.
• Profiling showed more than 2,000 small elementwise and dtype cast kernels per frame around the matrix multiplications. A frame went from 22.7 ms to 13.6 ms.
• Logits are only for the 4,097 tokens the audio head can sample (end of speech and the semantic codes) instead of the full 252k vocabulary.
• In speaker mode, the 256-expert router of every layer is now skipped, as it was previously computed and then ignored.

A pinned RAM tier holds the 6,000 experts that 300 traced Turkish sentences use most, chosen from results/trace_experts.pt. Per sentence, only about 40 experts still come from the drive. This setup keeps 18.6 GB in pinned RAM.

Evaluation and configuration

The project includes an evaluation script for the Freya-TR-Eval dataset, which contains 495 sentences. Users can generate the sentences with --sentences ref/freya_tr_eval. --out out/freya_tr_eval and then score them. The evaluation uses faster-whisper, jiwer, librosa, and soxr.

The project offers several configuration options. The TTS_COMPILE=max-autotune-no-cudagraphs flag compiles the per-frame step with torch.compile before it is captured in a CUDA graph, which adds a few minutes to start-up. The TTS_FP8=1 flag stores weights in block FP8, which is about 8% faster but raised the Freya-TR-Eval word error rate to 3.29% in the developer's run, so it is off by default.

The --cfg flag sets the direction guidance strength (default 4, 0 turns it off) and the --seed flag makes a run reproducible. The server.py script serves the engine over HTTP with a POST /tts endpoint. It reads TTS_HOT, TTS_COMPILE, and TTS_FP8 from the environment and caps each take's length by the length of the text.