dots.tts.edit

Precisely Controlled Speech Editing
with a Continuous Autoregressive Model

Hankun Wang1,2* Bohan Li1,2* Shi Lian1 Xiaoyu Gu1,2* Jing Peng1,2*
Da Zheng1 Yiwei Guo2 Colin Zhang1 Shuai Wang3 Kai Yu2†

1 Xiaohongshu Inc.

2 X-LANCE Lab, School of Computer Science, Shanghai Jiao Tong University

3 School of Intelligence Science and Technology, Nanjing University

wanghankun@xiaohongshu.com; {wanghankun,kai.yu}@sjtu.edu.cn

* Work done during an internship at Xiaohongshu Inc. † Corresponding author.

dots.tts.edit uses a transcript-grounded structural edit instruction with XML-style tags to specify what to change, where to change it, and the requested parameters. A continuous autoregressive model performs text, emotion, prosody, and pause edits, including their composition, while preserving the surrounding speech and speaker identity. Built on dots.tts.

First Glance

Two speakers. Two languages. Four edit types in one instruction.

Apply two text replacements, slow down an English phrase, insert a pause, and add melancholic delivery while preserving the surrounding speech and each speaker’s identity.

Loading the first example…

Overview

dots.tts.edit system architecture

doteBench Evaluation

doteBench V1.0 evaluates precise speech editing on 2,081 English and Chinese cases: Text (569), Emotion (612), Prosody (360), Pause (300), and Compositional (240). It measures three complementary dimensions: instruction following within the requested regions, local preservation of content, timing, pitch, and speaker identity, and audio quality of the result.

Distribution of 2,081 doteBench cases across five editing categories and operation types

Instruction following: edited-region WER/CER, emotion accuracy, pitch and duration errors, pause direction accuracy, and per-family compositional success.

Local preservation: content WER/CER, word-level duration and F0 preservation (WDTW-Dur and WDTW-F0), and WavLM ECAPA speaker similarity. Acoustic metrics use the eligible regions defined by each XML operation; F0 uses Praat.

Audio quality: UTMOS22 on the generated utterance.

One-take editing

Each complete instruction is compiled into one model call.

One-take results for dots.tts.edit, Step-Audio-EditX, Ming-UniAudio, MiMo-Audio-Instruct, and FireRedTTS3-Instruct across five editing categories
Five panels group metrics into instruction following, local preservation, and audio quality. Outward is better under fixed, model-independent normalization. Text uses the Hard shard; Emotion averages Neutral and Intense equally. The outer ring is 40% on Emotion Accuracy and Compositional Emotion Success. Open full-size figure.

Sequential editing

Each XML operation is executed in a separate model call; its output becomes the input for the next edit. Evaluation scores the final audio.

Sequential results for the same five models, categories, and metric groups
Panels, metric groups, aggregation, and normalization match the one-take comparison. Open full-size figure.

Reference results · all 2,081 cases

Operation success averages measurable requested operations. All-operations success requires every measurable operation in a case to succeed.

Loading reference results…

AuK uses Qwen3-ForcedAligner for local editing; the radar charts compare the five single-model systems.

Sources: doteBench release · Paper · Evaluation protocol.

Loading audio examples…