LM-SPT: LM-Aligned Semantic Distillation for Speech Tokenization

Audio demonstrations including reconstruction, semantic-only reconstruction, and zero-shot TTS results are provided
for various models (LM-SPT, Mimi (R), SpeechTokenizer) across multiple frame rates (25Hz, 12.5Hz, 6.25Hz).
Note that the semantic-only reconstruction results of LM-SPT were decoded exclusively using the main decoder (not auxiliary decoder).

25Hz 12.5Hz 6.25Hz

25Hz

Reconstruction

GroundTruth	LM-SPT	Mimi (R)	SpeechTokenizer

Reconstruction (Semantic Only)

GroundTruth	LM-SPT	Mimi (R)	SpeechTokenizer

Zero-shot Text-to-Speech

Text	LM-SPT	Mimi (R)	SpeechTokenizer
they were upon the verge of an open trap in the far corner of the hut and stuteley had tripped over the edge of the reversed flap mouth of this pit
were i in the warm room with all the splendor and magnificence
he seemed to wait for her reply but as she made none he proceeded
but now here is a subject of which you will wonder at first why turner drew it at all
without his scrapbooks his chemicals and his homely untidiness he was an uncomfortable man

12.5Hz

Reconstruction

GroundTruth	LM-SPT	Mimi (R)	SpeechTokenizer

Reconstruction (Semantic Only)

GroundTruth	LM-SPT	Mimi (R)	SpeechTokenizer

Zero-shot Text-to-Speech

Text	LM-SPT	Mimi (R)	SpeechTokenizer
they were upon the verge of an open trap in the far corner of the hut and stuteley had tripped over the edge of the reversed flap mouth of this pit
were i in the warm room with all the splendor and magnificence
he seemed to wait for her reply but as she made none he proceeded
but now here is a subject of which you will wonder at first why turner drew it at all
without his scrapbooks his chemicals and his homely untidiness he was an uncomfortable man

6.25Hz

Reconstruction

GroundTruth	LM-SPT	Mimi (R)	SpeechTokenizer

Reconstruction (Semantic Only)

GroundTruth	LM-SPT	Mimi (R)	SpeechTokenizer

Zero-shot Text-to-Speech

Text	LM-SPT	Mimi (R)	SpeechTokenizer
they were upon the verge of an open trap in the far corner of the hut and stuteley had tripped over the edge of the reversed flap mouth of this pit
were i in the warm room with all the splendor and magnificence
he seemed to wait for her reply but as she made none he proceeded
but now here is a subject of which you will wonder at first why turner drew it at all
without his scrapbooks his chemicals and his homely untidiness he was an uncomfortable man