LACE: Layer-Wise Compression for Dynamic Frame Rate Codecs
Neural audio codecs are a key component in speech language modeling. However, their high frame rates lead to long sequence lengths, increasing computational costs. Dynamic frame rate codecs mitigate this by reducing the effective frame rate using a compression step to merge multiple frames together. However, most prior methods either operate on single-codebook codecs or apply a
Reading in Plain English mode — toggle below to switch
Neural audio codecs are a key component in speech language modeling. However, their high frame rates lead to long sequence lengths, increasing computational costs. Dynamic frame rate codecs mitigate this by reducing the effective frame rate using a compression step to merge multiple frames together. However, most prior methods either operate on single-codebook codecs or apply a
This paper introduces a novel approach to dynamic frame rate codecs for neural audio, significantly reducing computational costs and sequence lengths. For ML engineers, understanding such compression techniques is crucial for deploying efficient and scalable audio models, especially in real-time applications.