Audio applications like search and editing require sound descriptions tied to precise times and speakers. Current audio AI models output free-form text that mixes sounds with reasoning, lacking temporal grounding.
TAC2 introduces composable timestamped captioning functions. Each task uses the audio, an instruction, and optional input, allowing tasks to be chained. Training data is built by aligning outputs from existing sound detectors. TAC2-S adds paralinguistic attributes using small datasets.
When paired with frozen text-only Large Language Models, TAC2 improves spoken understanding. For generation, it enables zero-shot control over event timing, speaker identity, and word emphasis. This demonstrates that structured, time-grounded audio representations can both aid understanding and enable controllable synthesis.