Overview
- TangoFlux is a fast text-to-audio model aligned with no human labels
- it uses CRPO: the model makes its own clips, CLAP ranks them, and the best and worst become a new preference pair each round
- the loop must stay online: reusing old data leads to reward over-optimisation
- the loss adds a flow-matching term on the winner, which keeps the model anchored
TangoFlux is a fast text-to-audio model that is aligned without one human label. It closes the preference loop: the model generates its own preference data, a frozen scorer ranks it, and the model trains on it, round after round.
Model
- 515M parameters, generates 30 seconds of 44.1 kHz audio in about 3.7 seconds
- uses rectified flow matching over the latent space of a Stable-Audio VAE
- pretrained on WavCaps and AudioCaps
- aligned with CLAP-Ranked Preference Optimization (CRPO)

TangoFlux gives better audio quality (higher CLAP score and lower FD score) than other text-to-audio models, while being about 2 times faster than the next fastest model and having fewer trainable parameters.
CRPO loop

At iteration :
- sample a set of prompts from the prompt pool
- generate audio clips per prompt from the current policy
- CLAP ranks the clips by the cosine similarity between the text and the audio
- the best clip becomes the winner and the worst becomes the loser
- train the next policy on this new dataset , then repeat
CLAP is a proxy reward model. The annotator is a frozen network, not a person. The authors describe this as resembling a self-improvement algorithm that can evolve by generating its own training dataset.
Why the loop must stay online
The loop only works if the data is regenerated every round.
- online CRPO: is regenerated from the current policy before every iteration; the CLAP score and the KL both improve over five iterations and then saturate, which is why training stopped at five
- offline CRPO: the same first-iteration dataset is reused five times; performance peaks early, then falls below where iteration 1 started
The KL here is not the KL penalty of RLHF. It is an audio evaluation metric, KL_passt, which compares the sound-label probabilities of the generated audio and the real audio. Lower is better, so "the KL improves" means it goes down.
The authors explain this by reward over-optimisation. In DPO, the reference model is the regulariser. Repeatedly updating against a fixed dataset destroys that regularisation, so the model starts to optimise the judge instead of the task.
Why the regulariser stops working:
- in each round, the reference model is reset to the latest policy, so the model is only held close to where it was one round ago
- with fresh data, each round learns from new pairs, so this is fine
- with the same pairs every round, each round pushes the same winners up and the same losers down again, so the model keeps moving in one direction, and nothing limits its total drift from the original model
- the model then learns the quirks of those fixed pairs and of the CLAP judge, not better audio
Fix inside the loss
where:
- is the total CRPO loss
- is the DPO loss written for a flow-matching model, which learns from the winner and loser pair
- is the ordinary flow-matching loss, computed on the winning audio only
The flow-matching term anchors the policy to the good sample, in the same way that the KL term anchors an RLHF policy to the SFT model. Self-improvement is not free: without fresh data, the model optimises the judge instead of the task.
Results
- TangoFlux reaches a CLAP score of 0.480 with 515M parameters and 3.7 seconds of inference
- Tango 2 scores 0.447 with 866M parameters and 22.8 seconds of inference
- the self-generated online data beats the static datasets BATON and Audio-Alpaca in a direct comparison
Note
TangoFlux continues the work of Tango 2, which built its preference data only once.