How Do Audio Codecs Work? The Psychoacoustic Model and Bitrate Logic

A WAV file takes about 10 MB per minute; the same audio drops to 1 MB as MP3. The data falls to a tenth but most people don't hear the difference. How is that possible? The answer lies less in compression math than in how the human ear works. This article explains the logic of lossy audio codecs and what should drive your conversion decisions.

What raw audio data is

A microphone converts changes in air pressure into an electrical signal. When that signal is digitized, two parameters are set:

Sample rate: how many measurements are taken per second. The CD standard is 44,100 Hz. That number isn't arbitrary: according to the Nyquist theorem, capturing a signal correctly requires sampling at twice its highest frequency. Since the human ear hears up to about 20,000 Hz, 44,100 provides a sufficient upper bound.

Bit depth: how many bits express each measurement. CD uses 16 bits, which means 65,536 distinct amplitude levels. 24 bits is common in studio recording.

The arithmetic is simple: 44,100 samples × 16 bits × 2 channels = 1,411,200 bits per second, or about 1,411 kbps. A minute of stereo audio takes about 10 MB uncompressed.

The WAV format stores exactly that raw data. FLAC holds the same data at about half the size with lossless compression — decompressed, exactly the same samples come back.

The basic idea of lossy encoding

Lossy encoders like MP3 ask a different question: how much of this data do we actually hear?

The answer: far less than you'd think. The human auditory system isn't a perfect recording device; under certain conditions it doesn't perceive certain sounds at all. The encoder finds those gaps and discards that information.

That's done through a calculation called a psychoacoustic model.

Masking: the sounds we can't hear

The foundation of the psychoacoustic model is the phenomenon of masking, and it comes in two kinds.

Frequency masking (simultaneous masking). A loud component makes quieter components at nearby frequencies imperceptible. It's like not being able to hear a whisper in a noisy room — but it happens on a much finer scale along the frequency axis.

A concrete example: while there's a strong tone at 1000 Hz, a tone at 1100 Hz below a certain threshold goes unheard. The encoder can discard that component or code it very coarsely; the ear doesn't notice the difference.

Temporal masking. Weak sounds arriving just before (about 20 ms) and just after (about 100-200 ms) a loud moment go unperceived. The ear stays briefly "deaf" after a strong stimulus.

Using both phenomena together, the encoder computes a masking threshold for every moment: at which frequency, below which level, information can be discarded.

The encoding process step by step

Roughly what an MP3 encoder does:

1. Split the audio into blocks. Usually windows of a few hundred to a thousand samples. Block size matters: long blocks give frequency resolution, short blocks give time resolution. The encoder switches to short blocks at sharp transitions (a drum hit, say).

2. Transform into the frequency domain. Time-domain data is decomposed into frequency components (subband filtering plus MDCT in MP3, direct MDCT in AAC).

3. Run the psychoacoustic model. A masking threshold is computed for each frequency band.

4. Distribute the bit budget. The available bitrate budget is distributed across how many bits each band gets. Masked bands get few bits or none.

5. Quantization. Each component is coarsened according to the number of bits allocated to it. This is where the loss happens — an approximate value is stored instead of the original's exact value.

6. Entropy coding. A lossless compression (like Huffman) is applied to the remaining data.

Decoding runs these steps in reverse, but the information discarded in step 5 doesn't come back. Quantization is irreversible.

Bitrate: how tight the budget is

Bitrate is the budget you give the encoder. At a low bitrate the encoder has to discard more information, and the loss starts to become audible.

What degrades most at low bitrate:

| Problem | Where it's heard | |---|---| | High frequency cutoff | Cymbals, high-pitched instruments, "s" sounds go dull | | Pre-echo | A faint "swish" before a drum hit | | Stereo degradation | A wide stereo image narrows | | Metallic timbre | Applause and crowd noise sound "watery" | | Quantization noise | Background hiss in quiet passages |

A rough reference (for MP3, with a good encoder):

| Bitrate | Perceived quality | |---|---| | 64 kbps | Acceptable for speech, obvious degradation in music | | 96 kbps | Enough for podcasts and audiobooks | | 128 kbps | General listening, noticeable to an attentive ear | | 192 kbps | Good for most uses | | 256 kbps | Close to transparent for most listeners | | 320 kbps | The practical ceiling, transparent for most ears |

The term "transparency" means the encoded audio can't be distinguished from the original in a blind listening test. It varies from person to person, content to content and equipment to equipment.

Constant and variable bitrate

CBR (constant bitrate): the same amount of data is used every second. Simple and predictable; advantageous for streaming.

VBR (variable bitrate): the encoder allocates as many bits as each moment needs. It saves during a quiet piano passage and spends during a dense orchestral section.

VBR is generally better because music's complexity isn't constant. At the same average bitrate, VBR gives better perceived quality than CBR. Its only downsides are that file size can't be predicted in advance and some older players have duration/seeking problems.

ABR (average bitrate) is a compromise between the two.

Why codecs differ in efficiency

AAC sounds better than MP3 at the same bitrate. The reasons:

• More flexible block sizes. It handles sharp transitions better and pre-echo is reduced.
• Better stereo coding. It exploits similarity between channels more efficiently.
• More efficient entropy coding.
• A more advanced psychoacoustic model.
• Design freedom. MP3 was designed with backward compatibility constraints; AAC started from scratch.

Opus goes further still and is noticeably good especially at low bitrates (32-64 kbps); it carries components designed for voice calls and podcasts.

The practical equivalence: 128 kbps AAC is perceived as roughly equal to 160-192 kbps MP3. That explains why the choice of target bitrate depends on the source format when converting.

Why transcoding loss accumulates

When you convert a lossy file to another lossy format, this happens:

1. The source file is decoded. The audio you get is already a lossy approximation of the original.
2. The new encoder applies its own psychoacoustic model to that audio and discards information again.

The problem is that the second encoder doesn't know what the first one discarded. It decides according to its own model and those decisions don't overlap with the first's. In the end two different sets of "discards" stack on top of each other.

On top of that, the quantization noise left by the first encoding looks like "real audio" to the second encoder, which spends budget trying to preserve it.

That's why chained conversions degrade quickly. On a path like MP3 → AAC → MP3 → OGG, at every step the high frequencies fade a little more and the transitions blur a little more.

The rule: always go from the best source you have to your target in a single step.

Going back to lossless formats

Converting MP3 to FLAC is a common misunderstanding. FLAC is lossless, but that means "stores what it's given without damaging it" — not "brings back what was lost."

The frequency components discarded during MP3 encoding are permanently gone. When you convert to FLAC you store that already-diminished audio losslessly. The result: a file at MP3 quality that's five times larger.

The one legitimate exception: if you're going to edit a file. To avoid re-encoding on every save during operations like cutting, joining and applying effects, converting temporarily to WAV makes sense. When the work is done you encode to your target format once.

In summary

Lossy audio codecs exploit the masking properties of the human ear to discard inaudible information; the loss happens at the quantization step and is irreversible. Bitrate is the budget for that operation — at a tight budget the first things sacrificed are high frequencies, stereo width and transition clarity. Different codecs have different efficiency; AAC and Opus give better results than MP3 at the same bitrate, which is why the target bitrate should be kept one notch above the source when converting. And the most important practical consequence: lossy conversions are cumulative and going back to lossless gains no quality — always go from the best source to your target in a single step.