Full amplitude sine wave causes distort output

[ccbruce0812]

Hi...

I found the following problem, can you help me to resolve it?

1. Input WAV file contains stereo, full amplitude (-1.0 ~ + 1.0), 10KHz, 44.1KHz S16LE sine waves
2. The amplitude of waveform of target MP3 file cannot be stable. It falls and raises randomly within few milliseconds. This problem doesn't show up if the amplitude don't touch upper and lower bound (ex. -0.8 ~ + 0.8)
3. The same problem shows up under different frequency (ex. 500Hz).

[toots]

Thanks for the report, that seems like an interesting issue, I'll have a look at it soon.

[ccbruce0812]

The problem is caused by int32_t arithmetic overflow...

Below is an example:
When performing one of below statements in the loop of shine_mdct_sub() of l3mdct.c:

muladd(vm, vm_lo, mdct_in[?], config->mdct.cos_l[k][?]);

with

vm=-0.920113(0x8a39bcbc)
mdct_in=0.234619(1e07fed1)
cos_l=-0.711309(a4f3d3a3)

vm will be 0.996444(7f8b7793) but not -1.003556 (0xffffffff7f8b7793) because there is no room for signature.

Many other points in the source code can cause the same problem. My dirty and fast solution is to replace all int32_t of fixed point number arithmetic with int64_t. It works fine with 10KHz full-amplitude input but not with 500Hz full-amplitude input. It is necessary to dig deeper.