Question on FullSNES BRR Sample Creation Recommendations

This is an archive of a topic from NESdev BBS, taken in mid-October 2019 before a server upgrade.
View original topic
Question on FullSNES BRR Sample Creation Recommendations
by on (#115227)
http://nocash.emubase.de/fullsnes.htm#s ... brrsamples

Look on the note about creating BRR Samples:
Quote:
When creating BRR data, take care that "new" does never exceed -3FFAh..+3FF8h, otherwise a number of hardware glitches will occur:


Um Ok, thanks FullSnes!! However...
Quote:
If new>+7FFFh then new=+7FFFh (but, clipped to +3FFFh below) ;\clamp 16bit
If new<-8000h then new=-8000h (but, clipped to ZERO below) ;/(dirt-effect)
If new=(+4000h..+7FFFh) then new=(-4000h..-1) ;\clip 15bit
If new=(-8000h..-4001h) then new=(-0..-3FFFh) ;/(lost-sign)
If new>+3FF8h OR new<-3FFAh then overflows can occur in Gauss section


and most importantly these two lines:
Quote:
If new>+7FFFh then new=+7FFFh (but, clipped to +3FFFh below) ;\clamp 16bit
If new=(+4000h..+7FFFh) then new=(-4000h..-1) ;\clip 15bit


If new=(+4000h..+7FFFh) then new=(-4000h..-1) :? ????
He flipped the signs, contrary to what the first line anticipates. (but clipped to +3FFFh below). Also, if you follow the common sense of "Clip 15 bit" to $7fff, you will get +$3fff as a result. So I think that's a mistake, you probably don't want to make that negative like it suggests. Please clarify it.
Re: Question on FullSNES BRR Sample Creation Recommendations
by on (#115230)
The document states that the samples passed on to the filter are 15-bit. A 15-bit signed (two's complement) value can represent -0x4000..+0x3FFF.

Quote:
If new>+7FFFh then new=+7FFFh (but, clipped to +3FFFh below) ;\clamp 16bit

What he's saying is that first any value that would grow above +0x7FFF due to the filter application is clamped, rather than wrapping around.

Quote:
If new=(+4000h..+7FFFh) then new=(-4000h..-1) :? ????

As mentioned above, the maximum positive value that can be represented as a 15-bit signed value is 0x3FFF. Beyond that it "wraps around" and you end up in the negative range, so e.g. 0x4000 will be interpreted as -0x4000 (the maximum negative value that can be represented). Just as 0x80 is -128 when viewed as a signed byte (0x81 is -127, 0xFE is -2, 0xFF is -1, and so on).
Re: Question on FullSNES BRR Sample Creation Recommendations
by on (#115231)
ugh, I thought the data was converted from BRR to 16-bit data. now it's 15?? WTF. Even the official manual says 16-bits, but we all know it's probably crap.

I'm so confused. My experience thus far is from converting a 16-bit PCM samples to BRR format successfully with no filter, (filter 0). That worked without any 15-bit treatment. How does that change with filters, and why did I not need to do these weird bounds checking in my filter 0 encoder?
Re: Question on FullSNES BRR Sample Creation Recommendations
by on (#115232)
Quote:
and why did I not need to do these weird bounds checking in my filter 0 encoder?

There's no accumulation involved with filter 0.

Filter 0: new = sample, and sample = (nibble SHL shift) SAR 1

The nibbles are viewed as signed (range -8..+7), and the maximum allowed shift is 12. So the largest positive value you'd been able to produce with filter 0 is (7 SHL 12) SAR 1 = 0x3800, which is in the safe range.

For negative values the max would be (-8 SHL 12) SAR 1 = 0xC000 = -0x4000 (16-bit), which when clipped to 15-bit should still be -0x4000. This would not be safe to use according to the fullsnes document, since it falls below -0x3FFA.
Re: Question on FullSNES BRR Sample Creation Recommendations
by on (#115233)
Quote:
For negative values the max would be (-8 SHL 12) SAR 1 = 0xC000 = -0x4000 (16-bit), which when clipped to 15-bit should still be -0x4000. This would not be safe to use according to the fullsnes document, since it falls below -0x3FFA.


Does that mean, that even for filter 0, that a -8 nibble in a range 12 packet ought to never be used?

Also I'm not too keen on the SHL and SAR shift operations.
Re: Question on FullSNES BRR Sample Creation Recommendations
by on (#115234)
for reference:

the SAR instruction sets or clears the most significant bit to correspond to the sign (most signif-icant bit) of the original value in the destination operand.
Code:
 In effect, the SAR instruction fills the empty bit position's shifted value with the sign of the unshifted value (see Figure 6-8 in the Intel Architecture Software Developer's Manual, Volume 1).


for people like me who like illustration:

take nibble 4bits, -8,
this is 1000 (you must know signed)
1000 << 12 = (do it out) = $8000

$8000 = just the 16th bit is set. here:
1000000000000000

NOW: ASR does this:

the MSBit is 1, ASR remembers that value and will restore the MSbit to 1 after the shift.
1) shift right 1
0100000000000000
2) restore MSbit sign
1100000000000000 = $c000

remember we are signed, so $c000 is actually -$4000. You can get this value by counting backwards from $FFFF (0 really). taking 2's complement (which can indirectly be achieved by 'value' XOR $FFFF + 1
Re: Question on FullSNES BRR Sample Creation Recommendations
by on (#115235)
Quote:
Does that mean, that even for filter 0, that a -8 nibble in a range 12 packet ought to never be used?


I haven't tested this myself. But based on nocash's description I'd say "Yes, avoid it".
Re: Question on FullSNES BRR Sample Creation Recommendations
by on (#115236)
But would the best approach be to simply downgrade the step to -7? Surely that's better than tossing the value the completely, I mean you need to use something right?


Lots of questions entailing... Such as what to do if the higher value is a good match for the sample's original value, but not in accordance with FullSnes' specifications. Do you just take step one lower than that?



I guess there's room for experimentation.

Plus I am a little ahead of myself, outside the scope of filter=0 anyways. Because I am sure I can make better decisions once I finally tackle actually filtering a sample.
Re: Question on FullSNES BRR Sample Creation Recommendations
by on (#115237)
Mic_ thanks for the time and help I appreciate it. Things are starting to come full circle
Re: Question on FullSNES BRR Sample Creation Recommendations
by on (#115360)
mic_ wrote:

Filter 0: new = sample, and sample = (nibble SHL shift) SAR 1



Mic_, Or anbody, Why does the sample need the SAR 1 after SHL shift.
Re: Question on FullSNES BRR Sample Creation Recommendations
by on (#115361)
From my angle, SAR 1 seems erroneous.

If i Have an original sample value of 28672, for instance, that is a max step value of +7 with range = 12.

7 << 12 = 28672. No SAR 1..

vice versa, if I had orig. sample -32768, that's step -8, range 12.

-8 << 12 = -32768. No SAR 1

Am I missing something?
Re: Question on FullSNES BRR Sample Creation Recommendations
by on (#115366)
Those formulas are descriptions of how the S-DSP handles the sample data internally. You'd have to ask nocash how he arrived at his conclusions since it's his document.

But if we accept what the document says as true, then the 16-bit samples are converted to 15-bit by means of throwing away the msb before they are sent to the DSP's filter. What happens when you throw away the msb of -32768? You get 0; probably not the desired result. What happens if you first shift it right one bit and then throw away the msb? You get -16384; the maximum negative value that you can represent with 15 bits, and probably the result you wanted.
Re: Question on FullSNES BRR Sample Creation Recommendations
by on (#115368)
mic_ wrote:
What happens when you throw away the msb of -32768? You get 0; probably not the desired result. What happens if you first shift it right one bit and then throw away the msb? You get -16384


Let's be Perfect.
for this DSP operation, it drops the 16th bit all the time. However, and I want you to clarify this, the MSbit of the actual SA operation, is kept, correct?

Let's try a more precise example that demonstrates the SAR operation. Hence, something that lands in bit 15 before the SA 1.

ok let's try 16384, that's a step value of 4, range =12
4 nibble = %100

let's do the DSP operation: (4 << 12) SAR 1

4 << 12 = %01000000 00000000 : A
A SAR 1 = %01100000 00000000

Am I doing that correctly?
Re: Question on FullSNES BRR Sample Creation Recommendations
by on (#115377)
Say I wanted to make a tool that goes the other way: BRR to WAV. I'd use this to verify that my converter is working correctly. Which BRR library should I base this on?
Re: Question on FullSNES BRR Sample Creation Recommendations
by on (#115401)
The "sample = (nibble SHL shift) SAR 1" means
multiply by "shift" times by 2.
Then divide once by 2.
But I guess you've figured out that part.

bazz wrote:
let's do the DSP operation: (4 << 12) SAR 1
4 << 12 = %01000000 00000000 : A
A SAR 1 = %01100000 00000000
Am I doing that correctly?

Almost, but there aren't any sign-glitches occurring in that formula.
+4 SHL 12 gives 0100000000000000 aka +40000h (as you said).
And after SAR 1 you get 0010000000000000 aka +2000h (what you've got was +6000h).

With filter 0 the only glitch will happen on -8 with shift=12, that due to the bug in the gauss interpolation (the fullsnes BRR Pitch chaper shows how the bugged waveform would look like; with two small unwanted spikes).

With filter=1..3, more things can go wrong. Sony tried to handle overflows automatically. But they've accidently saturated the values to 16bits (-8000h..+7FFFh) and did then cripple it to 15bits by removing the sign bit. What they wanted to do was saturating to 15bits (-4000h..+3FFFh). And even if they would have got that part right, values exceeding -3FFAh..+3FF8h would be still causing problems in the buggy gauss interpolation unit.

In result, you'll need to take care of that yourself: Be sure that the filtered values don't sum up to something that exceeds the -3FFAh..+3FF8h range.

But before blaming Sony too much: One should normally take care that the values don't exceed the the supported range anyways (ie. they have only failed to do some automatic error-handling... which would be needed only if the BRR samples contain crappy data).

If you use looped sounds combined with the (lossy) compression filters: Then incoming old/older values may grow bigger and bigger in each loop-cycle (causing overflows after repeating the loop too many times). The safest way to avoid that would be to put at least one BRR block with filter=0 into the loop (that forces the following old/older values back to clearly defined settings).
Re: Question on FullSNES BRR Sample Creation Recommendations
by on (#115421)
Thanks for all the protips nocash.

However, should the operation be a shr instead of sar, because sar as i have seen it defined preserves the preShifted MSbit. Shr would not preserve it.

Thus, why not just do nibble << (range-1) in that case.
Re: Question on FullSNES BRR Sample Creation Recommendations
by on (#115428)
bazz wrote:
why not just do nibble << (range-1) in that case.

Yes, that's the same. But for that notation, you could get "0-1" as shift amount, which would sound a but weird.

bazz wrote:
However, should the operation be a shr instead of sar, because sar as i have seen it defined preserves the preShifted MSbit. Shr would not preserve it.

SAR is for preserving the sign bit in signed numbers.
SHR is for unsigned numbers - that would be wrong for the signed BRR values.

You don't really have to mind much about that shifts. Just think about each shift-left step as "multiply by 2", and about each shift-right step as "divide by 2".
What need to care about: When using asm-code, use SAR/SHR for signed/unsigned numbers accordingly. When using HLL-code, define your variables as signed/unsigned variables accordingly.
Re: Question on FullSNES BRR Sample Creation Recommendations
by on (#115430)
bazz wrote:
why not just do nibble << (range-1) in that case.

Because some languages leave SHL by a negative number undefined, and others raise an exception when it occurs.
Re: Question on FullSNES BRR Sample Creation Recommendations
by on (#115433)
Thanks