Yeah this doesn't really apply to the NES since it doesn't have a decimal mode -- but it's still 6502, so I figured it was worth asking here since some of you may know.
I have two nagging questions concerning how ADC/SBC work when in decimal mode:
1) How are N and V affected? Do they change at all? Since most of the "negative" digits (A-F) are removed in BCD numbers I'd imagine they'd be pretty worthless if they even work at all.
2) What happens when you try to add/subtract one or more numbers that aren't BCD? $13 + $09 would get you $22 of course... but what would $13 + $0A yield? or $0F + $0A?
1. If my memory serves me right, the Z flag is set before any BCD fixes are made (i.e. based on the binary result). The N and V flags are updated after the lower 4 bits are fixed, but before the upper four. The C flag is updated after both nybbles have been fixed (and is thus the only reliable flag after the operation is completed). All flags are updated using the same logic you would normally use for binary ADC/SBC (i.e. the N flag gets a copy of bit 7 of the intermediate result). Note that on a 65C02, all flags are updated after the operation is completed, which is why the 65C02 requires an extra clock cycle (but at least the flags have more useful meaning, particularly the Z flag).
2. I'm not sure on this. I believe you add the lower 4 bits as if you were doing binary, and if the result is greater than 9, add an extra 6. You don't check if the original numbers were BCD-valid or even if the result is valid - you just add 6 if the result is greater than 9. Same with the upper four bits. For subtraction, you'd subtract 6 if the partial result drops below zero.
N always carries bit 7, and Z is set as if decimal mode wasn't on at all. For instance, $E0 + $20 would set Z, even if $60 is added to the first value before your code performs the operation. I have no idea how O works, but C is supposed to be the only thing that does work, presumably if the value is > 99. I have tested this kind of thing like mad on my Atari 2600.
I guess I probably should have mentioned that I'm really asking this because I'm looking at a 65816 (making a ghetto SNES emu). I figured since it's supposedly backwards compatible with a 6502 that the same logic would apply... but now that dvdmth is talking about flags being set between nybbles I figured I'd bring this up.
If N works predictably, then it makes perfect sense to me that V would work the same way, too (since V seems to work as kind of an extension of the N flag). I read about D mode taking an extra cycle on a 65C02 and wondered what that was for... dvdmth's explanation makes a lot of sense.
But you've tested this WedNESday? Awesome. If it indeed works like you're saying, that's good news -- it means I can use the same flag code for everything, rather than conjuring up some crazy D mode flag setting tricks.
One final Q:
$0F + $0F = $30? or $10?
$30 makes logical sense (15 + 15 = 30), but if it's only adding nybbles at a time, it's possible that only 1 carry bit was moved to the high nybble, which would result in $10.
Thanks a bunch guys!
The 65816 in the SNES (more accurately a 65C816) is based on the 65C02. It sets all flags (N, Z, C, and V) after BCD fixing, not before as the original 6502. Thus, all flags reflect the result that ends up in the accumulator at the end of the operation.
I looked at the source code for BSNES (which should have this behavior implemented correctly, since byuu is primarily concerned with accuracy). Here's the basic idea:
For ADC:
* Add the lower nybbles of the two operands and store in a temporary variable. Include the carry flag in the addition.
* Add the upper nybbles and store in another temporary variable.
* If the lower nybble result is greater than 9, subtract ten (or add six), AND the result with $F, then add 1 to the upper nybble result.
* If the upper nybble result is greater than 9, subtract ten (or add six), AND the result with $F, then set the carry flag (which should otherwise be clear).
* Set the accumulator to (upper nybble << 4) + lower nybble.
* Now set N, Z, and V as you would for binary ADC.
For SBC:
* Subtract the lower nybbles of the two operands and store in a temporary variable. Include the carry flag in the subtraction (subtract an extra 1 if carry is clear).
* Subtract the upper nybbles and store in another temporary variable.
* If the (unsigned) lower nybble result is greater than 9, add ten (or subtract six), AND the result with $F, then subtract 1 from the upper nybble result.
* If the (unsigned) upper nybble result is greater than 9, add ten (or subtract six), AND the result with $F, then clear the carry flag (which should otherwise be set).
* Set the accumulator to (upper nybble << 4) + lower nybble.
* Now set N, Z, and V as you would for binary SBC.
Note that the 65C816 has a 16-bit mode, in which you will need to repeat the steps for the upper byte, but the process is the same.