Strange comparison/evaluation issue

This is an archive of a topic from NESdev BBS, taken in mid-October 2019 before a server upgrade.
View original topic
Strange comparison/evaluation issue
by on (#226469)
Trying to handle a check for passing a threshold. In the positive direction, this works fine.

Code:

  LDA value
  CLC
  ADC additive
  BCC hasNotPassedThreshold
doThresholdStuff:



No problem.

Except sometimes, this is effectively adding a *negative* number (not signed, but where additive may have been 0-some value).

I tried this many different ways for proper evaluation, but I get anomalous results, such as it ALWAYS hitting the threshold stuff. I thought this would work:


Code:
  ;;; assuming this is only ever working in the 'negative' checks
  LDA value
  CLC ;; carry is set here
  ADC additive ;;;;(adding a var whose value was 0-some number, effectively subtracting)
        ;;; if abs additive is NOT bigger than value, carry should still be clear
        ;;; if it is, should it be set from underflow?
  BCC hasNotPassedThreshold
doThresholdStuff:
 


I've tried a few things, but that's one example. Can any one think of a better (fast) way to do this evaluation? It seems like there would be an easy way to figure out the underflow here for that instance where the threshold is crossed in the "negative" direction (even without using signed), but everything I'm trying is returning unexpected results.

Thanks!
Re: Strange comparison/evaluation issue
by on (#226472)
So you are adding a signed number to an unsigned number? What is the threshold when subtracting? Going negative? Either way, I don't think this will work if you don't know for sure if you are dealing with signed or unsigned numbers (If you need to handle values >127)
Re: Strange comparison/evaluation issue
by on (#226473)
(That was mostly just to demonstrate what i was trying to accomplish, not the method being used...sorry if that was confusing).

I fixed the problem. I SWEAR this is what I tried already, to wacky results, but just starting fresh today, it worked fine. Must've had a typo or something. Here's how I achieved it, for anyone who happens to find this:

Code:

   LDA additive
   EOR #$ff
   SEC
   ADC #$00
   STA temp

   LDA value
   SEC
   SBC temp
   BCC hasNotPassedThreshold
doThresholdStuff

Re: Strange comparison/evaluation issue
by on (#226474)
I don't understand what you're doing, but the overflow (V) flag is typically required for signed comparisons. Might look into that if you haven't already.
Re: Strange comparison/evaluation issue
by on (#226476)
You can also compliment a number by...

LDA #0
SEC
SBC value
Re: Strange comparison/evaluation issue
by on (#226486)
JoeGtake2 wrote:
Code:
   LDA additive
   EOR #$ff
   SEC
   ADC #$00
   STA temp

   LDA value
   SEC
   SBC temp
   BCC hasNotPassedThreshold
doThresholdStuff

Wait... You're negating a number (temp = -additive), and then subtracting that number from another value (a = value - temp), so the actual calculation this is doing is a = value - (-additive), which would be the same as a = value + additive. I don't get it. I mean, good that you somehow got this to work as expected, but I don't get it.
Re: Strange comparison/evaluation issue
by on (#226489)
Anyway, for the general problem of adding a signed number to an unsigned number, to find out if the number has overflowed, you want to know if the sign has changed as a result of the operation. Simplest way is to XOR the previous value with the new value, and check the negative flag (flag set by most significant bit of arithmetic result) of that.
Re: Strange comparison/evaluation issue
by on (#226490)
Code:
LDA additive
EOR #$ff
SEC
ADC #$00
STA temp
Is equivalent to:
Code:
LDA #0
SEC
SBC additive
sta temp
The method of negation with EOR #$FF has some useful applications but it doesn't seem like the best fit here.

But going further, since temp = 0 - additive:
Code:
LDA value
SEC
SBC temp ; temp = 0 - additive
BCC hasNotPassedThreshold
Is equivalent to:
Code:
LDA value
SEC
SBC additive
BCC hasNotPassedThreshold
And you don't have to use the first block of code to prepare temp at all. Additive can be negated directly in place with SBC.
Re: Strange comparison/evaluation issue
by on (#226491)
Dwedit wrote:
you want to know if the sign has changed as a result of the operation.

But an unsigned number changing from 127 to 129 when the valid range is (say) 8-247 shouldn't trigger clamping even if the sign bit has changed. Instead, to detect overflow when adding signed to unsigned, you can temporarily convert it to adding signed to signed and use the overflow bit logic.
Code:
clc
; not shown: add low bytes normally
lda unsigned_high_byte
eor #$80  ; convert 0..255 to -128..127
adc signed_amount_to_add
eor #$80  ; convert result back to 0..255 leaving V unchanged
sta unsigned_high_byte
bvc no_overflow_correction
  ; do clamping or whatever else
no_overflow_correction:
Re: Strange comparison/evaluation issue
by on (#226494)
I kind of think from the description of the problem that Joe is not really trying to do signed comparison. I think it might just be a value that in some cases is added (both unsigned) or in other cases is subtracted (again both unsigned). In this case, some other flag/variable somewhere may be deciding whether "additive" should be added or subtracted.

In the case given, if "value < additive", go to "hasNotPassedThreshold". (...and hopefully the result in A of value - additive is useful, but if you don't actually need it the SEC SBC can be replaced by a single CMP.)

Is there an parallel case for value + additive? Does that also have some sort of threshold branch?

There's probably also a way to use value (unsigned) + additive (signed) too, but maybe requires a better explanation of what we're trying to accomplish. Like if you just think about adding to a value from 0 to 255 another value from -128 to 127 that's fine, but think about the final value, and where it will wrap in 0 to 255. Things like the carry flag may or may not adequately catch what you need, but an additional CMP or two might do it. Think about the result of the add/subtract first, then decide afterward whether what you're hoping to make a branch on is already/automatically in the carry flag or not from that result.

(And if the problem really is trying to do a signed comparison, I generally always recommend this document as reference.)
Re: Strange comparison/evaluation issue
by on (#226498)
Order doesn't matter in addition. Order does matter in subtraction. For this reason, comparisons are done with subtraction. In fact, CMP uses the exact same logic as SBC, except for two differences.

1. It doesn't store the result.
2. It acts as if the carry flag was set before it runs.

Maybe some examples would help to show why (from my understanding of the problem) you're getting failures.

Code:
;;; assuming this is only ever working in the 'negative' checks
  LDA value
  CLC ;; carry is set here
  ADC additive ;;;;(adding a var whose value was 0-some number, effectively subtracting)
        ;;; if abs additive is NOT bigger than value, carry should still be clear
        ;;; if it is, should it be set from underflow?
  BCC hasNotPassedThreshold
doThresholdStuff:


Say value is 1. additive is 0. 1+0 is not greater than 255, so the carry stays clear.

Say value is 0. additive is 1. 0+1 is not greater than 255, so the carry stays clear.

Note that the carry stays clear in both cases. You can't check if values are greater than each other with addition, because again, order doesn't matter with addition. You can only check if both values together were greater than 255 (unsigned out of range), or out of range for signed.

Say value is 1. additive is 0. 1-0 is greater than or equal to 0, so the carry stays set.
Say value is 0. additive is 1. 0-1 is less than 0, so the carry gets cleared.

If you have additive set up as 0 minus some value, you can maybe avoid the subtract from zero and compare the number before you subtracted it from 0. A compare is a subtract anyway.

Depending on why you're subtracting from zero, you can potentially just change the order of comparisons (or the swap the branch condition) instead.
Re: Strange comparison/evaluation issue
by on (#226529)
Ok so the problem is while you think you are adding a negative number, you are not. If something is 0>255 or -128>127 is how we interpret something, not how the CPU does, the CPU doesn't give one hoot let alone two.

So

Lets you start with 5 and you add you magic value which lets say is 240 which gives you a range of 0-15 as valid

5 + 245 = 250 ; all is good, we didn't cross 255 so we didn't need to add the 9th bit so C is still clear
WHAT YOU THINK HAPPENS
-5 + 245 = 240 ; all is good we didn't cross 255 so we didn't need to add the 9th bit so C is still clear
But that is not true, what you are actually doing is
251+245 = 240 + C ; not good, walked forwards from 251, 245 times, got to 240 on the 8 bit but set the C flag as we went past 255, C is set

So if you want to check 0-some number less than 255, you are fine it will work. Pointless as you can just do
cmp #threshold
bcc
faster than
clc
adc #threshold
bcc
and will handle all the cases. But if you want to do -128<0<127 with your method, no. There is the overflow flag for doing signed compares but that has a pile of caveats with it and I think you are just trying to avoid doing a CMP and using more bytes and clocks to do it..