Well, in fact there *is* logic behind it. 80-FF are negative numbers because of the used notation,
two's complement. One of the main advantages about it is that you can consider a byte value unsigned or signed depending on the context; arithmetic operations will always make sense. If you substract 04 from 01, the result is FD ('cause past 0, you start over: 1->0->FF->FE->FD), which, in two's complement, is -3.
The N flag is just an aid to the programmer, and it is kind of copied from bit 7 of the value in the accumulator after certain instructions (as negative numbers, 80-FF, all have bit 7 set).