Zapper & LCD / Plasma Fix

Hi,

I know I am completely new to this forum but I've recently purchase a NES after never owning one. I had a lot of nostalgia for the NES purely because I wrote a NES emulator many years ago. I've since been looking at the problem related to modern LCD and Plasma TVs and I believe the Zapper problem on the NES can be addressed in one of two ways.

The main issue from what I can see is timing related. Modern TVs take a certain amount of time to process and display a frame - eg. 70ns, perhaps less or more depending on the TV and whatever image processing it does. I believe the PPU is displaying the black screen, then very soon after the CPU is reading the controller input from the Zapper, but as far as the TV is concerned the screen is still showing the game screen - not a black screen. The same then holds true a few ms later when the white boxes appear, the TV is still showing a normal game screen (or possibly black screen now) when the white boxes should be showing, but the TV is still processing that frame.

There are two options as far as I see here:-
1) Alter the game code (if possible) to increase any delay loop, to account for the TV frame processing time. It's going to vary from TV to TV and from game to game.
2) Produce some hardware (FGPA, or possibly a very fast PIC) to 'pause' the CPU clock for X ns / ms once the trigger is pressed. This could be enabled or disabled so it only affects Zapper games. The delay needs to be precise and timing set per game and TV, in order that the CPU is paused just before it checks the Zapper input vs what's displayed by the PPU. Could be difficult to get the timing precise.

For either of these the first thing anyone needs to know is what is the 'buffering' delay between the PPU outputing the image and the TV actually displaying it. I will look at this myself once I've dug out my old NES emulator code, because I will use that as a starting point. But considering there's bound to be a tonne of NES dev experts on here that could knock some NES code together quicker than I could right now, I thought I would float this and see if anyone has any ideas. I am proposing to write a small prog which could do no more than detect the Zapper trigger input (button held down) and then keep checking the Zapper light input vs changing the screen to black, and then to white and working out the time taken in nano seconds. This would be mega useful because it could be run on a cart or everdrive vs real hardware against both a CRT and HDTV. The timing difference there would be part of the solution to the problem rather either of the 2 fix approaches I've mentioned earlier.

This is based on a couple of assumptions, that there's not a hardware reason that after the trigger is pulled that a small limited window of detection is available due to the hardware itself. I am guessing that either the button needs to be held down, the NES can detect the light level detected by the Zapper at any time - I don't know, I've not looked yet and I never touched this regards emulation. Guessing its either a 0 or 1 for white or black?

The Zapper can detect light at any time. It's just that the light needs to be strobing at 15.7 kHz the way it does on a CRT, and I don't think LCDs and plasmas do that. I've pointed my Zapper at an LCD while running the Zapper test program Zap Ruder, and it registered the same way as if it weren't pointed at the screen at all. Even strobing an LED backlight at 15.7 kHz wouldn't help with games like Operation Wolf and ZapPing that not only need 15.7 kHz but also depend on the top-to-bottom scanning order, as they time from the top of the screen to when the sensor registers light.

Thanks, that's very interesting! So it looks like there are actually 2 issues here, well 3 if you include the fact that some games might depend on the top to bottom scanning order. I am thinking the 15.7 Khz issue could be solved by altering the light pickup side of things on the Zapper itself, perhaps for a different type of photo diode or something. I've got a Zapper on order so I will have a tinker and see if I can mod it in order to at least trick Zap Ruder to detect a hit (EDIT: That said, if the timing is an issue Zap Ruder might not detect a hit still - I've not looked at it yet but I will check it out). If I get that far I will try and take it to the next step.

I've also read that a few Sony LCD screens work with the Zapper - which is very interesting.

The zapper will fire any time it's pointed at a blinking light source—preferably 16kHz, but I've tested anything from 4kHz to 64kHz and had it work reasonably well too.

Modern LCD TVs with LED backlights often seem to trigger this.

The real problem with the Zapper (and all the other light pen -like input devices) is that they assume a feedback path (software → video → CRT → sensor → software) with a latency no longer than 15ms (for most games using the Zapper) to 1ms (ZapPing, Operation Wolf), or a couple of microseconds (light guns on other consoles that provide both X and Y coordinates). With a modern HDTV that adds up to a hundred ms latency for video processing...

Pausing the 2A03 is difficult... It's an NMOS 6502, so has a minimum speed, and the RDY input isn't exposed to the outside world. Additionally, the 2C02 has to be run at ≈full speed during vblank or its DRAM will lose its contents.

Doesn't the zapper register when its input goes from black to white? Adding a full black frame beforehand would fix that. You still need to deal with lag though - but even moreso, the fact that LCD screens have latency that's bound to make even that trick fail (as the zapper will see gray instead of black or white).

This is not an issue with LED screens since they should practically as instantaneous as CRTs (although it seems some early screens had issues with this), but there's still the signal-to-screen lag (dunno how much "game mode" fixes this), and there's still the fact the screen isn't scanned (so only games that don't check the position of the zapper could be made to work).

No, because the screen is basically working as a resonator which white will be easy to pick up and then also give you a whatever khz resonance you need. If the resonance is off, it won't work. Full on->Not working, Full off then on=60Hz resonance at best alternating=# khz needed.

I've considered a solution to this problem that involves using some other method to detect where the LCD-compatible Zapper substitute is pointing (something like a Wii Remote, perhaps?) and then uses those coordinates to filter the video signal from the NES into a "light" or "dark" signal.

The signal filter step would most likely need to be implemented in hardware, since the NES design assumes very low delay between the video output from the PPU and the signal received from the Zapper. I know enough about hardware design to have a basic idea of how this might be implemented, but not enough to come up with a specific design that would work.

The advantage of this sort of implementation is that it would work with all Zapper games and all TVs. As far as disadvantages, it would require slightly more setup before being able to play, and I can't imagine it being particularly cheap (though, depending on the design, it might cost nothing extra to make a single unit support two Zapper substitutes, which would be a nice feature).

I have been thinking about this, too. I was wondering what the screen actually displays when you pull the trigger in each case on a CRT, plasma, LCD, LED, projecter, and OLED. I would propose modding the light gun itself so that way it looks for what the screen is actually displaying while the trigger is pulled, and if a different detector can detect a hit/miss during the event for each of the display technologies above. I understand timing is key, and if the non-CRT displays are slower, then that would make everything impossible (which it sounds like it).

I just never saw anyone propose changing the detector in the gun itself.

What about some kind of "gun converter" that connects to the controller port and the composite cable? I think they had those things for Playstation 2.

It's not too hard to do something that would convert X,Y and RS170 in into a signal comparable to what the Zapper produces. Probably even doable in an Arduino, given the horizontal resolution of the Zapper... The only awkward part is providing a feedback path for those playing the game to match X,Y to what they see on-screen.

But even if the zapper knows what X,Y it is pointing at on the screen, how does it know if it hits something or not?

You're already going to need to synchronize against the video signal being output by the console just to make the zapper react to the proper timing. You may just snoop the brightness as well, then you can easily fake if the zapper "sees" a hit.

Yeah, it basically just works to "Y in → operate on scanline Y" and "X in → see at point X/Xmax through that scanline Y if the video signal is more than Z volts". You could probably do this with a microcontroller's compare port (convert X and Y into a timer value) and a little bit of analog circuitry... maybe even just a BJT, if its forward voltage happens to match.

You do need to keep the output from the fake-zapper true for approximately 4-16 scanlines, to be consistent with the real zapper, but no game relies on the lack of precision the true Zapper imposes on the X coordinate, or that the real zapper requires that it see light for several scanlines before it will trigger.

So it sounds like a possible way to do this would be to make a zapper that has a video input directly into it and a way to sense what coordinate it is being pointed at on a display. When you pull the trigger, the zapper compares the coordinate it is being pointed at to the electronic display signal (not optical from TV) and checks the brightness to see if there is a hit or not?

Why not take a page out of the Wii book and use IR light sources. You could maybe even use something reflective or distinctly colored. You could use 4 markers and put them on the corners of the display. With 4 markers you should be able to calculate where the gun is pointed no problem, right?

Yup.

The tricky part is figuring out how to make the UI on controlling X,Y usable.

I suppose it's not too hard to add a video overlay, and just allow visual servoing (the same feedback path used in the Wiimote, or a mouse)

Wouldn't that delay the product until roughly 2026 when the Wii patents expire? Or did Nintendo license the tech from someone else?

Here is one example of an IR tracking module, the host system sees it as a USB mouse.
http://www.ultimarc.com/aimtrak_module.html

Playstation 1 and 2 have a lightgun called the Namco GunCon (and GunCon 2). These lightguns use a direct connection to the composite video signal to aid in their accuracy. The GunCon 2 particularly is quite an accurate lightgun. I have not used the original GunCon 1, only a clone which worked well but was more shakey than the GC2.

Ofcourse it is still a lightgun, CRT required.

It certainly would if you were using the same exact tech. I think something similar to this:



would be a pretty interesting place to start. This mod would be from the outside in (zapper-> tv/signal -> nes) rather than other people suggesting modding from the inside out (game-> nes->TV->zapper).