Any-Yes: A Cycle-Accurate Java NES emulator

This is an archive of a topic from NESdev BBS, taken in mid-October 2019 before a server upgrade.
View original topic
Any-Yes: A Cycle-Accurate Java NES emulator
by on (#190748)
Wanting to learn more about the NES, I decided to write my own emulator, and today I'm releasing it into the wild. I plan to do a lot more work on it, and I have some interesting ideas that I hope will carve it into a niche of its own.

Some features of note include NTSC simulation, gameplay recording, mid-instruction cycle-accuracy, game genie support.
Currently Supported Mappers: 0, 1, 2, 3, 4, 5, 7, 9, 10, 11, 13, 16, 18, 19, 28, 34, 66, 69, 71, 79, 85, 118, 119, 140, 163, 185, 228, 232
Many difficult to emulate games work, such as BattleToads, Bee-52, Mig 29 Soviet Fighter, and Crystalis to name a few.

I attempted cycle-accuracy in the CPU and PPU, and have more or less achieved it. There is still a lot more work to be done, but it's totally playable currently.
I plan on putting the code up soon under an unrestrictive license (MIT maybe), after I clean it up a little.

Download Any-Yes 0.2.0

Let me know what you think, I certainly plan on improving it greatly! Thanks!

Instructions, from the readme:
===============================================================================
Operating Any-Yes
===============================================================================
To run: You must have Java installed. Simply double click the .jar file if you are in Windows.
If that doesn't work, or you are in Linux, run this command:
java -jar any-yes.jar

Put ROMs in the included ROMS folder, which is the easiest method of accessing them.
You can also navigate to whereever they are using the built-in file browser, but it's still somewhat of a work in prgress.

The file browser is simply a frontend. To set key mappings and other options, you must launch a game.
To navigate, click files or directories. To go up to a previous directory, click the folder name in the top bar.
If you need to change the start-up directory, edit last_dir.txt, which is in the loader/config/all/ subdirectory of the config location (see below)

You can make the window large by double clicking the title bar, or dragging the edges to resize. True fullscreen support will come soon.

Zip support is very temperamental, some zip files refuse to open currently. If you run into this problem, please unzip the rom and try again.

===============================================================================
Configuration
===============================================================================
To map controller controls:
Press Escape while in a game
Click "Map Controls"
Currently cannot remap keyboard controls.

All config data currently is put in:
Windows: <OS Drive>:/vnand/
Linux: ~/vnand/
(VNAnd is the name of a larger multi-emulator platform I am working on, of which Any-Yes is a module)
Any-Yes specific data and configs are in the engine_data/any-yes subdirectory.

===============================================================================
Recordings
===============================================================================
Some notes about the recording system:
Press R or click "Create" in the menu to begin recording.
Press R again or click "Finish" to complete the recording.
Press T to play the recording. Currently you can only have one recording at a time per game, to have more, move the recording file for that game.
You can make a recording over multiple play sessions by saving the state (F5) while recording. If you load the save state later, it will continue recording.

===============================================================================
Key list
===============================================================================
Escape: Toggle Pause / Menu

Q - toggle NTSC simulation
W - toggle linear screen filter
S - toggle scanline overlay

` - hold for fast forward

R - toggle recording
T - play recording

F1 - quick save state
F2 - quick load state

F5 - save state file
F6 - Load state file

Arrow key - D-Pad
Z - B
X - A
Tab - Select
Enter - Start

===============================================================================
Upcoming Features
===============================================================================
Some lofty and not-so-lofty upcoming features I plan on supporting:

- Rewind
- Game Manual / Cover archival
- Better frontend
- Android (Mostly working)
- Neural Network to learn to play games (via reinforcement learning, think Mar I/O)
- Javascript plugins
- External hardware simulation (plug and unplug virtual controllers, TVs, etc in a visual way)
- Fast cycle-innacurate core

===============================================================================
Changelog
===============================================================================
0.2.0:
- Command line launch support
- Added more conventional menu bar
- various optimizations
- Rewrote NMI triggering (Fixes Bases Loaded II)

0.1.3:
- Added mappers 85 and 34
- Adapted VRC7 sound chip from nesimulare (MIT licensed) for use with Any-Yes APU
- Partial Namco 163 sound chip support
- Audio high & low pass filtering
- Major performance increase (Located a huge source of Java garbage collection)
- Prevent joysticks that rapid-fire events from making key mapping impossible
- Save state serialzation rewrite
- Minor timing fixes
- Better handling of cleanup for audio and video threads

0.1.2:
- Added mappers 18, 79, and 185
- Fixed save state info in mapper 16 and 19
- NTSC simulation now attempts to use more CPU cores
- CHR-RAM banking, fixing Romancia and Lagrange Point
- Updated libGDX to 1.9.6
- Updated lwjgl to version 3

0.1.1:
- Added dummy write in INC and multi-write detection in MMC1 (Fixes Bill & Ted)
- PPU $2004 now returns more accurate values (Fixes Micro Machines) (Still needs a little work)
- Adjusted NMI timing (Fixes Battletoads)
Re: Any-Yes: A Cycle-Accurate Java NES emulator
by on (#190757)
Passes the Chu Chu Rocket test (Great job!), but fails the Bill & Ted test.
Re: Any-Yes: A Cycle-Accurate Java NES emulator
by on (#190758)
Dwedit wrote:
Passes the Chu Chu Rocket test (Great job!), but fails the Bill & Ted test.


Good catch, the multi-write latch that breaks Bill & Ted is on my short list of things to fix!
Re: Any-Yes: A Cycle-Accurate Java NES emulator
by on (#190762)
Doesn't seem to run a game I converted to MMC5 correctly but it could be my implementation. It doesn't seem to switch the BG-CHR banks right.

My MMC3 to MMC5 conversion works under the following emulators:
FCEUX
Nestopia
Nintendulator
Mesen
Re: Any-Yes: A Cycle-Accurate Java NES emulator
by on (#190763)
hackfresh wrote:
Doesn't seem to run a game I converted to MMC5 correctly but it could be my implementation. It doesn't seem to switch the BG-CHR banks right.

My MMC3 to MMC5 conversion works under the following emulators:
FCEUX
Nestopia
Nintendulator
Mesen


MMC5 is one I haven't tested a lot, I don't think the PRG-RAM banking is working correctly for instance (Uncharted Waters seems to be the only game that really uses it). Do you have an IPS patch for your conversion so I can test it? It would help me out a lot!
Re: Any-Yes: A Cycle-Accurate Java NES emulator
by on (#190767)
I PM'ed it to you.
Re: Any-Yes: A Cycle-Accurate Java NES emulator
by on (#190779)
Update:
After testing I discovered that I had managed to break Battletoads and hadn't noticed (probably in a recent round of timing corrections), but I managed to fix it; NMI triggering was off by a CPU cycle which was causing random mayhem.
I also discovered I was having the dreaded Micro Machines jittering (I had never tested the game). My $2004 read isn't pulling from secondary OAM during the proper cycles. I put a bandage on it with the recommended hack for now. Since I want this to be cycle-accurate I intend to put a proper fix in place.

I'll get a new release out later on today with those fixes and hopefully I'll have time to implement some more.
Re: Any-Yes: A Cycle-Accurate Java NES emulator
by on (#190803)
Made a small compatibility release:
Download Any-Yes 0.1.1

Changes:
- Added dummy write in INC and multi-write detection in MMC1 (Fixes Bill & Ted)
- PPU $2004 now returns more accurate values (Fixes Micro Machines) (Still needs a little work)
- Adjusted NMI timing (Fixes Battletoads)
Re: Any-Yes: A Cycle-Accurate Java NES emulator
by on (#190855)
If you will add a JavaScript plugins then you should need one that can use ES6 features (I think Rhino doesn't, but I am not sure). An important JavaScript feature inside of a NES emulator I think would be typed arrays, so you should use one that implements that. I don't know which ones that are compatible with Java do have such capabilities.
Re: Any-Yes: A Cycle-Accurate Java NES emulator
by on (#190877)
ES6 would definitely be great to support. It looks like Nashorn engine provides some level of support for it, though I haven't looked closely into it yet. Rhino has definitely been my goto in the past, but it's aging :)

One of my major goals for scripting support is on the fly programmatic manipulation of memory values.
For example I'd like to play with neural networks / reinforcement learning bots, and script would be a great middle layer to pull values from memory to feed to the AI.
Things like debugging plugins or game trainers would be relatively simple to write too, if I make the API good (and fast) enough :)
Re: Any-Yes: A Cycle-Accurate Java NES emulator
by on (#190901)
Also, do you implement support for NES 2.0 headers? If not, then you should. (Note: My recommendation is to never perform hash checks if the NES 2.0 header is present. If you perform hash checks at all, do them only if the NES 2.0 header is not present.)
Re: Any-Yes: A Cycle-Accurate Java NES emulator
by on (#190952)
I do not actually support 2.0 headers, but I really should add it to my short list.

As for hash checks, I have managed to get this far without game specific code / hacks, although I do need to add some; for instance Romancia and Lagrange Point with their surprise banked CHR-RAM.

When I add iNES 2.0 support, it sounds like a lot of things will be simpler! Thanks for the suggestion!
Re: Any-Yes: A Cycle-Accurate Java NES emulator
by on (#190953)
YoukaiCountry wrote:
for instance Romancia and Lagrange Point with their surprise banked CHR-RAM.
I'm pretty certain that all mappers that support banking and are used with CHR-RAM support banking that CHR-RAM.
Re: Any-Yes: A Cycle-Accurate Java NES emulator
by on (#190955)
[/quote]I'm pretty certain that all mappers that support banking and are used with CHR-RAM support banking that CHR-RAM.[/quote]

I had no idea. It doesn't seem to come up very often so I implemented a per-mapper chr-ram-banking boolean for optimization reasons. Turning it on for mapper 1 doesn't seem to break anything, and now Romancia and Lagrange Point show correctly. Thanks!
Re: Any-Yes: A Cycle-Accurate Java NES emulator
by on (#190957)
Yeah, very few games take advantage of it, since at the time cost reasons meant that almost all games only used 8 KiB of CHR RAM, and it's no-where near as obviously useful when you've only got 8 source banks.

But if you're only using 8x8 sprites, you're only going to be able to draw 320 different tiles in given refresh anyway, so very very careful arrangement of tiles leaves you with 3 banks for e.g. bankswitching animation.
Re: Any-Yes: A Cycle-Accurate Java NES emulator
by on (#190958)
Very few games banked CHR RAM around, but a few did. Lagrange Point did it. Rare's pinball games (TQROM) banked between CHR-RAM and CHR-ROM.
Re: Any-Yes: A Cycle-Accurate Java NES emulator
by on (#190959)
Some emulators kinda hard-code which mappers can have CHR-RAM or not, though I have no idea why. It's specified in the header, and CHR-RAM and CHR-ROM should be interchangeable, functionally. Boards with more than 8k of CHR-RAM are a problem, of course (without iNES 2 headers) so for those rare cases some sort of hard override was necessary, but otherwise "bankable" and "writeable" should be independent factors, I think?
Re: Any-Yes: A Cycle-Accurate Java NES emulator
by on (#190961)
There's certainly no reason they should be correlated. Just for whatever reason they usually are. Given that all PCBs had to re-route things to connect PPU /WR to get CHR RAM to work anyway, I can only assume either laziness (it is easier to rip up just one trace and route one new trace) or never bothering to reevaluate it.

And certainly a very few games do make use of it.

(edit: I confused myself, nevermind)
Re: Any-Yes: A Cycle-Accurate Java NES emulator
by on (#190964)
The fact that bankable CHR-RAM is so rare is certainly the reason why writeable and bankable become correlated, at least in my case. Romancia and Lagrange Point were just these strange anomalies that I didn't quite know how to handle.
Writing the TQROM mapper forced me to do a total refactor of how CHR was handled, so this revelation that RAM should be able to be banked always was thankfully as easy as switching a boolean.
And now I know better.
Re: Any-Yes: A Cycle-Accurate Java NES emulator
by on (#190972)
That is kinda the reason, I expect: in iNES, 0 in CHR-ROM field implies 8 KiB CHR-RAM, which is not a [usefully] bankable amount.
except MagicFloor's onechip but that's its own superspecial case anyway.
Re: Any-Yes: A Cycle-Accurate Java NES emulator
by on (#190973)
Yeah, there are a lot of different factors that make it seem (wrongfully) obvious that banking CHR-RAM just isn't done. Maybe some of the documentation needs to be updated making the whole issue a lot clearer.
Re: Any-Yes: A Cycle-Accurate Java NES emulator
by on (#190981)
Myask wrote:
That is kinda the reason, I expect: in iNES, 0 in CHR-ROM field implies 8 KiB CHR-RAM, which is not a [usefully] bankable amount.

It might be useful with MMC3 if the game uses 4K for four 1K frames of a background CHR animation loop and the other 4K for sprite and background tiles, 3K of which are shared ($0000-$0BFF mirrors $1000-$1BFF).

  • C bit set in $8000
  • Reg 0 (PPU $1000-$17FF): 4 (CHR RAM $1000-$17FF)
  • Reg 1 (PPU $1800-$1FFF): 6 (CHR RAM $1800-$1FFF)
  • Reg 2 (PPU $0000-$03FF): 4 (CHR RAM $1000-$13FF)
  • Reg 3 (PPU $0400-$07FF): 5 (CHR RAM $1400-$17FF)
  • Reg 4 (PPU $0800-$0BFF): 6 (CHR RAM $1800-$1BFF)
  • Reg 5 (PPU $0C00-$0FFF): 0, 1, 2, or 3 (CHR RAM $0000, $0400, $0800, or $0C00)

With there being only five licensed MMC3 + 8K CHR RAM games, only three outside Japan, I doubt that any game actually uses this layout.
Re: Any-Yes: A Cycle-Accurate Java NES emulator
by on (#190986)
It's true that most CHR-RAM​ games just used 8KB of it linearly, but the programmers were aware that it could be banked differently since the games do include code to initialize the CHR-RAM banks to the "default" setting.
Re: Any-Yes: A Cycle-Accurate Java NES emulator
by on (#191036)
Version 0.1.2 posted. This one adds 3 new mappers (18, 79, 185), fixes save state issues for mapper 16 and 19, grants more CPU threads to NTSC simulation, adds proper CHR-RAM banking, and has libGDX and lwjgl updates.

Download Any-Yes 0.1.2
Re: Any-Yes: A Cycle-Accurate Java NES emulator
by on (#191300)
Download Any-Yes 0.1.3

This one adds support for Lagrange Point, including sound! This is pretty exciting for me, as Lagrange Point is one of my favorite NES RPGs, and one of the most advanced games on the system.
Also in the sound department it adds some basic low and high pass filtering to the samples.

There was also a pretty major performance increase. I was generating a lot of Java garbage collection due to the way I was passing audio and video to my processing threads. I'm happy to report it is no longer an issue.

Besides those enhancements, this version comes with a lot of minor improvements including fixing a key mapping bug, some minor timing fixes, and better handling of audio and video thread cleanup.
Re: Any-Yes: A Cycle-Accurate Java NES emulator
by on (#192319)
Download Any-Yes 0.2.0

This version adds a much more conventional menu-based UI. It also allows command line rom launching (just pass the path to a rom as an argument).

There was another round of optimizations, and rendering in particular is significantly faster.

NMI triggering was rewritten, which fixes Bases Loaded II, and allowed me to remove a hack I had added to prevent Battletoads crashes.