Current state of programmable logic

I've started this thread to answer questions brought about programmable logic development software brought up here.


Your results may vary with the software, but here's a basic breakdown of my experiences with the softwares and current devices being offered.


XILINX

Xilinx ISE: Very stable, I've never had it fail to work as expected in synthesis without giving a meaningful error/warning. Editing pinouts via .ucf text file is very easy and pain free. IMO ISE is great especially if you're learning to use CPLDs, but unfortunately Xilinx no longer has 5v tolerant CPLDs recommended for new designs. ISE is the only software that supports legacy devices such as the powerpak's FPGA.

Xilinx Vivado: I tried it when it first came out a few years back and ISE recommended I migrate from ISE. I didn't feel like going through the annoyance of translating 30 some projects over especially because .ucf files didn't seem to be supported for pinout assignments. If you're starting fresh, hopefully vivado is just as good as ISE if not better.

Xilinx 5v tolerant devices:
3 years ago they announced EOL on the xc9500xl series & discontinued PLCC packages. VQFP packages are still available, but the prices were raised with the EOL announcement and their days are numbered. I wouldn't be surprised to see the entire xc9500xl line get the axe this year, but who knows..

Xilinx 3v only devices:
Coolrunner and older Spartan series are in the same boat as xc9500xl (see discontinuation notice). It doesn't matter much though, of the older Spartan 2 & 3 devices still in production for the time being, they're expensive starting at $8-10 with 400-1.5k logic elements on digikey.

The smallest Xilinx devices recommended for new designs looks to be the Spartan-6 series XC6SLX4. They offer ~4k LE (logic elements) and start around $10 with online pricing. This is beyond the scale of most NES mapper interest, perhaps it's not a bad option if you're looking for MMC5 scale capability and beyond.

Xilinx simply doesn't offer much in our target market of 32-1k macrocell/LE anymore, for this reason they're mostly dead to me..


ALTERA

Altera Quartus II & Prime: Which one you use depends on your family of CPLD. The max 10 uses Prime, and I believe max 5 uses QII. I have limited experience with these, what few projects I've used them for I've had no complaints. The software was stable and seemed on par with Xilinx ISE. From what I hear from peers and my limited experience I would recommend Altera for someone who's learning logic design.

Altera devices:
At the end of 2016, Altera announced EOL of their only remaining 5v tolerant family EPM3000. So they're very much in the same boat with XC9500XL series. But there's hope for Altera's 3v only devices.

Altera MAX V:
Their smallest current series is max 5 which is 3v only and comes in 32-1.7k 'equivalent macrocells'. They're a bit annoying for two reasons though. They all require 1.8v core supply, so you'll likely need both a 1.8v and 3.3v regulator on board. The smallest device 5M40 only comes in MBGA & EQFP (with 0.4mm pitch) packages. 0.4mm EQFP is better than it only being offered in MBGA, but that extra fine pitch on EQFP-64 packages. These facts paired with requirement for level shifters kinda makes the 5M40 unreasonable for multi-chip discrete to MMC1 scale mappers.

The 5M80/160/240/570 are offered in 0.5mm EQFP-100 packages though. The 5M80 should fit a full fledged MMC1 with room to spare for extras and costs under $2 online. The 5M160 wouldn't fit a full MMC3, but would fit a simplified version with CHR registers trimmed to 32KB CHR-RAM. The 5M240 would be required for full fledged MMC3. Online pricing for the 5M160/240 is $3-5, but I can say volume quoted pricing is pretty competitive to Lattice's offerings in this range.

Altera MAX 10:
From what I understand these are Cyclone FPGAs with flash that configures the logic at power up effectively making it comparable to a CPLD. They start at 2k logic elements and are offered in 0.5mm EQFP-144 and BGA packages. They're comparable to Xilinx's spartan6 series from what I can tell in logic density and online pricing. Unless you can get preferred pricing with a sizable EAU this device is probably out of reach though. It does have the benefit of only requiring a 3.3v supply though.


LATTICE

Lever classic:
This is the only software option for LC4000 series devices. I've been using the software routinely since 2015 when I migrated from Xilinx xc9500xl devices. From my experience it's terribly unstable, I've used it quite a bit and it crashes frequently. I think it's also safe to say that it's effectively no longer being maintained by lattice. If you find an issue it's unlikely it'll ever get repaired, your only option is to work around it.

Typically when it crashes you only need to restart the program, but this still happens frequently enough to where I find it unacceptable. There are some gotchas to watch out for as well. If you want to have the software create svf files you must select that option buried in the alternative installation options. If you forget to do that, you have to uninstall and reinstall all over again. Additionally it simply cannot handle spaces in your installation nor project path. Make sure you also don't save the project in something like a Dropbox folder. I forget if it just crashes, or won't allow your project to be saved. The error isn't meaningful for project path problems and can be very frustrating when getting started if you don't know to watch out for this.

For me, the inability to use cloud saves is terribly inconvenient when all other software supports them. Sure there are other backup options and I now use them, but they're inconvenient at best.

On windows you have to be very careful with your PC's environment variables especially when installing other make paths. A number of times I've broken leverclassic when setting up some other dev tools, or vice versa. I've never had any issues like this with other programmable logic design software.

The pinout assignment interface is terribly tedious and annoying to work with especially if you need to frequently make changes. I much prefer to have pinout options completed via text file and while I think it may be possible I still haven't managed to make it work so I know resort to the tedious GUI every time.

In the worst case I've had projects that no matter what I do they get in a state where it crashes during synthesis EVERY time. The only solution in this case is for me to create a new project, copy over my source files, redo all the pinout assignments in it's exceedingly tedious interface, and start over. No clue what goes wrong and you can't get an error report from software that crashed to try and find out what you may have done wrong.

Synthesis errors often offer little to no help. When I get in this situation I typically restart the program hoping it doesn't happen again. Or I have to create a fresh project, import source, redo all pin assignments, and now it just compiles no problem.

It's gotten to the point where I'm nearly ready to give up on the LC4032V despite it being the only 5v tolerant 32 macrocell device on the market. At times I've contemplated reverse engineering the configuration bit stream so I can manually design the device configuration. I'm guessing I could get the device to fit more than the software is capable of fitting and it would let me do away with the software's annoyances once and for all.

I've tinkered with the synthesis goals with varying results. Sometimes using 'Lattice LSE' delivers better results than 'synplify' in terms of resource utilization, sometimes 'synplify' is better. Sometimes one crashes and the other doesn't. Having problems and not getting meaningful error messages? Swap to the other synthesis tool and maybe they'll go away, or your design will now work or fit in your device.

Anyway lever classic can be made to work and in my experience if you slap it around enough it'll get the job done. It's been a thorn in my side for going on 3 years now though. If you already have experience with programmable logic design software I expect you'll have better experience. If your a newbie though, I would fear you'll give up on programmable logic all together if lever classic is your first experience.


Lattice Diamond:
This is Lattice's the actively maintained design software which is used for Mach-XO devices and beyond. It's relatively decent and I have a fair amount of experience with it over the past 8 years. I've found a number of problems with it but when I bring them up to Lattice they're quick to deliver a patch. Once it only took a day or two when I pointed out the software gave no means to reclaiming JTAG pins as user i/o as advertised in Mach-XO2 data sheets.

I have had issues which I can't fully blame lattice for, but more meaningful errors or warnings sure would be nice at times. I've mistakenly assigned two different drivers to the same signal and the software happily went along and synthesized the design without errors.

I've had some other issues as well where only one synthesis tool (synplify/latticeLSE) will create a physically working design in practice. But an update that came a year later corrected that issue to where both synthesis tools work. In the end I'll give the software the benefit of the doubt that my verilog was poorly defined. At the time I just didn't want to take the time to rewrite everything in my frustration. I'm not sure what the issue was exactly at the time, but now that I've looked back at my old code I can see room for improvement.

On the whole I'm happy with Lattice Diamond and would recommend it's use. I've never had stability issues, and Lattice is actively maintaining it.

5v tolerant Lattice devices:
Lattice is the only option for a 32-64 macrocell device that's 5v tolerant with the ispMACH LC4000 series which costs $1-3 each. I've been told by Lattice that there is no currently planned EOL. It's 5v tolerance is it's selling point and why they are expecting to manufacture for quite some time.

A MMC1 with 256KB PRG-ROM and 128KB CHR-ROM won't fit in 32 macrocells, but an MMC1 with CHR-RAM will. IMO the LC4032V is the next logical step up from a few discrete logic gates. There are 128-512 macrocell options available which could handle an MMC3/FME7 scale mapper, but their cost no longer makes sense if you can handle migrating to a 3v only device.

In general the LC4000 devices are great because of their 5v tolerance, and in practice I've had no issues with the devices. They're also very quick to program (only a few seconds) in comparison to the xc9500xl series which was exceedingly slow and can take a couple mins to program. Their biggest problem is the lever classic design software as you can't use Lattice diamond for the LC4000 series. If you have success with lever classic that's great, I'm envious, perhaps I'm doing something wrong... Personally I like the LC4032V just enough to put up with lever's software headaches. But it's also the ONLY option for 5v tolerance and 32macrocells with 32 gpio. I would have loved to use the xc9500xl for many more years as I prefer it greatly over the LC4000V, but Xilinx had different plans..

[url]3v only Lattice devices:[/url]
The Mach-XO 256 is the lowest cost device offering here and costs about $2. But it comes with the added cost of level translation which varies depending on how many signals you have to shift. If you can handle the 0.5mm fine pitch TQFP-100 package and level shifting it can actually be more cost effective than the 5v tolerant LC4064V. So in general this is my go to device when more than 32 macrocells is needed. It's actually a FPGA LUT/FF architecture and has roughly equivalent to 256 macrocells. It also has some nifty features not found in CPLDs such as a 18-26Mhz on chip oscillator. That becomes of use for things like getting around the M2-/ROMSEL delay. The 'free' oscillator also allows you to start designing mappers with fully synchronous logic which can avoid many of the pitfalls of asynchronous mapper design which can be problematic if not designed around the intricacies of FPGA architectures.

A full fledged MMC3 uses about 45% of available resources, and FME7 uses about 55%. So it's quite capable and I've yet to implement a mapper which comes close to fully utilizing the device but have plans to do so in the near future.

If you're looking for more than ~256 macrocells there are a number of options offered by Lattice. The best device depends on your i/o, logic density, and cost requirements. There are a number of cost effective offerings in the Mach-XO2, and iCE40 devices if you're looking to avoid BGA devices.


ALTERNATIVES:

Cypress has some PSoC offerings with a programmable logic block attached to an ARM mcu. Last time I researched them the amount of logic provided was rather minimal and the cost was rather high if you gain no value from the mcu. Microchip appears to also be offering small amounts of programmable logic attached to AVR cores. It's nice to see programmable logic becoming available in mcus, we can only hope this continues to grow in the future. As it is today, these aren't viable general use programmable logic solutions like the CPLD/FPGAs mentioned above.

Silego's greenpak devices are rather interesting. They only cost $0.20-0.50 but require volumes of 1-3k effectively. They can be powered from 5v, or 3v while being 5v tolerant. They currently have at most 18io, and 17 flipflops. But they also come with on chip oscillators, delay units, comparators, and state machines, aswell as LDOs in the near future. Unfortunately they currently only come in very small QFN packages, but there is a tssop20 package being released this summer. These are great for replacing a small handful of discrete logic ICs provided you can fit within the limited i/o count. They aren't capable of replicating a MMC1, but a custom mapper could be created with a relatively comparable feature set to MMC1.

Thanks for your write up. This seems to be a great summary of where things are at right now.

One more bonus CPLD:

Atmel (Microchip)'s ATF150x series. 5V natively, but definitely the wrong price for the 64 ($2.5-4/) and 128 ($4-17/ ???) macrocell versions. The 32 macrocell version has gotten cheaper since the last time I looked, now $1.5-2.5/ — which is actually sorta competitive.

The free option for programming for these is an ancient (effectively abandonware) copy of WinCUPL, with its own esoteric programming language. It's not a great programming language, but it works. I found the test framework to be vaguely annoying (in that the work flow was "generate stimulus, simulate, verify output" rather than anything more immediate).

The other option is called ProChip Designer and I decided dealing with licensing wasn't something I wanted to try. But it does actually support Verilog and VHDL workflows.

It feels like it "should" be practical to add support for this in YOSyS—the fusemap is fully documented.

There's a few options in the "even smaller" range, but I'm skeptical that they make sense if a Greenpak doesn't.
ATF750 is approximately two 22V10 PALs in a single package. Given that you can get bigger PLDs for cheaper, certainly not worth it.
ATF16V8 is exactly what it says it is. Pretty cheap. I don't know how you could fit anything useful into something this small.

Yeah, I saw that the the Atmel stuff went down in price not long after Microchip bought them. I don't know what their policy will be for PLDs (or how successful the product line is), but if it's anything like they do for PICs, that could be good news for availability. At my work we still produce some old products with the PIC16C54, and PIC16F84, those are ancient as far as MCUs go.

Silego does have a 32-pin GreenPAK already, 29 I/Os. I'm on their mailing list and they broke the news on that little over a month ago. So they are moving up in I/O count (internal resources are pretty much comparable to the previous parts though).

For those feeling adventurous, there are some Chinese companies making small CPLDs/FPGAs as well. Alta-Gate (AGM), and Anlogic. Neither are 5V tolerant (though the Anlogic datasheets show you how to calculate series resistors for 5V compatibility), and the only distributor I know carrying them (lcsc.com) has amazing quantities such as "less than 10" and "out of stock". I've been looking into those a bit, just out of curiosity.

Wow, WinCUPL, I remember that program. That was my first introduction to PLDs. I was able to create a JED file for a PAL, and load into the simulator in Proteus, which seemed pretty neat to me at the time.

I've used Lever Classic daily at work since 6 months and I have yet to see it crashing, period. I bet the problem is somewhere else on your PC...


For me it wasn't really a problem, there were minor annoyances here and there but nothing preventing me to do my work.


This is however very true, actually there's an option that should be enabled for the synthesis tool to automatically add buffers and most of the errors will go away, but when you come close to maxing the device usage they come back, and are very un-useful error message. Sometimes it even fails without having any displayed error at all. Thanfully you can get arround this by alternating between Symplify and Lattice's synthesis tools. This always saved my day so far, if one is annoying you too much use the other. Most of the time Lattice's is far better, faster and gives less errors, but sometimes it fits the design using more logic elements (but who care if you can afford them).

It has been more than 1 year since I looked at Atmel's stuff, but they seemed the nicest when you need 5V, low-density and easy to use CPLD. I'd forget it if I wanted to do anything advanced, but something like MMC1 or MMC3 it would be good. Note that a full MMC1 fits on 4xPAL22V10 or 2xATFv750 or single ATFv2500

You're welcome! Hope it's of use for others, I had fun writing it.


Ahh yes, I forgot about those. That pretty well summarizes my thoughts and limited things I've heard from peers on them as well. For NES use, a 3v CPLD with 5v tolerance is just as good as a 5v CPLD. Even if you did want 5v outputs the cost of adding 5v output buffers to a 3v CPLD should still be under atmel's cost..


Oh I didn't notice that! Although I think the reason I missed it was I hadn't considered the 4688x because it only had 5 flipflops. That's hardly enough to do anything useful.. Did they announce details of other high io count devices? Either way it's good to see the product line growing.

I also just learned that the SPI enabled greenpaks (46140 & 4662x) also include 8bit DACs. Not sure, but it looks like they may require an output buffer to drive famicom/NES audio though.


That's very interesting, I wasn't aware of these.

That's certainly possible, but I've had issues on a 3-4 different computers over the years so I'd have to be doing the same wrong thing everytime. I've wondered if I would have better luck with the linux version. Either way, I'm glad to hear you've had good experiences with it. Considering everything besides the instability, of all the design tools I've used it's easily the worst IMO. Perhaps I'm just spoiled by knowing how easy things can be, or perhaps I'm simply to impatient and easily frustrated..

I guess a little? The only 5V tolerant 3V CPLD you mentioned was Lattice's LC4k series, and the prices for the LC4032 (≈$1.10c/@100) are 30ish cents cheaper than the ATF1502 (≈$1.40/@100). Add 10c for the 3V regulator and I'd be pretty comfortable calling that 20c difference a wash.

Well, you can use any LUT3 (including the ones inside the ASM/DM cells) as a transparent latch...

Makes me wonder if there's any way to use the ASM in the context of the NES.

Yeah I'd have to agree with you on this. The fact you probably don't need 5v outputs still remains though..


My thought was to help keep i/o count down by having multiple registers at the same address. For example if only CPU D0-4 were connected you could have a 8bit register where each write alternated between updating the upper/lower nibble. Certainly has it's draw backs in terms of 6502 code, but anything to help alleviate the limited i/o could be of interest with the greenpaks.

If you're willing to mix CPLD models on one board, I guess you might be able to work around few flip-flops by having a large-I/O, small-flipflop device do combinational stuff like mirroring control and producing enable signals for the other CPLD or for latches such as 74HC377.

That's certainly within reason depending on the design. Personally the greenpaks are more of interest for supplementing a larger design instead of the greenpak being the only logic on board. The DAC would be a welcome addition to any design with expansion audio. I2C/SPI interface could serve as i/o expansion (and/or parallel interface) for a microcontroller mapper. A few spare i/o could be utilized as a level shifter for a 3v only CPLD/FPGA. Delay units may be useful for delaying M2 to avoid /ROMSEL delay problems. Devices with a LDO could replace the 3.3v regulator on board. Analog comparators could be useful to protect battery backed PRG-RAM from corruption during power up/down. Plus a number of other things I'm sure I haven't thought of.

Maybe not strictly connected with the topic, but when you are choosing the CPLD/FPGA family, please take care about some quirks (for example - the Power-up state of the internal registers).

I was implementing an MMC1 in MAX3000A family chip (EPM3064) to fix existing, non working PCB and lost some hours on really simple thing.


While my implementation worked flawlessly on EPM240 chip, it did not on MAX3000. For example, such statement like below is complely ignored in MAX3000:



So i had to wire up external reset signal (R+C+L) and do it like:

Would MMC1 games that have a stub in all 16K banks writing a reset value with bit 7 true have worked with the MAX3000 even without the external power-on reset?

Very true, the means of designating the power up state of registers varies significantly between tools and devices. Most allow some means to designate it, or should at least power up to known hi/lo state. With that you should be able to adapt the logic to work without external circuitry. Most SRAM based devices also have an internal global power up reset signal that can be used as well.

Yes, that's the fail safe solution.

Chip & Dale 1 was working so I guess this game has the stub in each bank.

Do you know how suitable iCE40 devices are?

In my opinion, they're too expensive per cart.

The iCE40xx384s don't have enough I/O, even though they're otherwise cheap enough and have enough logic inside.

The iCE40xx1K parts are about $4, plus a few dimes for its boot ROM and two regulators. They're not 5V tolerant, but the cost of that translation is probably balanced out by cheaper 3V parallel NOR flash. Might be possible to justify it for MMC5-class stuff, but it's really the wrong place for anything less.

(Note: the $3 iCE40LP1K has way too few pins)

In my original post I didn't elaborate much on higher density devices with 1k+ logic elements. If you have a good use/need for that much logic the iCE40 devices are comparable to Lattice MachXO2 & Altera max10 devices. Price and density wise they're all somewhat comparable, and you can avoid BGA packages with all of them. If you're looking for MMC5 scale hardware the iCE40-HX series would be a good fit as the 100-144pin TQFP packages are offered in 1k-2k LE options.



For a traditional design this is true. But if you want to get fancy and take advantage of the level shifters and actually make them work for you, the 39 i/o may just be enough. I've got a time multiplexed 'dual port' design I'm prototyping on right now that only requires 53 gpio and has full PPU/CPU bus decoding and 1MByte of ROM/RAM. While working on it I've recently realized some other tricks that might be able to be pulled off to get the i/o count down to 33 which would fit within the iCE5LP1K/2K/4K QFN-48 package that provides 39 io.

Sorry, I elided a lot of details. What I should have said was "Be careful that the thing you think is cheap actually has enough pins; there are iCE40LP1Ks that only have 16 pins and 10 I/O.

That said, my caveats about the iCE40xx384s appears to have been wrong; they are now(?) selling iCE40LP384-SG32 (QFN32) for a quite nice price, and it's a little I/O limited (17 to 21). Other iCE40LP384 parts with more I/O are available for not too much more money, but require dealing with BGA (and thus possibly with 4-layer boards)

iCEstorm even recently grew support for the small 384-cell ones.

I've been thinking of a design that would use a couple of small CPLDs like the Atmel ATF1502, each sitting on one of the address buses, with a resistor packs isolating the cartridge buses from the console. Since the address wires from the console become valid early in the cycle, typical cycles could start with the address wires floating, latch them in response to a slightly-delayed M2 signal, and then drive the proper address on the back half of the cycle. I don't think I'd bother connecting the CPU-side CPLD to the data bus; code can use addresses for signalling just as easily as data in most cases, and sometimes even more easily, and the pins could probably be better used for other things.

A key feature I'd include that I'd include, that I haven't seen in other mappers, would be a 74HC373 that would take input from the CPU-side data bus and output onto the PPU-side data bus. This would allow data to be copied into PPU-side cartridge RAM in response to CPU-side *reads*, at any time when the PPU isn't actively displaying picture content, thus doubling the speed of transferring data to PPU-side RAM. It might even be possible to include a mode that, when enabled, would cause the first half of each CPU cycle to use one address, latch data for the PPU data mid-cycle, while the second half (which would be seen by the CPU) used a slightly different address, thus allowing data transfer at a rate of one byte/cycle. The late-arriving /A15 would be a nuisance, but such a design could probably make something like the wire-frame graphics in Elite practical even on NTSC consoles with short VBLANK.

The PPU drives the AD0..7 pins continuously at every single moment that it is not itself asserting /RD. See Visual2C02 transistors t16479, t16501, t16516, t16511, and t16534.

I'd been planning on separating the cart-side address bus from the console via resistors; it sounds like i'd need to do likewise with the PPU data bus, but that shouldn't be a problem.

Continuing my last point: if a cart includes resistors (maybe 2.2Kish) separating the address and data buses, do you see any problem with it "doing its own thing" independent of the PPU any time the PPU doesn't need to fetch meaningful data [e.g. because background rendering is disabled and there are no sprites at the current screen location]? While a true dual-port memory system would probably be awkward to implement, increasing the length of time that the main CPU would have to draw the screen when games don't need to show stuff at the top of the frame would be a big win, as would increasing the rate of data transfer. Using a resistors as a crude sort of "multiplexer" is something I did in my Atari 2600 super-banking cartridge to save macrocells. While that was only 1.19Mhz, I think the same principle should probably be usable here. Reliable operation might require an active bus transceiver on the data pins to ensure the cartridge can feed data back to the console fast enough, but I still think the approach would open up a world of possibilities.

Also, what do you think of the idea of having a cartridge include a mode where some addresses 0x6000-0x7FFF would access a banked region of RAM while M2 is low and then a different region (perhaps set all upper address bits to ones) once M2 goes high? I think the effect would be that a mixture of 2-3 byte instructions would execute code located in the top part of memory while the corresponding region below was copied to PPU memory. Unlike OAMDMA, this could happen concurrently with digitized audio playback DMA or even--if the affected address range stopped short of $7FFA--an audio playback IRQ.

If it got garbage during unused name and pattern table fetches? No, I can't see why that wouldn't work.

I think the only tricksy part is making sure that any design doesn't violate the setup time for the PPU. Different parts of the PPU fetch cadence go to different places in the die, so it's not always obvious. Name table and sprite pattern fetches are synchronized through a transmission gate that opens halfway through /RD being asserted, and closes when /RD is deasserted. Attribute table and background pattern fetches use more complex logic but I'm pretty certain they ultimately use the same timing.

I vaguely remember there being a lot of bus capacitance here. When we were doing the initial research on the VRC6's extended nametable modes, I remember asking BootGod to solder eight resistors (10k) in lieu of the ROM and NTRAM, from (VRC6 CHR ROM A10..A15 and VRC6 CHR ROM /CE and VRC6 NTRAM /CE) to (PPU D0..D7), and he only got garbage out.

What would be around to use the RAM while M2 is low?

Sounds like a bus driver would probably be required then, which could be enabled using PPU /RD. If software uses the "outside" '373 for writing data, there would be no need for the cart to care about anything PPU /WR or anything the PPU does with the data bus.



The '373 (or--thinking about it--maybe a '374) feeding data to the PPU-side memory. A typical usage scenario would be something like:


The CPU would think it was fetching 127 "CMP #$C9" instructions followed by "RTS" and a dummy byte from addresses $6n00-$6nFF, but it would actually see byte values fetched from the last 256 bytes of RAM. Meanwhile, the 256 bytes banked at $6n00-$6nFF would be sent to the PPU-side memory.

Oh, the ZX80 trick. Read the byte in the first half, replace it with something else (a 1-cycle NOP there, a NOP-slide here) as long as it's strictly in-order and no bytes are refetched... I wonder if there is any actually useful computation that would have PC doing that.

What's the receiving side look like?

If there's no need for the main CPU to signal the PPU-side CPLD during rendering, one could have two control wires between the two CPLDs, called "/control" and "/wppu", the latter of which could also be tied to latch enable (active high) and output enable (active low) on the '373 connecting the buses as well as /WE on the PPU RAM. The "/control" input would act as an active-low output enable and active-high latch enable for the address pins on the CPLD(*), so while it's high the PPU would have control of the bus in normal fashion. When /control goes low, the CPLD would latch the current address into a counter and output that counter on the bus until /control goes high again. Each rising edge of /wppu would increment the counter.

(*) The latch enable could be implemented using asynchronous set/asynchronous reset product terms on a register that would otherwise respond to rising clock edges. Banking control options could be controlled when rendering is disabled by reading addresses in the 3xxx range.

It sounds interesting, if it can work. What concerns me is that I don't think you can safely rely on the address lines being stable early, from discussions I've had with kevtris (while he was designing the Hi-Def NES), the address lines and M2 are going to be problematic if you want to be compatible with all consoles. Just be aware, if something tricky like this works on the G-revision CPU, doesn't mean it will work just as well on all the other revisions. It had been a while, but I had seen some of kevtris' logic captures for comparison, I seem to remember especially M2 being different-looking on the older revision CPUs. Also, the M2 cycles during operations like OAM DMA and DPCM playback might be really different from a "normal" cycle. It's enough to cause bus conflicts when the PRG bus is outputting fast enough. I'd seen OAM DMA get corrupted when using a ROM emulator, and the earliest version of the NES PowerPak had a similar problem. Both were solved by putting (IIRC) 300 ohm resistors in series with the data bus.

Once I have a testbed for it, one thing I'd like to try is using OAM DMA to copy to multiplexed CHR-RAM. 2 cycles per byte that way, not too bad. I hadn't thought of that before seeing this thread, but maybe one could take over both read/write cycles of the DMA?

The Game Genie is also doing some multi-operation-per-cycle type stuff. Because it does that data comparison, it must be outputting data from the cartridge, doing the comparison which if it matches, shuts off the cartridge bus and outputs it's own data. Not really any relation to this, other than being unusual things on the CPU bus..

I would expect that delaying M2 slightly would be a good idea so as to ensure that /ROMEN becomes available before the the CPLD decides what to do with addresses. On the Atari 2600, the address bus gets driven early in the cycle, but I could imagine that OAMDMA could be problematical. If so, however, that may simply mean that one can't use DMA audio playback during data transfers. My idea of being able to use audio IRQs probably wouldn't work without extra logic in the CPLD because IRQ handling would start with two ignored code fetches.

A two-cycle-per-byte approach would be to add logic to the CPLD so that the first access in the zone will use a normally-banked address while latching data, and the next will use a modified address but not latch. From outside the zone, load the first byte of the zone and then jump there. The CPU would then fetch the first byte, ignore the second (allowing it to be used to supply PPU data), re-fetch the second as an opcode, ignore the third, etc. If a single-cycle-per-byte approach couldn't work, the two-cycle approach could probably be used instead, but I think the single-cycle approach would probably be simpler and better.

BTW, audio IRQs with the single-cycle approach could probably be accommodated by having the CPU-side CPLD keep track of whether the last byte came from an even or odd address and only pay attention to accesses of the other kind. If DMA addresses are resolved late, they may still porse unsolvable problems, but fast graphics would be cool even if they couldn't coexist with audio playback.

Can you point me to any timing measurements or other precise specifications about the sub-cycle behavior of the CPU and PPU?

One possibility is OAM DMA twice during vblank: once with the data to send to the pseudo-dual-port video memory on the cartridge, and again with the actual data to send to OAM. This way, CPU A7-A0 could act as the counter, and the cart would snoop the first DMA to transfer one byte per two cycles.

I'm not sure I really see the advantage here unless the timing of OAMDMA is more favorable (I see no reason to expect it to be, but without seeing scope traces I can't tell). During the second cycle of each transfer the main bus is going to hold an address that's useless to the cartridge, so the cart would have to latch the LSB of the first-cycle address in order to do anything useful in the second cycle. A key aspect of my intended approach is that the console would drive the low part of the address bus so the cart wouldn't have to output it at all.

For compatibility with the widest range of consoles, I would think it would probably best to ignore OAMDMA except for ensuring that the cart is fast enough to process those cycles "normally".

Hijacking OAMDMA is guaranteed to safely handle any analog timing issues, because it's only running at the same data rate as the CPU, not twice the rate as the CPU.

I don't know of any useful repository of analog timing measurements. I've recorded a very short list of random things: PPU A12 vs MMC3 IRQ and M2 vs Mclk.

As far as digital bits:
The 2A03 runs off two synchronous biphase clocks; the standard 6502 50% φ1/φ2 for everything internal (visual2a03 node "clk0"), and a 15/24 (revision E and newer) duty M2 for everything external (visual2a03 node 11200). We've seen several bugs come from M2 being asserted before φ2 on write cycles.

We used to think that the original letterless 2A03 had a 18/24 duty cycle, but looking more closely at the die shots it looks like it might be 17/24 instead.

The 2C02 runs off a single 50% biphase clock at the pixel clock, which I've personally taken to calling "left half dot" and "right half dot"; Visual2C02 calls it "pclk0" and "pclk1". Normal PPU fetch cadence during rendering is {ALE idle AssertRD doRead}.

I don't think we know anything beyond the divider ratios for the 2A07 (16 master clock cycles/instruction cycle) and 2C07 (5 master clock cycles/pixel).

I was under the impression that the weird controller-port interactions with DMA were caused by the fact that DMA cycles didn't use the same timing as normal CPU accesses.

There are byte-stuffing approaches that run at one byte of data per two CPU cycles (make opcode fetches get NOPs, and use the dummy operand fetch for data transfer using the CPU address), which would allow the low-order address bits to be used by the memory directly. I don't see any advantage of OAM-DMA over such approaches sufficient to justify the extra address latching that would be required to do anything useful during the second cycle. If one wanted to use OAM-DMA to feed data that would be meaningless to the OAM and then later send meaningful data, that would be possible, but one would lose the ability to extend the "useful" part of vblank by disabling background rendering and ensuring that no sprites are too high on the screen.

Only when DMA preempts normal execution, for the DPCM fetch. OAMDMA is triggered directly by the CPU itself, so its timing is guaranteed safe. (it works extremely similar to 2600 WSYNC)

Writing to $2004 after rendering has started seems to have only temporary (within those scanlines) effects, no changes to the contents of primary OAM DRAM.

Using OAM-DMA to fetch data that isn't meaningful to the OAM would corrupt the OAM if done at any time when the system isn't rendering the frame. If 1- or 2-cycle-per-byte CPU-based transfer methods don't work, OAM-DMA might be usable as a substitute, but relying upon the OAM to ignore transfers would seem hokey unless there was no other way to get good transfer speeds. If one wants a 20-line top border, having to wait until the end of vblank before starting a transfer would forfeit about half of one's potential update window.