First of all, I want to thank everyone who worked on the SPC7110. It's the unexpected success of that thread that led me to post this one.
The SNES was a classic console, but an underpowered one that featured an unusually large amount of coprocessors. I hope that one day all licensed games from this system are historically understood and playable without issue in emulators. There are still a few obscure SNES coprocessors that have not been entirely reverse engineered, the DSP3, DSP4, and some Seta chips. They're not too glamorous, only a single game uses each one.
Overload was mostly who worked on these back in the day. Not sure what motivated him or if it was his style to work with others, but developments were fairly rapid until ending abruptly.
http://users.tpg.com.au/advlink/dsp/
Andreas Naive wrote:
I love to reverse engineer stuff, and most of my hobby time goes to such activities... the challenge itself that makes it fun.
Neviksti said something similar, and it's obvious to me that this is the only kind of attitude that could possibly choose to work on these chips. Because the work involved just doesn't promise much in terms of gameplay. So yeah, if anyone is looking to gang up on something again, I recommend one of these.
I like DSP3 cause I'm a fan of the SD Gundam games on SNES. No idea if ZSNES emulates it very accurately but ZSNES does have some support for it.
It does, but there are still problems with it as "Gundam GX still buys the farm despite most of the registers being bit perfect," to quote FitzRoy. Is it known what could be causing these problems?
Ver Greeneyes wrote:
It does, but there are still problems with it as "Gundam GX still buys the farm despite most of the registers being bit perfect," to quote FitzRoy. Is it known what could be causing these problems?
How is the "Gundam GX" now? are they still buys the farm??
_________________
aluminum plate
Uh, SD Gundam GX is SA-1, not DSP3. SD Gundam X is DSP3.
I helped with some of the DSP-1 reverse engineering, and briefly with DSP-4 decoding. It was fun figuring out a bunch of the details, but that fun quickly wears off for some of the truely complicated calculations .. the reason is that these are (usually) lossy calculations, so even after you make the break-through of realizing what the calculation is, it takes forever to figure out the precise software algorithm they chose (silly things like order of operation suddenly matter if you want bit perfect responses).
That being said. The processor is known for the DSP-# chips. If we could figure out how to stain NAND rom, we could just read out the data directly. I always thought that would be fun.
Another option, is some dopant profiling technique. Many large universities have a facility that users can pay for equipment training and time. Would anyone be interested in learning how to use one of those machines? It could be a really fun learning experience, and we'd all get something productive out of it as well.
I've used an AFM setup before. They are usually quite user friendly and easy learn. One technique called "scanning capacitance microscopy" allows for, effectively, taking a 2-D image of dopant levels on the surface. You'll see the data pop right out on the screen!
I'd be willing to chip in some money for that.
FitzRoy, do you live near any large universities?
Anyone else?
Also, if someone knows (or can ask around for) people that could explain how to stain the ROM, that would work too. (I can stain NOR rom, and have successfully read out the data on a chip this way before, but I have never been able to stain NAND rom ... I'm not sure why. I even bought a silicon processing book to read up on it, but I clearly need help from someone knowledgeable about possible problems/pitfalls as I can't get it to work myself.)
----------------
Oh, and for those wanting to do software reverse engineering, to answer Fitzroy's implied question of "what gets these things rolling?" ... the answer is usually data. SDD1 cracking didn't get rolling until TheDumper built a hardware setup, and released data and custom tests. DSP-1 really got rolling when Overload supplied lots of data for everyone to play with. SPC7110 wouldn't have been cracked unless someone built a way to access the real hardware (in this case, me) so we could get our hands on custom data sets. And so on and so forth.
I don't have a cartridge with a DSP-3. A hardware setup doesn't even need to be built for DSP tests, we just need a good SNES copier with an external cartridge slot. (I believe Overload used a SWC DX2. I found that a GameDoctor SF7 can't respond fast enough, but an SF3 _almost_ responds quick enough (I can play MarioKart in the external slot, but there are occasional graphics glitches). Caitsith2 got his to work reliably after some good cleaning ... although this is a timing issue, so it may be just that his responds slightly faster).
Anyway, the point is, the hardware barrier for this one is not very large. If someone starts running tests, and sharing data along with some summaries to attract interest ... you'll probably have people fiddling with the fun puzzle on their free time.
The $1E command is still not bit perfect to Overload's DSP3 log. Some commands that are used during battles don't show up in the log. I fixed $1E in the current ZSNES tree, so it doesn't crash when the computer does a turn but it's still not bit perfect. I uploaded the current dsp3 source code to rapidshare.
Download it here:
http://rapidshare.com/files/216279520/dsp3.zip.html
If anyone ever bothers to look into the Seta DSP like ST010 (ie Exhaust Heat 2) I have a development cart for this and one for GSU1A (Power Slide SFX). They are real socketed development boards so if anyone wants to put something on EEPROM and test it, let me know.
jbo_85 wrote:
The $1E command is still not bit perfect to Overload's DSP3 log. Some commands that are used during battles don't show up in the log. I fixed $1E in the current ZSNES tree, so it doesn't crash when the computer does a turn but it's still not bit perfect. I uploaded the current dsp3 source code to rapidshare.
Wow, that's really amazing! Thanks a million for doing this.
I don't suppose you'd mind if I used your fixes eventually? I'll wait for ZSNES v2.0 before-hand, as you guys certainly deserve the full credit for playable DSP-3.
If you decide to research this further (not asking you to if you don't want to), do you have access to a copier to test on hardware? If not let me know, and I'll see what I can arrange.
MatthewCallis wrote:
If anyone ever bothers to look into the Seta DSP like ST010 (ie Exhaust Heat 2) I have a development cart for this and one for GSU1A (Power Slide SFX). They are real socketed development boards so if anyone wants to put something on EEPROM and test it, let me know.
Thanks, I just got your e-mail.
The ST010 should be easy enough with a copier to run our own tests. The SFX chip could be interesting though.
The SFX and SA-1 allow execution from the cart-side (ROM+RAM), and we have no easy way to time code running from ROM without hardware mods. But I believe we can infer most ROM timings from observing how things work with RAM, so I* should be okay hopefully.
(* will let you know if not.)
You certainly find some neat hardware though :D
Awesome with sockets, the only kind of IC I can replace, heh.
byuu wrote:
jbo_85 wrote:
The $1E command is still not bit perfect to Overload's DSP3 log. Some commands that are used during battles don't show up in the log. I fixed $1E in the current ZSNES tree, so it doesn't crash when the computer does a turn but it's still not bit perfect. I uploaded the current dsp3 source code to rapidshare.
Wow, that's really amazing! Thanks a million for doing this.
I don't suppose you'd mind if I used your fixes eventually? I'll wait for ZSNES v2.0 before-hand, as you guys certainly deserve the full credit for playable DSP-3.
If you decide to research this further (not asking you to if you don't want to), do you have access to a copier to test on hardware? If not let me know, and I'll see what I can arrange.
I don't mind if you use the fixes. I don't have a copier but it would certainly be nice to get some battle logs to get the remaining commands implemented so the game doesn't crash in a battle. I don't want to get $1E bit perfect right now. You can get less errors by changing the end of DSP3_OP1E_B to
Code:
int i=16;
while(i--) {
op1e_x = op3e_x;
op1e_y = op3e_y;
op1e_lcv_radius = 1;
op1e_search = 0;
DSP3_OP1E_B1();
}
SetDSP3 = &DSP3_OP1E_C;
}
The game appears to be using a non-optimal algorithm to get the smallest weights for the cells. We would need some special data sets to get this bit perfect.
MottZilla wrote:
Uh, SD Gundam GX is SA-1, not DSP3. SD Gundam X is DSP3.
Uh, no. I have an SD Gundam GX cart right here next to me (scored it for $20 from a japanese import store at vgxpo '07), and it most definitely has a DSP-3 in it.
Chip is marked:
Nintendo
DSP3
013
9423A3001
If anyone wants to make a rom which runs on the GDSF7 and sends commands to and logs data from the cart, I can run tests like that. Be warned: the top-cart-port dsp-x access on the gdsf7 has 'issues'; mario kart for instance will run for a bit, go all unstable/wacky and unplayable after a while when running the code from copier and dsp-1 from a cart stuck on top (it works fine with the "blue passthrough" between gdsf7 and snes, which has a die-cloned dsp1b on it); similarly, if i read sd gundam gx into the copier and stick the cart in the top slot, it runs for a bit but gfx end up getting corrupt and the game crashes during the setup phase.
There may be some hardware mod i need to do to the gdsf7 (perhaps reconnect a disconnectted clock line somewhere?) but I don't know where/how to do that.
LN
I appreciate the continued interest in this. I wouldn't know the first thing about where or how to stain chips. It seems that more RE work could be sufficient before resorting to that. These will eventually become the most important things yet done, but they're not today.
Heh, I don't know whether it will be a positive or negative thing when getting Quick-move Shogi Match with Nidan Rank-holder Morita 2 perfected is the most pressing concern in SNES emulation.
Still, that only two games hold us back from full 100% commercial software compatibility is quite annoying.
Lord Nightmare wrote:
MottZilla wrote:
Uh, SD Gundam GX is SA-1, not DSP3. SD Gundam X is DSP3.
Uh, no. I have an SD Gundam GX cart right here next to me (scored it for $20 from a japanese import store at vgxpo '07), and it most definitely has a DSP-3 in it.
Sorry about that. I got that mixed up. SD Gundam X is regular, SD Gundam GX is DSP3, SD Gundam G-Next is SA-1 I think. Easy to get confused when they aren't really named very imaginatively.
DSP-3 OP1E:
I know I've heard awhile back that it uses the SD Gundam X path-finding algorithm (exactly) to bit-match Overload's logs.
In case anyone missed it, the latest bsnes version has added limited support for the ST011 and ST018 chips. If anyone wants to look at the progress and possibly help improve them, take a look at the source.
The ST011 code in ZSNES is bit-perfect for the demo. We either need more logs to get it bit-perfect or someone who is experienced with writing a nice shogi AI to get it somewhat playable.
The ST018 code I added was very preliminary and really does nothing more than get past the first few lock-up points. I mostly just wanted to see why everyone kept calling it a RISC processor. It may be internally, but it's irrelevant since you can't upload your own instructions. Well, at least until someone dumps the program ROM from it.
The way it works is to upload the board state in a 96(?)-byte structure, and then send a command and get back the results. The command requests are usually things like "will this move result in checkmate?", "has player 1 or 2 won?", etc. Most of it we can emulate with generic Shogi algorithms, but there are a few that control where the computer AI moves that will require chip logs.
As for ST011, I guess if I can get my hands on the cart, I can use blargg's serial interface to dump some logs. Seems like a lot of work, but if someone is willing to RE them, I suppose getting the logs is the least I can do.
For both the ST011 and ST018, I don't have the hangups that others in the past have had with using a generic Shogi algorithm as a placeholder to better understand the chip. It's definitely simulation in place of emulation, though.
And proper OP1E support for SD Gundam GX should at least get the game fully playable, right? If someone needs help converting the Gundam X routines from 65816 to C, I can lend a hand there.
byuu wrote:
And proper OP1E support for SD Gundam GX should at least get the game fully playable, right? If someone needs help converting the Gundam X routines from 65816 to C, I can lend a hand there.
We need more OPs, that aren't used in the available logs, to get the battles right. Most of the failures with the current OP1E algo are minor.
For what it's worth, the MESS team and I have raised money and obtained a cart to have the DSP-1B decapped and scanned. If we are able to emulate the NEC uPD77C25 and get the program ROM, then we can decap the DSP-2, DSP-3 and DSP-4 and get instant bit-perfect, timing-perfect emulation of said chips.
The same should be possible for ST-010+ST-011, ST-018 and Cx4. But we don't know which three CPUs are used, and they are most likely custom, eg undocumented. Meaning it may even be harder to emulate the PROMs than it is to simulate the opcodes.
And for those who enjoy the simulation approach, having the PROMs should allow us to RE the algorithms much easier.
I have some side projects that I may be willing to pay for decap and rom reading. How much and where are you going for such services? I never had much luck looking previously.
TheGuru is secretive about his deal (or at least was last time I tried to ask him about it), so I'm not sure what all we can expect. (Unlike several images of metal layer masked rom he's shown, the DSP1 ROM cannot be read by merely decapping and imaging ... either staining or probing is necessary.) However I see some things on his successful list that must have been probed or something, so who knows. $200 is ridiculously cheap for such work, so there must be some heck of a deal.
I've talked with Chris at flylogic.net and exchanged some stuff with him before. Unfortunately, if he isn't interested in the project personally (ie doing it for free or pitance), I would never be able to afford his rates. He's quite amazing at what he does.
Did you read the links I posted? The questions are answered about the nature of the deal. Even probing equipment being used is mentioned.
http://decap.mameworld.info/about-2/
neviksti, could you ask Chris what his rates would be for taking on a MaskROM like the DSP-n series? It may be a good idea to have him take a shot at this as well, perhaps with the DSP-1A or somesuch. I've heard estimates that MaskROMs typically cost ~$350-500 on the open market, which also seems rather low.
HJRodrigo wrote:
Did you read the links I posted? The questions are answered about the nature of the deal. Even probing equipment being used is mentioned.
http://decap.mameworld.info/about-2/No, I'm sorry I didn't.
I saw mameworld and thought the decap page you listed was just this page:
http://guru.mameworld.info/decap/index.htmlThank you for pointing out your link to me (again).
Anyway, it is nice to know they are not as secretive about it anymore. The remaining secrecy probably stems almost more from pricing issues (in his real job) than security issues. The page doesn't go into technique details, but it is clear probing is used when necessary.
byuu wrote:
neviksti, could you ask Chris what his rates would be for taking on a MaskROM like the DSP-n series? It may be a good idea to have him take a shot at this as well, perhaps with the DSP-1A or somesuch. I've heard estimates that MaskROMs typically cost ~$350-500 on the open market, which also seems rather low.
The prices I told you about earlier, one data point was from prices he told me he charged. I was asking about a more recent chip though. Um, I should probably say the rest in a PM.
The MaskROM estimate you write is echoed on the page HJRodrigo linked. I'm not sure where that is coming from. Previously with some searching I remember finding it was possible to get images at roughly that cost, so they must be referring to the ROM made by changing the top metal interconnect layer mask. In older devices, everything was large enough that you could easily read out the data after decapping such devices (you could probably even do the whole thing with just sandpaper, and a geological microscope from Toys R Us).
But many kinds of maskrom cannot be read in an image (and even the metal layers ones wouldn't be readible if there were additional metal layers above). So newer devices may require additional processing and/or techniques. Also newer devices just have a lot more memory to read out, which costs more. Some devices I've looked at have 4 metal layers, which obscures much of the lower circuitry (sometimes there is a layer purposely for obscuring). So just in general, the newer the device the more expensive.
I was very discouraged by the prices I was getting from asking around, so I by no means exhaustively searched for places that do such work. Plus that was at least a couple years ago, so maybe prices have dropped as well.
And also:
http://board.byuu.org/viewtopic.php?f=3&t=1263
all 3 within a 12 hour period, too!
LN