Why do existing PC emulators suck so bad?

OK, I'd better back up that statement real quick, or else fanboys will start trolling me.

All the PC emulators I've encountered thus far have some sort of semi-fatal flaw. They either have unnecessary HLE (nearly all of them), they virtualize (VirtualBox, QEMU, Virtual PC, VMWare), they have code that's ugly as all fuck (Bochs), or they only focus on popular shit (DOSBox). In other words, there are currently no good PC emulators, except for (maybe) MESS, and it's still not up to spec entirely (e.g. VGA emulation). I don't know why PC emulation is like this... Another thing that bothers me is that the 8086 microcode is out there in die shots now, but nobody has dared to dump it... I tried, but I couldn't bring myself to finish it. Anyways, I hope that I find answers to these questions soon.

A: Because existing PC emulators satisfy the wants of most users.

Hmm... Good point. Somebody still needs to make everything actually work better, and I guess that has to be me as nobody else who has a remote clue of the PC hardware cares.

more about preserving this particular PC hardware as compared to all the other things they care about in their lives.

What features are you looking for that you can't find?

I am looking for accuracy, and clean code.

Still not sure what you are missing, (Do you feel better about using an emu if you know it is coded properly even if the results are the same as a emu coded in a way you don't approve of ?) but perhaps you'd find this project interesting: http://fake86.rubbermallet.org/

No. Sorry, but it does HLE, and I don't approve of that, unless absolutely necessary.

... So what, you want someone to take the original 8086, 80286, 80386, EGA, VGA, &c microcode and rebuild you a cycle-accurate PC simulator despite the huge variety of different other hardware (chipsets, sound cards, all their knock-offs...) that needs to be supported? And the code has to be pretty? That makes bsnes look trivial by comparison.

The only problems I've had with DOSbox have been raster effects on pre-VGA machines. Oh, and trying to emulate tran's Luminati and Ambience demos, which require accurate emulation of a multisync monitor and perceptual frame merging. Even MESS's CGA-NTSC composite color emulation is lacking.

For DOSBox, I have no idea by what you mean by "only the popular shit", because it certainly looks to me like they've got perfect compatibility with more than 85% of all games known to them. Is this the equivalent of bitching about how no famicom emulator accurately handles X random Waixing game?

How about starting with just the PC (5150), PC XT (5160), PCjr (4860), and PC AT (5170), with the most common floppy, serial, and parallel adapter cards made by IBM (MDA, CGA, EGA)? Once the 386 and cache come into the picture, cycle accuracy becomes less necessary, as games have to adapt to a multitude of clock speeds and video cards available on the market.

ObNES: For extra geek points, emulate a PC connected to an NES through a CopyNES.

I never said anything about cycle-accuracy.

I think its safe to assume you mean cycle accuracy when you seem to be asking for accurately emulated microcode... ?

How about emulating a PC emulating a Commodore Amiga emulating an Atari ST emulating a NES?

Low level versus high level is a very arbitrary and subjective ...subject. I don't feel you are being specific enough about what it is you are looking for.

This thread has revealed another impediment to PC emulation as compare to game consoles: the multitude of hardware. Harder to focus lots of people on the same goal when there are so many differing configurations interest might be in. Sort of like how consoles are easier to develop for due to the generally singular hardware configuration (tepples will fill us in on deviances from this in the next post).

There are three consoles: PC, XT, and AT. The differences among these are analogous to the differences between a Game Boy and a Game Boy Color. Yet VisualBoyAdvance emulates not only the Game Boy and Game Boy Color but also the Game Boy Advance, a completely different platform.

I want to emulate emulating.

What? Anyways, I do understand that there is a multitude of hardware available, but what about the VGA? I mean, can't people at least get THAT right?

When you say VGA, are you referring to video adapter emulation in general or do you quite literally mean classic 320x200x256 (i.e. segment A000) mode? Or are you talking about VESA?

To me, the funny part is that you're wanting accuracy and clean code -- yet PC architecture in general is far from clean. In fact, it's a complete clusterfuck. Do not even for a minute think that just because we're into the 21st century that somehow PC emulation is "easy" -- it's not. My point is that "PC emulation" -- the term -- is simply too vague. And don't get me started on the utter disgusting mess that is the ""evolution"" (double-double quotes!) of x86.

That said, I have zero complaints about VMware Workstation other than it's fairly slow (it doesn't offer paravirtualisation to the same degree that, say, ESXi does). But I wouldn't use it to play old DOS games, for example.

Honestly I think that most folks who go apeshit over use of HLE are usually wanting to waste CPU time emulating hardware down to the transistor/gate level for little-to-no gain. There are emulators which do this of course, and the system requirements of such emulators are preposterous; the trade off isn't worth it in most cases.

What is incorrect about DOSbox's emulation that is not a raster effect? (Best raster effects "abuse the video card and monitor" I've seen is this demo: Copper by Surprise!, or either of Tran's demos that I previously linked to). If you can't find anything wrong other than raster effects, what games use them?

I loved running Copper on my 386

Look, there are some bugs that even DOSBox has. For example, one the MESS team found in both DOSBox, and MESS, is that 0x3C0 bit 5 on the address portion of the register is not emulated. At all. This made it to where some stuff that would run in DOSBox and MESS, would not work on real hardware (e.g. something like the PS/2 Model 70 with the original IBM VGA).

I think it's hilarious how your list leaves out the one that actually tried to be a console (the PCjr)

The very fact that the MESS team unconvered that inaccuracy surely implies that they do care, and are working on it, and will fix it eventually? If not, why are you spending energy trying to convince us that it matters? Surely that effort would be better spent convincing MESS's &// DOSbox's dev teams instead.

Anyway, is retrodev for the XT a real thing? I guess Trixter wrote Monotone and 8088 Corruption, but beyond his contributions?

What is it that you're trying to do that all existing emulators are failing you for? You've described a few things about how you would like it to be implemented, but what actual function are you missing because of this implementation difference? Is there a particular program you are trying to run that you can't?

No, it is not a particular program. What I am looking for is the ability to have an emulator, test a program on it, and if it works, be sure it works on real hardware.

^ baloney

even if you lived in DOS era and wanted to be sure that the program you made in REAL hardware would behave the same in all configs you NEEDED to test it in many hardware configurations, from different vga cards to different sound cards and stretch as far as to test with different memory managers on the same machines. And then make adjustments on your program to make it compatible with said hardware/software if issues arose or advise how to run your program with that combo.
You obviously missed all that back then and made this boo-hoo thread that borderlines with trolling.

According to #nesdev comments, he's 14. So yeah...missed it by miles and miles. I remember it taking longer to configure a game's settings for your combo than it did to actually complete the game. That was while I was an Amiga die hard, though, so I was mostly standing on the sidelines laughing. Not that Amiga didn't have its own combo problems -- OCS, ECS, AGA, PAL, NTSC, not to mention the Kickstart/Workbench mayhem.

I meant the original Video Graphics Array, introduced in 1987 alongside the Personal System/2. Both the VGA and the MCGA support the mode 13h to which you refer, but only VGA supports tweaked "mode X" variants of this mode.


I agree with you that especially on battery-constrained hardware, emulators with anal-retentive accuracy provide little benefit to end users. But to developers, a laptop with a perfect emulator is a way to ensure no reliance on emulation defects that's easier to lug around than the original machine and its monitor. I think that's what Alegend45 is talking about.

Yeah, you can't even write a program on most hardware, run it on a machine, and be sure it'll run the same again on the same machine, let alone another machine of the same model, or another PC model/manufacturer, because there can be unpredictable effects. So forget having it run on an emulator and then run exactly the same on hardware. Even hardware fails at this.

^this.

Your problem is not the lack of a "good emulator" (by *your* standards ofcourse) but the lack of knowledge from your part. So yeah, a "good emulator" won't offer you something , create your super awesome/useful program FIRST and then rest assure the communities will help you debug it IF it's super awesome/useful indeed ... at this point your posts here worth nothing.

Sorry, but unlike the Famicom the PC is a lot more than just a games platform. There are plenty of small businesses using what would be considered legacy DOS software thats both incompatible with modern operating systems and too expensive for such a business to replace. Tee repair section of the business I work for use a DOS version of a application called Servicebase to manage all the repair jobs. Running it on DOSBox on a newer machine just kept corrupting it's database. DOSBox can run newer DOS games quite well but can't handle a older textmode database.

The attitude that some people have with DOSBox would be like people saying that (insert name of Nintendo 80s 8 bit home console emulator here) is fine with just supporting the US releases...

DOSBox is not made to run applications , it is made to run games. It is stated by the authors a gazillion times. It is not to be trusted for serious stuff.

I hate to say it, but DOSBox does have the best VGA emulation out of all of them.

I understand that now but back then the name 'DOSbox' made me think thats what it did, emulate a DOS box as in a DOS environment. Perhaps the name "Old-non-windows-pc-game"box would be more descriptive.

At least the printer emulation worked.

Anyhoow, anybody tried PCem? Seems capable enough, the developer even says it can run Win98.

I doubt that highly. Making a PC emulator that runs Win98 is not a one-man job.

So have you told them? Have you tried any of the other x86 emulators? Do you know whether the corruption issue wouldn't have happened with a cyrix/amd x86 cpu? Or whether it's sensitive to the version of DOS? I encountered a "problem" where DOSbox always presented the directory entries in alphabetic order, and $person had been using the file entry order of the disk to organize things.

At no point have I said that DOSBox is the pinnacle of PC emulation. I'm simply pointing out that they can't fix what they can't directly debug.

For retrodev, even byuu says that bsnes is not sufficient to make sure that your game will work correctly on hardware. Even Trixter agrees that emulation can never replace the original hardware.

DOSbox works well enough for me to run windows 3.1 software. (Although, honestly, so does dosemu using its v86 mode on an ia32 host).

After reading through their forums and seeing the various responses to the "can DOSBox run my non-game application" I got the impression that they didn't really want to know.

No. At the time I didn't know of any alternatives and we needed that system working yesterday. Eventually we managed to source a DOS compatible laser printer and built a new FreeDOS powered box. Still the downtime did cause some aggro for the service guys.

I can't really comment on the CPU (anyhoow, you'd think that the DOSBox team would be striving for intel compatibility and the old box was a genuine intel 486 and the new one a early Pentium) but Servicebase is happy running in FreeDOS as it was in MS-DOS 6.22

And at no point did I make any suggestion that you did. More like I was taking a stab at the general idea of "its good enough". It's good enough meant that Mario Kart was emulated for ages before Pilotwings was for example. Its good enought is partly the reason why Nesticle lasted so long...
Preaching to the choir there

Game consoles are all the same. Every NES you come across is going to have the exact same hardware, functions, and capabilities. Same with every Sega Genesis, every Xbox, every Astrocade, every Atari 2600, every Nintendo 64, etc.

This makes them easy to emulate, because all of the software is designed for this one particular set of features.

PCs are customizable and expandable to no end. One PC will have wildly different features and capabilities compared to another. There's no "standard behavior" because, for example, nVidia graphics hardware and Intel GMA graphics hardware, while adhering to several standards (such as OpenGL and SVGA), will still differ significantly in clockspeeds, bandwidth, etc.

So! PCs wildly vary from one to another, NESes do not.

I guess the most plausible PC emulator would be one that emulates a generic IBM PC compatible with Soundblaster-compatible sound hardware, a generic VGA graphics board, and maybe a 486 CPU. So basically, the most generic computer imaginable. This would definitely work for anything pre-Windows 95, and possibly early Windows 95.

Yeah, but I'd try to emulate as many different video cards and sound cards and chipsets as I possibly could.

Surprisingly, the vast majority of them seem to boil down to VGA compatible or Soundblaster compatible. Provide generic VGA and generic Soundblaster emulation, and you'll support a very large portion of PC software right then and there.

Assuming we're talking about all aspects of PCs here (that's what your OP implied) and not just classic DOS-era hardware:

I don't think you understand the situation in full (I'd border on saying you definitely don't understand the situation in full). Emulating mainstream video cards made after roughly 1996 (around the time the 3dfx Voodoo was introduced) is amazingly difficult without someone who can actually sit down and reverse-engineer the device at the hardware level. Why the complexity? Because all product documentation was (and still is -- the situation has only gotten worse) under NDA.

I speak the truth and from personal experience as well. Short story: in 1993 I wanted to give a shot at using actual hardware-accelerated features of the video card I happened to own (a VLB card using the ET4000/W32 chipset). However, as I expected, there was no documentation for the acceleration features of this card. Sure, the card manufacturer offered 3D Studio drivers (whoop dee doo), but most every other application I found (including X drivers) just used VESA -- i.e. no actual hardware acceleration capabilities of the card. I knew the card had features like smooth screen panning, hardware sprite support, masking, and some basic 3D capabilities (reminder of what year this story takes place), and I wanted to tinker with those, yet I had no idea how. I was used to classic 320x200x256 (segment A000) mode.

I decided to contact Tseng Labs directly -- I did this because in the past I had contacted companies (specifically Zilog) to get information on their chips and was greeted by folks who sounded thrilled that I wanted docs, so I moronically assumed that it'd be the same with PC hardware manufacturers. My first call with Tseng consisted of telling them the truth: I was just a generic hobbyist wanting to tinker. I still remember the call: I was handed to some manager somewhere and was politely yet very directly told "sorry we don't do that". A few weeks later I tried a different approach: I explained to the gal at the front desk that I was working on a combination of a hardware/software project and that I needed product documentation since we were considering the ET4000/W32 chip for our device. I was handed to some other manager, who asked me to explain in detail the project and how many chips we'd be looking at using, so I made some shit up on-the-fly (I said for starters a few hundred chips). After an uncomfortable delay the person responded that they could ship technical documentation to me, assuming -- and I quote -- that "I gave them my word that I would not distribute or copy it". A week later I had the documentation (around 40 double-sided pages if I remember right).

Was I able to accomplish anything? Not really -- the documentation was mainly for hardware implementation (i.e. stuff someone designing a circuit or card would need for interfacing with the chip), and the software side of things was tiny (a few pages at most). The software bits were obviously translated and badly at that (that year I was a 3rd-year Mandarin student and was fairly certain the conversion was from Chinese to English, but I still remember second-guessing myself sometimes). I wasn't going to get any actual support from Tseng -- think about what schmooze I had to go through just to get what I did? -- so I was out of options. I gave up. The end.

Fast forward to today and think about NDA, IP, and DMCA. Now ask yourself the question: do you really think it's possible for you to "emulate as many video chipsets as possible" when you have access to absolutely no official documentation? If you think the answer is "yes", you've got your head buried deep in the (hopefully sweet-tasting) sand. Reverse-engineering is a very lengthy process, and is in no way/shape/form easy. Hell, you're on a nesdev forum, where all public documentation for the console itself has consisted of reverse-engineering efforts -- that console came out in 1983 and folks here are STILL trying to figure out its behaviour. Even open-source operating systems like Linux end up getting handed binary blobs from companies like nVidia (graphics chips) and Broadcom (wireless NICs and switching fabrics), citing IP. AMD has pulled PR stunts insisting they "want to help Linux and provide the documentation" yet the documentation they've given has been sub-par (from what I understand). OpenBSD has tons of songs talking about this predicament (see the descriptions for songs "100001 1010101", "Puffy Baba and the 40 Vendors", "Blob", "Hackers of the Lost RAID", and "Wizard of OS").

So how's that sand? :-)

Now, assuming we're talking about DOS-era hardware: above still applies, just to a lesser degree; see below.


Assuming we're still talking about classic DOS-era stuff -- correct, but don't forget about VESA/VBE. I feel sorry for anyone having to emulate that.

I wouldn't be surprised if these un-publicly-documented video chips were full of bugs. Since the supplier provides the software driver, and updates it many times at first, it's likely that only the combination of the driver and video chip has a low bug count. They can work around bugs and avoid features which turned out too buggy to use (at a performance cost). Since only their driver uses the chip, they don't even need very good documentation at all.

Well, koitsu, I do at least plan to start with the small stuff, like the MDA, clones, CGA, clones, etc. The only problem I need some help on with clones, is how to scan for extra registers. My first step would be to find them.

EDIT: I also know that this will be a lifelong goal I will never reach. Some obscure cards have probably already gone to the sands of time. However, I can't stop until I'm dead, if I want to preserve the PC. My project will perhaps never be finished. However, I will at least do the best I possibly can, for as long as I possibly can.

@Koitsu: Yeah, I was thinking mainly DOS era PCs. Anything newer than DOS-era is probably better left to compatibility rather than emulation.

So, the other reason I suggested what I did is because if you get a 486 processor and basic VGA emulation working, you'll already have something working and usable, and chances are, there will be other people out there who'll take an interest in your project, so you won't be working alone.

I'd say processor and graphics first, and then work on the SB. Try to get QBASIC working, and you can use that to debug your VGA emulation. Then show off some popular shareware titles, like Doom, Duke, Commander Keen, Scorched Earth, ZZT (if you can still find the DOS version), etc. Sorry, I mainly used DOS for games way back then, so I don't know what kinds of business software you can show off.

Again, this will help get the project some interest.

SuperZZT/ZZT and Kroz works on most major emulators, Even if they do not have font or graphics support

Also, Kroz needs speed adjustments unlike SuperZZT/ZZT.