Transition from WLA

This is an archive of a topic from NESdev BBS, taken in mid-October 2019 before a server upgrade.
View original topic
Transition from WLA
by on (#141867)
I wanted to ask some things and I can't find an appropriate thread for it so here's one. I hate WLA. I can't find any real documentation of its syntax and it gives me problems trying to get the right opcode in constantly. All I know about it I've learned by example.

I hear everyone saying to switch to ca65. From what I've seen it seems radically different from WLA. My project at this point is getting massive, and the thought of rewriting the entire thing just to transfer to a new assembler is painful. I also have a bit of a deadline coming up where I'd like to have something to show for all my work. It took me a good month just to understand WLA to begin with; if it takes another month to learn ca65 then I'm screwed and better off patching around WLA's stupidity for the time being.

Can anyone comment on how difficult it is to transfer code between the two, and exactly what is required to do so? Is there official documentation on ca65 so that I can just look up my answers and never deal with this guessing ever again? Does it have annoying bugs too? Is there anything better than ca65?

How come I can't make WLA use opcode 9D: STA addr,X (Absolute Indexed,X), with A in 8-bit mode? It defaults to direct page, x.
Re: Transition from WLA
by on (#141869)
It seems ca65 has some trouble with direct page. It only knows about zero page, so you can't move it. I haven't gotten around to trying ca65 yet, so I don't know if this is a hard limitation or a convenience issue. (I mean, it can't be a hard hard limitation, as long as you can Define Byte, but that's kinda...)

And I know nothing whatever about bass, except that it probably makes a lot more sense to byuu than either WLA DX or ca65.

Khaz wrote:
How come I can't make WLA use opcode 9D: STA addr,X (Absolute Indexed,X), with A in 8-bit mode? It defaults to direct page, x.

How big is the operand? WLA tends to assume a value is as small as it can be, but you can force it to use a certain addressing mode by appending .b, .w, or .l to the opcode to specify byte, word, or long (using the suffix on the operand itself doesn't always work).

We did recently run across a weird bug where it was using 8-bit addressing for addresses that didn't fit into 8-bit numbers, which probably calls for the same fix...
Re: Transition from WLA
by on (#141871)
1. The documentation for WLA DX is here. It is badly organised, and as I've ranted about here already, is very unclear in many regards: https://github.com/vhelin/wla-dx/blob/master/README

2. I would suggest given your "deadlines" that you continue to stick to WLA DX. Use ca65 for your next project. Don't get hung up on creating blockers (the need to move to ca65) right now, especially if time is important. If you want to migrate to ca65 for this project later, i.e. after it's release, then wait until after its release. Stay focused on what you're already doing. :-)

3. I can comment on moving from WLA DX to ca65 because I did so for Espozo and his SNES project recently. I mention this because if I remember right *your* project is SNES, and that matters. His very small project took me probably 3-4 full hours. Most of that is because I lacked familiarity with ca65 (huge thanks to Tepples and his ca65 LoROM template), but some of it was also because ca65 has "mostly good" support for the 65816 but not entirely (its concept of direct page is not very good; ca65 is very much "6502-centric").

You aren't being very forthcoming with WLA DX features you're using. For example, WLA DX has the abiliyt to do "overwrap maps" or something, where you can literally use WLA DX to patch/replace/inject code into existing ROM files. ca65 does not have this. This is just one of many things.

WLA DX and ca65 use substantially incompatible pseudo-ops (assembler directives), and addressing mode directives (for things like getting the lower 16-bit portion of an address, vs. getting bank, etc.) differ greatly as well.

ca65 has "complexities" understanding direct page fully -- I've complained about this in other recent threads several times and am getting tired of digging up the posts -- but my most recent feedback/view of this is here: viewtopic.php?p=141784#p141784 -- I'm thus of the opinion while its memory/segment stuff doesn't have full comprehension of direct page, the actual assembly it generates is okay.

The one huge (and I mean HUGE HUGE HUGE) advantage ca65 has over WLA DX is that it's listing generation mode *actually works*, while WLA DX acts like a complete moron when you use .include (or is it .incsrc) within an assembly file -- the listing generation contains interspersed output (and this is still the case on the most recent WLA DX github build too). I STRONGLY advocate using listing files, because they help you diagnose issues in your code; "I expected the assembler to do this, but it's doing this instead".

And finally, there are known, very scary bugs in WLA DX.

4. WLA DX is trying to be "extra smart" about optimising. See the WLA DX documentation, section 14.5 ("65816"), and read it in full. If you need additional insights, see the .8BIT and .16BIT directives. .ACCU and .INDEX are also worth looking at. Just be aware of what it is you're doing with those directives or size modifiers (e.g. .w) if you use them -- use assembly listings (oh wait those are crap on WLA DX if you use include ;-) ) to determine if it's doing what you want.
Re: Transition from WLA
by on (#141872)
Okay, thank you 93143, the .w fixed it. I thought I tried that already, maybe I didn't hit save or something... I guess I will be sticking with WLA for the time being.
koitsu wrote:
The one huge (and I mean HUGE HUGE HUGE) advantage ca65 has over WLA DX is that it's listing generation mode *actually works*, while WLA DX acts like a complete moron when you use .include (or is it .incsrc) within an assembly file -- the listing generation contains interspersed output (and this is still the case on the most recent WLA DX github build too). I STRONGLY advocate using listing files, because they help you diagnose issues in your code; "I expected the assembler to do this, but it's doing this instead".

Can you explain a bit more what you mean by listing generation/files? This is new to me. So far the only way I've really been directly examining my code is by running it and reading a trace log.

As for the very scary bugs, I read that thread over several times and as far as I can tell it's not so much a "bug" as that WLA required the .w to specify which operation? (It's definitely still an error-likely design choice...)
Re: Transition from WLA
by on (#141873)
No, that was the one where it was apparently taking a value of $1808 and deciding it was fine to just use one byte to represent it. Pretty egregious on the face of it...
Re: Transition from WLA
by on (#141874)
Ah, I see what you mean. I, uh, guess I just got used to that behaviour...
Re: Transition from WLA
by on (#141876)
Khaz wrote:
Can you explain a bit more what you mean by listing generation/files? This is new to me. So far the only way I've really been directly examining my code is by running it and reading a trace log.

Full details (you need to slowly read my entire post): viewtopic.php?p=139495#p139495

Short version: if any of your programs use .include, then listing file generation will be completely bogus, with interspersed hex bytes into the "main file" that actually correlate with code from the included files. I can understand from a programming perspective how this could happen (it indicates extremely poor design choices) -- and yes it can be fixed but it isn't my problem to fix. This, compounded with other bugs, just make me dismiss the entire assembler. I like ca65 a lot more now that I've gotten more used to it, but its memory/segment template stuff (while extremely useful) still makes me shake my head a bit. (What can I say, I like KISS/bare-bones stuff)

That bug psychopathicteen found is an absolute total disgrace. There's nothing to "get used to" -- it's a downright bug and I feel sorry for anyone getting bit by it.
Re: Transition from WLA
by on (#141878)
93143 wrote:
It seems ca65 has some trouble with direct page. It only knows about zero page, so you can't move it.

I'm pretty sure you can move direct page, as my init code does just that for writes to the PPU MMIO ports at $002100-$00213F, and I verified its correctness in NO$SNS's debugger. You just have to subtract the value in D from each address you use with a direct page addressing mode, either with subtraction or with the < (low 8 bits) operator.

koitsu wrote:
WLA DX has the abiliyt to do "overwrap maps" or something, where you can literally use WLA DX to patch/replace/inject code into existing ROM files. ca65 does not have this.

There's no explicit for this "overwrap" as you call it in ca65, but I've got it to work in my UNROM hack of ccovell's Solar Wars.
Re: Transition from WLA
by on (#141879)
Hm. Is there a way to bypass this garbage data problem, while still using WLA? Something other than .INCLUDE? (Other than just put all my code in one text file)

I'm also not sure how you get a listing file in the first place. The wla setup I'm using just creates then deletes a .prj and an .obj file, neither of which seems very legible.
Re: Transition from WLA
by on (#141881)
Khaz wrote:
I'm also not sure how you get a listing file in the first place.

ca65 outputs them whenever you assemble a file. I don't remember anyone talking about listing files on WLA. Basically from what I've seen (and experienced) ca65 > WLA.
Re: Transition from WLA
by on (#141882)
Khaz wrote:
Hm. Is there a way to bypass this garbage data problem, while still using WLA? Something other than .INCLUDE? (Other than just put all my code in one text file)

I'm also not sure how you get a listing file in the first place. The wla setup I'm using just creates then deletes a .prj and an .obj file, neither of which seems very legible.

Make sure you don't .include any files that actually contain code (e.g. a file filled with equates should be safe), and assemble all files with actual code in them individually (i.e. a separate call to wla-65816.exe for each file -- don't forget the -i flag so that listing file generation can be done by wlalink.exe later). This should (if the person writing the build script did it right/specified the right arguments to wla-65816.exe) generate a whole series of .o files. These should be linked together with wlalink.exe (don't forget the -i flag) and through use of its "link file" of what objects/libraries/etc. it should refer to (see docs, section 10, "Linking", for all of this -- you must read the entire section, not skim it).

Again: nobody realises WLA DX can do any of this because the docs are so badly formatted and disorganised.

Earlier when you said "So far the only way I've really been directly examining my code is by running it and reading a trace log", I took that to mean some kind of "log" that WLA DX was generating, but now I realise you're probably talking about a SNES9x trace log. That's a fine approach if you enjoy sifting through enormous log files (I deal with it when doing romhacking, albeit painfully). I prefer to use an emulator to see what the behaviour is, use its debugger to figure out what might be going on, and if something looks totally amiss with the code that was run, start looking at the generated listing files of the assembler to find out what it (or I) did wrong.

I cannot stress enough the importance of assembly listing generation especially during development, or when a person is not familiar with all the quirks/behaviours of their assembler (or when learning (or migrating to) another assembler). When something anomalous begins happening in your work, you can go immediately to the assembly listing and say "...there! Damn, that should have been a 24-bit load, not 16-bit" or "Gah, I obviously meant rep #$20 not rep #$10".

ca65 (not ld65 (linker)) will output listing files when using the -l (dash-lowercase-ELL) argument with a specified filename, e.g. ca65.exe -l main.lst main.asm -o main.obj will generate an assembly listing (main.lst) and a resulting object file (main.obj) for use with ld65 (linker) later.

@Espozo -- what are you talking about, re: "I don't remember anyone talking about listing files on WLA" -- that's the entire reason I got you off WLA DX and over to ca65, and I even linked in my previous post here in this thread the post in *your* thread about that problem in WLA DX. How can you say you don't remember anyone talking about listing files when they were what I talked about for a page or more?
Re: Transition from WLA
by on (#141883)
koitsu wrote:
@Espozo -- what are you talking about, re: "I don't remember anyone talking about listing files on WLA" -- that's the entire reason I got you off WLA DX and over to ca65, and I even linked in my previous post here in this thread the post in *your* thread about that problem in WLA DX. How can you say you don't remember anyone talking about listing files when they were what I talked about for a page or more?

What I meant to say is that WLA doesn't output them.
Re: Transition from WLA
by on (#141884)
Espozo wrote:
What I meant to say is that WLA doesn't output them.

It doesn't by default, but it does if you give it the arguments I just described. ca65 doesn't output listing files by default either, unless you give it the arguments I also just described. *confused look*
Re: Transition from WLA
by on (#141885)
I'm guessing the arguments are in build.bat? I'm really not thinking very well at the moment.
Re: Transition from WLA
by on (#141893)
For your project: where else does ca65.exe or ld65.exe get run?! Of course. :P You really need to get familiar with that script. It's actually legible and followable since I rewrote it for you. DOS batch is never pretty, but what I wrote you is pretty clean. But I'm not going to get into discussing that here because it's unrelated to what this thread is about. (There have been _WAY_ too many threads in the past few weeks which have derailed)
Re: Transition from WLA
by on (#141901)
Please, by all means, go off topic. I'd never have created this thread in the first place if I'd only actually tested the .w. And I meant it to be more of a multiple-questions thing but by the time I was done ranting about WLA I forgot what else I was going to ask.

Anyways. I tried to set up WLA.bat to output the listing as follows:
Code:
wla-65816 -io %1.asm %1.obj
wlalink -vir temp.prj %1.smc

(I just stuck the 'i's in there.) When trying to execute wlalink it gives "LISTFILE_WRITE_LISTFILES: Out of memory error."

And regarding the traces, I've been using Bizhawk for the last while and it's generally been good, it has a lot of very useful features for debugging. I've gotten used to reading the trace logs if only because that's how I first started learning SNES code. It's time-consuming though, I try to solve my problems just watching the emulator's hex editor and thinking about it first if I can.
Re: Transition from WLA
by on (#141912)
WLA DX's argument parser is one of the worst I've dealt with in a while. It is EXTREMELY SENSITIVE to where you place arguments and in what order. You really have to read the docs and look at the usage syntax from the docs very very carefully. Its error messages when you violate this are bizarre and don't make obvious sense.

Try:

Code:
wla-65816 -oi %1.asm %1.obj
wlalink -ivr temp.prj %1.smc

This should generate files named %1.lst (where %1 is the base name of the file/thing you've passed that function in DOS batch or on the command line, e.g. wla.bat main should result in a main.lst file).

If this doesn't work and you continue to get weird errors, you should open up an Issues request on github with the WLA DX author to report the problem. You will probably be asked for the full source code to everything you are doing/working on. I can confirm listings do work (the stuff I linked/said earlier is proof), so if this happens then there may be another bug.

And you should also try the latest WLA DX builds if you can find them. The most recent ones I was able to find were made by nicklausw here on nesdev. You can find them in this post: viewtopic.php?p=139589#p139589 -- or if you need 32-bit binaries, you can find them in this post (same thread) -- viewtopic.php?p=139741#p139741
Re: Transition from WLA
by on (#141918)
Huh. The version of wlalink.exe I have is almost five times larger than the one you linked, and your wla-65816 is smaller too. Tried running it with the code you posted in wla.bat, and wlalink crashed somehow...

Wish I could tell you where I got the version I've been using from. Was possibly one of the ones included in bazz's tutorials, can't remember.

EDIT: Okay so I also tried the 32-bit version and that one won't work either, it gives an error with a stack trace. I tried your commands with the version I was using before and it still says Out of memory error...
Re: Transition from WLA
by on (#141919)
The binaries I linked (at least the 32-bit binaries) work okay (they're how I was able to test/verify for nicklausw). You absolutely need cygwin1.dll (included in the .zip) in the same directory as those executables -- failure to do that may result in crashing or a missing DLL error. These binaries aren't built using the same compiler or framework as the ones mentioned below (above = Cygwin and gcc, below = Visual C++).

There are "unsupported" Win32 binaries provided by the WLA DX author any more, as per his website: http://www.villehelin.com/wla.html -- the binaries can be found here, the latest being 9.5 built a year and a half ago -- http://www.villehelin.com/wla-win32.html

The binaries I linked you from nicklausw are from fairly recent WLA DX code on github.

Edit: okay, then go ahead and file an official Issues report with the WLA DX author about the bug. It sounds like something you're doing in one of your files or some feature you're using does not work correctly with listing generation. Let us know of the issue (provide a link to it) and let us know what the outcome is when its resolved.
Re: Transition from WLA
by on (#141921)
Ah, okay, I remember now. I did download the latest from that page, I've been using "Version 9.5 of WLA DX based on revision 8 of the code in SVN with extra magic by William Jones (02-Nov-2013)." Mostly because I can't say no to 'extra magic'.

I will file a report sometime after I've had sleep. Don't know how comfortable I am with handing over the entire source code. Will probably try to reduce the project to the simplest case that still gives the error and narrow down what's causing it.
Re: Transition from WLA
by on (#141924)
Well file the issue first, and then see what sorts of information/source/etc. the author asks for. The good thing is the problem is 100% reproducible, rather than intermittent.
Re: Transition from WLA
by on (#141935)
tepples wrote:
93143 wrote:
It seems ca65 has some trouble with direct page. It only knows about zero page, so you can't move it.
I'm pretty sure you can move direct page, as my init code does just that for writes to the PPU MMIO ports at $002100-$00213F, and I verified its correctness in NO$SNS's debugger.

Sorry, I was unclear. What I meant was that you can't change where ca65 thinks direct page is, which is of course not at all the same thing...

Quote:
You just have to subtract the value in D from each address you use with a direct page addressing mode, either with subtraction or with the < (low 8 bits) operator.

So, a convenience issue rather than an outright 'you can't do this'.

I'm hazy on what exactly is going on here because I only know WLA, but I apparently need to learn ca65 for Super FX support, and once I've done that I'll be in a better position to comment (not that that's stopped me so far)...
Re: Transition from WLA
by on (#141937)
The latest WLA builds (v9.6) can be found here. :)
Re: Transition from WLA
by on (#141951)
Ramsis wrote:
The latest WLA builds (v9.6) can be found here. :)

...And they still haven't fixed this: viewtopic.php?f=12&t=12268&start=45
Re: Transition from WLA
by on (#141970)
Espozo wrote:
Ramsis wrote:
The latest WLA builds (v9.6) can be found here. :)

...And they still haven't fixed this: viewtopic.php?f=12&t=12268&start=45

You're welcome. I mean, to file a bug report. :)
Re: Transition from WLA
by on (#142108)
https://github.com/vhelin/wla-dx/issues/new

Since I always hear people complain about WLA but not even try to work with Ville...

And sorry that the 32-bit binaries I made have to have cygwin1.dll with them. I legit could not get mingw to work with any batches they made. Now William and Neonman (or something) are trying to switch the project over to Cmake for building...why does it have to be so freaking hard to compile this thing!? XD
Re: Transition from WLA
by on (#142112)
The biggest issue, in my experience, is that the development environments in Windows, when making CLI tools, tend to be garbage. You're either forced into using Visual Studio -- which has a good compiler but is bloated and in a weird way almost requires you to use their IDE and project files and other whatnots (yes I'm aware you can do it all with their makefiles and so on, but man cmd.exe is awful) -- or you're forced to use Cygwin (a travesty for a completely different set of reasons). Then there's MSYS/MingW, which is (IMO) better than Cygwin by far, but the last time I used that thing its development suite (particularly the include/.h files for Win32 development) was completely busted. Every single thing I've seen about the WLA DX code indicates that the author seems to have a hard-on for *IX systems, which is fine (remember I'm a UNIX SA), but most people doing dev are going to be on Windows.

That said: multi-platform and multi-arch support is a complete nightmare no matter what suite you go with. Just looking through the makefiles/ directory makes me want to gag. (I have no idea why the guy designed it that way, by the way, nor am I left with the impression that the guy knows how to write actual GNU makefiles -- but then again a lot of people write crappy ones because the gmake documentation is a tome and not so greatly organised unless you know exactly what you're looking for).

All that rambling aside, I haven't taken the time to really look at the actual WLA DX code in detail, but a quick skim of some of the most key files indicate the author uses comments super, *SUPER* sparingly. That doesn't bode well for any open-source project, especially one as complex as a compiler. The probability of others getting involved greatly diminishes then. Comparatively, I tend to be pretty lenient with comments in both closed- and open-source projects, all because I operate off the assumption that 6 months from now I'll have to re-visit some code and will certainly say "why the heck did I do it this way? *reads comments* Oh right! Right right right". But that's just my style, I guess. I'm also very clear about what I'm changing in commit messages. (Both are in contrast to my workplace, where when I try to inject comments explaining why something is the way it is, I am often told "remove that". Most of the code I've worked on at work is devoid of comments. It's disheartening)

Finally, moving to CMake isn't going to solve code issues or bugs. In fact, CMake is nightmare fuel on a completely different level -- if you think gmake is bad, you're entering a world of pain and hurt with CMake. It is absolutely possible to write coherent and decent Makefiles that work with both GNU make and BSD make, while with CMake I hope you enjoy variable names that take up the entire width of your terminal window and are both documented and undocumented at the same time (this is hard to describe/explain in text). I can get around inside of a classic Makefile within a few minutes; with CMake this could take me days.
Re: Transition from WLA
by on (#146348)
nicklausw wrote:
https://github.com/vhelin/wla-dx/issues/new

Since I always hear people complain about WLA but not even try to work with Ville...

https://github.com/vhelin/wla-dx/issues/32 :(
Re: Transition from WLA
by on (#146351)
...ca65...
Re: Transition from WLA
by on (#146356)
Espozo wrote:
...ca65...

Truth: ca65 does in fact address the problem mentioned here, as I understand it. ca65 has the following directives to assist with this:

  • .BANKBYTE -- returns upper 8 bits (d23-d16) of an address; synonymous with ^ operator; e.g. sep #$20 / lda #.BANKBYTE(label)
  • .LOWORD -- returns lower 16 bits (d15-d0) of an address, e.g. rep #$20 / lda #.LOWORD(label)
  • .LOBYTE -- returns lower 8 bits (d7-d0) of an address; synonymous with < operator, e.g. sep #$20 / lda #.LOBYTE(label)
  • .HIWORD -- returns upper 16 bits (d31-d16) of an address, e.g. rep #$20 / lda #.HIWORD(label)
  • .HIBYTE -- returns upper 8 bits (d15-d8) of an address; synonymous with > operator, e.g. sep #$20 / lda #.HIBYTE(label)

ld65's template configuration, which helps the linker know what addresses (including banks) should be involved with specific pieces of code, ultimately is what (I think) alleviates this. But I wanted to make clear the above directives.

Reference: http://cc65.github.io/doc/ca65.html

Note: .HIWORD at first might seem stupid, but it could actually be useful when referring to actual pre-defined raw table data that might be in a 32-bit format.

Footnote: remember to stay on topic, as I can see this post of mine causing all sorts of thread diversions.