Please add .UNF extension to the list of allowed files

.NES is allowed, so I think .UNF should be allowed too.

I've added .unf extension and a few others. Now I need a ROM that plays "The Terrible Secret of Space", called "pak_chooie.unf".

Thank you tepples

How about adding .IPS next?

For extensions in a category that did not exist, I first have to add the category ("Patch files"). What other patch formats in this category are desired?

IPS, UPS, XDELTA, BPS, anything else?
As for what is actually used, IPS is by far used the most, since many patches don't need to move data around.

edit: List of all patching tools from Romhacking.net: http://www.romhacking.net/?page=utiliti ... lsearch=Go

File extensions:
.ips
.xdelta3
.bps (beat)
.bdf (bdiff)
.pat/.ffp (fileflower patch)
.ups
.ppf (playstation patch format)
.jfp (just F patch)
.rup (NINJA patch)
.gdiff

I've added support for these extensions, under the extension group called "Patch Files".

Attached a test .ips file just as proof.

Can I request the .py file extension? I use python to build simple tools for working with NES code sometimes and they might be helpful to share. (Is there a security risk with .py or is everything just disallowed by default?)

Correct; it's a whitelist. But py.zip should be fine because it sees only the extension of the outermost container.

Rephrased: the forum software only looks at everything past the last dot found in the filename. ("Container"? Damn Python freaks... ;P) And yes, by default nothing is permitted. I can't find a good screenshot of the control panel that controls this (all the stuff I find is either outdated or obfuscated or in a foreign language) else I'd show you and you'd understand the UI/model a bit better.

I've added the following extensions as permissible:

.py
.php
.bmp
.dmg
.lha
.lzh

Thanks!

Could you add .nesproject files for NESICIDE?

Beat me to it.

Is there a spec for .nesproject files? I'm trying to find why not to just zip them.

I've added .nesproject (for Tepples: I added it to "Downloadable Files" because it's the only description/section/behaviour that makes the most sense for what it is. I didn't want to add it to Plain Text, etc...)

Yeah, tepples is right, there's probably not going to be a case where we need a loose .nesproject file by itself. Sorry for the bother.

Koitsu added it.

Can we add .nsfe?

Added .nsfe in category "chip music".

Another extension request: .lua

Would be good for sharing FCEUX lua scripts.

.lua has been added to the same group that contains .js and .py.

Does anyone know if there is a way to get phpBB to display a warning for a particular extension, such as a warning that many people won't be able to open it and that another format is superior in the vast majority of cases? Like PCX vs. PNG, or some obscure archive format vs. ZIP and 7Z, or AI vs. SVG?

There's no reason to treat the user like a baby that doesn't know what they're doing. If they want to upload a file, that's a deliberate choice they made to upload that specific file. There's no reason to warn them that some webmaster out there likes some other kind of file better.

Also, I find it kinda pointless to lecture people who are clueless about file formats, since they're probably the least qualified to do proper conversions. We don't want to get files with the wrong extensions (i.e. simply renamed) or catastrophic .ai to .svg conversions.

If you really think that a file format lecture is in order, a sticky post somewhere would probably work better, because you could explain everything in detail and why certain formats are preferred for certain purposes. Then we could point people to this post whenever necessary (which probably won't happen that frequently), rather than showing annoying warnings every time a file is attached.

Request for extension: .pal

Useful for discussions that benefit from sharing palette files.

Added .pal (color lookup table)

Appreciated.

Requesting .SAV for sharing PowerPak (or various emulator) 8k battery saves.

added .SAV

Request: FCEUX .fm2 files

Request: Processing .pde files

They're a text format, basically just a java source code. Useful for writing visual examples, especially interactive ones.

.pde extension was added

Really, why annoying users and making them need of additioanl effort for double uploading some files (if first time it fails, then he needs to pack it as .zip/.rar)

AVI also is not alowed.

A nontrivial AVI is probably too big in MB to attach anyway.

Requests:

• .bat (I use them with NES assemblers)
• .java (I probably won't need them but there is one at http://nesdev.com/NESrev.java)
• .xz (compressed file)

.xz seems good, .bat and .java though both make me a little paranoid. If it's not a filetype I would feel safe opening myself from a random unknown source, I wouldn't want the forum to allow someone else to, also. Of course I'm not worried about stuff actual forum users would post, but anybody can easily make an account. Unfortunately, we don't have an easy way of stopping someone from posting attachments and spam URLs as soon as they sneak through the registration process, wouldn't be a concern otherwise.

People should be compressing stuff like that using .zip or some other format if possible.

I'm getting the point: source code should be in a zipfile or something so that the user has to take a deliberate action to execute it even on an exploited browser, especially if it's in a scripting language like .bat. I may have made a mistake in the past by adding scripting language source code; that mistake may need to be reversed sometime.

I've added .xz.

I agree about .bat, but .java is just Java source code. It won't run by default on any operating system (.js is a different deal).

My proposal: Remove .lua, .php, .py, .js, and .swf; add .java. Anyone second or object?

Could we change the "you can't upload that" message to instead say "put it in an archive and then upload that" ?

I think .py and .lua are probably OK along the lines of .java, AFAIK they're not something one could just mindlessly click on and launch, right? I'm not a big user of any of those languages so it's a real question. At least not without purposefully installing some kind of support for those in your OS. We see those used enough around here that's it's probably not worth trading the convenience for the safety. Removing php, js and swf sounds reasonable to me, though.

The installer of Python for Windows associates the .py extension with itself.


PHP and Python are roughly in the same boat here: the interpreter isn't installed by default on Windows. Python is more likely to be installed on FreeBSD and GNU/Linux, but those systems use the execute bit to decide when to treat something as a program. And anyone who has started with my NES and Super NES project templates has Python and Pillow installed alongside cc65.

I like both .py and .lua here very much, and would not want to see them removed. I also previously requested .pde which is tantamount to .java already.

I can understand trepidation about .bat, though I'd also say that they're not a particularly versatile script format, and rarely useful without accompanying files (thus already covered by zip/etc), so having loose .bat prevented doesn't seem like a big deal to me.

I don't really think .js or .php is a problem. I've definitely seen .bat as a vector for malware installation, because any Windows user can run it. The others require separate installs. I don't think there's any value in trying to prevent all executable code from being attached (quite the opposite), I thought the goal was just to stop up typical malware/spam points?

Same deal with .swf... but has someone actually wanted to attach an .swf here?

I agree; .bat would be dangerous and quite useless. .swf seems unnecessary, too.

However, I think most source files (.py etc.) should be accepted. After all, they're not more dangerous than an .exe inside a .zip, which can be executed even on an out-of-the-box Windows.

More suggestions:

• .bas (BASIC source; there might be some old utilities written in it, although I don't remember seeing any)
• .svg (Scalable Vector Graphics; for posting e.g. sketches of game graphics/website logos)
• .xhtml (some documents might be in that format, although again I don't remember seeing any)

I'd worry about allowing html/xhtml permitting XSS attacks. (I don't know that it would, it's just the obvious failure mode)

DO NOT allow html/xhtml/shtml/php/js/anything else along these lines. You make blind assumptions about the infrastructure of the systems used to serve the website. All it takes is one Content-Type (MIME type) header and suddenly something is being run vs. downloaded (and not necessarily by the client; see: reverse proxying). DO NOT DO IT. I shouldn't have to state stuff like this. The last thing you want is WhoaMan finding out there's been a security hole open for some time due to someone allowing a silly extension through.

People wanting to use the formats: archive/zip up your work and upload it. It's not hard (either through GUI or CLI). The end.

I'll take that as a second, koitsu. Thank you. And for that reason, I don't even feel safe adding .svg, which also has a <script> element.

I have made changes to the following categories of allowed extensions.

• "Active content" that may accidentally be executed in the browser context, causing cross-site scripting:
  Removed .swf, .js
  Removed .html, .htm, .xml (because <script> element and on* attributes)
  Did not add .svg, .xhtml (because <script> element and on* attributes)
• Common CGI languages that may accidentally be executed in the server context:
  Removed .php, .py, .lua
• Compiled languages:
  Added .cs, .java
• Chip music scores:
  Added .0cc, .ly, .pently

Changes look sound to me. Thumbs up.

Ah, but who allowed .php in the first place?
https://forums.nesdev.com/viewtopic.php?p=127134#p127134

I'm a little disappointed to see .lua removed (there's been lots of cool FCEUX lua scripts shared in the past), and .py but whatever. More zips I guess. (I was the person who asked for both of those in the first place.)

Was either .js or .php ever actually requested? .swf?

You can still upload Lua scripts. Just zip them up first so that they don't accidentally get executed on the server.

Yes, that's what I was disappointed about. The friction of un-zipping propagates also to each person who wants to download it too.


I understand that part. Whatever you feel is necessary to protect the server is fine. I don't know anything about what your server's configuration looks like, so I'm in no position to tell you what's safe for the server, but as an end user I'm still disappointed that something I liked using (both up and down) is being removed.

Especially because this makes several old posts inaccessible, without even being shown a filename or any information to cross reference what might have appeared there with files I might still happen to have downloaded. It's a bit frustrating to have content you uploaded to the BBS for archival purposes suddenly effectively "deleted" with no identifying reference...

Like I would have a hard time finding my affected posts at this point, and then also knowing what content is actually missing is also a problem with this interface, and then even if I knew the filename I'd have to hope I still have a copy somewhere else that I can zip up and edit back into the post.

So... my disappointment is a bit more than "just" having to zip some files up in the future.


If you need to have them disabled for security reasons, I'm not trying to fight about that, you can weigh that as you need to, I'm just telling you how I feel about it as an end user, but is there anything you can do about old posts, at least? From my side I have no way of finding or recovering the now blocked content. That stuff is actually quite frequently useful to me. (Plus even if I had, e.g. a grace period and list of my own affected posts... that still doesn't work for anyone else's old posts who's not currently watching the issue and actively working to update with zips.)

Just in case the problem isn't visible to moderators (tepples has mentioned interface differences in the past), this is what an attached lua file currently looks like for me:


Apparently for my own old posts, I can edit them and it will at least tell me the filename, but can't get back the content. (Attempting to download it will say it's blocked.) Edit: it seems I can get the filename for other peoples' posts by using the Quote button, but only if it was placed inline, I think.

I think the server infrastructure changed between then and now (including the webserver, IIRC; it used to be Apache, now it's nginx, and I think there's a reverse proxy involved now). What I knew to be true then I don't think is true now.

MIME types can be treacherous territory; server-side they seem innocent enough ("it's just a Content-Type header!"), but when reverse proxying is involved or potentially other devices like load balancers, all of which tend to inspect content, it becomes risky. Apache's mod_mime_magic can be a blessing and a curse too. Often feels that the days of basic web hosting/content serving are long gone. Things were simpler back then (code directly on an Apache webserver which was directly on the Internet, no intermediary anything).

Reviewing the download links from phpBB (example), we can see that the Content-Type returned (at least for a .zip) is application/octet-stream -- good -- and a Content-Disposition type of attachment-- which is correct and VERY important -- but the rest of that header looked bizarre to me (those are two apostrophes next to one another BTW, not a double-quote; the asterisk also made me go "?!?!"):


For Content-Disposition, it looks like filename* is an RFC 5987 extension... from 2010, so no wonder I'm not familiar with it. Reading (well, I skimmed) that RFC, it looks as if the syntax is in fact correct. Learned something new. Though, it does make me wonder what happens if you upload a .txt that's in something other than ASCII or UTF-8, ex. JIS). I'd have to check.

Finally, client-side MIME type association is often a crap shoot as well -- you have no control over how someone's browser is set up/configured, so you don't know what will happen if the client receives a true/literal Content-Type that matches a MIME type that they've configured to allow to auto-run (e.g. "Download as..." vs "Open file"; scarily, a lot of people still do the latter, either automatic or manual). For example, we don't know if someone has .bat set to automatically run cmd.exe on it, and some jackass uploads one that does @echo off\rrmdir /q /s C:\WINDOWS. The idea is to minimise the chance of something like that happening. TMK, phpBB doesn't do any kind of "filtering" or "scanning of content" on uploads -- and I tend to fear stuff like that anyway (false positives causing failures that drive the uploader crazy).

These days, all it takes is an intermediary (ex. reverse proxy on the server side, and sometimes even a caching proxy on the client's network (semi-common at workplaces)) to cause a bit of mayhem with MIME types or filtering out certain headers (the latter is VERY common with reverse proxies). The time to worry is when Content-Type: application/octet-stream becomes, for example, Content-Type: application/javascript and there's no Content-Disposition header.

I wasn't aware that this feature of phpBB 3 removed previously uploaded attachments from view. As an administrator, I cannot download them either. In order to preserve both availability (no removal of legitimate downloads from public view) and integrity (no unexpected execution on the server), am I now expected to spend time writing a script that spiders the entire forum looking for attachments with extensions that have been deactivated, temporarily enable them, download them, use my moderator powers to upload them with the attachments properly zipped, and re-disable them?

Hrm that's both a positive and a negative feature I'd say. Yeah, hrm. There can't be *that* many attachments of now-excluded extensions. Maybe you'd be better off doing it server-side and doing your best to go through the phpBB MySQL tables and see if you can pull out what you need from there. Might be quicker, and certainly less rude on the HTTP server.

Out of idle curiousity (not anything actionable), do any of the python scripts you've uploaded show up in the Manage attachments list?

Ah, yes they do. At least there's a list of my own posts I can access then. (...and yeah, can see the filename and thread but can't download.) I thought I'd uploaded more lua scripts than python, but apparently it's the other way around.


You can decide how and whether to work on this. I'd volunteer to help, if I could, but I don't think I can really do much about it as a non-administrator. (If there is work I can do to facilitate this, though, let me know.)

I would suspect/hope that for most of them, the number of affected files is actually zero, but .py and .lua specifically are ones I'd been using and seen others using too. (It's possible this affects my posts more than anyone else's... I know I'm responsible for requesting these two formats in the first place.)

For whatever it's worth, attachments with the forbidden extension return "404 Forbidden" in response to a HEAD request. (In contrast, "403 Forbidden" for PM attachments and "404 Not Found" for stuff that's actually gone).

With 12500-ish current attachments on the forum that's a little too big to just manually check without explicitly getting WhoaMan's OK.