Frames: A nomenclature discussion

Preamble:
We've had discussions on what to call entities, instances and objects which i found very informative and productive. There was also the discussion on what to call colours which might've helped some artist-dev coops? I'd like to talk and have a discussion about a time-domain nomenclature, and especially frames.

Body of topic:
In animation circles, we (ideally) distinguish keyframes from frames.

-A frame is the atom of animation in the time domain; as in for example 60fps or 50fps: You can't put a picture between two frames when frames have a fixed rate (even though analogue slew effects may smooth the transition by some amount, percievable or not)

-A keyframe is a special frame somewhere in the span of time that holds all the relevant data that causes the animation to happen: What picture to show, the duration of it, its opacity, colour scheme, or what have you. It may also contain information on automated interpolation sequences and what not. It got more advanced when software like AfterEffects matured, with a 'layer' for each property of each object holding its own set of keyframes, but the distinction heralds back to pen and paper animation and video fx development. My guess is video editing in general used this term too.

I wrote ideally, because in informal everyday talk, we might just as well say 'frame' offhandedly when meaning 'keyframe', because the distinction is assumed to be encoded in the topic.

This has caused me a bit of confusion from time to time in software dev environments. More specifically things NesDev-related; there's a few causes for that confusion on my part.

1)Famitrackers' (and possibly other trackers') use of the word frame. It ought to mean: one tick/update. But it also means a cell in the sequence of patterns.

I can't change the way tracker culture or software uses these terms, but in my mind a "frame" in the cell in the sequence of patterns-sense of is really just a keyframe. I've been confused more than once when there's talk of frames in nes music, so i've decided for myself to become more actively clear on calling this a keyframe. Or if i suspect there's risk of misunderstanding, a famitracker frame to denote the difference and still reference it to something immediately recognizable, if animation lingo isn't an option.

2)When animating a metatile or metasprite.
In previous conversations, i think i've made myself very much guilty of just causually saying 'frame' for keyframe, even though i think it's bad practice in communication. It leads to slightly confusing sentences like "that frame should be one frame longer". This would be the perfect application for the distinction between keyframes and (video) frames, imo.

<digression>Maybe an offhand short for keyframe could be key, rather than frame...? But that assumes all participants understand where key is coming from.
</digression>

What do you think?

"keyframe" isn't very meaningful in the NES sprites; there is generally not enough memory to spend on frames for some to have "key" importance and others to be drawn interpolations between them. Even if that weren't the case, you might think of such things when animating but there's no context beyond that once the sprites are rendered; a sprite engine doesn't have "frames" vs "keyframes". (It's an extensively used term in games with 3D animation though, where the engine is doing the interpolation.)

I don't think applying the term "keyframe" to mean a sprite held for several frames is common or particularly intuitive, but sure I guess it's a word you could co-opt. I generally just call this a "hold" or a "wait" instead, though, which I think more directly states what it is?

"Hold this sprite for 3 frames." <- I think this is about as clear as I could say this, and the three terms "hold", "sprite", and "frame" should be unambiguous with this usage.

If you're using an engine that is tracing (interpolating) paths between points to move along, those definitely can and do get called keyframes. These kind of paths are less common in NES games though. Enemies in a scrolling shmup could use animated paths like this, for example.


Famitracker's term "frame" is inherited from other trackers. It's an unrelated word. If I'm going to use that word I try to be clear about the context, but I think mostly I try to tiptoe around it because it's awkward terminology.

In the case of nes graphics, I'd prefer just the term metasprite instead of keyframe, because in animation software keyframe usually means that the other frames will be interpolated. If you mean the background you can say "palette update" or "scroll" or "nam update", depending on the case. I'm curious which case specifically led to the weird sentence in your example.

I have used "cel" for a particular set of pixels that are displayed for one or more frames. For example: "Many enemies in The Curse of Possum Hollow advance to the next cel every 8 frames based on the low bits of NMI count because their animation time variable is used for another purpose." Or "For each of Donny's cels, it stores a hitbox position, physics flags, and duration in frames."

The 64-row "frame" in a tracker is a hypermeasure.

Frame - as generated by the PPU ie update every 4 frames
Sprite/obj/bob/blob - image that is displayed as sprites ie display this sprite for 4 frames
Software Sprite - tiles or chars that are uses to replace sprites ie uses these tiles and display for 4 frames
Animation Frame - a sprite/char/tile or set of sprites/chars/tiles that is displayed for 1 or more frames to represent part of an animation. ie where in memory are the jump animation frames. There are 6 animation frames for jump which plays over 24 frames

A tracker will use Frame because that is the core time base at which everything will and can be updated upon. It will be able to update the music registers on frame intervals, where a frame may be 1/50, 1/60, 1/30 or 1/25 depending on how your code works and what system you are running on.

This is the terminology I use as well.



I'm not sure I understand your definition, maybe I'm saying the same thing, but I'd use:
Sprite - a hardware sprite
Meta-sprite or logical sprite or software sprite - in-game entity that gets rendered as one or more sprites

Sprites use hardware and are sprites. Although the NES has 8x8 I would still call a 2x3 grid of Sprites "a" Sprite, as the sprites are "bolted", which gets the odd case where the Mario Sprite uses 6 sprites. If I wanted to refer to an individual piece I would call them "Sub Sprites". So the Mario Sprite is made up of 6 "Sub Sprites". This Animation frame only needs to update 2 Sub Sprites on the Mario Sprite.

Software sprites do not use Sprite hardware and are made up of chars/tiles and may or may not be blitter accelerated. On an A500 you have B(L)OBs and Sprites, they are not the same. While all sprites on a Spectrum are software so in that context one might drop the 'Software' due to its implied nature and lack of hardware acceleration to make any distinction. While on a C64 in IK+ 2 fighters are Sprites but the 3rd fighter is built using Chars and is hence a Software Sprite. On the SNES large bosses in RPGs like Illusion of Time/Gaia tend to have most of the bosses body on a layer and only have some parts as sprites. This makes the boss both Sprites and Software Sprites.

The tracker terminology is "pattern".

I've never seen "hypermeasure" before, but there are common musical terms for this: "phrase", and slightly less frequently: "period".

I thought the "pattern" was a 64-row piece of music data, and a "frame" was one of the slots of the order table in which a 64-row piece of music data can be played. The idea is that if you reuse a "pattern" in multiple "frames", you save on music data size, which can prove important when trying to fit your soundtrack in 16K or 31K or whatever.

Yeah, in a pattern sequencer (which any standard tracker is more or less a computer software implementation of), typically any such part is a pattern, and any string of patterns is a sequence. Frame here is the time-domain order of patterns. Frames are fixed; patterns programmable. For example a pretty common figure in an 80s pop song could be

Frame 1 may contain pattern 1,
frame 2 pattern two,
f3 p1 again and
f 4: p3
then
f5: p1
f6: p2
f7: p1
f8: p4 (a variation)

<story>
The same goes for pattern sequencers used in wheft machines; except i've never seen or heard the usage of the term "frame" here, but rather "step", which makes sense. Steps in musical sequencing, on the other hand, is what trackers call rows. The longest measure in weaving or textile printing is a "report" (if direct translation would work for once, but many local terms are directly imported from the english textile industry, so i'll try my luck), which is the length before the macropattern repeats itself.
</story>

I'll get back to the rest when i have more time, so i don't rush it.

Famitracker and tracker terminology:

Yes, the "order" contains a sequence of "frames" and each frame designates a collection of "patterns" to play.

The frame doesn't have any explicit length, it inherits it from the pattern(s) it references. This is why I think "64-rows" applies to the pattern, not the frame-- the pattern contains rows, the frame contains patterns-- but this is an academic point.

Patterns are variable length, but the default is 64. There's different ways of customizing pattern lengths depending on the tracker, e.g. a global length, a per-pattern length, or a shortening effect like Dxx/Bxx in Famitracker.

In many trackers the "frame" contains only one multi-channel pattern, rather than several one-channel patterns like Famitracker.


As far as reusing patterns to save space, it works on a coarse level, but I think it's more useful just for organization of the composition. There are a lot of ways for an exporter to automate better reuse (e.g. combine matching pattern data and not just explicit reuse, try subdividing patterns to see if this allows additional reuse, alternative loop/reference methods). For example: 4klang, which has its primary goal of creating very tiny music programs, works directly from a MIDI stream and does not require manual indication of reuse.


I don't recommend using "hypermeasure" if you want musicians to understand you without looking it up. I think it's an obscure term, which is why I suggested "phrase", which means approximately the same thing in most practical circumstances. If you really want to split that hair, it probably requires more work to express than you can with already-known terms.

This is the terminology I use in my music programs :

Famitracker's double use of the word is interesting and seems to be mostly accidental. And in my opinion poorly chosen.

Frame as meaning one call to the player routine makes sense as in the context of a typical NES music routine where it is in fact called once every VBlank. But the mistake is in extrapolating that to cover playback at a different rate and for example saying "240 frames/second". Imo, the correct term when not called at frame intervals would be tick.

Frame as meaning one screen in the pattern editor is an odd one. Especially since the word pattern is used a lot throughout the manual. I guess the word pattern is used to mean one column, whereas frame is used to mean the combination of phrases for all channels. I think I would have used different terms, for example FT2's pattern/channel duality. Another choice might have been pattern/phrase.

To put it another way, the distinction between "pattern" and one of the meanings of "frame" (order table entry) parallels the distinction between a "tile" and an "OAM entry". Just as one tile may be used in multiple OAM entries, one pattern may be reused in multiple "frames".

I understand and agree with the need for different terminology for the two, but I still think "frame" was a poor choice.

I propose these:

Music

Video field: A cycle of a 60.1 Hz or 50.0 Hz* time base
CD sector: A cycle of a 75 Hz time base
Jiffy: A cycle of a multitasking operating system's process scheduler time base, from 1 to 10 ms
Tick: One update of a music engine, which may be synchronized to video fields, CD sectors, or another time base on the order 20-200 Hz
Row: A cycle of the smallest significant rhythmic time base in a piece of music, consisting of a small (usually 2-15) whole or rational number of ticks. Also called tatum.
Groove: A repeating sequence of row lengths measured in ticks, used for rational or swung tempo
Pattern: A sequence of notes that can be repeated
Order table element: The use of a pattern at a given time in a piece

Graphics

Tile: An 8x8 or 8x16 pixel* image
Metatile: A small set of tiles to be displayed together in a grid shape next to other metatiles in a coarser grid
Cel: An image in an animated sequence, displayed for a small (usually 1-32) number of video fields
Metasprite: An arbitrarily sized set of tiles to be displayed together at some relative distance from one another, representing one cel
Actor: An object that moves during gameplay and is usually drawn by adding metasprites to a sprite display list sent to the PPU every video field

Which if any are inaccurate or clunky?


* May vary based on platform

I really like the use of the word cel as you described. For a non-english person the etymology* might not be obvious, but it's the best we have, i think. Unlike keyframe, it describes any one picture in a sequence/ of pictures, not just the ones holding special meta information (in digital animation) or representing original frames distinguished from interpolated (in hand drawn/film development-mixed animation).

*celluloid painting for use in animation against a fixed or scrolling background in classic animation.


I think i have a reservation, and some additions.
"A cycle of the smallest significant rhythmic time base in a piece of music" <-- is this how time base should be used? Wouldn't that make one row a full note? Or wait, i'm confusing time base with measure, am i not?

The term 'row' seems drawn from rows and columns in text mode graphics, where a row is abs(yposinpixels/tileheight).
When speaking of a row in a tracker, i think it, just like "order table element" is a fixed measure, as it would be in an old school text editor (compare with 'lines' when speaking of code). Except rows wrap around at the pattern length boundary.

A usable phrase could be: "on tri channel, what if you moved all notes in pattern 00, 01 and 03 on row 1C to row 1E?"
Of course, this doesn't exclude it being used as a relative measure: "I think C#4 would be stronger two rows off in either direction."

When speaking of Rows and Cols in nes graphics, the above formula works. Speaking of at which row or column a line of text should appear could be practical, for example. Or when dealing with scrolling issues or attributes.


An 8x16 pixel image is two tiles, isn't it? Unless we want to differentiate tiles from characters and patterns, which seems to have been used interchangeably. That'd actually make sense to do but might cause confusion.


Doesn't roll off the tounge and isn't directly understood, IMO. Are there alternatives? Pattern tick?


It might just be me, but i didn't even understand the description despite reading it over and over.

====

That makes sense.

I'm looking for a way to distinguish... how should i put this? Ah well, let's say we've got 4 cels in a cycle running over 16 frames.
Each cel updates what ppu patterns are used. In between them, there's also position-domain animation. Metasprite goes up, then right, then down, then left one pixel. So there's two animation domains across the time domain. We could theoretically add more, like subpalette animation, even character masking...

Keys here could hold info on what to set timers and temps(or what arrays to look into) for the next alteration in the corresponding animation domain. Each time a timer reaches 0, the "step sequencer" advances one step in the appropriate domain, in a way described by arrays or maybe even some temp vars.

So the distinction is between the key that governs the total sum of animation on one hand, and the various components of animation on the other: picture, position, mask, priority, subpalette assignment, subpalette colours, what have you.

Surely you mean 59.94? And video field is a discussion in and of itself. In regular TV broadcasts, the video is encoded as 25/~30 frames per second, but 2 fields per frame. Gaming systems however often tricked TV sets into displaying progressive scan, by only sending odd or only even fields. (As I'm sure you're well aware.) What for example a SNES transmits could trivially be seen as 50/~60 FPS, whereas a regular broadcast could not.

Do you know of examples where a music engine is synced to CD sectors. (CD-DA doesn't count imo since it's streaming audio and not tempo syncing note data to that rate.)

The potential problem with this use is that it's ambiguous whether this is referring to a row in a pattern (for one channel only) or across all available channels. This would depend on the layout of the tracker in question. The word could also deliberately omit to define this.

Again, tying into the potential ambiguity depending on the tracker layout:

• In Fast Tracker II (and, I believe in Pro Tracker before it) the order table contains one object per row. Each object contains one sequence of notes, and a row spans across all channels.
• In Famitracker, the order table contains one object per channel, per row. Each object contains a sequence of notes, for only one channel.
• In LSDj, the order table (called song) contains one object per channel. This object in turn contains a second table of objects, which then contains a single sequence of notes, for one channel.

It might be difficult to find a truly universal useful nomenclature.

Shouldn't that be "usually"? That would depend on the hardware. Perhaps there are consoles with a 16x16 tile size or even weirder ones.

I think I'd prefer animation frame, despite it being a longer word.

I don't think it's that hard. Famitracker follows the (slightly loose) standard of every pattern sequencer i've ever used in this regard; be it yamaha, alesis, korg or some eurorack boutique module, and pattern sequencers are the standard (in what sense there is a standard) which some other trackers seem to have chosen (actively or by pathtaking chance) to deviate from.

Most often, they do call the totality of pattern sequences a song. (Edit: Actually; as does famitracker too.)

The small differences between pattern sequencers is often in the ui/gui (and mostly it's then about navigation, confirmation and stuff like that), but not really in the underlying structure.

In 4/4 time, each measure is a whole note. A 16th note is smaller than a whole note.


It behaves as one in the NES's 8x16 pixel sprite mode.


Say you have a tick rate matching the NTSC NES field rate (60.10 Hz), and you want a tempo of 124 quarter notes per minute, with a row equaling a sixteenth note. A tempo of 496 rows per minute implies 60.1*60/496 = close to 7.25 ticks per row. If you alternate among 8, 7, 7, and 7 ticks per sixteenth note, it averages out to 7.25 ticks per sixteenth note, or 124.3 quarter notes per minute.

Or you might set a grid of swung eighth notes by alternating between 12 and 8 or 13 and 7 ticks. This averages out to 20 ticks per quarter note or 180 quarter notes per minute, but the on-beat eighth note is longer than the off-beat one, creating a swing feel.

For more information on grooves in 0CC-FamiTracker, see 0CC-readme.txt.


Video following the NTSC standard has a field rate of 60000/1001 = about 59.94 Hz.
The NES PPU and Super NES S-PPU produce nonstandard video with a field rate of 39375000/655171 = about 60.10 Hz.


I haven't seen it used in practice, but it's a well-known time base that a CD-based system might have used for audio engine updates.


A row considered as a unit of time is independent of how many patterns happen to be advancing every row. The most prominent use case I can see for different row periods on different channels is sound effects and music playing at once.


Pro/Scream/Fast/Impulse have multi-channel patterns because that's convenient with the comparatively large memory of a 16-bit Amiga or IBM PC. By contrast, NT2, FT, and Pently have single-channel patterns because on the smaller memory of an 8-bit machine, it becomes more important to reuse e.g. bass and drum patterns across different melody patterns. The order table in NT2 and FT is coarse, aligning pattern starts along all channels; Pently's is finer, allowing a pattern to restart on any row. I haven't used LSDJ so I'm not familiar with its architecture.


Sprites on TG16 and Neo Geo are made of 16x16-pixel tiles, for instance. But I was defining these relative to the NES so as not to confuse readers with excessive generality. That's also why I used 60.1 Hz.

Sure, in more general animation editors basically everything is a timeline with keframes on it, and the animation engine interpolates between keyframes for each of them. In a context like that, it would probably be natural to call changes on the sprite timeline keyframes.

The timelines could be position, colour, velocity, gravity, blend mode, etc. whatever's useful. I've used/worked on animation editors like this. For the "sprite" timeline maybe the only available interpolator is just a hold (or perversely you might call it a "nearest neighbour" interpolation).

So I think in a context like that to provide analogy I probably would call it a keyframe. Looking at just the sprites changing over time, I wouldn't have, though.


On another note, I usually feel like "metasprite" is an awkward and unnecessary distinction from "sprite". I get that it's trying to differentiate a hardware sprite (single tile) from the overall thing, but it seems to invert what's important to me. The word "sprite" should just be the one you talk about most (the metasprite), and the hardware sprite is the more specific technical thing that should be made awkward with more attached words. So in most cases I'd say "sprite tile" for the hardware sprite where it should be disambiguated, and just "sprite" instead of metasprite.

...tons of people here seem to love typing "metasprite" out over and over, so maybe I'm a weirdo in this preference.


edit: additional note moved to post below, tokumaru intervened

The problem is that a single hardware sprite isn't always a single tile, so "sprite tile" doesn't make much sense. Even on the NES they can be 2 tiles (8x16 sprites), and the Genesis can do any combination from 1x1 to 4x4 tiles.

Heh, that's perhaps another layer of ambiguity about what "tile" means. I'd say all of those are still one "sprite tile". Maybe they're multiple "CHR tiles" though. Hey, there's also people that like to call CHR "pattern", just to mingle with the tracker terms more too.

There's an SNES case where the distinction of multiple 8x8 tiles in a single hardware sprite matters, to do with overflow, but describing that properly is going to require lots of words even in the best of times.


edit: was trying to post this above, moved down here

If they're evenly spaced that would have traditionally just been called animating "on fours". ...though on the NES I think the memory limitations make uneven timings a huge advantage in a lot of cases, so this traditional terminology becomes less useful.