Geometry (Metroid Prime): Difference between revisions

From Retro Modding Wiki
Jump to navigation Jump to search
VisualWikitext
imported>Antidote
mNo edit summary
(Surface display list size)
 
(33 intermediate revisions by 5 users not shown)
Line 1: Line 1:
The format for '''geometry''' appears in both the [[CMDL (File Format)|CMDL]] and [[MREA (File Format)|MREA]] formats, and is identical between both. This format is used in all three Prime games, and is largely unchanged between all of them (with the exception of an extra vertex attribute that was added in Echoes and Corruption).
The format for '''geometry''' appears in both the [[CMDL (File Format)|CMDL]] and [[MREA (File Format)|MREA]] formats, and is identical between both. This format is used in all three Prime games, and is largely unchanged between all of them (with the exception of an extra vertex attribute that was added in Echoes and Corruption).


{{researchminor|reason=There's a couple unknowns in the submesh headers.}}
{{research|1|There's two unknowns in the surface headers that appear in Prime 2.}}


__TOC__
__TOC__
Line 7: Line 7:
== Format ==
== Format ==


An important thing to note off the bat is that both CMDL and MREA files are split up into a number of 32-byte-aligned ''sections''; their respective headers both declare a section count, and list the sizes of each section contained in the file. This is required to navigate to different parts of the file, and is required in order to read the geometry data correctly. Sections are generally split based on the type of data contained in them, so one section might contain vertex coordinates, while another contains a submesh definition. For the purposes of this article, each subheader will mark the start of a new section; use that as your cue to advance to the next one. Check the CMDL or MREA pages for more info on how the sections work.
An important thing to note off the bat is that both CMDL and MREA files are split up into a number of 32-byte-aligned ''sections''; their respective headers both declare a section count, and list the sizes of each section contained in the file. This is required to navigate to different parts of the file, and is required in order to read the geometry data correctly. Sections are generally split based on the type of data contained in them, so one section might contain vertex coordinates, while another contains a surface (submesh) definition. For the purposes of this article, each subheader will mark the start of a new section; use that as your cue to advance to the next one. Check the CMDL or MREA pages for more info on how the sections work.


For vertex attributes, note that there's no count value present anywhere. If you want a rough count, you can divide the size of that attribute's section by the attribute's size.
In the CMDL format, and in the MP1/2 MREA format, all data sections appear in the order that they are listed in this page. In the Prime 3/DKCR MREA format, the surface definitions, surface group IDs, and surface lookup table are shifted to a different part of the file.
 
For vertex attributes, note that there's no count value present anywhere. If you want a rough count, you can divide the size of that attribute's section by the attribute's size; this approach doesn't account for padding, however.


=== Vertex Coordinates ===
=== Vertex Coordinates ===
Line 17: Line 19:
=== Normals ===
=== Normals ===


Normals can appear as either floats or shorts. In CMDL, this usually corresponds to the 0x2 bit in the flags value, and in MREA they will always appear as shorts; this is technically supposed to be set by the vertex format flag in the primitive data, though. If they appear as shorts, they should be divided by the mantissa value that appears on each submesh (that value is almost always 0x8000).
Normals can appear as either floats or shorts. In CMDL, this usually corresponds to the 0x2 bit in the flags value, and in MREA they will always appear as shorts; this is technically supposed to be set by the vertex format flag in the primitive data, though. If they appear as shorts, they should be divided by the mantissa value that appears on each surface (that value is always 0x8000).


=== Vertex Color ===
=== Vertex Color ===


These are theoretically stored as 32-bit RGBA values, but vertex color isn't actually used by any mesh in any Retro games, so this section usually appears completely empty.
These are theoretically stored as 32-bit RGBA values, but vertex color isn't actually used by any mesh in any Retro games, so this section always appears completely empty.


=== Float UV Coordinates ===
=== Float UV Coordinates ===
Line 29: Line 31:
=== Short UV Coordinates ===
=== Short UV Coordinates ===


These are stored as a sequence of two shorts. Like normals, these should be divided by the submesh's mantissa value. The short UV array is generally only used by terrain lightmaps, and as such this section always appears completely empty (listed with a size of 0) on CMDL. This array can only be used by the first UV coordinate on a given vertex, and whether it's used or not depends on one of the flags on the material. The rest will use the float array.
These are stored as a sequence of two shorts. Like normals, these should be divided by the surface's mantissa value. The short UV array is generally only used by terrain lightmaps, and as such this section always appears completely empty (listed with a size of 0) on CMDL. This array can only be used by the first UV coordinate on a given vertex, and whether it's used or not depends on one of the flags on the material. The rest will use the float array.


=== Submesh Definitions ===
=== Surface Offsets ===


This section is very small and simple; it simply declares a submesh count and then lists the offsets to the end of each submesh. Following this section, there'll be one additional section per submesh.
This section is very small and simple; it simply declares a surface count and then lists the offsets to the end of each surface. Following this section, there'll be one additional section per surface.


{| class="wikitable"
{| class="wikitable"
! Offset
! Offset
! Size
! Type
! Description
! Count
! Name
! Notes
|-
|-
| 0x0
| 0x0
| 4
| u32
| '''Submesh count''' (SC)
| 1
| '''Surface Count'''
|
|-
|-
| 0x4
| 0x4
| 4 × SC
| u32
| '''Submesh offsets''' (relative to the start of the first submesh)
| ''Surface Count''
| '''Surface End Offsets'''
| Relative to the start address of the first surface.
|-
|-
| colspan=3 | Submesh sections begin
| colspan=5 {{unknown|Surface sections begin}}
|}
|}


=== Submesh ===
=== Surface ===


There will be one of these sections per submesh. Each submesh starts with a small header; following the end of the header, the file is padded with 0s to the next 32-byte boundary, then the primitive data begins.
There will be one of these sections per surface. Each surface starts with a small header; following the end of the header, the file is padded with 0s to the next 32-byte boundary, then the primitive data begins.


{| class="wikitable"
{| class="wikitable"
! Offset
! Offset
! Size
! Type
! Description
! Count
! Name
! Notes
|-
|-
| 0x0
| 0x0
| 0xC
| float
| '''Position''' (or pivot location; unknown purpose)
| 3
| '''Center Point'''
| Used for depth sorting on CMDL and used as the eye position for reflections
|-
|-
| 0xC
| 0xC
| 4
| u32
| '''Material ID'''
| 1
| '''Material Index'''
|
|-
|-
| 0x10
| 0x10
| 2
|u32
| '''Mantissa''' - divide short normals/UVs by this value; always 0x8000
|1
|-
|'''Display List Size and Normal Hint'''
| 0x12
|The top bit contains an unused "Normal Hint" while the remaining bits are used to specify the display list size.
| 2
| '''Primitive table size''' (important note: ''this value is not always reliable'')
|-
|-
| 0x14
| 0x14
| 4
| u32
| Unknown - always 0?
| 1
| '''Parent Model Pointer Storage'''
| Always 0; the game uses this space at runtime to store a pointer to the parent CCubeModel instance
|-
|-
| 0x18
| 0x18
| 4
| u32
| Unknown - always 0?
| 1
| '''Next Surface Pointer Storage'''
| Always 0; the game uses this space at runtime to store a pointer to the next surface
|-
|-
| 0x1C
| 0x1C
| 4
| u32
| '''Extra data size''' (ES); this isn't 100%, but seems to be correct
| 1
| '''Extra Data Size'''
| Amount of space taken up by extra data. Always 0 on CMDLs, 0x18 on MREAs.
|-
|-
| 0x20
| 0x20
| 0xC
| float
| A normalized vector; unknown purpose
| 3
| '''Surface Normal'''
| On materials that use the Samus reflection, this is used to project the reflection onto the surface
|-
|-
| 0x2C
| 0x2C
| ES
| u16
| Extra data; usually empty on CMDL, contains a bounding box used for depth sorting on MREA
| 1
| {{unknown|Unknown}}
| Not in Prime 1.
|-
|-
| colspan=3 | End of submesh header; pad to multiple of 32 before primitive data starts
| 0x2E
| u16
| 1
| {{unknown|Unknown}}
| Not in Prime 1.
|-
| 0x30
| u8
| ''Extra Data Size''
| '''Extra Data'''
| Empty on CMDL; contains a bounding box used for depth sorting on MREA.
|-
| colspan=5 {{unknown|End of surface header; pad to multiple of 32 before primitive data starts}}
|}
|}


The primitive data is a standard GX display list. Each primitive begins with a byte that contains the primitive type in the upper 5 bits, and the vertex format setting in the lower 3; see below for possible settings. After that is a 16-bit vertex count, followed by a series of 16-bit vertex attribute indices.
The primitive data is a GX display list. Each primitive begins with a byte that contains the primitive type in the upper 5 bits, and the vertex format setting in the lower 3; see below for possible settings. After that is a 16-bit vertex count, followed by a series of 16-bit vertex attribute indices.


Note that you'll need to check some settings on the material to read the primitive data properly; the material determines which vertex attributes are present. The material also determines whether the submesh is using lightmaps; if it is, then the first UV coordinate will be read out of the short UV array instead of the float one.
Note that you'll need to check some settings on the material to read the primitive data properly; the material determines which vertex attributes are present. The material also determines whether the surface is using lightmaps; if it is, then the first UV coordinate will be read out of the short UV array instead of the float one.


There is no primitive count value; you'll need to continue reading data until you hit the end of the primitive table. There are a couple values you can use for reference; either the submesh's section size, or the submesh's end offset. '''Do not use the primitive table size setting from the submesh header, though'''; it's usually accurate, but on some models it will give you a size smaller than the table actually is, and you'll miss a bunch of data. See Metroid3/4a11192a.CMDL from Prime 1 (icicle Ridley) or Metroid1/05778239.CMDL from Echoes (Samus's gunship) for an example of this.
There is no primitive count value; you'll need to continue reading data until you hit the end of the display list. There are a couple values you can use for reference; either the surface's section size, or the surface's end offset. It was originally believed that the display list size was incorrect in a very small number of cases, however it's since been determined that the display list size '''is''' accurate is actually a u32 with the top bit being used as a "Normal Hint", the name of the bit was determined from symbols, but it's actual use has not been determined. Masking the field with 0x7FFFFFFF gives you the correct display list size that works in all cases.


There are 7 primitive types supported by GX, indicated in the upper 5 bits of the flag value. Note that the game only ever actually uses triangles, triangle strips, and triangle fans; however, all of these primitives are supported by GX and are therefore supported by the game, and so they could be used in custom model files.
There are 7 primitive types supported by GX, indicated in the upper 5 bits of the flag value. Note that the game only ever actually uses triangles, triangle strips, and triangle fans; however, all of these primitives are supported by GX and are therefore supported by the game, and so they could be used in custom model files.
Line 118: Line 152:
|-
|-
| 0x98
| 0x98
| Triangle strip
| Triangle Strip
|-
|-
| 0xA0
| 0xA0
| Triangle fan
| Triangle Fan
|-
|-
| 0xA8
| 0xA8
Line 127: Line 161:
|-
|-
| 0xB0
| 0xB0
| Line strip
| Line Strip
|-
|-
| 0xB8
| 0xB8
Line 151: Line 185:
| GX_NRM_XYZ / GX_S16
| GX_NRM_XYZ / GX_S16
| GX_TEX_ST / GX_S16
| GX_TEX_ST / GX_S16
|}
=== Surface Group IDs ===
This section (and the following one) appears in [[MREA (File Format)|MREA files]] starting in Metroid Prime 2. Starting in MP2, surfaces in world geometry are now merged together to create as few meshes as possible (meshes are now only split when the vertex count exceeds 65535). Two extra data sections are present to manage the resulting super-meshes to enable the game to extract smaller meshes out when needed.
This first section assigns a group ID to each surface, forming smaller meshes. This ID is used in various places that operate on world geometry, including in the [[AROT (MREA Section)|area octree]] as well as in the [[EGMC (File Format)|EGMC format]], among others.
{| class="wikitable"
! Type
! Count
! Name
! Notes
|-
| u16
| 1
| '''Surface Count'''
| Matches the world model surface count
|-
| [[#Surface Group ID|Surface Group ID]]
| ''Surface Count''
| '''Surface Group ID Array'''
| Sequentially assigns a surface group ID to every surface in the mesh.
|}
==== Surface Group ID ====
{| class="wikitable"
! Offset
! Type
! Name
! Notes
|-
| 0x0
| u16
| '''Model-Relative Surface Group ID'''
| This is the ID of the group this surface belongs to, relative to the groups in this model (starting from 0).
|-
| 0x2
| u16
| '''Surface Group ID'''
| Same ID, except area-relative (starting where the previous world model's group IDs left off). This is generally the ID that appears in other places to reference this group.
|}
=== Surface Lookup Table ===
This is the second new [[MREA (File Format)|MREA]]-exclusive section introduced in Metroid Prime 2. It provides a mapping of each surface group to the surfaces contained in it, so that surfaces can be quickly accessed by group ID.
{| class="wikitable"
! Type
! Count
! Name
! Notes
|-
| u16
| 1
| '''Surface Group Count'''
|
|-
| u16
| ''Surface Group Count''
| '''Lookup Table Index Array'''
| The value of each element corresponds to the end index of the surfaces for this group in the lookup table. Each element is equal to the previous element + the number of surfaces contained in this group.
|-
| u16
| ''Surface Count''
| '''Surface Lookup Table'''
| List of surface indices ordered by group ID.
|}
|}


Latest revision as of 06:45, 22 October 2024

The format for geometry appears in both the CMDL and MREA formats, and is identical between both. This format is used in all three Prime games, and is largely unchanged between all of them (with the exception of an extra vertex attribute that was added in Echoes and Corruption).


This file format is almost completely documented
There's two unknowns in the surface headers that appear in Prime 2.


Format

An important thing to note off the bat is that both CMDL and MREA files are split up into a number of 32-byte-aligned sections; their respective headers both declare a section count, and list the sizes of each section contained in the file. This is required to navigate to different parts of the file, and is required in order to read the geometry data correctly. Sections are generally split based on the type of data contained in them, so one section might contain vertex coordinates, while another contains a surface (submesh) definition. For the purposes of this article, each subheader will mark the start of a new section; use that as your cue to advance to the next one. Check the CMDL or MREA pages for more info on how the sections work.

In the CMDL format, and in the MP1/2 MREA format, all data sections appear in the order that they are listed in this page. In the Prime 3/DKCR MREA format, the surface definitions, surface group IDs, and surface lookup table are shifted to a different part of the file.

For vertex attributes, note that there's no count value present anywhere. If you want a rough count, you can divide the size of that attribute's section by the attribute's size; this approach doesn't account for padding, however.

Vertex Coordinates

These are stored as a sequence of three floats.

Normals

Normals can appear as either floats or shorts. In CMDL, this usually corresponds to the 0x2 bit in the flags value, and in MREA they will always appear as shorts; this is technically supposed to be set by the vertex format flag in the primitive data, though. If they appear as shorts, they should be divided by the mantissa value that appears on each surface (that value is always 0x8000).

Vertex Color

These are theoretically stored as 32-bit RGBA values, but vertex color isn't actually used by any mesh in any Retro games, so this section always appears completely empty.

Float UV Coordinates

These are stored as a sequence of two floats.

Short UV Coordinates

These are stored as a sequence of two shorts. Like normals, these should be divided by the surface's mantissa value. The short UV array is generally only used by terrain lightmaps, and as such this section always appears completely empty (listed with a size of 0) on CMDL. This array can only be used by the first UV coordinate on a given vertex, and whether it's used or not depends on one of the flags on the material. The rest will use the float array.

Surface Offsets

This section is very small and simple; it simply declares a surface count and then lists the offsets to the end of each surface. Following this section, there'll be one additional section per surface.

Offset Type Count Name Notes
0x0 u32 1 Surface Count
0x4 u32 Surface Count Surface End Offsets Relative to the start address of the first surface.
Surface sections begin

Surface

There will be one of these sections per surface. Each surface starts with a small header; following the end of the header, the file is padded with 0s to the next 32-byte boundary, then the primitive data begins.

Offset Type Count Name Notes
0x0 float 3 Center Point Used for depth sorting on CMDL and used as the eye position for reflections
0xC u32 1 Material Index
0x10 u32 1 Display List Size and Normal Hint The top bit contains an unused "Normal Hint" while the remaining bits are used to specify the display list size.
0x14 u32 1 Parent Model Pointer Storage Always 0; the game uses this space at runtime to store a pointer to the parent CCubeModel instance
0x18 u32 1 Next Surface Pointer Storage Always 0; the game uses this space at runtime to store a pointer to the next surface
0x1C u32 1 Extra Data Size Amount of space taken up by extra data. Always 0 on CMDLs, 0x18 on MREAs.
0x20 float 3 Surface Normal On materials that use the Samus reflection, this is used to project the reflection onto the surface
0x2C u16 1 Unknown Not in Prime 1.
0x2E u16 1 Unknown Not in Prime 1.
0x30 u8 Extra Data Size Extra Data Empty on CMDL; contains a bounding box used for depth sorting on MREA.
End of surface header; pad to multiple of 32 before primitive data starts

The primitive data is a GX display list. Each primitive begins with a byte that contains the primitive type in the upper 5 bits, and the vertex format setting in the lower 3; see below for possible settings. After that is a 16-bit vertex count, followed by a series of 16-bit vertex attribute indices.

Note that you'll need to check some settings on the material to read the primitive data properly; the material determines which vertex attributes are present. The material also determines whether the surface is using lightmaps; if it is, then the first UV coordinate will be read out of the short UV array instead of the float one.

There is no primitive count value; you'll need to continue reading data until you hit the end of the display list. There are a couple values you can use for reference; either the surface's section size, or the surface's end offset. It was originally believed that the display list size was incorrect in a very small number of cases, however it's since been determined that the display list size is accurate is actually a u32 with the top bit being used as a "Normal Hint", the name of the bit was determined from symbols, but it's actual use has not been determined. Masking the field with 0x7FFFFFFF gives you the correct display list size that works in all cases.

There are 7 primitive types supported by GX, indicated in the upper 5 bits of the flag value. Note that the game only ever actually uses triangles, triangle strips, and triangle fans; however, all of these primitives are supported by GX and are therefore supported by the game, and so they could be used in custom model files.

Value Primitive
0x80 Quads
0x90 Triangles
0x98 Triangle Strip
0xA0 Triangle Fan
0xA8 Lines
0xB0 Line Strip
0xB8 Points

In addition, the game uses three vertex formats, indicated in the lower 3 bits of the flag value (0x7). The only attributes that change between formats are nrm and tex0, so those are the only ones listed in the table below.

Format GX_VA_NRM type/size GX_VA_TEX0 type/size
0 GX_NRM_XYZ / GX_F32 GX_TEX_ST / GX_F32
1 GX_NRM_XYZ / GX_S16 GX_TEX_ST / GX_F32
2 GX_NRM_XYZ / GX_S16 GX_TEX_ST / GX_S16

Surface Group IDs

This section (and the following one) appears in MREA files starting in Metroid Prime 2. Starting in MP2, surfaces in world geometry are now merged together to create as few meshes as possible (meshes are now only split when the vertex count exceeds 65535). Two extra data sections are present to manage the resulting super-meshes to enable the game to extract smaller meshes out when needed.

This first section assigns a group ID to each surface, forming smaller meshes. This ID is used in various places that operate on world geometry, including in the area octree as well as in the EGMC format, among others.

Type Count Name Notes
u16 1 Surface Count Matches the world model surface count
Surface Group ID Surface Count Surface Group ID Array Sequentially assigns a surface group ID to every surface in the mesh.

Surface Group ID

Offset Type Name Notes
0x0 u16 Model-Relative Surface Group ID This is the ID of the group this surface belongs to, relative to the groups in this model (starting from 0).
0x2 u16 Surface Group ID Same ID, except area-relative (starting where the previous world model's group IDs left off). This is generally the ID that appears in other places to reference this group.

Surface Lookup Table

This is the second new MREA-exclusive section introduced in Metroid Prime 2. It provides a mapping of each surface group to the surfaces contained in it, so that surfaces can be quickly accessed by group ID.

Type Count Name Notes
u16 1 Surface Group Count
u16 Surface Group Count Lookup Table Index Array The value of each element corresponds to the end index of the surfaces for this group in the lookup table. Each element is equal to the previous element + the number of surfaces contained in this group.
u16 Surface Count Surface Lookup Table List of surface indices ordered by group ID.
Retrieved from "https://wiki.axiodl.com/index.php?title=Geometry_(Metroid_Prime)&oldid=2828"