CSNG (File Format): Difference between revisions

From Retro Modding Wiki
Jump to navigation Jump to search
VisualWikitext
>MrSinistar
(Thanks to jackoalan for helping crack this format even further!)
imported>Jackoalan
 
(23 intermediate revisions by 3 users not shown)
Line 1: Line 1:
The '''CSNG format''' contains MIDI data. It appears in Metroid Prime 1 and 2.  It is essentially MusyX's SON music format.
The '''CSNG format''' contains MIDI data. It appears in Metroid Prime 1 and 2.  It is essentially MusyX's GameCube SNG music format, with a custom header.  


{{research|3|Nothing is known about this format.}}
__TOC__


== Format ==
All offsets are relative to the start of the main header (after the custom header).
Overall, SNG functions similar to a Type-1 (multi-track) MIDI file, with (up to) 64 tracks
merging their events into 16 General-MIDI sequencer channels. In addition to the multi-track
structure, SNG supports storing MIDI regions as indexed ''patterns'', so songs with repetitive
refrains can save on memory by referencing patterns more than once.
Timings are represented in ''ticks''. Unlike MIDI, the tick-resolution is fixed at 384
ticks per beat (e.g. 120 beats-per-minute works out to <code>384 * 120 / 60 = 768</code> ticks-per-second).
=== Custom Header ===
This 0x14-byte header isn't part of the MusyX format; it appears at the start of the file. After parsing this the rest of the file is copied into a buffer and then passed to the MusyX functions.


{| class="wikitable"
{| class="wikitable"
Line 15: Line 30:
| 0x4
| 0x4
| 4
| 4
| '''Sequence Index'''
| '''MIDI Setup ID'''
|-
|-
| 0x8
| 0x8
| 4
| 4
| '''Voice Count'''
| '''SongGroup ID'''
|-
|-
| 0xC
| 0xC
Line 27: Line 42:
| 0x10
| 0x10
| 4
| 4
| '''File Length'''
| '''SNG File Length'''
|-
|-
| 0x14
| 0x14
| colspan=2 {{unknown|MusyX data starts}}
|}
=== Header ===
{| class="wikitable"
! Offset
! Size
! Description
|-
| 0x0
| 4
| 4
| '''Voice Header Table Offset''' (start of MusyX SON data)
| '''Track Index Offset'''; usually 0x18 for GCN games
|-
| 0x4
| 4
| '''Region Data Index Offset'''
|-
| 0x8
| 4
| '''Channel Map Offset'''
|-
| 0xC
| 4
| '''Tempo Table Offset'''; 0x0 if tempo doesn't change
|-
| 0x10
| 4
| '''Initial Tempo'''; (commonly 0x78 or 120 beats per minute)
|-
| 0x14
| 4
| {{unknown|'''Unknown'''}}
|-
|-
| 0x18
| 0x18
| colspan=2 {{unknown|End of header}}
|}
===Region Info===
The ''track index'' has offsets relating the first '''region info''' for each track.
There is a sequence of at least 2 region info structures populating each track, last one acting as terminator:
{| class="wikitable"
! Offset
! Size
! Description
|-
| 0x0
| 4
| 4
| '''Voice Sequence Offsets Table Offset'''
| '''Start Tick'''; time-point to begin executing region data
|-
| 0x4
| 4
| {{unknown|'''Unknown'''}}; commonly 0xffff0000
|-
| 0x8
| 2
| '''Region Data Index'''; used to select region data when positive; -1 value terminates track; -2 value loops jumps to ''Loop Region Index'' at this region's tick
|-
| 0xA
| 2
| '''Loop Region Index'''; index of region to jump to if ''Region Data Index'' set to -2
|-
| 0xC
| colspan=2 {{unknown|End of region info}}
|}
 
===Region Data===
 
Here begins a free-form blob of indexed region data. It starts with a variable-length
'''u32 array''' of SNG offsets for each region, then the region data itself.
 
====Region Data Header====
 
{| class="wikitable"
! Offset
! Size
! Description
|-
|-
| 0x1C
| 0x0
| 4
| 4
| '''Wave Index Table Offset'''
| '''Region Data Header Size'''; size of the header ''after'' this field (always 0x8)
|-
|-
| 0x20
| 0x4
| 4
| 4
| {{unknown|'''Unknown'''}}; (always 0...probably another offset)
| '''Pitch Wheel Data Offset'''; (absolute SNG-offset) 0x0 if no pitch-wheel messages in region
|-
|-
| 0x24
| 0x8
| 4
| 4
| '''Initial BPM Rate'''; AKA Tempo (always 0x78 = 120 beats per minute)
| '''Mod Wheel Data Offset'''; (absolute SNG-offset) 0x0 if no mod-wheel messages in region
|-
| 0xC
| colspan=2 {{unknown|End of region data}}
|}
 
====Region Commands====
 
After the region data header, the actual playback commands begin. There are only 3 types of commands
in SNG: ''note'', ''control change'', and ''program change''.
 
=====Delta Time Encoding=====
 
Just like MIDI, each command starts with a '''delta time''' value telling the sequencer
how many ticks to wait after the previous command. Unlike MIDI, Factor5 uses a fixed 16-bit
value to store the delta times. In case of a delta time greater than 65535, no-op commands
with a 65535 delta time are inserted into the command stream until the target time is attained.
 
In a theoretical Python streaming API, the delta time can be decoded with no-ops automatically consumed like so:
 
<syntaxhighlight lang="python" line="1">
def DecodeDeltaTime(streamIn):
    total = 0
    while True:
        term = streamIn.ReadU16()
        next = streamIn.PeekU16()
        if next == 0:
            # Automatically consume no-op and continue accumulating time value
            total += 0xffff
            streamIn.seekAhead(2)
            continue
        total += term
        return total
</syntaxhighlight>
 
=====No-Op Command=====
 
When the two bytes following the delta-time are both zero, this is a no-op command.
 
As mentioned before, no-ops are useful for accumulating delta times greater than 65535.
 
=====Note Command=====
 
When the two bytes following the delta-time != 0xffff, and the high-bits of both bytes are ''unset'',
this is a '''note command'''.
 
Unlike MIDI, which has separate commands for note-on/note-off, SNG attaches a ''note length'' value
to a note-on command, which is then able to track its own lifetime.
 
{| class="wikitable"
! Offset
! Size
! Description
|-
|-
|
| 0x0
|
| 1
| {{unknown|WIP}}
| '''Note'''; AND with 0x7f for the value
|-
| 0x1
| 1
| '''Velocity'''; AND with 0x7f for the value
|-
| 0x2
| 2
| '''Note Length'''; count of ticks before note-off issued by sequencer
|-
| 0x4
| colspan=2 {{unknown|End of note}}
|}
|}


=====Control Change Command=====
When the two bytes following the delta-time != 0xffff, and the high-bits of both bytes are ''set'',
this is a '''control change command'''.
See the [https://www.midi.org/specifications/item/table-3-control-change-messages-data-bytes-2 standard MIDI control numbers] for details.
{| class="wikitable"
! Offset
! Size
! Description
|-
| 0x0
| 1
| '''Value'''; AND with 0x7f for the value
|-
| 0x1
| 1
| '''Control'''; AND with 0x7f for the value
|-
| 0x2
| colspan=2 {{unknown|End of control change}}
|}
=====Program Change Command=====
When the two bytes following the delta-time != 0xffff, and the high-bit of the first byte is ''set'',
this is a '''program change command'''.
See the [https://www.midi.org/specifications/item/gm-level-1-sound-set standard MIDI program numbers] for details.
{| class="wikitable"
! Offset
! Size
! Description
|-
| 0x0
| 1
| '''Program'''; AND with 0x7f for the value
|-
| 0x1
| 1
| '''Padding'''; always zero
|-
| 0x2
| colspan=2 {{unknown|End of program change}}
|}
=====End Of Region=====
When the two bytes following the delta-time == 0xffff, this region has no more commands.
====Continuous Pitch / Modulation Data====
If the pitch or mod offsets in a region are non-zero, they point to a buffer of
(delta-tick, delta-value) pairs. The delta times are encoded as unsigned 7/15 bit values
and the delta values are signed 7/15-bit values. The values are promoted to 15-bit if they
exceed their 7-bit range. In this case, the highest bit is set on the first byte in the pair
(0x80). The decoder must track the absolute time and value, summing each consecutive update
for the current time/values.
The byte sequence 0x80 0x00 is a special case that signals the end of the stream.
Similar to the command stream, when the delta time exceeds 15 bit range (>32767), extra pairs
with a zeroed value delta are inserted.
<syntaxhighlight lang="python" line="1">
def SignExtend7(value):
    return (value & 0x3f) - (value & 0x40)
def SignExtend15(value):
    return (value & 0x3fff) - (value & 0x4000)
def DecodeSignedValue(streamIn):
    byte0 = streamIn.ReadU8()
    if byte0 & 0x80:
        byte1 = streamIn.ReadU8()
        return SignExtend15(byte1 | ((byte0 & 0x7f) << 8))
    else:
        return SignExtend7(byte0)
def DecodeUnsignedValue(streamIn):
    byte0 = streamIn.ReadU8()
    if byte0 & 0x80:
        byte1 = streamIn.ReadU8()
        return byte1 | ((byte0 & 0x7f) << 8)
    else:
        return byte0
def DecodeDeltaPair(streamIn):
    ret = [0,0]
    while ret[1] == 0:
        if streamIn.PeekU16() == 0x8000:
            # The stream has ended
            break
        ret[0] += DecodeUnsignedValue(streamIn)
        ret[1] = DecodeSignedValue(streamIn)
    return ret
</syntaxhighlight>
===Channel Map===
This is a simple '''u8 table''' mapping 64 SNG tracks to 16 MIDI channels for instrument selection via the [[AGSC (File Format)#MIDI Setup Entry|SongGroup MIDI-Setup]].
===Tempo Table===
When the SNG has a non-zero tempo table offset, this song features tempo changes.
The change events are simple absolute-tick / BPM pairs.
{| class="wikitable"
! Offset
! Size
! Description
|-
| 0x0
| 4
| '''Tick'''; absolute time-point to perform tempo change
|-
| 0x4
| 4
| '''Tempo'''; new tempo in BPM
|-
| 0x2
| colspan=2 {{unknown|End of tempo change}}
|}


[[Category:File Formats]]
[[Category:File Formats]]
[[Category:Metroid Prime]]
[[Category:Metroid Prime]]
[[Category:Metroid Prime 2: Echoes]]
[[Category:Metroid Prime 2: Echoes]]

Latest revision as of 05:32, 4 September 2018

The CSNG format contains MIDI data. It appears in Metroid Prime 1 and 2. It is essentially MusyX's GameCube SNG music format, with a custom header.

Format

All offsets are relative to the start of the main header (after the custom header).

Overall, SNG functions similar to a Type-1 (multi-track) MIDI file, with (up to) 64 tracks merging their events into 16 General-MIDI sequencer channels. In addition to the multi-track structure, SNG supports storing MIDI regions as indexed patterns, so songs with repetitive refrains can save on memory by referencing patterns more than once.

Timings are represented in ticks. Unlike MIDI, the tick-resolution is fixed at 384 ticks per beat (e.g. 120 beats-per-minute works out to 384 * 120 / 60 = 768 ticks-per-second).

Custom Header

This 0x14-byte header isn't part of the MusyX format; it appears at the start of the file. After parsing this the rest of the file is copied into a buffer and then passed to the MusyX functions.

Offset Size Description
0x0 4 Magic; (always 0x2)
0x4 4 MIDI Setup ID
0x8 4 SongGroup ID
0xC 4 AGSC ID
0x10 4 SNG File Length
0x14 MusyX data starts

Header

Offset Size Description
0x0 4 Track Index Offset; usually 0x18 for GCN games
0x4 4 Region Data Index Offset
0x8 4 Channel Map Offset
0xC 4 Tempo Table Offset; 0x0 if tempo doesn't change
0x10 4 Initial Tempo; (commonly 0x78 or 120 beats per minute)
0x14 4 Unknown
0x18 End of header

Region Info

The track index has offsets relating the first region info for each track.

There is a sequence of at least 2 region info structures populating each track, last one acting as terminator:

Offset Size Description
0x0 4 Start Tick; time-point to begin executing region data
0x4 4 Unknown; commonly 0xffff0000
0x8 2 Region Data Index; used to select region data when positive; -1 value terminates track; -2 value loops jumps to Loop Region Index at this region's tick
0xA 2 Loop Region Index; index of region to jump to if Region Data Index set to -2
0xC End of region info

Region Data

Here begins a free-form blob of indexed region data. It starts with a variable-length u32 array of SNG offsets for each region, then the region data itself.

Region Data Header

Offset Size Description
0x0 4 Region Data Header Size; size of the header after this field (always 0x8)
0x4 4 Pitch Wheel Data Offset; (absolute SNG-offset) 0x0 if no pitch-wheel messages in region
0x8 4 Mod Wheel Data Offset; (absolute SNG-offset) 0x0 if no mod-wheel messages in region
0xC End of region data

Region Commands

After the region data header, the actual playback commands begin. There are only 3 types of commands in SNG: note, control change, and program change.

Delta Time Encoding

Just like MIDI, each command starts with a delta time value telling the sequencer how many ticks to wait after the previous command. Unlike MIDI, Factor5 uses a fixed 16-bit value to store the delta times. In case of a delta time greater than 65535, no-op commands with a 65535 delta time are inserted into the command stream until the target time is attained.

In a theoretical Python streaming API, the delta time can be decoded with no-ops automatically consumed like so: 

def DecodeDeltaTime(streamIn):
    total = 0
    while True:
        term = streamIn.ReadU16()
        next = streamIn.PeekU16()
        if next == 0:
            # Automatically consume no-op and continue accumulating time value
            total += 0xffff
            streamIn.seekAhead(2)
            continue
        total += term
        return total
No-Op Command

When the two bytes following the delta-time are both zero, this is a no-op command.

As mentioned before, no-ops are useful for accumulating delta times greater than 65535.

Note Command

When the two bytes following the delta-time != 0xffff, and the high-bits of both bytes are unset, this is a note command.

Unlike MIDI, which has separate commands for note-on/note-off, SNG attaches a note length value to a note-on command, which is then able to track its own lifetime.

Offset Size Description
0x0 1 Note; AND with 0x7f for the value
0x1 1 Velocity; AND with 0x7f for the value
0x2 2 Note Length; count of ticks before note-off issued by sequencer
0x4 End of note
Control Change Command

When the two bytes following the delta-time != 0xffff, and the high-bits of both bytes are set, this is a control change command.

See the standard MIDI control numbers for details.

Offset Size Description
0x0 1 Value; AND with 0x7f for the value
0x1 1 Control; AND with 0x7f for the value
0x2 End of control change
Program Change Command

When the two bytes following the delta-time != 0xffff, and the high-bit of the first byte is set, this is a program change command.

See the standard MIDI program numbers for details.

Offset Size Description
0x0 1 Program; AND with 0x7f for the value
0x1 1 Padding; always zero
0x2 End of program change
End Of Region

When the two bytes following the delta-time == 0xffff, this region has no more commands.

Continuous Pitch / Modulation Data

If the pitch or mod offsets in a region are non-zero, they point to a buffer of (delta-tick, delta-value) pairs. The delta times are encoded as unsigned 7/15 bit values and the delta values are signed 7/15-bit values. The values are promoted to 15-bit if they exceed their 7-bit range. In this case, the highest bit is set on the first byte in the pair (0x80). The decoder must track the absolute time and value, summing each consecutive update for the current time/values.

The byte sequence 0x80 0x00 is a special case that signals the end of the stream.

Similar to the command stream, when the delta time exceeds 15 bit range (>32767), extra pairs with a zeroed value delta are inserted. 

def SignExtend7(value):
    return (value & 0x3f) - (value & 0x40)

def SignExtend15(value):
    return (value & 0x3fff) - (value & 0x4000)

def DecodeSignedValue(streamIn):
    byte0 = streamIn.ReadU8()
    if byte0 & 0x80:
        byte1 = streamIn.ReadU8()
        return SignExtend15(byte1 | ((byte0 & 0x7f) << 8))
    else:
        return SignExtend7(byte0)

def DecodeUnsignedValue(streamIn):
    byte0 = streamIn.ReadU8()
    if byte0 & 0x80:
        byte1 = streamIn.ReadU8()
        return byte1 | ((byte0 & 0x7f) << 8)
    else:
        return byte0

def DecodeDeltaPair(streamIn):
    ret = [0,0]
    while ret[1] == 0:
        if streamIn.PeekU16() == 0x8000:
            # The stream has ended
            break
        ret[0] += DecodeUnsignedValue(streamIn)
        ret[1] = DecodeSignedValue(streamIn)
    return ret

Channel Map

This is a simple u8 table mapping 64 SNG tracks to 16 MIDI channels for instrument selection via the SongGroup MIDI-Setup.

Tempo Table

When the SNG has a non-zero tempo table offset, this song features tempo changes. The change events are simple absolute-tick / BPM pairs.

Offset Size Description
0x0 4 Tick; absolute time-point to perform tempo change
0x4 4 Tempo; new tempo in BPM
0x2 End of tempo change
Retrieved from "https://wiki.axiodl.com/index.php?title=CSNG_(File_Format)&oldid=2496"