CSNG (File Format): Difference between revisions

From Retro Modding Wiki
Jump to navigation Jump to search
imported>Jackoalan
imported>Jackoalan
No edit summary
Line 1: Line 1:
The '''CSNG format''' contains MIDI data. It appears in Metroid Prime 1 and 2.  It is essentially MusyX's SON music format, with a custom header.  
The '''CSNG format''' contains MIDI data. It appears in Metroid Prime 1 and 2.  It is essentially MusyX's SON music format, with a custom header.  
{{research|3|Nothing is known about this format.}}


__TOC__
__TOC__
Line 7: Line 5:
== Format ==
== Format ==


All offsets are relative to the start of the main header (after the custom header).
All offsets are relative to the start of the main header (after the custom header).
 
Timings are represented in ''ticks'', like MIDI. Unlike MIDI, the tick-resolution is fixed at 384
ticks per beat (e.g. 120 beats-per-minute works out to <code>384 * 120 / 60 = 768</code> ticks-per-second).


=== Custom Header ===
=== Custom Header ===
Line 55: Line 56:
| 0x4
| 0x4
| 4
| 4
| '''Channel Index Offset'''
| '''Track Index Offset'''; (absolute SON-offset)
|-
|-
| 0x8
| 0x8
| 4
| 4
| '''Channel Map Offset'''
| '''Channel Map Offset'''; (absolute SON-offset)
|-
|-
| 0xC
| 0xC
| 4
| 4
| '''Tempo Table Offset''', 0x0 if tempo doesn't change
| '''Tempo Table Offset'''; (absolute SON-offset) 0x0 if tempo doesn't change
|-
|-
| 0x10
| 0x10
Line 75: Line 76:
| 0x18
| 0x18
| 256
| 256
| '''Channel Offsets'''; 64 elements
| '''Track Header Offsets'''; (absolute SON-offsets) 64 elements, 0x0 if track not present
|-
|-
| 0x118
| 0x118
Line 81: Line 82:
|}
|}


==Data==
===Track Header===
 
This is a variable-length table of headers for each track
 
{| class="wikitable"
! Offset
! Size
! Description
|-
| 0x0
| 4
| '''Start Tick'''; time-point to begin executing track data
|-
| 0x4
| 4
| {{unknown|'''Unknown'''}}; commonly 0xffff0000
|-
| 0x8
| 2
| '''Track Data Index'''
|-
| 0xA
| 2
| '''Padding'''
|-
| 0xC
| 4
| '''Start Tick'''; copy of start tick
|-
| 0x10
| 4
| {{unknown|'''Unknown'''}}; commonly 0xffff0000
|-
| 0x14
| 4
| {{unknown|'''Unknown'''}}; commonly 0xffff0000
|-
| 0x18
| colspan=2 {{unknown|End of header}}
|}
 
===Track Data===


The data in the files is very similar to MIDI, however it is formatted by word, rather than MIDI's byte formatting.
Here begins a free-form blob of indexed track data. It starts with a variable-length
'''u32 array''' of SON offsets for each track, then the track data itself.


After the data for the instruments are defined, there is a table of offsets for data through out the SON data.  Each chunk of data starts with a long 0x8, usually followed by another absolute offset that points to the end of the chunk.
====Track Data Header====


Notes durations are controlled by 96 ticks per beat.
{| class="wikitable"
! Offset
! Size
! Description
|-
| 0x0
| 4
| '''Track Data Header Size'''; size of the header ''after'' this field (always 0x8)
|-
| 0x4
| 4
| '''Pitch Wheel Data Offset'''; (absolute SON-offset) 0x0 if no pitch-wheel messages on track
|-
| 0x8
| 4
| '''Mod Wheel Data Offset'''; (absolute SON-offset) 0x0 if no mod-wheel messages on track
|-
| 0xC
| colspan=2 {{unknown|End of header}}
|}
 
====Track Commands====
 
After the track data header, the actual playback commands begin. There are only 2 types of commands
in SON: ''note'' and ''control change''.
 
=====Delta Time RLE=====
 
Just like MIDI, each command starts with a '''delta time''' value telling the sequencer
how many ticks to wait after the previous command. Unlike MIDI, Factor5 uses a custom
[[wikipedia:Run-length encoding|RLE scheme]] to adaptively scale the value's precision
to reduce the value's size.
 
The RLE operates on 16-bit words, with the value 0xffff triggering a continuation,
then a 'dead' 16-bit word skipped over, then the 0xffff is summed with the following RLE value,
looping the decode algorithm.
 
In Python, decoding works like so:
 
<syntaxhighlight lang="python" line="1">
def DecodeDeltaTimeRLE(in):
    total = 0
    while True:
        term = in.ReadU16()
        if term == 0xffff:
            total += 0xffff
            dummy = in.ReadU16()
            continue
        total += term
        return total
</syntaxhighlight>
 
=====Note Command=====
 
When the two bytes following the delta-time != 0xffff, and the high-bit of the first byte is ''set'',
this is a '''note command'''.
 
Unlike MIDI, which has separate commands for note-on/note-off, SON attaches a ''note length'' value
to a note-on command, which is then able to track its own lifetime.
 
{| class="wikitable"
! Offset
! Size
! Description
|-
| 0x0
| 1
| '''Note'''; AND with 0x7f for the value
|-
| 0x1
| 1
| '''Velocity'''; AND with 0x7f for the value
|-
| 0x2
| 2
| '''Note Length'''; count of ticks before note-off issued by sequencer
|-
| 0x4
| colspan=2 {{unknown|End of note}}
|}
 
=====Control Change Command=====
 
When the two bytes following the delta-time != 0xffff, and the high-bit of the first byte is ''unset'',
this is a '''control change command'''.
 
{| class="wikitable"
! Offset
! Size
! Description
|-
| 0x0
| 1
| '''Value'''; AND with 0x7f for the value
|-
| 0x1
| 1
| '''Control'''; AND with 0x7f for the value
|-
| 0x2
| colspan=2 {{unknown|End of control change}}
|}
 
=====End Of Track=====
 
When the two bytes following the delta-time == 0xffff, this track has no more commands.
 
====Continuous Pitch / Modulation Data====
 
If the pitch or mod offsets in a track are non-zero, they point to a buffer of RLE-compressed
(delta-tick, delta-value) pairs, decoding to signed 16-bit precision. The decoder must track
the absolute time and value, summing each consecutive update for the current time/values.
 
The algorithm for this RLE is different than the delta-time one for commands. It may
scale down to a single byte if able.
 
<syntaxhighlight lang="python" line="1">
def DecodeRLE(in):
    term = in.ReadU8()
    total = term & 0x7f
    if term & 0x80:
        total *= 256 + in.ReadU8()
    return total
 
def DecodeContinuousRLE(in):
    total = 0
    while True:
        term = DecodeRLE(in)
        if term == 0x8000:
            total += 0xffff
            dummy = in.ReadU8()
            continue
        total += term
 
        if total >= 0x4000:
            return total - 0xffff
        else:
            return total
</syntaxhighlight>
 
===Channel Map===
 
This is a simple '''u8 table''' mapping 64 SON tracks to 16 MIDI channels for instrument selection via the [[AGSC (File Format)#MIDI Setup Entry|SongGroup MIDI-Setup]].
 
===Tempo Table===
 
When the SON has a non-zero tempo table offset, this song features tempo changes.
The change events are simple absolute-tick / BPM pairs.
 
{| class="wikitable"
! Offset
! Size
! Description
|-
| 0x0
| 4
| '''Tick'''; absolute time-point to perform tempo change
|-
| 0x4
| 4
| '''Tempo'''; new tempo in BPM
|-
| 0x2
| colspan=2 {{unknown|End of tempo change}}
|}


[[Category:File Formats]]
[[Category:File Formats]]
[[Category:Metroid Prime]]
[[Category:Metroid Prime]]
[[Category:Metroid Prime 2: Echoes]]
[[Category:Metroid Prime 2: Echoes]]

Revision as of 20:35, 19 May 2016

The CSNG format contains MIDI data. It appears in Metroid Prime 1 and 2. It is essentially MusyX's SON music format, with a custom header.

Format

All offsets are relative to the start of the main header (after the custom header).

Timings are represented in ticks, like MIDI. Unlike MIDI, the tick-resolution is fixed at 384 ticks per beat (e.g. 120 beats-per-minute works out to 384 * 120 / 60 = 768 ticks-per-second).

Custom Header

This 0x14-byte header isn't part of the MusyX format; it appears at the start of the file. After parsing this the rest of the file is copied into a buffer and then passed to the MusyX functions.

Offset Size Description
0x0 4 Magic; (always 0x2)
0x4 4 MIDI Setup ID
0x8 4 SongGroup ID
0xC 4 AGSC ID
0x10 4 SON File Length
0x14 MusyX data starts

Header

Offset Size Description
0x0 4 Version; always 0x18
0x4 4 Track Index Offset; (absolute SON-offset)
0x8 4 Channel Map Offset; (absolute SON-offset)
0xC 4 Tempo Table Offset; (absolute SON-offset) 0x0 if tempo doesn't change
0x10 4 Initial Tempo; (commonly 0x78 or 120 beats per minute)
0x14 4 Unknown
0x18 256 Track Header Offsets; (absolute SON-offsets) 64 elements, 0x0 if track not present
0x118 End of header

Track Header

This is a variable-length table of headers for each track

Offset Size Description
0x0 4 Start Tick; time-point to begin executing track data
0x4 4 Unknown; commonly 0xffff0000
0x8 2 Track Data Index
0xA 2 Padding
0xC 4 Start Tick; copy of start tick
0x10 4 Unknown; commonly 0xffff0000
0x14 4 Unknown; commonly 0xffff0000
0x18 End of header

Track Data

Here begins a free-form blob of indexed track data. It starts with a variable-length u32 array of SON offsets for each track, then the track data itself.

Track Data Header

Offset Size Description
0x0 4 Track Data Header Size; size of the header after this field (always 0x8)
0x4 4 Pitch Wheel Data Offset; (absolute SON-offset) 0x0 if no pitch-wheel messages on track
0x8 4 Mod Wheel Data Offset; (absolute SON-offset) 0x0 if no mod-wheel messages on track
0xC End of header

Track Commands

After the track data header, the actual playback commands begin. There are only 2 types of commands in SON: note and control change.

Delta Time RLE

Just like MIDI, each command starts with a delta time value telling the sequencer how many ticks to wait after the previous command. Unlike MIDI, Factor5 uses a custom RLE scheme to adaptively scale the value's precision to reduce the value's size.

The RLE operates on 16-bit words, with the value 0xffff triggering a continuation, then a 'dead' 16-bit word skipped over, then the 0xffff is summed with the following RLE value, looping the decode algorithm.

In Python, decoding works like so:

def DecodeDeltaTimeRLE(in):
    total = 0
    while True:
        term = in.ReadU16()
        if term == 0xffff:
            total += 0xffff
            dummy = in.ReadU16()
            continue
        total += term
        return total
Note Command

When the two bytes following the delta-time != 0xffff, and the high-bit of the first byte is set, this is a note command.

Unlike MIDI, which has separate commands for note-on/note-off, SON attaches a note length value to a note-on command, which is then able to track its own lifetime.

Offset Size Description
0x0 1 Note; AND with 0x7f for the value
0x1 1 Velocity; AND with 0x7f for the value
0x2 2 Note Length; count of ticks before note-off issued by sequencer
0x4 End of note
Control Change Command

When the two bytes following the delta-time != 0xffff, and the high-bit of the first byte is unset, this is a control change command.

Offset Size Description
0x0 1 Value; AND with 0x7f for the value
0x1 1 Control; AND with 0x7f for the value
0x2 End of control change
End Of Track

When the two bytes following the delta-time == 0xffff, this track has no more commands.

Continuous Pitch / Modulation Data

If the pitch or mod offsets in a track are non-zero, they point to a buffer of RLE-compressed (delta-tick, delta-value) pairs, decoding to signed 16-bit precision. The decoder must track the absolute time and value, summing each consecutive update for the current time/values.

The algorithm for this RLE is different than the delta-time one for commands. It may scale down to a single byte if able.

def DecodeRLE(in):
    term = in.ReadU8()
    total = term & 0x7f
    if term & 0x80:
        total *= 256 + in.ReadU8()
    return total

def DecodeContinuousRLE(in):
    total = 0
    while True:
        term = DecodeRLE(in)
        if term == 0x8000:
            total += 0xffff
            dummy = in.ReadU8()
            continue
        total += term

        if total >= 0x4000:
            return total - 0xffff
        else:
            return total

Channel Map

This is a simple u8 table mapping 64 SON tracks to 16 MIDI channels for instrument selection via the SongGroup MIDI-Setup.

Tempo Table

When the SON has a non-zero tempo table offset, this song features tempo changes. The change events are simple absolute-tick / BPM pairs.

Offset Size Description
0x0 4 Tick; absolute time-point to perform tempo change
0x4 4 Tempo; new tempo in BPM
0x2 End of tempo change