GFF List Index Corruption

Summary

A binary GFF writer can silently corrupt list mapping if it writes list index entries in a way that allows recursive nested-list writes to interleave with the parent list’s index block.

This is a compatibility-critical issue for KOTOR data because many resources depend on stable list ordering and correct struct index mapping.

How GFF Lists Work

In binary GFF, a List field stores:

1. A relative offset into the list_indices table.
2. At that offset:
   1. count (u32)
   2. count struct indices (u32 each), each pointing into the struct table.

If these indices are wrong, the parser will load the wrong list structs.

Failure Mode

The bug class occurs when a writer:

1. Starts writing a parent list.
2. Recursively builds child structs.
3. Appends list indices directly while recursion is still producing nested list index data.

Because nested lists also write into the same list_indices buffer, parent and child index blocks can interleave and the parent list can point at unintended structs.

Observable Symptoms

• Struct IDs in list entries change after roundtrip.
• Expected fields are missing from entries after roundtrip.
• Mod compatibility breaks for list-heavy resources due to reordered/remapped entries.

Where a changed struct ID is not cosmetic

In most lists a struct ID is a position or a constant, so a remap shows up as entries in the wrong order and someone notices. Equip_ItemList is the one worth knowing about: a creature’s equipment slot is the list element’s struct ID, read off the element header, and it is recorded nowhere else. See UTC.

So a remap there reorders nothing. It moves items between slots, with every field of every entry intact and no missing data to notice. A robe can come back equipped in the creature hide slot.

The same reasoning applies to any list whose struct ID carries meaning rather than position, which is why a writer getting this right is not only about ordering.

And the damage takes two shapes, not one. Where a loader uses the ID, a remap moves the element somewhere it does not belong, as above. Where a loader validates it, the element is dropped instead: a GIT Creature List element whose ID is not the value that list expects is skipped silently, so the creature is not in the area and nothing says why. Both come from the same writer defect and they look nothing alike from the outside.

Correct Writer Strategy

For each list field:

1. Write list count.
2. Reserve contiguous slots for all struct indices up front.
3. Build each child struct recursively.
4. Backfill each reserved slot with the final struct index.

This guarantees parent list index layout is stable even when nested lists write their own index blocks.

Implementation Status

In this repository:

• rakata-formats/src/gff/writer.rs reserves list index slots and backfills them.
• Regression tests cover:
  • synthetic list order + struct-id stability
  • UTC fixture roundtrip stability on lists like FeatList, ItemList, ClassList.
• rakata-generics/src/utc.rs includes a no-op rebuild test to ensure typed conversion does not drift list order/IDs.