Quake data structures

Raphael.Quinet@eed.ericsson.se
Mon, 4 Mar 1996 10:55:16 +0100

Date: Mon, 4 Mar 1996 10:55:16 +0100
Message-Id: <199603040955.KAA17680@ebcw405.ebc.ericsson.se>
From: Raphael.Quinet@eed.ericsson.se
To: quake-editing@nvg.unit.no
Subject: Quake data structures

Hello!

As some of you already know, I have been working on the Quake data files,
trying to decode all of them and understand their format. My intent is
to provide a standard library for reading and writing Quake files, which
can be used in any Quake editor. Included here is a list of functions
which are provided by the current QEU library for reading and writing the
various file formats.

The main differences between QEU 0.2 and QEU 0.3 are a few bug fixes and
improvements, and the files f_sprite.h and f_sprite.c which allow you to
read and write the sprite files (*.spr). The format of the sprites was
a bit more complex than I thought at first, but now I think I got most of
it.

The text below is taken from the file "FORMATS", as included in QEU 0.3.
The detailled structure of all Quake files can be found in the corresponding
header file (*.h). You should also take a look at the "README" file for
more information about the QEU library.

Quake data formats
==================

Here is a list of the various data structures used by Quake, and a guide to
the functions of the QEU library that can be used to work on these structures.

PACK (magic: "PACK", extension: ".pak")
---------------------------------------

This is a simple archive file, which includes other files.

* Files: f_pack.h, f_pack.c

* Status: Completely understood.

* Reading PACK files:

ReadPACKDirectory()
Reads the directory of a PACK files and stores it in memory. The individual
files in the archive can be read with the appropriate Read*() function, using
the offset found in the directory.

* Creating PACK files:

WritePACKHeader()
Writes the header of the PACK file and creates an empty directory in memory.

AddPACKEntry()
Adds a new entry to the PACK directory, after the data has been written to
the file using the appropriate Save*() or Write*() function.

WritePACKDirectory()
Updates the PACK header and writes the directory to the file. Like the
other Save*() and Write*() routines, it returns the total number of bytes
taken by the whole PACK data (header + all entries + directory).

* Other functions:

FindPACKEntry()
Returns the index in the PACK directory of the first entry matching a given
name.

DumpPACKDirectory()
Prints the contents of the PACK directory in a human-readable form. Useful
for debugging.

UnPACKFile()
Creates individual files from a PACK archive.

WAD2 (magic: "WAD2", extension: ".wad")
--------------------------------------

The WAD2 format is also an archive format, which includes several chunks.
Each chunk has a type (raw data, bitmap with header, raw bitmap) and an
optional compression method, although none of the chunks are compressed in
the test version of Quake.

* Files: f_wad2.h, f_wad2.c (+ f_bitmap.h, f_bitmap.c)

* Status: Completely understood.

* Reading WAD2 files:

ReadWAD2Directory()
Reads the directory of a WAD2 files and stores it in memory. The individual
chunks in the archive can be read with the appropriate Read*() function,
using the offset found in the directory. Most of the chunks are bitmaps that
can be read with ReadBitmap(). Two chunks (CONBACK, CONCHARS) are raw
bitmaps that can be read with ReadRawBitmap(), using the appropriate width
and height. The palette (PALETTE) is stored as raw data and can be read with
ReadBytes() or copied to a file with CopyBytes().

* Creating WAD2 files:

WriteWAD2Header()
Writes the header of the WAD2 file and creates an empty directory in memory.

AddWAD2Entry()
Adds a new entry to the WAD2 directory, after the data has been written to
the file using the appropriate Save*() or Write*() function.

WriteWAD2Directory()
Updates the WAD2 header and writes the directory to the file. Like the
other Save*() and Write*() routines, it returns the total number of bytes
taken by the whole WAD2 data (header + all entries + directory). If the WAD2
file is included in a PACK file, this number can be given to AddPACKEntry().

* Other functions:

FindWAD2Entry()
Returns the index in the WAD2 directory of the first entry matching a given
name.

DumpWAD2Directory()
Prints the contents of the WAD2 directory in a human-readable form. Useful
for debugging.

UnWAD2File()
Creates individual files from a WAD2 archive.

BSP (extension: ".bsp")
-----------------------

The BSP files contain the information about the maps (level data) and the
static objects that can be included in the maps. Each BSP file contains 14
unnamed chunks of data which define a level and its contents. More
information about the contents of the BSP files can be found in the
"Unofficial Quake Specs" by Olivier Montanuy.

Obviously, the hard part is not to work on the structure of the BSP file,
but on its contents. This will be done by other routines (not written yet).

* Files: f_bsp.h, f_bsp.c

* Status: Structure of the file completely understood, but the format of the
14 entries is still unknown to me. Some of them are easy to
understand (the first entry is plain text), but the other ones are
more complex. I have a rough idea of what each one does, but I
haven't had the time to investigate this in detail.

* Reading BSP files:

ReadBSPDirectory()
Reads the directory of the BSP file and stores it in memory. It seems that
all BSP files in Quake contain 14 entries, but this could change in future
versions of the game (the first 4 bytes of the BSP files contain a version
number). This function only reads the offsets to the individual data
chunks, which have to be read with an approriate Read*() function (not done
yet).

* Creating BSP files:

(not done yet, although it's rather simple).

* Other functions:

DumpBSPDirectory()
Prints the contents of the BSP directory in a human-readable form. Useful
for debugging.

UnBSPFile()
Creates individual files from a BSP file. Useful for debugging.

Models (magic: "IDPO", extension: ".mdl")
-----------------------------------------

These 3D models define the animated objects (players, monsters and moving
objects). They contain the skin texture of the object (flat picture), a
set of triangles defining the object and a sequence of frames for the
animation.

* Status: I'm still working on that. I have understood most of it, but I
didn't include the source code because there are still some bugs
in it.

Sprites (magic: "IDSP", extension: ".spr")
------------------------------------------

The sprite files are simplified models (flat 2D pictures). They contain
multiple frames, which can contain several pictures. I understand the format
of the sprites, but not the purpose of the multiple pictures.

* Files: f_sprite.h, f_sprite.c (+ f_bitmap.h, f_bitmap.c)

* Status: Almost completely understood.

* Reading sprite files:

ReadSprite()
Reads all the frames for a sprite and stores them in memory.

* Creating sprite files:

SaveSprite()
Saves a sprite to a file (all frames).

* Other functions:

NewSprite()
Creates a new, empty sprite.

FreeSprite()
Discards a sprite and frees memory.

AddSpriteImage()
Adds an image (bitmap) to a sprite.

Sounds (magic: "RIFF", extension: ".wav")
-----------------------------------------

These are standard WAV files, which can be read and saved by most sound
editors. I do not intend to re-invent the wheel and include a sound
editor in QEU.

Bitmaps
-------

These are "flat" bitmaps, consisting of an array of 8-bit pixels (using a
palette of 256 colors). The bitmaps are included in the WAD2 file (gfx.wad)
and are part of some other structures, such as the frames for a sprite or an
alias model. In all these bitmaps, the reference to the Quake palette is
implicit. This is probably the simplest data structure in Quake.

* Files: f_bitmap.h, f_bitmap.c

* Status: Completely understood (that was simple enough!).

* Reading bitmap files:

ReadBitMap()
Reads a bitmap, including its width and height.

ReadRawBitMap()
Reads a raw bitmap (without header). The width and height of the bitmap
must be supplied to this function, since they are not included in the file.

* Creating bitmap files:

SaveBitMap()
Saves a bitmap to a file, including its width and height. The number of
bytes written is returned, so that it can be given to AddWAD2Entry() if the
bitmap is included in a WAD2 file.

SaveRawBitMap()
Saves a bitmap to a file as raw data, without its width and height.

* Other functions:

NewBitMap()
Creates a new, empty bitmap.

FreeBitMap()
Discards a bitmap and frees memory.

DumpBitMap()
Prints the contents of the bitmap in a human-readable form. Useful for
debugging, although not practical for large bitmaps.

SavePPM()
Saves a bitmap as a PPM file (uses the Quake palette).

SaveBMP()
Saves a bitmap as a BMP file (uses the Quake palette).

Other formats
-------------

To be added later...

-Raphael