Re: Distributing Quake maps

Alex R. Moon (amoon@odin.mdn.com)
Sun, 10 Mar 1996 13:07:44 -0500 (EST)

Message-Id: <m0tvpWy-0004wVC@odin.mdn.com>
From: amoon@odin.mdn.com (Alex R. Moon)
Subject: Re: Distributing Quake maps
To: quake-editing@nvg.unit.no
Date: Sun, 10 Mar 1996 13:07:44 -0500 (EST)
In-Reply-To: <11239@tsys.demon.co.uk> from "Tom Wheeley" at Mar 10, 96 01:16:25 pm

Tom Wheeley wrote:
>
> In article <Pine.LNX.3.91.960310075200.979D-100000@portal.atak> you write:
>
> > On Sun, 10 Mar 1996, Tom Wheeley wrote:
> >
> > > At the minimum I am proposing is a tool not dis-similar to DeuSF or DeuTex
> > > which is run locally by the end user. This will process a list of textures
> > > which the program `statically links' into a final BSP file.
> >
> > This is a good idea, but it shouldn't be too slow, otherwise we will get
> > 'binaries' distributed versions of levels as well as 'source' versions.
>
> The test maps are only a meg, so my guess is a max of about 10s on my 386sx
> (I'm in a similar position to Raphael -- can't play the damn game ! :)
> This time would be larger for more textures. There is (currently) a max of
> 256 textures now. I don't think it is a significant step, timewise. Slower
> than PKUNZIPping the file, probably, but not meaningfully so.

This depends on how compiled you expect this format to be. Do you expect it
to contain only geometry data, or do you expect it to contain all the bsp
trees and visilists as well? If the former, be aware that id has said that
visilist calculation can take over an hour on a 4 processor 200MHz Alpha
machine. If the latter, why not just use a .BSP format with the textures
replaced with placeholders (null textures with just a texture name). Then
some tool could go through and rifle the id .BSP files (or any other .BSP
files) looking for the appropriate textures and chuck them in. Personally
I'd like to see both, a textureless .BSP file AND a human-friendly, BSPless
and visilistless geometry format.

> > > This idea of processing at the users locale could possibly be extended to the
> > > creation of the various abstract stuctures which comprise most of the rest of
> > > the .BSP file.
> >
> > However, I think that this would slow down the local processing time too
> > much, leading to separate binary and source distributions.
>
> It depends on the how fast it is to create some of them. f.e. if a
> particularly large lump was quick to process, it would be prefereable to
> process this at the end, rather than distribute a large lump. Remember that
> `binary' distributions are illegal, and are currently blocked from ftp.cdrom.com
> This is just a _possible_ extension, anyway.

Compiling the BSP trees, light maps, and visilists sounds like it'll take
forever. Definitely not something an end user is going to want to mess with.
On the other hand, Doom showed us that dedicated BSP builders can get better
results than many editors' built-in BSP builders. I hate to think how much
more that will be true with visilist builders. So, how about a generic
text format which all editors and compilers can use which simply gives the
geometry for the level which can then be compiled into a .BSP file (with or
without actual textures, see above)?

> > > The MakeTex itself would be in the PAK as Entry 'MakeTex'. Other (possibly
> > > standardized) entries could be made such as `Author', 'Description', 'Licence'> > etc. These could be displayed by the MakeTex program.
> >
> > I'm not sure if this is completely necessary. If the format of the levels
> > within the distributed PAK file is going to be different than in standard
> > Quake BSP files, why not simply put the data as part of the new
> > distribution level format?
>
> Umm, I'm not entirely sure what you mean. The MakeTex will build a `standard'
> BSP file from the more general information in the PAK file. There is no
> good place to put `unneccesary' information in the BSP file, as there is in
> the PAK files, which have a more complicated directory. You can only put
> things in gaps, (like QEU does with version info)

If this is all you want it to do, just use a .BSP format with a wierd version
and placeholders for the texture data. Searching other .BSP files or .TEX
files for the appropriate textures should be simple.

Here are my thoughts on how an uncompiled level format should be constructed.
There are essentially three elements which make up a quake map:
1) It's geometry (the surfaces),
2) It's inhabitants (the entities), and
3) It's textures.

Everything else can be calculated given those three. Therefore, the
obvious solution (to me) is to have three file types, one for each
element of a level. Each level would have three files (optionally
PACKed) which could be compiled into one .BSP file. The only question
is what format each file should adhere to.

The entities are stored in the .BSP file in a nice simple text format, so why
not use that for the .ENT file? We could make it conform to DOS CR-LF format
instead of its UNIX-style LFs and just strip the CRs when it's compiled into
a BSP file.

The textures also have their own format in the .BSP file which we can use
for the .TEX (or .MIP, or whatever) files, perhaps with an additional header
giving a magic and any other info we want to add.

The only question then is what format we should use for storing the geometry
data. In my opinion, there's no question that we should use as human-readable
a text format as possible. The files probably won't be huge in the first
place and compression will seriously shrink the text files such that file size
shouldn't be a deciding factor in favoring a binary format. Other than
size, text files have all the advantages; they're easy to read, they're easy
to manipulate, they're easy to understand, etc, etc. Some time ago, iD
was using a format like the following format for this purpose (snarfed from
an old ddt irc log, everything has, no doubt, changed):

:
4 WALL14_5 0 0 0 0 0 0
(0,912,176,144) (0,928,176,144) (16,928,176,144) (16,912,176,144)
4 WALL14_5 0 0 0 0 0 0
(192,928,144,0) (192,1040,144,0) (208,1040,144,0) (208,928,144,0)
:

In the generic case (using the variables from qkspec30), this format is:

numedge texname sofs(?) tofs(?) flips(?) light(?) unknown0(?) unknown1(?)
(v1_x,v1_y,v1_z1,v1_z2) (v2_x,v2_y,v2_z1,v2_z2) ... (vn_x,vn_y,vn_z1,vn_z2)

Note that they're using a simple solid geometry approach instead of giving a
separate entry for each surface. Each entry instead represents a sort of
extruded polygon which includes numedge+2 surfaces. This has advantages for
editors in that they can use the solid geometry information in their user
interface and for CSM instead of forcing the user to deal with each surface
separately. And for editors or files which have no solid information,
you can simply set vn_z1 = vn_z2 for every vertex.

Basically, if we want a text and human-readable geometry format, we need to
decide whether that format should contain only surface information, or if
it should contain some sort of additional solid geometry information. If the
former, a format such as:

numedge texname sofs tofs flips light unknow0 unknown1
(v1_x,v1_y,v1_z) (v2_x,v2_y,v2_z) ... (vn_x,vn_y,vn_z)

should be sufficient. In the latter case we need something more complex,
either id's format above (which is awfully limited), or perhaps something
like this:

# Surfaces
# --------
numsurfaces
:
numedge texname sofs tofs flips light unknow0 unknown1
(v1_x,v1_y,v1_z) (v2_x,v2_y,v2_z) ... (vn_x,vn_y,vn_z)
:
# Solid Objects
# -------------
numobjects
:
numsurfaces surface1 surface2 ... surfacen
:

This format is just the surfaces format from above plus a number of surface
lists (similar to the surface lists used for BSP leaves) which represent
solid objects, for use by editors in CSM and user-level manipulation.

In summary, we've got an entities format. We've got a texture format (though
we may wish to add a header). The only decision we've left to make is for
a geometry format, and deciding on that should be relatively simple.

-- 
Alex R. Moon                | "A university is what a college become when the
amoon@odin.mdn.com          |  faculty loses interest in students."
71513.1453@compuserve.com   |                                     -- John Cardi