Content: Slate Blackcurrant Watermelon Strawberry Orange Banana Apple Emerald Chocolate Marble
Background: Slate Blackcurrant Watermelon Strawberry Orange Banana Apple Emerald Chocolate Marble
Pattern: Blank Waves Notes Sharp Wood Rockface Leather Honey Vertical Triangles
Welcome to Xbox Chaos: Modding Evolved

Register now to gain access to all of our features. Once registered and logged in, you will be able to contribute to this site by submitting your own content or replying to existing content. You'll be able to customize your profile, receive reputation points as a reward for submitting content, while also communicating with other members via your own private inbox, plus much more! This message will be removed once you have signed in.

  • entries
    11
  • comments
    66
  • views
    21,251

About this blog

Programming, research, and random interesting information.

Entries in this blog

AMD

Reach - Map Header Resigning

Reach map files have a hash in them which is computed based on the data in the file header. This is kinda irrelevant since Zedd's XEX patches disable the header signature checks, but I figure I'd document this anyway because he asked me about it today. Here's some quick info on how to update the hash that's computed on the map header.

First, you need to prepare the header by removing certain bits of data. Make a copy of the first 0xA000 bytes (header) of the map file. Then, zero out the following areas of the header:

  • 4 bytes at 0x320
  • 8 bytes at 0x328
  • 0x3C bytes at 0x330
  • 0x100 bytes at 0x36C
  • 0x100 bytes at 0x1C
  • 4 bytes at 0x144
  • 4 bytes at 0x148
  • 4 bytes at 0x16C
  • 4 bytes at 0x170
  • 0x18 bytes at 0x174
  • 4 bytes at 0x49C
  • 4 bytes at 0x4A0
  • 4 bytes at 0x4A4
  • 4 bytes at 0x4A8

Next, you need to salt the hash. Put the following data at the beginning of the header you prepared:

ED D4 30 09 66 6D 5C 4A 5C 36 57 FA B4 0E 02 2F 53 5A C6 C9 EE 47 1F 01 F1 A4 47 56 B7 71 4F 1C 36 EC

Finally, compute a SHA-1 checksum over your modified header (this should be 0xA022 bytes after adding the salt). The new hash goes at offset 0x330.

So yeah. While not too useful, I suppose this could be used to help map out unknown parts of the header based on the offsets that get zeroed. I'm also pretty sure that more hashes would have to be updated and not just that one, because cache files have at least three hashes in the header.

Anyway, there you go. Have fun.

-- Aaron

AMD

I'm going to start doing some mini blog posts from now on to keep everyone updated on our research, because it seems people don't notice some of the stuff we do otherwise.

Zedd's been doing a lot of work on cross-map BSP injection lately, and he's been having some trouble with the terrain shaders. Even though Assembly has support for shader extraction and injection, the map was still crashing. One shader that was causing problems for us in m70_bonus.map was levels\solo\m70_bonus\shaders\ground\m70_bonus_cinematic_ground.rmtr. Its actual pixl tag is shaders\terrain_templates\_0_0_0_0_0_0_0.pixl.

If you go to that tag in a newer version of Assembly (one that supports shaders), you might notice that all of the shader pointers in it are null, even though the shader works fine and certainly isn't just a null shader. How can this be? Well, we noticed that just above the shader pointer in one of the blocks, there was an int32 with a value of 321. This is usually set to -1 for shaders with valid code pointers. Obviously, there must be something to this.

We first speculated that this integer was an index into the resource table in zone. (That's a pretty safe guess when you're dealing with stuff like this.) At index 321 in zone, the parent resource was...a bitmap of a spartan helmet. Zedd did some zedding around with Assembly's injection code anyway, but couldn't get anything to work reliably. OK, that's probably not any good. Time to move on.

Stumped, I decided that we should set a memory read breakpoint on that integer and see what the game does with it. The breakpoint hit at address 0x82182E2C in the TU1 default.xex. Opening that up in IDA, you see this (not including irrelevant code):

lwz	 %r8, 0x50(%r3)...mulli   %r10, %r8, 0x58

So this means that the value is an index into an array with an element size of 0x58. And it just so happens that shader tag blocks are that large, but there weren't 300-something entries in the pixl tag.

Turns out that, starting in Reach, there's a global_cache_file_pixel_shaders ('gpix') tag which contains a bunch of commonly-used shaders, and this is an index into a block in that tag. The actual shader code can be found there. This means that the corresponding block entry needs to be extracted and injected into the target map in order to make the shader work.

Well, looks like we have some work ahead of us.

AMD

Enabling Ai In Multiplayer

UPDATE: Zedd was kind enough to make PPF patches for this. You can download them here: http://www.xboxchaos.com/files/file/60-halo-reach-blue-flames-xex-ppf-patches-and-ai/

UPDATE 2: Added values for Reach TU1.

UPDATE 3: Added values for Halo 3.

ivXuems.jpg

(Credit to OrangeMohawk for the screenshot)

I've been working on this for the past few sleepless nights days and finally figured it out. If you have a map in multiplayer that contains script code to spawn AI (hopefully someone can write a tutorial on how to convert campaign maps to MP), this will show you what to poke in order to force the AI to spawn. It's also possible to do the opposite and force AI to never spawn.

Note that this will NEVER work in a networked game. Don't even try. This is because the multiplayer engine is asynchronous and doesn't ensure that the game state (and hence AI state) is synced between clients. This is the very reason why AI was disallowed in multiplayer in the first place. If you want to use AI in a networked game, then you will have to convert your map to Firefight. No exceptions.

These values must be poked while at the main menu.

Halo 3 Non-TU

Enable AI Everywhere:

At 0x8203A7BC, poke 0x00000000 (uint32)

Halo 3 TU3

Enable AI Everywhere:

At 0x8203ADBC, poke 0x00000000 (uint32)

(If you want the addresses for disabling AI everywhere, let me know and I'll find them.)

Reach Non-TU

Enable AI Everywhere:

At 0x820388EC, poke 0x00000000 (uint32)

At 0x8213E4C3, poke 0x01 (byte)

At 0x821438E3, poke 0x01 (byte)

At 0x821F3B1F, poke 0x01 (byte)

At 0x823DB083, poke 0x01 (byte)

At 0x824D7970, poke 0x39600001 (uint32)

At 0x824F14D7, poke 0x01 (byte)

At 0x8261FB8B, poke 0x01 (byte)

At 0x826797C3, poke 0x01 (byte)

At 0x8267C4E3, poke 0x01 (byte)

At 0x826A705F, poke 0x01 (byte)

At 0x826FC167, poke 0x01 (byte)

Disable AI Everywhere:

At 0x821ADD20, poke 0x60000000 (uint32)

Reach TU1

Enable AI Everywhere:

At 0x820389D4, poke 0x00000000 (uint32)

At 0x8213E643, poke 0x01 (byte)

At 0x82143A63, poke 0x01 (byte)

At 0x82143B13, poke 0x01 (byte)

At 0x821F3A97, poke 0x01 (byte)

At 0x823DC653, poke 0x01 (byte)

At 0x824D96C0, poke 0x39600001 (uint32)

At 0x824F31C7, poke 0x01 (byte)

At 0x82622193, poke 0x01 (byte)

At 0x8267BDF3, poke 0x01 (byte)

At 0x8267EB13, poke 0x01 (byte)

At 0x826A9647, poke 0x01 (byte)

At 0x826FE78F, poke 0x01 (byte)

Disable AI Everywhere:

At 0x821ADD20, poke 0x60000000 (uint32)

I might make a blog post later which explains how I found these addresses in the first place and what everything does.

Special thanks go out to Zedd and Gamecheat (for helping me test) and to Kornman (for explaining how AI enabling worked in H2, among other things).

AMD

DjwERLx.png

So this post is a bit of a follow-up to a post I made last year (man, time sure does fly) about expanding the tag data section of a cache file. A lot of work has done since then (especially fairly recently), and things have changed enough that I feel the topic needs revisiting. Everything that I will cover here has already been implemented into the injection branch of Assembly, so the intent of this is to document the changes that were made and how they work.

Also, most of the stuff covered in here wasn't my work, so don't credit me for it.

But first...story time!

So...why revisit this?

Good question. Last year, I hacked together discovered a way to expand the tag data section of a cache file. However, of course, that only works on the tag data section. You can't use it to expand the resource table, you can't use it to expand the debug string data (tag names and stringIDs), and you can't use it to expand the localization data (even though I figured out a hackish way and never documented it).

The problem with this is that it makes tag injection difficult. Injected tags can't have names (because we can't rely upon free space being available) and stringIDs have to get approximated when injecting. On top of that, Gamecheat has been yelling at me because locale stringIDs can't be edited for the same reason (sorry). So some work needed to be done.

Blazing a trail

Well, Zedd figured out a way to expand the resource table by setting it to wumbo comparing a leaked version of a map he had with the release version and then coming up with a list of offsets that need to be changed. It worked and he was able to (sort of) inject a custom bitmap. Pretty cool.

This leaves the debug string section (the section containing tag names and stringIDs). OrangeMohawk made a tutorial describing how to add new stringIDs and tag names, but this relies upon the fact that the section sizes are rounded to multiples of 0x1000 bytes and that there's a high chance of there being free space. But this doesn't guarantee that space will be available and it isn't consistent. So I challenged him to find a way to expand the section.

Again, he was successful. However, something didn't make sense. He found that the value at 0x474 had to be increased by the amount of extra space injected in order for the map to load, and this value had been previously undocumented by me. I checked the BlamLib project (part of Open-Sauce) to see if Kornman knew anything about this value, and he had it labeled as "RuntimeBaseAddress," describing it as "Base address for runtime (Debug, Tag and Language pack) sections" but then also left a comment stating that it's unused in release builds. Wait, what?

Cornification

Considering that there was an obvious contradiction here, I contacted Korn and told him about what Mohawk found. He found this interesting and decided to turn to looking in the XEX. He came across this bit of code in the Reach beta:

.text:82551218 loc_82551218:						   # CODE XREF: cache_file_read_tag_section+148j.text:82551218				 lwz	   r6, s_map_file_info.header.memory_buffer_size(r30).text:8255121C				 clrlwi    r11, r6, 20.text:82551220				 cmpwi	 cr6, r11, 0.text:82551224				 beq	   cr6, loc_82551230.text:82551228				 ori	   r11, r6, 0xFFF.text:8255122C				 addi	  r6, r11, 1		    # buffer size.text:82551230.text:82551230 loc_82551230:						   # CODE XREF: cache_file_read_tag_section+164j.text:82551230				 lwz	   r11, 0(r29).text:82551234				 li	    r4, 2			 # section type (tags).text:82551238				 mr	    r3, r29	   # sub_8260F390.text:8255123C				 lwz	   r7, s_map_file_info(r30)	    # buffer (tag memory address).text:82551240				 lwz	   r5, s_map_file_info.header.memory_buffer_offset(r30) # cache offset.text:82551244				 lwz	   r10, 0(r11).text:82551248				 mtspr   CTR, r10

Now, I get it, most of you probably can't understand that. But what it essentially does is determines the offset of the tag data in the file by taking the memory buffer offset (located at offset 0x14) and adding the value at 0x474 (accounting for 32-bit integer overflow). This essentially makes the value at 0x474 an offset mask. Or more formally:

tag data offset = memory buffer offset + tag data offset masktag data address mask = tag data address - tag data offset

So, yes, that's right. The method that's been used to calculate "map magic" for the past six years is totally wrong.

Damn.

Putting it all together

Given this, we can look around the tag data offset mask to find that starting at 0x46C, there's an array of offset masks, one for each section of the file. In order, they are for the debug, resource, tag, and localization sections:

0x46C debug offset mask0x470 resource offset mask0x474 tag offset mask0x478 localization offset mask

Now, following this, we have the "section interop" table. This table lists virtual address and size pairs for each section of the file. (Note: the virtual addresses are not memory addresses. Don't even try.) The section addresses follow after each other (with a few exceptions, which I'll explain below), and adding an offset mask to a section's corresponding address yields its offset in the file. So:

0x47C debug virtual address0x480 debug size0x484 resource virtual address0x488 resource size0x48C tag data virtual address0x490 tag data size0x494 localization virtual address0x498 localization size

Finally, this leaves rebuilding that data. If this can be done, then any section of the file can be resized (provided that all pointers are recalculated accordingly, which Assembly's backend takes care of automatically). Based off of some observations I made about how these values are set up, I came up with the following formulas:

resource virtual address = 0localization virtual address = resource virtual address + resource sizetag virtual address = resource virtual address + resource size + localization sizedebug virtual address = resource virtual address + resource size + localization size + tag data size if tag data size > 0:    debug virtual address -= first tag partition size debug offset mask = debug data offset - debug virtual addressresource offset mask = resource data offset - resource virtual addresstag offset mask = tag data offset - tag data virtual addresslocalization offset mask = localization data offset - localization virtual address

Of course, there are a few things to note about this:

  1. If a section's size is zero, then its offset mask and virtual address should also be zero (however, the converse is not true - a virtual address or offset mask of zero does not imply that the section doesn't exist).
  2. The virtual addresses in Halo 3 are allocated differently, but it doesn't matter.
  3. The official cache files have a "gap" between the resource and localization sections which is likely due to removed debugging data. It doesn't matter and doesn't need to be accounted for.
  4. Doing this requires rebasing any string or localization pointers in the file. Deal with it.

So...there we have it. Using this, any part of a cache file can be expanded, and any part of a cache file can be loaded. Properly.

Hey look, that program again!

Mapexpand has been updated to accomodate this. A new feature allows you to optionally specify a section to expand (instead of just expanding the tag data, which is the default). For example:

mapexpand forge_halo.map stringiddata 1

adds one page to the string ID data in forge_halo.map. Valid sections include "stringidindex", "stringiddata", "tagnameindex", "tagnamedata", "resource", and "tag". If no section is specified, the tag name data is expanded (so usage is backwards-compatible). Should be pretty self-explanatory.

Download it here.

Virus scan here.

Source code for this is available here.

So, let's see what people can do with this.

Love you guys. wink.png

Here's to the next year.

-- AMD

AMD

I just figured out how to dump address information for built-in BlamScript functions from Reach's XEX. Basically, this information can be used to disassemble BlamScript functions (for example, physics_set_gravity) and look at how they work. It's extremely useful for researching the game's memory/saved game layout. I've already found some cool stuff which I plan to add into the next Liberty update.

If you extract the .exe basefile (XexTool's -b option) from default.xex, then the function table starts at offset 0xA31040 in non-TU and 0xA31070 in TU1. It's a list of 0x7BD memory addresses, each right after the other. Function opcodes in .map scripts are just indices into this table.

If you follow the memory address at an entry in the table (just subtract 0x82000000 to get the file offset), then you'll be taken to a structure with the following layout:

int16 return typeint16 flagsuint32 memory address of subroutineuint32 pointer to usage description string (null for most functions)int16 number of parametersfor each parameter:    int16 parameter typeint16 zero

Function names or parameter type names aren't stored anywhere, but Kornman built a list of them which you can find here: http://code.google.c...x_Scripting.xml

So you don't have to dump this table yourself, though, I did the dirty work for you and dumped the info (alongside data pulled from Korn's XML file) into a couple of Excel spreadsheets. One sheet is for TU and the other is for non-TU. Use this to quickly look up memory addresses of BlamScript functions.

Enjoy!

Download: http://www.mediafire...kqxf5o0p2d4os57

AMD

Since a lot of people have been asking us the same questions about Assembly over and over again and have also been giving us crap for making certain decisions, I decided to write this post to just clear some things up.

The Release Date

Although we do not currently have a specific release date set, the current plan is to have it finished by December 1st at the latest (read: it can come out earlier than that, and this may also change). However, it will not be ready by November 6th. We would much rather release a quality product later than release a crappy one earlier.

Beta Testing

There may be a private machine-locked beta test for Heroic members on this site, but other than that, there will be no public "beta test." Don't PM us and ask to get in, either.

SUBZ N VIEWZ

We are not making Assembly just to get scene fame. The program actually has useful tools in it which no other map editor has (you'll just have to wait and see what they are ;)). DeadCanadian and Lord Zedd can't wait to show you what's going to be possible even in Reach and Halo 3. Not to mention that we're making Assembly open-source just so that people will be able to learn from it.

Xbox Chaos Account

Unlike what some people are thinking, you will not need an Xbox Chaos account in order to use Assembly. There is only one feature which will require one (network poke doesn't even need it anymore!), and needing an account in order to use that feature should just make sense (you'll see why).

Supported Games

The first release of Assembly will only support editing map files from third-generation games (Halo 3 Beta - Halo 4). However, we may eventually add in support for older games if we see enough demand for it.

Halo 4 Tag Names

Yes, they are still encrypted. Yes, this means that people will still need to write taglists. Nuff said.

Halo 4 Plugins

Lord Zedd already has some object plugins mapped, but for the most part, plugins will need to be re-done because the meta formats for most tags have changed.

Tag Injection

;)

Asset Manipulation

The first release of Assembly will not support texture, sound, or model viewing, let alone texture/model/sound injection.

The Plugin Format

The plugin format will differ slightly from Ascension and Alteration, but we will be providing a tool to convert old plugins to our new format.

Megalo Editing

Assembly is a .map editor, not a gametype editor. Stop asking us about Megalo. Almost nobody (except for kornman?) knows anything about it.

BLF/Mapinfo Editing

Assembly will support editing .blf and .mapinfo files.

Well, hopefully this cleared some things up. If you have any more questions for us, feel free to ask by posting a comment below.

P.S.

NYSHNC.png
AMD

1OHR9H.png

yolo.

One of the features that we would love to add into Assembly is model script injection. However, in order to do this, we need to be able to add extra entries into a "scripts" reflexive in scnr. There aren't any "stub" script entries that we can just replace (well, technically there can be, but they don't appear in retail maps), and there also isn't any free space after the reflexive because tags are packed pretty tightly. This means that we need to move the reflexive to a different place that has room for the extra entries.

Traditionally, modders have "resized" reflexives by simply finding one large enough in an "unused" tag and overwriting that by copying the offset over. It works pretty well, and it's really simple to do...manually. The problem is that we're making a map editor here, and Assembly has no way of knowing what tags you don't plan on using (unless we ask you to mark them in some sort of dialog, but that's just plain stupid), so we actually need to find space in the file which isn't used by anything. And that's a problem in itself, because doing so normally would require us to scan every single tag's metadata and build a map of the file's memory usage. That just isn't feasible.

We finally decided that we have no other choice but to actually inject extra space into the file. It is really the only reliable method of expanding a chunk infinitely, because it allows us to know for certain where a large-enough space in the file is. After doing lots of research on the subject, we were able to figure out how this works, and today I'm going to be sharing that.

Injecting extra space into the file involves taking the following steps:

  1. Figuring out where to inject the data
  2. Figuring out how much data to inject
  3. Actually injecting the data
  4. Adjusting the file so that it loads the new data

Without any further ado, let's get started.

Step 1: Finding the location to inject data to

Obviously, we can't inject any data if we don't know where to put it. Sure, we need to expand the area of the file where tag meta is stored, but how do we know where that is? Let's take a little look into the .map format.

First of all, .map files have the following general layout (not drawn to scale):

QZ2T7H.png

Now, the two main blocks that we need to focus on are the file header and the tag metadata. The "tag metadata" block is the section of the file where tags and reflexives are stored, and that's the section that we're interested in expanding. In order to know where it is, though, we need to look at the file header. Obviously, that's at the very beginning of the file, and it's the first thing that you will see when you open the .map in a hex editor.

I currently have the following header values mapped out (these offsets are for Reach, but they're very similar in the other games):


0000 int32 magic = 'head'
0004 int32 game version
0008 int32 file size

0010 uint32 tag table header address
0014 uint32 index offset of the end of the file (file size - locale address mask)
0018 uint32 virtual size (sizes of all partitions added together, see below)

013C int16 type

0158 int32 number of stringIDs
015C int32 size of stringID data
0160 int32 pointer to the stringID offset table (not a memory address, but irrelevant here)
0164 int32 pointer to the stringID data

018C asciiz internal name
01B0 asciiz scenario name

02B4 int32 number of tag names
02B8 int32 pointer to tag name data
02BC int32 size of tag name data
02C0 int32 pointer to tag name offset table

02E8 uint32 virtual base (memory address of partition 0, see below)
02EC int32 xdk version
02F0 uint32 partition 0 address
02F4 int32 partition 0 size
02F8 uint32 partition 1 address
02FC int32 partition 1 size
0300 uint32 partition 2 address
0304 int32 partition 2 size
0308 uint32 partition 3 address
030C int32 partition 3 size
0310 uint32 partition 4 address
0314 int32 partition 4 size
0318 uint32 partition 5 address
031C int32 partition 5 size

0470 int32 file offset of asset data
0478 int32 locale address mask (aka "index offset magic," controls where the locale tables are located)
047C int32 stringID/tagname table address mask (minus the header size)
0480 int32 offset of asset data, relative to the end of the file header
0488 int32 asset data size
048C int32 index offset of the end of the file
0490 int32 virtual size (size of the tag metadata, or all partition sizes added together)
0494 int32 pointer to the first locale table (file offset of the first locale table - locale address mask)
0498 int32 size of the locale data

As you might have noticed, however, there isn't a direct pointer to where the metadata is - we'll need to do a bit of math. But it's pretty simple. If you look back at the .map layout diagram above, you'll see that the tag metadata immediately follows the asset table. And guess what: the header stores the file offset and the size of the asset data! This means that all we have to do is add the two values together:


offset of tag metadata = file offset of asset data + asset data size

(note: subtract the virtual base from that result and you have the map magic!)

All right, so we have the file offset of the metadata block. We just need to determine where to put the injected data inside of that block. Ideally, we would want to insert the data in a way that doesn't throw off any memory addresses in the file, because going back and adjusting everything would be a huge pain and would be very error-prone. My first thought was to put the injected data at the end of the metadata, because logically that would be pretty easy, right?

Wrong. It turns out that doing so makes the game completely reject the .map file even if you adjust the header correctly. It doesn't even bother trying to RSA check it; loading will simply stop at 0%. See, all of the .map files vary in size, and Bungie decided to play things safe. Rather than putting every .map file at a fixed base address in memory and setting a size limit on them so that they don't overflow into other data, they build the files to always end at a certain address (in Reach, this is 0xBFC00000). This means that injecting our extra data at the end of the file will cause the file to surpass that memory address and fail to load.

Taking this into consideration, our only choice is to inject data at the very beginning of the metadata. And at a first glance, this might seem like a radical thing to do: wouldn't that push back every single memory address in the file? Let's take a closer look at the .map format.

Something that you might have noticed in my map of the file header is that there are a bunch of "partition X address" and "partition X size" values. This is because .map files are split into what are known as "partitions" (this presentation made by Bungie briefly explains this). Tag metadata is actually divided into blocks based upon purpose (and probably for debugging reasons), and those partition values control how large each block is and what the block's memory address should be. This means that we can actually inject data at the beginning of the meta without changing any memory addresses - we just decrease the memory address of the first partition.

After all of this, we finally know exactly where we want to inject our extra data: at the very beginning of the meta partition (see my "offset of tag metadata" calculation above). I know that I talked a lot there even though the calculation of this offset is very simple, but I wanted to share information on how .map files work because understanding that is key to understanding how injection works and why using that offset is the best choice.

Step 2: How much?

Okay, so this is a bit of a trivial question. But the answer actually isn't as obvious as we initially thought it was. Just because we might only need 0x200 or so bytes for our new chunk doesn't actually mean that we only need to expand the file by that much. In fact, doing so actually makes the game freeze up.

The reason behind this is because the game (and the Xbox?) expects the meta partitions to be aligned to a multiple of a certain "page size." If you've ever looked at the meta section of a .map file in a hex editor, you might have noticed that there are a lot of zero bytes in that section of the file. The purpose behind those is to pad the partition out so that its size is properly aligned.

But what is the alignment? Let's whip out our handy-dandy hex editor and take a look at the partition table:

WZ131J.png

Now, if you take a close look at that diagram, you might notice something: every single address and size always has two zero bytes at the end, making everything a multiple of 0x10000. This is our alignment! Given this, we need to adjust the size of the data we want to inject so that it is a multiple of 0x10000 and satisfies the game's alignment requirements. This formula does that for us:


size of data to inject = (size actually needed + 0xFFFF) & 0xFFFF0000
(where & is bitwise and)

And sure, this can result in a lot of wasted space because needing 0x10010 bytes means that we would actually need to inject 0x20000 bytes, but we can set up a memory allocation system to fix that problem. Now, let's inject our data!

Step 3: Bangarang

Actually injecting the space into the file is pretty easy. All we have to do is start at the end of the file and move everything back in blocks (we settled on copying 1MB at a time for now) until enough space has been made available at the beginning of the metadata.


// C++

const long BufferSize = 0x100000; // 1 MB
uint8_t* buffer = new uint8_t[BufferSize];

// Push everything back in blocks
long moved = 0;
long moveSize = header.fileSize - injectOffset;
while (moved < moveSize)
{
long readSize = std::min(moveSize - moved, BufferSize);
reader.seekTo(header.fileSize - moved - readSize);
long read = reader.readBlock(readSize, buffer);

writer.seekTo(header.fileSize + injectSize - moved - readSize);
writer.writeBlock(buffer, read);
moved += read;
}

It's also important to zero out the injected pages so that they don't cause problems when read by a meta editor, but I'm not going to show that here.

Step 4: Loading... Done

We're almost there! All we need to do now is adjust the file header so that the newly-injected pages will actually load into memory. Remember that we injected extra pages into the beginning of the meta partition. Looking back at the header layout I posted above, this means that we need to do the following:

  1. Increase the file size value
  2. Increase the virtual size value (to resize the metadata)
  3. Decrease the virtual base and the memory address of partition 0 (so that no memory addresses change)
  4. Increase the size of the first meta partition (since we injected our data there)
  5. Finally, increase the locale address mask since the locale tables got pushed back (see the diagram showing the general .map layout)

This code accomplishes all of that:


// C++

header.fileSize += injectSize;
header.virtualSize += injectSize;
header.virtualBase -= injectSize;
header.localeAddressMask += injectSize;
header.partitions[0].address -= injectSize;
header.partitions[0].size += injectSize;

Congratulations!

After all of this, we've finally managed to inject empty space into the .map file that we can use for anything meta-related (injecting new tags, resizing reflexives, etc.). It's up to you how you want to use the new data.

Since this involved a fair amount of work, I have created a program called "mapexpand" (link below) which can do all of this for you. It's a command-line program because I like C++ and I was too lazy to make a GUI for it, but that shouldn't be a problem since this program is targeted at advanced modders anyway. You use it like this:


mapexpand <path to .map file> <number of 0x10000-byte pages to inject>

The program will then print the memory address and file offset of the injected data. Don't close the console window, because you'll probably need that. Just sayin.

So, for example, to add 0x20000 bytes into forge_halo.map, you can just run


mapexpand "forge_halo.map" 2

There are a few things that you need to be careful of though:

  • This only adds extra meta to the .map. You can't use this program to add models or textures - that's an entirely different process.
  • This program only works with the retail version of Reach for now. However, the method described in this blog post should work for Halo 3 as well, aside from differing header offsets.
  • You must patch your .xex with Zedd's blue flames patches. Using a .xex patched with Reachunlock or a similar program makes the game reject the map file for a currently-unknown reason.
  • Don't inject anything while Ascension or another editor is open. Injecting changes the map magic because it involves decreasing the virtual base address of the metadata.
  • Ascension can't handle patching between maps of different sizes. To create a patch which requires an expanded map file, you need to set an expanded .map as the original map and instruct users to use this program (perhaps through a .bat file or a similar script) before patching.
  • Finally, only inject the number of pages that you actually need. You can always add more later. Adding 50 MB to the file just because you think you might need it is stupid and will probably slow your Xbox down or make the game just crash.

Download mapexpand (includes C++ source code)

So, let's see what people can do with this.

Love you guys. ;)

-- AMD

AMD

Welcome To My Blog

Well, it's about time that I kick off the exciting new IP.Blog system by creating a blog of my own. Here, I'll be posting about random programming techniques or problems that I find amusing, research that I do, and other interesting things that I happen to find or wonder about. It's going to be a rather technical blog, so if you ever have any questions, please feel free to comment and ask away.

Occasionally, I might also post information related to how different parts of Argon (or my other projects) work. While we do have the official "Building Argon" blog, we're going to be talking more about design stuff there and not so much about how everything works.

So...yeah. That's about it for now.

bunny_pancake.jpg