Steel Penny Games

It's been one of those rare days when things go from bad to worse just when you think the problems are licked.  But a fun day, nonetheless.  Let me explain.

I'm writing my second article for Game Engine Gems, one about an advanced palettizer technique that works for colors and non-colors alike, and does a remarkable job... but I'll save the details for the article.  What is important is that today, I had to generate a couple of sample images to compare with Photoshop.  My good friend and editor, Eric Lengyel, asks that all images be 600 dpi so they have flexibility during layout in sizing images up or down without losing quality. 

Okay, sure.  So I scale up my test image and do a little hacking in my texture processing tool so that it accepts images that large (5333x4000).  Yes, that's a big image!  21 million pixels worth, which is about 385MB when cracked into floating point for processing.

Due to my inefficient organization of memory, my poor 32-bit operating system ran out of memory at 2GB, and simply gave up.  So, an hour later, I rewrote that a bit and now it fits in memory fine, but I've introduced a bug which causes the processing to take forever, then another bug where the colors produced are simply wrong.  Half the day is spent and I don't have any test images to show for it, but the tools are faster and take less memory, and can definitely process massive textures.

The final bug?  The .tga file that I wrote out was 83MB without compression, and for some reason was a file full of zeros.  Eh? What!  After quite a bit of digging into the code, I found nothing wrong with the data or my handling of it.  It turns out that I had to dig into the C-Runtime Library source to track down why a simple fwrite() is failing.  It's an 83MB fwrite().  Deep in the bowels of Microsoft's CRT, fwrite() calls WriteFile().  This is important, because it has a limitation that fwrite() does not have. Here's a snippet from the documentation:

"Named pipe write operations across a network are limited to 65,535 bytes. "

 Suddenly, it clicks.  I've been working on a drive letter that is actually mapped from the network, and it appears the code in the C Runtime Library does not automatically loop and write in blocks of 64K.  Sigh.

Now, I get to go through all my tools and make sure I'm never using a raw fwrite() anywhere important, and fix the one place most of my tools-side file accesses go through so that it loops and writes out the correct size for me.  Ancient zombie MS code, finding new ways to crawl out of the crypt and curse unsuspecting developers.

Bewarned!