[Runtime Bug]: colorTexture2 Color & Alpha Not Blending with colorTexture1 - Star Wars Empire at War (2006)

Describe the bug

Context:

For the game Star Wars Empire at War (Directx9.0c), we have a odd issue. There is multiple shaders which use fixed function calls to blend textures to produce one output.

1. Space Fog of War: A fog of war mask texture is generated by the CPU to indicate in real time using a alpha channel, where units can see on a 2d plane. This is seen as a corresponding white texture with an alpha of where units can see is generated. This is combined with a generic diffuse texture for a grid which should have its color and alpha "blended" with the fog of war texture.

2. Land Terrain Textures: A mask texture is generated by the CPU to indicate on a cell of the map where a certain terrain texture should be painted. This is seen as a corresponding white texture. This is combined with a generic diffuse texture for the type of terrain (grass/rock/dirt/etc.) which should have its color "blended" with the mask texture. The game splits a "cell" of a terrain into chunks based on where a certain terrain texture is painted. Hence a separate grid like mesh to created, split from the original square cell. The texture is then painted onto the new mesh within the now split cell.

Issue Observed:

When handling two color textures. Remix will read colorTexture1 and will display said colorTexture1 without blending the color or alpha of colorTexture2.

• For Space Fog of War this means the white mask texture (colorTexture1) with the alpha which indicates where units currently see, becomes the only visible texture. The grid texture's (colorTexture2) color and alpha is not blended with the mask texture.

• For Land Terrain Textures the mask texture (colorTexture1) generated to indicate where to display a terrain diffuse texture (colorTexture2) within a cell becomes the only visible texture. The terrain diffuse texture color is not blended with the mask texture.

colorTexture1 & colorTexture2 will be read by Remix and present in the list of textures on the side for Space Fog of War. However for Land Terrain Texture only colorTexture1 will be present. If you ignore the skydome texture, then mark the terrain white mask texture as Skybox, the proper blended Terrain Texture becomes visible. However it is not read by Remix at said point and hence no render calls (lighting) are made on it, so this "workaround" is pointless.

I ultimately believe these two issues are very similar in scope and hence could share a single fix.

Code:

As mentioned prior, Empire at Wars shaders are all purpose built with fallback fixed function techniques to ensure compatibility with less-powerful graphics cards of the time. See below for each of the above issues, the corresponding shader technique code, to get an idea of the fixed function calls used.

1. Space Fog of War (Shader Name: SpaceFogOfWar.fx):

technique t1
<
	string LOD="FIXEDFUNCTION";
>
{
	pass t1_p0
	{
        SB_START

    		ZWriteEnable=false;
    		ZFunc=lessequal;	
    		AlphaBlendEnable=true;
    		DestBlend = INVSRCALPHA;
    		SrcBlend = SRCALPHA;
    		
            CullMode=none;
    		
    		// FF Vertex pipeline
    		Lighting=false;
    		
    		// FF Pixel pipeline
    		AddressU[0] = CLAMP;
    		AddressV[0] = CLAMP;
    
    		AddressU[1] = WRAP;
    		AddressV[1] = WRAP;
    
            TexCoordIndex[1] = CAMERASPACEPOSITION;
    		TextureTransformFlags[1] = COUNT2;
    		ColorOp[0]=SELECTARG1;
    		ColorArg1[0]=TEXTURE;
    		AlphaOp[0]=SELECTARG1;
    		AlphaArg1[0]=TEXTURE;
    
    		ColorOp[1]=SELECTARG1;
    		ColorArg1[1]=CURRENT;
    		ColorArg2[1]=TEXTURE;
            
            AlphaOp[1]=MODULATE;
    		AlphaArg1[1]=TEXTURE;
    		AlphaArg2[1]=CURRENT;
    				
    		ColorOp[2]=Disable;
    		AlphaOp[2]=Disable;

        SB_END        

        VertexShader = NULL;
        PixelShader = NULL;
        
		Texture[0] = (m_FOWTexture);
        Texture[1] = (GridTexture);
		TextureTransform[1] = 
		(
			mul(
				m_viewInv,
				float4x4(
					float4(200*m_FOWTexU.x,200*m_FOWTexV.x,0,0),
					float4(200*m_FOWTexU.y,200*m_FOWTexV.y,0,0),
					float4(0,0,1,0),
					float4(0,0,0,1))
				)
		);
	}
	
	// cleanup pass
	pass t1_cleanup < bool AlamoCleanup = true; >
	{
        SB_START

    		AddressU[0] = WRAP;
    		AddressV[0] = WRAP;

    		TexCoordIndex[0] = 0;
    		TextureTransformFlags[0] = DISABLE;
    		TexCoordIndex[1] = 1;
    		TextureTransformFlags[1] = DISABLE;

        SB_END        
	}

}

2. Land Terrain Textures (Shader Name: TerrainRenderBump.fx):

technique t3
<
	string LOD="FIXEDFUNCTION";
>
{
	pass t3_p0
	{
        SB_START

    		// General Render States
    		ZEnable=true;
    		ZWriteEnable=true;
    		ZFunc=lessequal;
    		
    		// Blend Texture
    		MinFilter[0]=LINEAR;
    		MagFilter[0]=LINEAR;
    		MipFilter[0]=LINEAR;

    		TexCoordIndex[0] = CAMERASPACEPOSITION;
    		TextureTransformFlags[0] = COUNT2;
    		
    		ColorOp[0]=SELECTARG1;
    		ColorArg1[0]=TEXTURE;
    		AlphaOp[0]=SELECTARG1;
    		AlphaArg1[0]=TEXTURE;
    				
    		// Diffuse+Gloss Texture
    		MinFilter[0]=LINEAR;
    		MagFilter[0]=LINEAR;
    		MipFilter[0]=LINEAR;
    
    		TexCoordIndex[1] = CAMERASPACEPOSITION;
    		TextureTransformFlags[1] = COUNT2;
    		
    		ColorOp[1]=MODULATE2X;
    		ColorArg1[1]=DIFFUSE;
    		ColorArg2[1]=TEXTURE;
    		AlphaOp[1]=SELECTARG1;
    		AlphaArg1[1]=CURRENT;
    		
    		ColorOp[2]=Disable;
    		AlphaOp[2]=Disable;
    		
    		/* 
            Attempt at gloss-masked specular...
    		ColorOp[2]=MODULATEALPHA_ADDCOLOR;
    		ColorArg1[2]=CURRENT;
    		ColorArg2[2]=SPECULAR;
    		AlphaOp[2]=SELECTARG1;
    		AlphaArg1[2]=CURRENT;
    		*/

        SB_END        

        VertexShader = NULL;
        PixelShader = NULL;

		Texture[0] = (blendTexture);
		TextureTransform[0] = 
		(
			mul(
				m_viewInv,
				float4x4(
					float4(blendTexScale.x,0,0,0),
					float4(0,blendTexScale.y,0,0),
					float4(0,0,1,0),
					float4(blendTexOffset.x,blendTexOffset.y,0,1))
				)
		);

		Texture[1] = (diffuseTexture);
		TextureTransform[1] = 
		(
			mul(
				m_viewInv,
				float4x4(   float4(diffuseTexU.x,diffuseTexV.x,0,0),
                            float4(diffuseTexU.y,diffuseTexV.y,0,0),
                            float4(diffuseTexU.z,diffuseTexV.z,1,0),
                            float4(diffuseTexU.w,diffuseTexV.w,0,1))
				)
		);
		// Material colors
		MaterialAmbient = (materialDiffuse);
		MaterialDiffuse = (materialDiffuse);
		MaterialEmissive = (float4(0,0,0,0));
		MaterialSpecular = (materialSpecular);
		MaterialPower = (16.0f);
	}
	
	// cleanup pass
	pass t3_cleanup < bool AlamoCleanup = true; >
	{
        SB_START

    		TexCoordIndex[0] = 0;
    		TexCoordIndex[1] = 1;
    		TextureTransformFlags[0] = DISABLE;
    		TextureTransformFlags[1] = DISABLE;

        SB_END        
	}
}

Helpful Info:

From a conversation with NV_Mark on the RTX Remix Showcase Discord, he mentioned within the opaque material, specifically:

    chooseTextureArgument(textureArg1, surface.textureColorArg1Source, albedo, surfaceInteraction.vertexColor.rgb, tFactor.rgb, f16vec3(1.0));
    chooseTextureArgument(textureArg2, surface.textureColorArg2Source, albedo, surfaceInteraction.vertexColor.rgb, tFactor.rgb, f16vec3(1.0));
    chooseTextureOperationColor(albedo, surface.textureColorOperation, textureArg1, textureArg2);

There is an example of colorTexture1 & colorTexture2 blending. Perhaps however it is not being handed right/is bugged in chooseTextureOperationColor of src\dxvk\shaders\rtx\concept\surface_material\opaque_surface_material_blending.slangh.

The ray portal material for Portal was also mentioned as an example of colorTexture2 usage.

Image Breakdown:

For further context for the above two issue cases, see the below images from the DirectX SDK (June 2010) debugger alongside images from in-game with RTX Remix enabled.

1. Space Fog of War:
   Fog of War Mask Texture (colorTexture1):
   
   Grid Texture (colorTexture2):
   
   Fog of War Mask Texture combined with Grid Texture:
   
   Remix's interpretation of the Shader. Failing to combine the two textures. Only showing Fog of War Mask Texture (colorTexture1)
   

2. Land Terrain Textures:
   Land Terrain Mask Texture (colorTexture1):
   
   Terrain Diffuse Texture (colorTexture2):
   
   Land Terrain Mask Texture BEFORE combination with Terrain Diffuse Texture:
   
   Land Terrain Mask Texture AFTER combination with Terrain Diffuse Texture:
   
   Remix's interpretation of the Shader. Failing to combine the two textures. Only showing Land Terrain Mask Texture (colorTexture1)
   

Useful Files:

StarWarsG_d3d9.log
rtx.conf.txt
dxvk.conf.txt

How do you reproduce the bug?

1. Launch Star Wars Empire at War
2. Enter into any Space Skirmish Map
3. Observe if Fog of War is blended with Grid Texture color + alpha.

• If solid gray plane observed then issue reproduced.
• If grid like plane observed then issue not reproduced, working as expected.

1. Launch Star Wars Empire at War
2. Enter into any Land Skirmish Map
3. Observe if Terrain texture mask is blended with Terrain diffuse map.

• If solid white plane observed then issue reproduced
• If terrain texture (such as grass or dirt) is visible, then issue not reproduced, working as expected.

What is the expected behavior?

For Space the runtime should properly blend the mask texture whos alpha indicates visible space with the grid texture which indicates not visible space.

For Land the runtime should properly blend the mask texture with the terrain texture. The mask texture color indicates where a terrain texture should be present.

Version

0.5.1

Start in src/dxvk/shaders/rtx/concept/surface_material/opaque_surface_material_blending.slangh, especially chooseTextureArgument and chooseTextureOperationColor. Reproduce the issue in Star Wars Empire at War using the Space Skirmish and Land Skirmish steps, then compare the colorTexture1/colorTexture2 behavior with the Portal ray portal example. Done means both mask and diffuse textures blend with the expected color and alpha.