Question about create stub D3D9Query return data

[Venis]

Hi gho!
Many thanks to your simple and tiny code dxwnd tool,very easy to modify the code but i still met some difficulty in some routines,i am trying if you can give some advice,
Before the story,due to the lazy Intel,they still provides intergrated graphics that most based on the first 2004 Gen3 GMA900 even in 2009-era,which turned out to be the GMA950/3100 mainstream consumer motherboard grahphics for quite a long peroid,as the intel Gen3 arc most based on the previous 865G which is DX7/8 hardware feature level but with some minor mod to support the DX9/SM2.0 software driver interface,which caused quite a lot games failed to run on such platform.but it's still true DX9 capable,
Unfoutnatly,i am using GMA3100 druing those years which caused a lot of regrets for me that failed to play many title games.after almost 20 yrs,I decided to dig out the root cause that made many games failed,which turned out to be the story now.
The major target is IW enigne CoD4 while it's predecessor CoD2,with the same engine runs fiine on Gen 3 graphics,after modding the dxwnd proxy code,i managed to solve several problem that stop it from running on gen3 graphics,but due to lack of understanding of the directx COM objects data structures,some ugly hack was used to temporay solve the problem.
The biggest barracade is the D3D9Query type,as the intel Gen3 graphics is actually based on DX7-level hardware design,it lacks routine for modern feature like occlusion query,so it's not p;ossbile to using the native d3d9query DDI in the GMA driver as it non-exist at all ,so i am trying to make stub query9 routine in dxwnd but obviously the hack-style workaround works not very good.the dxwnd completely lacks code for this routine,and due to it's just proxy for real d3d9 interface,i don't know how to directly emualte and return d3d9 query data back to the game instead of the hackish using some similar debugging type to emulate the return pointers, and such methos turned out to be not good for performance with the wrong return value.
So,is it possble to add some simple code to emulate these d3dquery routines without sending to the real DDI, especially for

typedef enum D3DQUERYTYPE { 
  D3DQUERYTYPE_EVENT              = 8,
  D3DQUERYTYPE_OCCLUSION          = 9,

these two types that heavliy used by these games,which just refuse to run even it can run with such query.

and Here is a simple demo of running CoD4 on GMA3100

[Venis]

Many details are exist in codes and difficult to post here directly

[Venis]

i tried to copy from the swiftshader routines in Query9,but the codes from transgaming is completely rubbish to understand

[gho]

So,is it possble to add some simple code to emulate these d3dquery routines without sending to the real DDI, especially for ...

I suppose it's possible and hopefully it shouldn't be too hard. But I need to know which answer should be returned from a query intercepted by a DxWnd hook.
Redirecting the queries to an internal logic should give simple answers, like these:

D3DQUERYTYPE_EVENT: Query for any and all asynchronous events that have been issued from API calls.

Could return no events.

D3DQUERYTYPE_OCCLUSION: An occlusion query returns the number of pixels (or samples when multisampling is enabled) that pass z-testing. These pixels/samples are for primitives drawn between the issue of D3DISSUE_BEGIN and D3DISSUE_END. This enables an application to check the occlusion result against 0. Zero is fully occluded, which means the pixels/samples are not visible from the current camera position. To get the number of pixels when a multisampled render target is used, the result should be divided by the sample count of the target.

Here I don't know which could be best, but the query could return either 0, 1 or a big number like WxH of the screen...

[Venis]

Could return no events.

It has been a few months that i modify the code almost forgot about the details,but I remember the EVENT type is far more difficult to handle,simply return D3D_OK to the game dosen't meet all of its demand.

Here I don't know which could be best, but the query could return either 0, 1 or a big number like WxH of the screen...

The dev guide suggest that occlusion query returns the pixels that were visible,greater than 0 is enough,but more investigate into the field game will be needed.

The biggest chanlenge is the as Dxwnd is just an 'proxy' layer for D3D APIs,it relys on the return data pointer from the real driver DDI,I tried to hardcode it but ends in illegal access at most time,this is what i was stuck at

[Venis]

I suppose it's possible and hopefully it shouldn't be too hard. But I need to know which answer should be returned from a query intercepted by a DxWnd hook.

It's not some simple 'return value",Direct3D generate a lot of C++ style COM dynamic pointers,which you cannot manual generate one,you have to using it's own pointer manager to allocate one pointer for those query,that's why i saidd i redirect it to some debugging query type in debug build of D3D runtime.,this is biggest problem that I can not solve

[Venis]

I will try to recover the errors and solving code to here,hope it will envoled to a general workaround fix in Dxwnd for the vast GMA950 users which is still active for such years

[gho]

Inside DxWnd source code there are several examples of fake COM objects. For instance, the virtual Joystick in the COM wrappers for DirectInput, or the "Bypass DirectSound" feature that builds void DirectSound objects.
You can download the dxwnd.dll sources and try to implement the COM objects by yourself, or share your patched code and give me the possibility to fix it and integrate it in DxWnd solution (credits for the feature will be yours).

[Venis]

I will upload the mod codes as soon as possible after clean a bit of many debug printf lines,refer to the microsoft reference code,it shows the COM pointer that difficult to emulate

IDirect3DQuery9* pOcclusionQuery = NULL;
m_pD3DDevice->CreateQuery( D3DQUERYTYPE_OCCLUSION, &pOcclusionQuery );
//**<---games check the validation of pOcclusionQuery pointer which return from real d3d9 runtime.**

// Add a begin marker to the command buffer queue.
pOcclusionQuery->Issue( D3DISSUE_BEGIN );

... // API calls

// Add an end marker to the command buffer queue.
pOcclusionQuery->Issue( D3DISSUE_END );

// Avoid flushing and letting the CPU go idle by not using a while loop.
// Check if queries are finished:
DWORD dwOccluded = 0;
if( S_FALSE == pOcclusionQuery->GetData( &dwOccluded, sizeof(DWORD), 0 ) )
{
    // Query is not done yet or object not occluded; avoid flushing/wait by continuing with worst-case scenario
    pSomeComplexMesh->Render();
}
else if( dwOccluded != 0 )
{
    // Query is done and object is not occluded.
    pSomeComplexMesh->Render();
}

[Venis]

I worked on this build,and it's a bit painful to fix the missing Query9 interface,for reference

[gho]

I also turned the request to Narzoul, the author of DDrawCompat, to get his opinion.
You can read the thread here https://github.com/narzoul/DDrawCompat/issues/619#issue-5448489408 .
He focused on the problem of defining a valid answer to the occlusion query: reasonably the only possibility to give an answer with no internal knowledge would be to reply that nothing was occluded, but this would heavily impact on the performances because all polygons would be processed. If you are aware of the problem and accept the consequences (poor FPS) I will try to implement something.
But this will require some patience and your collaboration (I don't think I have any suitable video card to make the tests), so please stay tuned on this topic (I suggest that you subscribe on the topic, see the envelope icon on the top of the thread page).

[Venis]

but this would heavily impact on the performances because all polygons would be processed

In fact,the 2004-2006 era games indeed request for occlusioin query,but they actually works well without it,one example is the CoD2.
Morever,the core solution to this is how to generate an valid pointer like the real D3D runtime returns,i didn't find anyway to emulate such windows handle-like pointer back to the game,nor any github users did some similar research on this,maybe due to D3D9 is history,
the only good reference is swiftshader,but its code are really hard to understand

[gho]

Well, in the meantime I tried to align your code updates to my last release, also adding some logs and using the debug flag EXPERIMENTAL2 to enable your changes.
This way I should be able to easily compare the D3D9 behavior with and without the query hacks. At the moment I didn't add the COM objects generation yet, I want to be sure first of all that the added hooks are not impacting on the normal behavior.
Do you suggest CoD2 as a suitable test program? The smaller and simpler it is, the better for the tests!
Soon I will upload the modified sources so that you could be able to align your code: the current WIP release is much newer, v2.06.16.

I also noted that in the extD3DGetDeviceCaps wrapper you forced specific values for all the fields of the DevCaps structure. In a general tool this should not happen and I didn't copy your modification about it. Was it necessary? Can I just omit that?

Also, could you explain the purpose of the "D3DFMT_R32F hack" ?

Same clarification request for the following code in the CheckDeviceFormat wrapper:

    res = (*pCheckDeviceFormat)(lpd3d, Adapter, DeviceType, AdapterFormat, Usage, RType, CheckFormat);
    // here a D3DERR_NOTAVAILABLE return code is a normal case
    if (Usage == 0x2 & RType == 3){
        res = D3DERR_NOTAVAILABLE;
    }
    if (Usage == 0x2 & CheckFormat == 83){
        res = D3DERR_NOTAVAILABLE;
    }
    /*
    if (Usage == 0x1 & CheckFormat == 114){
        res = D3D_OK;
    }
    */

[gho]

more on the subject: I downloaded the "Call of Duty 2" game demo to start testing the new hook, but (unless I made some big mistake) CoD2 doesn't call the CreateQuery method. I am now downloading the CoD4 demo, hopefully I'll have a better luck.

edit: I couldn't get any evidence of CreateQuery on "Call on Duty 4" either. I can't help unless I can build a testbed. Why do you think I can't get a CreateQuery invocation here?

[Venis]

Do you suggest CoD2 as a suitable test program? The smaller and simpler it is, the better for the tests!

I mentioned the CoD2 just want to suggest that it use the merely the same IW engine for CoD4,and it works well on the Intel Gen3 graphics with out occlusion query,which means that the occlusion query is not that neccessary at least for this engine,it will not be slower without queries ,same for CoD4 i guess.

Soon I will upload the modified sources so that you could be able to align your code: the current WIP release is much newer, v2.06.16.
I also noted that in the extD3DGetDeviceCaps wrapper you forced specific values for all the fields of the DevCaps structure. In a general tool this should not happen and I didn't copy your modification about it. Was it necessary? Can I just omit that?

There are many debug test lines for CoD4 may not be suitable for merge into mainline,i didn't clean the codebase in time.these debug lines were used to find out the actual root cause that making the D3D runtime error when I test the CoD4,it will only gives you an "DirectX encontered an unrecoverbale error"with out any debug information,its inner routine in the program were even found with reverse-engineering plus some assumntion.

Also, could you explain the purpose of the "D3DFMT_R32F hack" ?

CoD4 will have only 1 time call of render an surface of R32F format,which turns out to be the famous "Create2DTexture($floatz,800,600,0,114) failed' error on Gen3 graphics,but I remember it can be turned off by using r_floatz configuration,I just use this to test its logiic.

Same clarification request for the following code in the CheckDeviceFormat wrapper:

The IW engine in CoD4 checks the D3DFMT support and it has at least 2 different routine for different support,its complex,i remember one fixed 24bit D3DFMT routine get into unsolveable situation as i didn't fine the cause of the last directx error,i used the R32F routine i guess,although it was called R32F routine,R32F were never called in its routine at all

more on the subject: I downloaded the "Call of Duty 2" game demo to start testing the new hook, but (unless I made some big mistake) CoD2 doesn't call the CreateQuery method. I am now downloading the CoD4 demo, hopefully I'll have a better luck.

edit: I couldn't get any evidence of CreateQuery on "Call on Duty 4" either. I can't help unless I can build a testbed. Why do you think I can't get a CreateQuery invocation here?

it should both have queries,but appear as an "check" for different type of queries,i still a bit confused about its actaull routine,i didn't go far last time on this aspect,just proved that at least the rendering material is full compatiable onf Gen3 grahpics.

Yet another good target for one occlusion query is Battlefield 2,it calls the query at startup without any pre-condition,i think it's much easier for you to test this function out.CoD's engine will check the D3DCaps and performs completely different on modern hardware unless you have an physical Gen3 grahphics machine.
You can get it from here
https://download.nvidia.com/downloads/nZone/demos/BF2Demo.zip

[gho]

It would be good to align your version to the current DxWnd sources.
If you want to do it, you can download the sources of dxwnd.dll from the "v2.06.16 work in progress" thread here https://sourceforge.net/p/dxwnd/discussion/general/thread/e7aec06d6f/#59ba , then replace the file hd3d.cpp with the modified copy here in attach.
This new hd3d.cpp is aligned to the last release and includes your changes inside the #ifdef HOOKD3D9QUERYINTERFACE clause, then enables part of your hacks when setting the Debug flag EXPERIMENTAL2 in the DxWnd Debug panel (see screenshot).

[Venis]

ause, then enables part of your hacks when setting the Debug flag EXPERIMENTAL2 in the DxWnd Debug panel (see screenshot).

Great work

[gho]

Update: I can't see logs from extCreateQuery9 also with "Battlefield 2". Maybe I should select another video card (my portable has a dual card, integrated Intel and nVidia) or maybe that depends on the unpatched GetCaps wrapper ...

[Venis]

Are you checking using Dxwind itself?I forget to say that Dxwnd will not caputure quite a lot of procedures even with log function,that's the misery again of C++ style COM objects

[Venis]

Besides,there exists Direct3D9 and Direct3DDevice9 type,I forget where the log function is,but if it's in the first one,no log will generated

[gho]

I found the problem, you conditioned this code line to the TRANSFORMANDLIGHT flag:

SetHook((void *)(**(DWORD **)ppD3Ddev9 + 472), extCreateQuery9, (void **)&pCreateQuery9, "CreateQuery(D9)");

deleting the if(dwFlags11 & TRANSFORMANDLIGHT) statement activate the hook (and the d3d error), so now I can investigate.
This is the error in dxwnd.log:

IDirect3DDevice9::CreateQuery: d3dd=0x26a20880 replacing D3DQUERYTYPE_EVENT->D3DQUERYTYPE_RESOURCEMANAGER
IDirect3DDevice9::CreateQuery: res=0x8876086a

where 0x8876086A is D3DERR_NOTAVAILABLE

[Venis]

I have made lot of switch,unfountrunately,forgot about most of it after such months,it's Jan 2026,that's why i said this code base is only suitable for test

[Venis]

D3DQUERYTYPE_RESOURCEMANAGER is and type available in debug build of d3d9 runtime,kind sort of hack style methods i have used :)

[gho]

I think the problem is here: the CreateQuery wrapper should not replace the query if the operation fails, it should create a fake COM object only in case of failure. The implementation could be like this:

HRESULT WINAPI extCreateQuery9(void *lpd3dd, D3DQUERYTYPE Type, IDirect3DQuery9** ppQuery)
{
    HRESULT res;
    ApiName("IDirect3DDevice9::CreateQuery");
    OutTraceD3D("%s: d3dd=%#x Type=%d(%s)\n", ApiRef, lpd3dd, Type, sQueryType(Type));

    res = (*pCreateQuery9)(lpd3dd, Type, ppQuery);

    if(res && (dxw.dwDFlags2 & EXPERIMENTAL2)){
        // create a fake IDirect3DQuery9 COM object
        res = D3D_OK;
    }

    if(!res) HookQuery9(*ppQuery);
    OutTraceD3D("%s: res=%#x\n", ApiRef, res);
    return res;
}

where obviously the comment line marks where the operation should be done ...

[Venis]

That's why i said that used an hack-style method as an temporary workaround,i redirect it to another debug type that decoupled with the real intel driver interface