Monday, April 25, 2011

Problem debugging shell extension in Vista (IShellFolder)

Hi,

I have created a Windows Shell Extension using ATL (Visual Studio 2008). It has the following modules and each module is a separate ATL Simple Object with its own .rgs file for registration:-

  1. IShellFolder/IShellView -> For a virtual drive in windows explorer
  2. IContextMenu/IShellExtInit -> For a popup menu files and folders
  3. IShellIconOverlayIdentifier - To display overlay icons on files and folders
  4. IShellPropSheetExt/IShellExtInit -> For a custom property page in File & Folder's properties

The above work fine in WinXP and I am able to debug this shell extension in WinXP. But a soon as I switch to Vista, I only get method calls for (2) and (4). The drive gets created in Windows Explorer but the underlying methods are not called when I click it. Although when I right click I get called for (2) and when I select "Properties" for a file or folder I get into the code for (4).

Is there some interface that I am missing for (1) and (3) on Vista. I could not find full and detailed documentation on writing Shell Extensions for Vista on MSDN. I had already gone through the CodeProject articles on shell extensions.

From stackoverflow
  • Make sure that your extensions are handling the initialization properly, otherwise the verclsid.exe won't let your extension getting loaded. verclsid.exe first loads the shell extensions and does a quick check on them before it sends an 'ok' to the shell indicating that it's safe to load the extension.

    See here for some details about verclsid.exe.

    A9S6 : I could not quite get the solution right. I included the verclsid.exe in the firewall's "Allowed" list but when I start debugging, it shows 'explorer.exe': Loaded '...\Debug\ShellExtension.dll' 'explorer.exe': Unloaded '...\Debug\ShellExtension.dll' And the breakpoint is not hit
    Stefan : It has nothing to do with a firewall. Unless you do some network stuff in your dll init function (which you *never* should do). the "(un)loaded" lines in the debug window don't help here. You can check in your init functions in which process the dll is loaded.
    A9S6 : I apologise for the late reply, I was away. By Init function, do you mean dllmain()? In there, I am not doing much ::CoInitialize(NULL); if (dwReason == DLL_PROCESS_ATTACH) { _AtlModule.Init(); ::DisableThreadLibraryCalls(hInstance); } hInstance; return _AtlModule.DllMain(dwReason, lpReserved);
    A9S6 : I doubt if IShellFolder and IPersistFolder are the only interfaces that needs to implemented in Vista. There might be another *required* interface
    Stefan : Not just dllmain - verclsid also does some testing on QueryInterface() and depending on what extension it is also on some other interfaces. And no: Vista does not require any interface to be implemented that wasn't on XP.
    A9S6 : Where can I find this QueryInterface() initialization?
    Stefan : QueryInterface is part of the IUnknown interface
    A9S6 : ya but I could not find the implementation.
  • The problem was because of a Xml SMART Pointer usage. It was has problems when Release() was called on it.

    Check this link http://stackoverflow.com/questions/641896/internet-explorer-crashes-when-msxml2ixmldomdocumentptr-release-is-called

  • Hi, I also have such problem, I have written a program to map the directory on ssh server to local desktop. It works well on XP, but not on Vista and win7( when click a folder, the items in the folder cannot display. Do you get the resolution? my email: wlm@veryopen.org, thanks

    A9S6 : There was a nice site on Shell programming - ShellRevealed. The site has been shut down but you can find some articles in the web archive. See this question: http://stackoverflow.com/questions/1237174/where-is-shellrevealed

0 comments:

Post a Comment