OLE Automation Dependency

[cristianbuse]

Recently, there were 2 issues raised regarding IUnknown not being defined. See #3 and #4

The IUnknown interface definition is part of OLE Automation (stdole2.tlb). Although the OLE Automation reference should be on by default, this does not seem to always be the case for Mac users. The easy fix was to ask the users to add the needed reference.

Should we continue using IUnknown?

It is currently used in 2 separate places in this repo:

• As the return interface for NewEnum. This was initially IEnumVariant but IEnumVariant is not present in the Mac OLE Automation so IUnknown is used instead
• As a means to get a pointer for keys of vbObject or vbDataObject var type. This is to enforce an object instance to be hashed to the same value regardless of what interface of that instance is being used. See here

For the return type of NewEnum, we can simply replace with Variant and everything will continue to work fine.

However, getting a common interface for any object instance is more problematic. For example, let's assume we have a Class1 which Implements Class2:

Sub TestObjPtr()
    Dim c1 As New Class1
    Dim c2 As Class2: Set c2 = c1
    '
    Debug.Print ObjPtr(c1)
    Debug.Print ObjPtr(c2)
End Sub

will print something like:

 2190324407568 
 2190324407680 

which would lead to 2 different hash values, and we definitely do not want that. Scripting.Dictionary.HashVal returns same hash value for both interfaces.

The current approach of asking for the IUnknown interface and only then calling ObjPtr will return the same pointer regardless of what the initial interface was.

The problem in VBA is that we do not have such a common interface like IUnknown defined in the VBA library itself. We have Object but that is basically IDispatch. As explained here, whenever we ask for the Object interface, a call to IDispatch::QueryInterface is made only if the interface being queried is not derived from IDispatch (i.e. it's derived from IUnknown).

What is the alternative?

As explained at the end of the above Stack Overflow answer, we can use the VBEGlobal interface which is only derived from IUnknown to force a QueryInterface call:

Public Function GetDefaultInterface(ByVal obj As Object) As Object
    Dim g As VBEGlobal
    MemLongPtr(VarPtr(g)) = ObjPtr(obj)
    Set GetDefaultInterface = g
    MemLongPtr(VarPtr(g)) = NULL_PTR
End Function

This completely removes the need for having OLE Automation's IUnknown interface but introduces the following 2 problems:

• Instead of the simple Set iUnk = Key assignment, we now need an extra pointer memory accessor. This is not really a big deal as one is already available for hashing numbers on x64 and it's easy to update the code to reuse it for the purpose of forcing a QueryInterface call
• What if the incoming Key is not derived from IDispatch? We would simply not be able to assign to a variable of type Object without raising an error. A quick example:

Sub ErrTest()
    Dim c As New Collection
    Dim o As Object
    '
    Set o = c.[_NewEnum] 'Raises error 13 - Type mismatch
End Sub

The error is raised because [_NewEnum] (IEnumVariant) is not derived from IDispatch

The only way I can think of, to avoid this problem, is to use error handling but this will greatly impact performance when it comes to hashing keys of object type.

Goal of this discussion

It would be very helpful to see other's opinion on this and if it's worth removing the OLE Automation dependency or to just simply add a note in the main Readme to ask users to make sure the reference is on.

Thanks!

[guwidoe]

So as I understand it we have 3 Options:

1. Leave everything as is and add the reference issue to the documentation
2. Use VBEGlobal and just ignore the problem of Keys not derived from IDispatch hoping that this will occur only very rarely
3. Use VBEGlobal and implement error handling for Keys not derived from IDispatch and accept a significant performance impact for Object Keys

If I understood that correctly, I would tend towards option 1, but I might very well have misunderstood something 😅

[MarioAnthes]

As the OLE Automation reference is usually on by default I would also leave everything as is and add the reference issue to the main Readme.

[Greedquest]

Option 4

Write a little bit of code to invoke query interface on the key manually with iid_iunknown

This could be implemented as a method of the dictionary class with a vtable swap and written in assembly for performance -it's a very simple method. The iid_iunknown data could be a static variable so only calculated once.

Alternatively calling the query interface with dispcallfunc, although I guess that would be less performant since it requires passing arguments as variant arrays, but you would have to test

[cristianbuse]

Thanks @Greedquest ,

I did think of calling QueryInterface but gave up. I basically discarded DispCallFunc because of the API overhead in VBA7. I also discarded a vtable swap because I could not think of a way to swap the pointer. However, it did not occur to me that this can be done via assembly.

Are you saying it is possible to have a Private function inside Dictionary class that can receive the key (or the key pointer) and the iid_iunknown and that we can swap that method with an assembly method that can discard the implicit dictionary pointer and can call QueryInterface on the key pointer? That would be awesome.

The only downside I could think of is - would that work on Mac?

[Greedquest]

Does Mac have the VBA7 dll call issue? But dispcallfunc is not available on mac so probably a non-starter anyway see sancarn/stdVBA#2 (comment) and https://stackoverflow.com/q/76027766/6609896

VTable swap with a private function, substituting out a pointer to some assembly to circumvent dispcallfunc was what I was thinking, and should work on Mac although I'm not sure whether there is an equivalent of VirtualAlloc to allocate executable region, or even VirtualProtect to overwrite the existing function instructions (which typically would just be an entry point to the p-code interpreter). But in principle Mac is intel cpu so assembly bytecode would be the same as on windows IIUC.

However unfortunately bit of a catch: https://stackoverflow.com/a/77210739/6609896 vtable patching doesn't work on 64 bit vba for vba defined classes (I haven't tested mac, but Wayne didn't specify if mac was in the same situation, I imagine yes but you might get lucky!).

So then you get into defining a custom class from scratch to hold your static method. You would have to identify an interface that has the correct signature Function XXX(byref key as Variant) As Long so you could declare a static variable of that type and have your custom class implement that interface and method using the assembly bytecode. I think you did something similar once maybe, reusing a standard interface to call something entirely different.

[cristianbuse]

Does Mac have the VBA7 dll call issue?

It does not.

But dispcallfunc is not available on mac so probably a non-starter anyway

Indeed.

You would have to identify an interface that has the correct signature

I already looked for one and could not find one that matches exactly, unfortunately.

Many thanks!

[Greedquest]

Alternative I think since VBA can execute non-compiling code provided you don't enter that procedure, you could add an error in class_initialize the first time Ole Automation is not found in the activevbproject.references, and instruct the user to add it (even offer programmatically). Not sure if that requires access to the project to be enabled if you are just reading the references, probably does...

[cristianbuse]

Well, I avoided having a Class_Initialize event on purpose. I actually did all the initialization at runtime based on when it's needed. This was to avoid wasting memory when parsing JSON like [{},{},{},{}...], where the Dictionary is actually never used.

Not sure if that requires access to the project to be enabled if you are just reading the references, probably does

Probably does.

For now, I added a note that OLE Automation is needed but will keep this thread open. Many thanks!

[Greedquest]

I think it's entirely good practice and sets a precedent, to require standard dependencies that are usually enabled by default