Issue with GNNExplainer and TransformerConv Layer in PyTorch Geometric

[muchen1453]

Hi Everyone!

I'm using the GNNExplainer from the torch_geometric.explain module to interpret the nodes in a GNN model that includes several 'TransformerConv' layers. However, I'm encountering an issue with the following error: AttributeError: type object 'TransformerConv' has no attribute '_orig_propagate'. When I switched the 'TransformerConv' to a 'GCNConv' layer, the error disappeared, so I'm assuming my code is actually correct.

Here are the details of the error. I would greatly appreciate any assistance!

Best regards,

---

AttributeError                            Traceback (most recent call last)
Cell In[9], line 23
     10 explainer = Explainer(
     11     model=model_saved,
     12     algorithm=GNNExplainer(epochs=200),
   (...)
     20     ),
     21 )
     22 node_index = 10
---> 23 explanation = explainer(data.x, data.edge_index, index=node_index,batch_index=data.batch )
     24 print(f'Generated explanations in {explanation.available_explanations}')
     26 path = 'feature_importance.png'

File ~/.conda/envs/pytorch/lib/python3.11/site-packages/torch_geometric/explain/explainer.py:205, in Explainer.__call__(self, x, edge_index, target, index, **kwargs)
    202 training = self.model.training
    203 self.model.eval()
--> 205 explanation = self.algorithm(
    206     self.model,
    207     x,
    208     edge_index,
    209     target=target,
    210     index=index,
    211     **kwargs,
    212 )
    214 self.model.train(training)
    216 # Add explainer objectives to the `Explanation` object:

File ~/.conda/envs/pytorch/lib/python3.11/site-packages/torch/nn/modules/module.py:1511, in Module._wrapped_call_impl(self, *args, **kwargs)
   1509     return self._compiled_call_impl(*args, **kwargs)  # type: ignore[misc]
   1510 else:
-> 1511     return self._call_impl(*args, **kwargs)

File ~/.conda/envs/pytorch/lib/python3.11/site-packages/torch/nn/modules/module.py:1520, in Module._call_impl(self, *args, **kwargs)
   1515 # If we don't have any hooks, we want to skip the rest of the logic in
   1516 # this function, and just call forward.
   1517 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks
   1518         or _global_backward_pre_hooks or _global_backward_hooks
   1519         or _global_forward_hooks or _global_forward_pre_hooks):
-> 1520     return forward_call(*args, **kwargs)
   1522 try:
   1523     result = None

File ~/.conda/envs/pytorch/lib/python3.11/site-packages/torch_geometric/explain/algorithm/gnn_explainer.py:87, in GNNExplainer.forward(self, model, x, edge_index, target, index, **kwargs)
     83 if isinstance(x, dict) or isinstance(edge_index, dict):
     84     raise ValueError(f"Heterogeneous graphs not yet supported in "
     85                      f"'{self.__class__.__name__}'")
---> 87 self._train(model, x, edge_index, target=target, index=index, **kwargs)
     89 node_mask = self._post_process_mask(
     90     self.node_mask,
     91     self.hard_node_mask,
     92     apply_sigmoid=True,
     93 )
     94 edge_mask = self._post_process_mask(
     95     self.edge_mask,
     96     self.hard_edge_mask,
     97     apply_sigmoid=True,
     98 )

File ~/.conda/envs/pytorch/lib/python3.11/site-packages/torch_geometric/explain/algorithm/gnn_explainer.py:123, in GNNExplainer._train(self, model, x, edge_index, target, index, **kwargs)
    121     parameters.append(self.node_mask)
    122 if self.edge_mask is not None:
--> 123     set_masks(model, self.edge_mask, edge_index, apply_sigmoid=True)
    124     parameters.append(self.edge_mask)
    126 optimizer = torch.optim.Adam(parameters, lr=self.lr)

File ~/.conda/envs/pytorch/lib/python3.11/site-packages/torch_geometric/explain/algorithm/utils.py:34, in set_masks(model, mask, edge_index, apply_sigmoid)
     30 if (not isinstance(mask, Parameter)
     31         and '_edge_mask' in module._parameters):
     32     mask = Parameter(mask)
---> 34 module.explain = True
     35 module._edge_mask = mask
     36 module._loop_mask = loop_mask

File ~/.conda/envs/pytorch/lib/python3.11/site-packages/torch/nn/modules/module.py:1747, in Module.__setattr__(self, name, value)
   1745     buffers[name] = value
   1746 else:
-> 1747     super().__setattr__(name, value)

File ~/.conda/envs/pytorch/lib/python3.11/site-packages/torch_geometric/nn/conv/message_passing.py:747, in MessagePassing.explain(self, explain)
    742     self.inspector.inspect_signature(self.explain_message, exclude=[0])
    743     self._user_args = self.inspector.get_flat_param_names(
    744         funcs=['message', 'explain_message', 'aggregate', 'update'],
    745         exclude=self.special_args,
    746     )
--> 747     self.propagate = self.__class__._orig_propagate.__get__(
    748         self, MessagePassing)
    749 else:
    750     self._user_args = self.inspector.get_flat_param_names(
    751         funcs=['message', 'aggregate', 'update'],
    752         exclude=self.special_args,
    753     )

AttributeError: type object 'TransformerConv' has no attribute '_orig_propagate'

[akihironitta]

I see some similar regressions in recent versions of PyG, e.g., #9035 and #9092 (and I see your comment there, too).

Can you try the latest release 2.5.3 (and also try master if 2.5.3 doesn't work)? The installation guide is here: https://github.com/pyg-team/pytorch_geometric/?tab=readme-ov-file#nightly-and-master

Would be very nice if you could provide us with your script to reproduce this.

[muchen1453]

Thanks for the suggestion, I'm currently using PyG 2.5.2. I'll try with the latest release!

Best

[muchen1453]

GNNExplainer now works in pyg-nightly 2.6.0, thanks again!

[rusty1s]

Added a test for this: #9451

Otherwise, I agree with @akihironitta. Can you make sure you are on latest 2.5.3 version?

[muchen1453]

I'm currently using PyG 2.5.2. I'll try with the 2.5.3 version and let you know the result!

Best

[muchen1453]

GNNExplainer now works in pyg-nightly 2.6.0, thanks again!