FLOPs Calculation

[OctoNinja9]

Hi, @Haiyang-W! You have done a very interesting work. However, I encounter a problem when calculating the FLOPs of the GiT model. When I run python tools/analysis_tools/get_flops.py, it output 0 FLOPs.

loading annotations into memory...
Done (t=0.16s)
creating index...
index created!
The tokenizer class you load from this checkpoint is not the same type as the class this function is called from. It may result in unexpected tokenization. 
The tokenizer class you load from this checkpoint is 'BertTokenizer'. 
The class this function is called from is 'BlipTokenizer'.
10/08 15:06:10 - mmengine - WARNING - The following submodules of the model were never called during the trace of the graph. They may be unused, or they were accessed by direct calls to .forward() or via other python methods. In the latter case they will have zeros for statistics, though their statistics will still contribute to their parent calling module.
backbone, backbone.drop_after_pos, backbone.layers.0, backbone.layers.0.attn, backbone.layers.0.attn.proj, backbone.layers.0.attn.qkv, backbone.layers.0.ffn, backbone.layers.0.ffn.dropout_layer, backbone.layers.0.ffn.layers, backbone.layers.0.ffn.layers.0, backbone.layers.0.ffn.layers.0.0, backbone.layers.0.ffn.layers.0.1, backbone.layers.0.ffn.layers.0.2, backbone.layers.0.ffn.layers.1, backbone.layers.0.ffn.layers.2, backbone.layers.0.ln1, backbone.layers.0.ln2, backbone.layers.1, backbone.layers.1.attn, backbone.layers.1.attn.proj, backbone.layers.1.attn.qkv, backbone.layers.1.ffn, backbone.layers.1.ffn.dropout_layer, backbone.layers.1.ffn.layers, backbone.layers.1.ffn.layers.0, backbone.layers.1.ffn.layers.0.0, backbone.layers.1.ffn.layers.0.1, backbone.layers.1.ffn.layers.0.2, backbone.layers.1.ffn.layers.1, backbone.layers.1.ffn.layers.2, backbone.layers.1.ln1, backbone.layers.1.ln2, backbone.layers.10, backbone.layers.10.attn, backbone.layers.10.attn.proj, backbone.layers.10.attn.qkv, backbone.layers.10.ffn, backbone.layers.10.ffn.dropout_layer, backbone.layers.10.ffn.layers, backbone.layers.10.ffn.layers.0, backbone.layers.10.ffn.layers.0.0, backbone.layers.10.ffn.layers.0.1, backbone.layers.10.ffn.layers.0.2, backbone.layers.10.ffn.layers.1, backbone.layers.10.ffn.layers.2, backbone.layers.10.ln1, backbone.layers.10.ln2, backbone.layers.11, backbone.layers.11.attn, backbone.layers.11.attn.proj, backbone.layers.11.attn.qkv, backbone.layers.11.ffn, backbone.layers.11.ffn.dropout_layer, backbone.layers.11.ffn.layers, backbone.layers.11.ffn.layers.0, backbone.layers.11.ffn.layers.0.0, backbone.layers.11.ffn.layers.0.1, backbone.layers.11.ffn.layers.0.2, backbone.layers.11.ffn.layers.1, backbone.layers.11.ffn.layers.2, backbone.layers.11.ln1, backbone.layers.11.ln2, backbone.layers.12, backbone.layers.12.attn, backbone.layers.12.attn.proj, backbone.layers.12.attn.qkv, backbone.layers.12.ffn, backbone.layers.12.ffn.dropout_layer, backbone.layers.12.ffn.layers, backbone.layers.12.ffn.layers.0, backbone.layers.12.ffn.layers.0.0, backbone.layers.12.ffn.layers.0.1, backbone.layers.12.ffn.layers.0.2, backbone.layers.12.ffn.layers.1, backbone.layers.12.ffn.layers.2, backbone.layers.12.ln1, backbone.layers.12.ln2, backbone.layers.13, backbone.layers.13.attn, backbone.layers.13.attn.proj, backbone.layers.13.attn.qkv, backbone.layers.13.ffn, backbone.layers.13.ffn.dropout_layer, backbone.layers.13.ffn.layers, backbone.layers.13.ffn.layers.0, backbone.layers.13.ffn.layers.0.0, backbone.layers.13.ffn.layers.0.1, backbone.layers.13.ffn.layers.0.2, backbone.layers.13.ffn.layers.1, backbone.layers.13.ffn.layers.2, backbone.layers.13.ln1, backbone.layers.13.ln2, backbone.layers.14, backbone.layers.14.attn, backbone.layers.14.attn.proj, backbone.layers.14.attn.qkv, backbone.layers.14.ffn, backbone.layers.14.ffn.dropout_layer, backbone.layers.14.ffn.layers, backbone.layers.14.ffn.layers.0, backbone.layers.14.ffn.layers.0.0, backbone.layers.14.ffn.layers.0.1, backbone.layers.14.ffn.layers.0.2, backbone.layers.14.ffn.layers.1, backbone.layers.14.ffn.layers.2, backbone.layers.14.ln1, backbone.layers.14.ln2, backbone.layers.15, backbone.layers.15.attn, backbone.layers.15.attn.proj, backbone.layers.15.attn.qkv, backbone.layers.15.ffn, backbone.layers.15.ffn.dropout_layer, backbone.layers.15.ffn.layers, backbone.layers.15.ffn.layers.0, backbone.layers.15.ffn.layers.0.0, backbone.layers.15.ffn.layers.0.1, backbone.layers.15.ffn.layers.0.2, backbone.layers.15.ffn.layers.1, backbone.layers.15.ffn.layers.2, backbone.layers.15.ln1, backbone.layers.15.ln2, backbone.layers.16, backbone.layers.16.attn, backbone.layers.16.attn.proj, backbone.layers.16.attn.qkv, backbone.layers.16.ffn, backbone.layers.16.ffn.dropout_layer, backbone.layers.16.ffn.layers, backbone.layers.16.ffn.layers.0, backbone.layers.16.ffn.layers.0.0, backbone.layers.16.ffn.layers.0.1, backbone.layers.16.ffn.layers.0.2, backbone.layers.16.ffn.layers.1, backbone.layers.16.ffn.layers.2, backbone.layers.16.ln1, backbone.layers.16.ln2, backbone.layers.17, backbone.layers.17.attn, backbone.layers.17.attn.proj, backbone.layers.17.attn.qkv, backbone.layers.17.ffn, backbone.layers.17.ffn.dropout_layer, backbone.layers.17.ffn.layers, backbone.layers.17.ffn.layers.0, backbone.layers.17.ffn.layers.0.0, backbone.layers.17.ffn.layers.0.1, backbone.layers.17.ffn.layers.0.2, backbone.layers.17.ffn.layers.1, backbone.layers.17.ffn.layers.2, backbone.layers.17.ln1, backbone.layers.17.ln2, backbone.layers.2, backbone.layers.2.attn, backbone.layers.2.attn.proj, backbone.layers.2.attn.qkv, backbone.layers.2.ffn, backbone.layers.2.ffn.dropout_layer, backbone.layers.2.ffn.layers, backbone.layers.2.ffn.layers.0, backbone.layers.2.ffn.layers.0.0, backbone.layers.2.ffn.layers.0.1, backbone.layers.2.ffn.layers.0.2, backbone.layers.2.ffn.layers.1, backbone.layers.2.ffn.layers.2, backbone.layers.2.ln1, backbone.layers.2.ln2, backbone.layers.3, backbone.layers.3.attn, backbone.layers.3.attn.proj, backbone.layers.3.attn.qkv, backbone.layers.3.ffn, backbone.layers.3.ffn.dropout_layer, backbone.layers.3.ffn.layers, backbone.layers.3.ffn.layers.0, backbone.layers.3.ffn.layers.0.0, backbone.layers.3.ffn.layers.0.1, backbone.layers.3.ffn.layers.0.2, backbone.layers.3.ffn.layers.1, backbone.layers.3.ffn.layers.2, backbone.layers.3.ln1, backbone.layers.3.ln2, backbone.layers.4, backbone.layers.4.attn, backbone.layers.4.attn.proj, backbone.layers.4.attn.qkv, backbone.layers.4.ffn, backbone.layers.4.ffn.dropout_layer, backbone.layers.4.ffn.layers, backbone.layers.4.ffn.layers.0, backbone.layers.4.ffn.layers.0.0, backbone.layers.4.ffn.layers.0.1, backbone.layers.4.ffn.layers.0.2, backbone.layers.4.ffn.layers.1, backbone.layers.4.ffn.layers.2, backbone.layers.4.ln1, backbone.layers.4.ln2, backbone.layers.5, backbone.layers.5.attn, backbone.layers.5.attn.proj, backbone.layers.5.attn.qkv, backbone.layers.5.ffn, backbone.layers.5.ffn.dropout_layer, backbone.layers.5.ffn.layers, backbone.layers.5.ffn.layers.0, backbone.layers.5.ffn.layers.0.0, backbone.layers.5.ffn.layers.0.1, backbone.layers.5.ffn.layers.0.2, backbone.layers.5.ffn.layers.1, backbone.layers.5.ffn.layers.2, backbone.layers.5.ln1, backbone.layers.5.ln2, backbone.layers.6, backbone.layers.6.attn, backbone.layers.6.attn.proj, backbone.layers.6.attn.qkv, backbone.layers.6.ffn, backbone.layers.6.ffn.dropout_layer, backbone.layers.6.ffn.layers, backbone.layers.6.ffn.layers.0, backbone.layers.6.ffn.layers.0.0, backbone.layers.6.ffn.layers.0.1, backbone.layers.6.ffn.layers.0.2, backbone.layers.6.ffn.layers.1, backbone.layers.6.ffn.layers.2, backbone.layers.6.ln1, backbone.layers.6.ln2, backbone.layers.7, backbone.layers.7.attn, backbone.layers.7.attn.proj, backbone.layers.7.attn.qkv, backbone.layers.7.ffn, backbone.layers.7.ffn.dropout_layer, backbone.layers.7.ffn.layers, backbone.layers.7.ffn.layers.0, backbone.layers.7.ffn.layers.0.0, backbone.layers.7.ffn.layers.0.1, backbone.layers.7.ffn.layers.0.2, backbone.layers.7.ffn.layers.1, backbone.layers.7.ffn.layers.2, backbone.layers.7.ln1, backbone.layers.7.ln2, backbone.layers.8, backbone.layers.8.attn, backbone.layers.8.attn.proj, backbone.layers.8.attn.qkv, backbone.layers.8.ffn, backbone.layers.8.ffn.dropout_layer, backbone.layers.8.ffn.layers, backbone.layers.8.ffn.layers.0, backbone.layers.8.ffn.layers.0.0, backbone.layers.8.ffn.layers.0.1, backbone.layers.8.ffn.layers.0.2, backbone.layers.8.ffn.layers.1, backbone.layers.8.ffn.layers.2, backbone.layers.8.ln1, backbone.layers.8.ln2, backbone.layers.9, backbone.layers.9.attn, backbone.layers.9.attn.proj, backbone.layers.9.attn.qkv, backbone.layers.9.ffn, backbone.layers.9.ffn.dropout_layer, backbone.layers.9.ffn.layers, backbone.layers.9.ffn.layers.0, backbone.layers.9.ffn.layers.0.0, backbone.layers.9.ffn.layers.0.1, backbone.layers.9.ffn.layers.0.2, backbone.layers.9.ffn.layers.1, backbone.layers.9.ffn.layers.2, backbone.layers.9.ln1, backbone.layers.9.ln2, backbone.patch_embed, backbone.patch_embed.adaptive_padding, backbone.patch_embed.projection, data_preprocessor, embed, embed.LayerNorm, embed.dropout, embed.position_embeddings, embed.word_embeddings, synt_att, synt_att.out_proj, synt_position_embeddings
==============================
Use size divisor set input shape from (480, 640) to (1120, 1120)
==============================
Compute type: dataloader: load a picture from the dataset
Input shape: (1120, 1120)
Flops: 0
Params: 0.159G
==============================

[Haiyang-W]

I'm sorry, we haven't tested this part of the code. If I have time later, I will improve this part of the code.