[Feature] Add BERT SWAG Multiple Choice application (Ref: #ID9IPX)

[Magnus0100]

Purpose: Added a new NLP application case for Multiple Choice tasks using BERT on the SWAG dataset.

Issue: Ref: #ID9IPX

Approach: Implemented using MindSpore 2.7.1 and Mindhf 0.6.0

Changes
Added applications/nlp/multiple_choice/swag_bert_run_result.ipynb.

Updated applications/nlp/README.md to include the new case in the application list.

Inference demo runs successfully on 910B2.

ref:https://gitee.com/mindspore/community/issues/ID9IPX

[moyu026]

运行出错：

ValueError Traceback (most recent call last)
Cell In[8], line 14
11 tok = tokenizer(first, second, truncation=True, padding=True, return_tensors="ms")
12 inputs = {k: v.reshape((1, 4, -1)) for k, v in tok.items()}
---> 14 outputs = model(**inputs)
15 logits = outputs["logits"] if isinstance(outputs, dict) else outputs.logits
16 pred = int(np.argmax(logits.asnumpy(), axis=1)[0])

File /usr/local/python3.10.14/lib/python3.10/site-packages/mindtorch/nn/modules/module.py:826, in Module._wrapped_call_impl(self, *args, **kwargs)
824 if self._compiled_call_impl is not None:
825 return self._compiled_call_impl(*args, **kwargs) # type: ignore[misc]
--> 826 return self._call_impl(*args, **kwargs)

File /usr/local/python3.10.14/lib/python3.10/site-packages/mindtorch/nn/modules/module.py:840, in Module._call_impl(self, *args, **kwargs)
835 return forward_call(*args, **kwargs)
837 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks
838 or _global_backward_pre_hooks or _global_backward_hooks
839 or _global_forward_hooks or _global_forward_pre_hooks):
--> 840 return forward_call(*args, **kwargs)
842 try:
843 result = None

File /usr/local/python3.10.14/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py:1605, in BertForMultipleChoice.forward(self, input_ids, attention_mask, token_type_ids, position_ids, head_mask, inputs_embeds, labels, output_attentions, output_hidden_states, return_dict)
1598 position_ids = position_ids.view(-1, position_ids.size(-1)) if position_ids is not None else None
1599 inputs_embeds = (
1600 inputs_embeds.view(-1, inputs_embeds.size(-2), inputs_embeds.size(-1))
1601 if inputs_embeds is not None
1602 else None
1603 )
-> 1605 outputs = self.bert(
1606 input_ids,
1607 attention_mask=attention_mask,
1608 token_type_ids=token_type_ids,
1609 position_ids=position_ids,
1610 head_mask=head_mask,
1611 inputs_embeds=inputs_embeds,
1612 output_attentions=output_attentions,
1613 output_hidden_states=output_hidden_states,
1614 return_dict=return_dict,
1615 )
1617 pooled_output = outputs[1]
1619 pooled_output = self.dropout(pooled_output)

File /usr/local/python3.10.14/lib/python3.10/site-packages/mindtorch/nn/modules/module.py:826, in Module._wrapped_call_impl(self, *args, **kwargs)
824 if self._compiled_call_impl is not None:
825 return self._compiled_call_impl(*args, **kwargs) # type: ignore[misc]
--> 826 return self._call_impl(*args, **kwargs)

File /usr/local/python3.10.14/lib/python3.10/site-packages/mindtorch/nn/modules/module.py:840, in Module._call_impl(self, *args, **kwargs)
835 return forward_call(*args, **kwargs)
837 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks
838 or _global_backward_pre_hooks or _global_backward_hooks
839 or _global_forward_hooks or _global_forward_pre_hooks):
--> 840 return forward_call(*args, **kwargs)
842 try:
843 result = None

File /usr/local/python3.10.14/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py:936, in BertModel.forward(self, input_ids, attention_mask, token_type_ids, position_ids, head_mask, inputs_embeds, encoder_hidden_states, encoder_attention_mask, past_key_values, use_cache, output_attentions, output_hidden_states, return_dict, cache_position)
933 else:
934 token_type_ids = torch.zeros(input_shape, dtype=torch.long, device=device)
--> 936 embedding_output = self.embeddings(
937 input_ids=input_ids,
938 position_ids=position_ids,
939 token_type_ids=token_type_ids,
940 inputs_embeds=inputs_embeds,
941 past_key_values_length=past_key_values_length,
942 )
944 if attention_mask is None:
945 attention_mask = torch.ones((batch_size, seq_length + past_key_values_length), device=device)

File /usr/local/python3.10.14/lib/python3.10/site-packages/mindtorch/nn/modules/module.py:826, in Module._wrapped_call_impl(self, *args, **kwargs)
824 if self._compiled_call_impl is not None:
825 return self._compiled_call_impl(*args, **kwargs) # type: ignore[misc]
--> 826 return self._call_impl(*args, **kwargs)

File /usr/local/python3.10.14/lib/python3.10/site-packages/mindtorch/nn/modules/module.py:840, in Module._call_impl(self, *args, **kwargs)
835 return forward_call(*args, **kwargs)
837 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks
838 or _global_backward_pre_hooks or _global_backward_hooks
839 or _global_forward_hooks or _global_forward_pre_hooks):
--> 840 return forward_call(*args, **kwargs)
842 try:
843 result = None

File /usr/local/python3.10.14/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py:179, in BertEmbeddings.forward(self, input_ids, token_type_ids, position_ids, inputs_embeds, past_key_values_length)
176 token_type_ids = torch.zeros(input_shape, dtype=torch.long, device=self.position_ids.device)
178 if inputs_embeds is None:
--> 179 inputs_embeds = self.word_embeddings(input_ids)
180 token_type_embeddings = self.token_type_embeddings(token_type_ids)
182 embeddings = inputs_embeds + token_type_embeddings

File /usr/local/python3.10.14/lib/python3.10/site-packages/mindtorch/nn/modules/module.py:826, in Module._wrapped_call_impl(self, *args, **kwargs)
824 if self._compiled_call_impl is not None:
825 return self._compiled_call_impl(*args, **kwargs) # type: ignore[misc]
--> 826 return self._call_impl(*args, **kwargs)

File /usr/local/python3.10.14/lib/python3.10/site-packages/mindtorch/nn/modules/module.py:840, in Module._call_impl(self, *args, **kwargs)
835 return forward_call(*args, **kwargs)
837 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks
838 or _global_backward_pre_hooks or _global_backward_hooks
839 or _global_forward_hooks or _global_forward_pre_hooks):
--> 840 return forward_call(*args, **kwargs)
842 try:
843 result = None

File /usr/local/python3.10.14/lib/python3.10/site-packages/mindtorch/nn/modules/sparse.py:192, in Embedding.forward(self, input)
191 def forward(self, input: Tensor) -> Tensor:
--> 192 return F.embedding(
193 input,
194 self.weight,
195 self.padding_idx,
196 self.max_norm,
197 self.norm_type,
198 self.scale_grad_by_freq,
199 self.sparse,
200 )

File /usr/local/python3.10.14/lib/python3.10/site-packages/mindtorch/nn/functional.py:184, in embedding(input, weight, padding_idx, max_norm, norm_type, scale_grad_by_freq, sparse)
183 def embedding(input, weight, padding_idx=None, max_norm=None, norm_type=2.0, scale_grad_by_freq=False, sparse=False):
--> 184 return execute('embedding', input, weight, padding_idx, max_norm, norm_type, scale_grad_by_freq)

File /usr/local/python3.10.14/lib/python3.10/site-packages/mindtorch/executor.py:6, in execute(func_name, *args, **kwargs)
5 def execute(func_name, *args, **kwargs):
----> 6 out, device = dispatcher.dispatch(func_name, *args, **kwargs)
7 # if MS27:
8 # return out
10 if not isinstance(out, (tuple, list)):

File /usr/local/python3.10.14/lib/python3.10/site-packages/mindtorch/dispatcher.py:42, in Dispatcher.dispatch(self, func_name, *args, **kwargs)
40 devices = {tensor.device for tensor in tensors}
41 if len(devices) > 1:
---> 42 raise ValueError("All tensor arguments must be on the same device.")
44 device = next(iter(devices), mindtorch.device("cpu"))
46 if DEVICE_TARGET == "Ascend" and device.type == "cuda":

ValueError: All tensor arguments must be on the same device.

[Magnus0100]

已修复问题并且修改格式，请重写测试。

[moyu026]

MindSpore 2.7.0和MindSpore NLP 0.5.1版本可以运行

[Magnus0100]

clean and rename SWAG multiple choice notebook

[xing-yiren]

1. 呃其实这个严格来说应该属于llm领域的，放到llm路径下吧，同步路径改成模型名称，更新下readme
2. 多余的这种.ds_Store的Mac上的数据文件删掉
3. ipynb内容中涉及MindNLP建议写成MindSpore NLP全名

整体挺好的，赞

[Magnus0100]

移动并清理文件[Move BERT SWAG notebook to llm]

[xing-yiren]

我这边没问题了，辛苦 @wang-hua-2019 也把把关