SequenceEmpty parameter dtype seems not work

[vincentme]

behavior

When create an empty sequence by SequenceEmpty, the dtype parameter seems does not work. Although in the code the type of the sequence and tensor to be inserted are the same. But sequence dtype seems always be float.

In the following code, the second and third parts will fail. The first work because the tensor is also float.

code to reproduce

from onnxscript import script, opset20 as op
import onnx
import numpy as np

@script()
def test(img_in):
    seq = op.SequenceEmpty(dtype=onnx.TensorProto.FLOAT)
    seq = op.SequenceInsert(seq, img_in)
    return seq
    
img_in = np.random.randn(128, 256).astype(np.float32)
res = test(img_in)


@script()
def test(img_in):
    seq = op.SequenceEmpty(dtype=onnx.TensorProto.FLOAT16)
    seq = op.SequenceInsert(seq, img_in)
    return seq
    
img_in = np.random.randn(128, 256).astype(np.float16)
res = test(img_in)

@script()
def test(img_in):
    seq = op.SequenceEmpty(dtype=onnx.TensorProto.DOUBLE)
    seq = op.SequenceInsert(seq, img_in)
    return seq
    
img_in = np.random.randn(128, 256).astype(np.float64)
res = test(img_in)

error message

Fail: [ONNXRuntimeError] : 1 : FAIL : Node () Op (SequenceInsert) [TypeInferenceError] Input Sequence and Tensor are expected to have the same elem type. Sequence=1 Tensor=11

[justinchuby]

Thanks for reporting! I remember this is a known issue that needs to be fixed.

[gramalingam]

Adding some info/context here: this is a problem with execution of the op.SequenceInsert (in trace/debug mode, as opposed to generation of model/function proto). The source of the issue is here ... we don't currently have a typed representation of runtime (sequence) values, as we simply reuse python lists ... and when the list is empty we cannot figure out a suitable onnx type for it (as it is a polymorphic value).

Two possible fixes are: using a runtime representation for sequences that combines a type with the value ... or, we need a smarter inference that can figure out the type from other values (which is possible in this example, but not sure about a general implementation of this).