Datatype conversion

[wangjiawen2013]

Hi,
This is a snapshot from https://github.com/tracel-ai/burn/blob/main/examples/custom-image-dataset/src/data.rs

The conversion is too complicated and cause confusions to users who are not familiar with burn. I converted the datatype of TensorData like this and got the same results

[wangjiawen2013]

And I also converted the TensorData datetype like this:

[wangjiawen2013]

In both cases we can get the same conversion. Why not using the latter ? It's easy to understand and more concise than that in the example. Are are any special consideration in the example ?

[AsherJingkongChen]

In both cases we can get the same conversion. Why not using the latter ? It's easy to understand and more concise than that in the example. Are are any special consideration in the example ?

I think it is just not updated. You can open a PR for it.

[laggui]

Actually the associated type is used to be compatible across backends. Not all backends are guaranteed to support the same element type for ints or float. For example, IntElem for wgpu is i32 but for tch it is i64.

So changing that to i64 for all won't work since the example can use both backends. But in practice, if you only need to support one backend or you know what element types are ok then you can simply make the changes as you did 🙂

[wangjiawen2013]

I browsed the source code of IntElem and Element but have not found how burn determines the element types corresponding to each backend automatically. Could take about the mechanism ? Knowing this could make users understand the data type conversion better and could write more robust code.

[wangjiawen2013]

Besides, when the ran the following code in evcxr_jupyter to construct a polars dataframe, it worked:

let tensor1: Tensor<NdArray, 1> = Tensor::from_data([1, 2, 3], &device);
let tensor2 = tensor1.into_data().to_vec::<<NdArray as Backend>::FloatElem>().unwrap();

// Construct a polars dataframe
let mut results = df![
    "targets" => tensor2,
].unwrap();

println!("{:?}", results);

But when I ran the same code in main.rs, It's very weired that this error occurred:

[laggui]

I browsed the source code of IntElem and Element but have not found how burn determines the element types corresponding to each backend automatically. Could take about the mechanism ? Knowing this could make users understand the data type conversion better and could write more robust code.

The element types are set in the backend implementation. Take a look at the NdArray backend. The default NdArray uses f32 and i64.

For wgpu, the default uses f32 and i32 and the element types are also in the backend implementation.