Export - Custom Attributes

[white3km]

For the Export CSV and JSON feature, some of our entities have attributes similar to your AddressFragment (https://github.com/yahoo/elide/tree/master/elide-datastore/elide-datastore-jpa/src/test/java/example) using both a Type and an AttributeConverter.

public class Example{
   private String name;
   @Convert(converter = CommaDelimitedStringConverter.class)
   private Set<String> alternatives;
   @Type(
      type = "json-object",
      parameters = {@Parameter(name = "class", value = "example.AttributeDto")})
   private AttributeDto attributes;
}

I've noticed the csv produced skips the json attribute completely which throws off the column values. The set of strings seems to be listed last in the csv even though the header column is in a different spot. For the json produced the set of strings is handled properly and the json attirbute is omitted from the final json. Is there any way to provide a custom formatter for these fields to be exported properly?

[aklish]

You can provide your own ExportFormatter in this bean:

https://github.com/yahoo/elide/blob/master/elide-spring/elide-spring-boot-autoconfigure/src/main/java/com/yahoo/elide/spring/config/ElideAsyncConfiguration.java#L101

You can clone the CSVExportFormatter, make any fixes, and use the clone instead. Let me know if that solves this issue. Also, if you have a patch to the CSVExportFormatter to address this issue, please submit. We'll review and merge it. Thanks.

[justin-tay]

I have created a repro test case at https://github.com/justin-tay/elide/tree/csvexport. Given the current design and implementation I'm not sure what is an appropriate approach.

The output from the test is

"name","alternatives","address"
"1", null, "a", "b"
"2", "a", "b", "c", "d", "My Street 1", "My State 1", null
"3", "a", "My Street 2", "My State 2", "123", "4444"

The header is actually being generated just from the attributes in the projection, but Json2Flat actually needs to process the data to determine the headers. The actual headers generated by Json2Flat for the 3 records are.

[name, address, alternatives_0, alternatives_1]
[name, alternatives_0, alternatives_1, alternatives_2, alternatives_3, address_street, address_state, address_zip]
[name, alternatives_0, address_street, address_state, address_zip_zip, address_zip_plusFour]

The current design of the TableExportFormatter is to process row by row but Json2Flat would need the complete JSON document to determine the header and the data format. I don't really think this can be avoided if you needed to serialize a collection of JSON objects. Possibly needing to load all the results into memory is going to be an issue.