What client shaded plugin we should provided?

[maobaolong]

A goal from mine is to provided a shaded plugin:

• With all compile scope dependencies and relocated for shaded purpose so upstream app should never encounter MethodNotDefException and any other exception about class conflict
  • We should also provided a script to verify there have no shaded result existing in the shaded client jar.
• Without any provided dependencies, since we want to use the provided dependencies provided from runtime environment, we should never include them and relocate them.

Free free to discuss with me about this topic?

For now, the layout of shaded client jar looks like this:

This is what celeborn looks like as a reference:

[maobaolong]

@rickyma @EnricoMi @advancedxy @jerqi @zhengchenyu How about your idea?

[zhengchenyu]

This discussion originated from #2030.
After #1878, many class is relocated. For example org.apache relocated to ${rss.shade.packageName}.org.apache. This will cause a problem. For example, uniffle indirectly introduced commons-collections3 through hadoop. But after pr, org.apache.commons.collections was relocated to {rss.shade.packageName}.org.apache.commons.collections. Since collections3 was introduced through hadoop and is provided, collections3 is not included in the fat shaded jar. Therefore, when executing, NoClassDefFoundError will be thrown.