[FEA] Support Model A/B Testing

[mdemoret-nv]

Goal: To support model; A/B testing. There are many ways this could be done, some ideas:

• Utilize an ML Platform to run inference requests and track the results for each model in the ML Platform
• Use ControlMessage to enqueue 2 inference tasks, comparing the results of both outputs
• Create a ModelABStage which can split incoming requests up to multiple models and concat the output

[tgrunzweig-cpacket]

Just a thought on this:

First: Great and would be helpful for the stated objectives.

But also, one of the strengths of the morpheus pipeline is that its easy to re-train. Especially for something like DFP where thousands of models need to retrained regularly. AB testing would be a great addition, because it addresses a basic question: "are the retrained models better than the older ones?". Really, either morpheus does it in "out of the box", or us, the users, have to implement it somehow - I see no way around this.

Cheers,
Tzahi