How to use this with cmyk color profile based image ?

Hi @tancik can you please help me.. i want to use this technique for cmyk color profile images but its showing color differences... even in rgb images there is small color shifts... So is it expected or there is any issue with the model i have trained?

Please help i want to make this project ....