Avoid setup-dev.py writing to generated/

[kevin85421]

Why are these changes needed?

docker build --progress=plain --build-arg BUILD_DATE="$(date +%Y-%m-%d:%H:%M:%S)" -t $RAY_IMAGE -f ./python/ray/tests/kuberay/Dockerfile . || exit

#48793 assumes that the ray/serve/generated directory does not exist in the Ray directory, which means that users have not compiled Ray in that directory.

However, this approach is incorrect for developers who want to test code changes in KubeRay. The workflow is like:

• Develop Ray locally, meaning you continuously update the code and compile it as needed. The directory ray/serve/generated should be created after compilation.
• After finishing development, the developer wants to build a Docker image using this Dockerfile and then test it with KubeRay. The Dockerfile copies the local Ray directory into the image and runs python3 setup-dev.py -y to replace the Ray package with your local Ray code.
• Preserve ray/serve/generated when creating softlink for serve package #48793 will move the existing .../site-packages/ray/serve/generated directory to a temporary directory, and then move it back after the soft link is created. However, in this case, ray/serve/generated directory has already existed.

In addition, in the Dockerfile, the ray directory is added by the command ADD . ray, so the owner is root instead of ray. That's why we see the permission error above.

Related issue number

Checks

• I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
• I've run scripts/format.sh to lint the changes in this PR.
• I've included any doc changes needed for https://docs.ray.io/en/master/.
  • I've added any new APIs to the API Reference. For example, if I added a
    method in Tune, I've added it in doc/source/tune/api/ under the
    corresponding .rst file.
• I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • Unit tests
  • Release tests
  • This PR is not tested :(

[kevin85421]

For observability, this is to print the output of each step.

[kevin85421]

cc @rueian @ryanaoleary this is for autoscaler development.