Flyte Add Enable/Disable Options for Agent Services

[Future-Outlier]

TL;DR

• Added options to enable (default) or disable the Flyte agent.

Describe your changes

@pingsutw and I have introduced two options for users to configure the Flyte agent - it can now be either enabled or disabled.

We believe this enhancement will offer more flexibility and cater to diverse user needs and deployment scenarios.

After a thorough consideration, we have decided to set the "enable agent" option as the default setting.

Enable agent service (Default)

Enable the agent deployment by updating helm and kustomization.
You can use my image on the docker hub to test.

cd flyte/docker/sandbox-bundled/
make build
docker tag flyte-sandbox:latest futureoutlier/flyte-sandbox:flyteagent-v2
docker push futureoutlier/flyte-sandbox:flyteagent-v2

./bin/flytectl demo start --image futureoutlier/flyte-sandbox:flyteagent-v2

Disable agent service (Use flag)

./bin/flytectl demo start --image futureoutlier/flyte-sandbox:flyteagent-v2 --disable-agent

Tracking issue

#3936
flyteorg/flytectl#420

Check all the applicable boxes

• I updated the documentation accordingly.
• All new and existing tests passed.
• All commits are signed-off.

Screen Shot

• New Pod Addition: The newly introduced pod is named "flyteagent". You can view the snapshot of its presence here.
  

• Config Map Update: The "flyte-sandbox-config" config map situated in the "flyte" namespace has been updated.
  Its current snapshot can be viewed here.
  

Notably, there's an addition of the enabled-plugins key, which can be viewed here.

[Future-Outlier]

Hello Reviewers,

I appreciate @pingsutw's suggestion; it's quite insightful! I learned a lot!

For someone who's new or at the entry level, I believe enhancing the key agent-service structure would make it more user-friendly. Consider the transition:

Current Configuration:

agent-service:
  defaultAgent:
    endpoint: "dns:///flyteagent.flyte.svc.cluster.local:8000"
    insecure: true

Proposed Configuration:

agent-service:
  defaultAgent:
    endpoint: "dns:///flyteagent.flyte.svc.cluster.local:8000"
    insecure: true
    timeouts:
      GetTask: 100s
    defaultTimeout: 100s
  agents:
    custom_agent:
      endpoint: "dns:///flyteagent.flyte.svc.cluster.local:8001"
      insecure: false
      defaultServiceConfig: '{"loadBalancingConfig": [{"round_robin":{}}]}'
      timeouts:
        GetTask: 100s
      defaultTimeout: 100s
  agentForTaskTypes:
    # Overrides the default agent for custom_task. This means propeller will send the request to this agent.
    - custom_task: custom_agent

I'm eager to hear your thoughts on this. Thanks a lot!

[Future-Outlier]

After a discussion with pingsu, we decide to left

agent-service:
  defaultAgent:
    endpoint: "dns:///flyteagent.flyte.svc.cluster.local:8000"
    insecure: true
    timeouts:
      GetTask: 10s
    defaultTimeout: 10s

like this!
And it will be far more user friendly!

[Future-Outlier]

@pingsutw @jeevb Please take a look, and approve the PR here: flyteorg/flytectl#420
Thanks a lot!