Skip to content

[Feature][KubeAI] Creating BKC of Auto Scaling for Model Profiles on Gaudi. #288

@joshuayao

Description

@joshuayao

Priority

P0

OS type

Ubuntu

Hardware type

Xeon-GNR

Running nodes

Single Node

Description

  • Tune the parameters for each model profiles on Gaudi.

    • minReplicas
    • maxReplicas
    • targetRequests
    • scaleDownDelaySeconds
  • Benchmark on Gaudi.

Metadata

Metadata

Assignees

Labels

Backlogfeatures in backlogfeatureNew feature or request

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions