feat: Adding RestrictedFeatures Support to the Python Frontend Bindings#7775

Open

KrishnanPrash wants to merge 3533 commits intomainfrom kprashanth-tritonfrontend-rfeatures

+221,735-31,795

This pull request is big! We're only showing the most recent 250 commits

Commits on Apr 9, 2024

Commits on Apr 10, 2024

Updated vllm version (#7095 )
oandreeva-nv
authored

Commits on Apr 12, 2024

Validate the memory requested for the infer request is not out of bounds (#7083 )
jbkyang-nvi
authored
Add copyright for tritonclient_api (#7109 )
Tabrizian
authored

Commits on Apr 13, 2024

Commits on Apr 23, 2024

Use a lower concurrency with more repetition for L0_memory_growth (#7127 )
krishung5
authored

Commits on Apr 24, 2024

Replace deprecated tritongrpcclient package (#7061 )
Tabrizian
authored

Commits on Apr 25, 2024

Commits on Apr 26, 2024

Commits on Apr 30, 2024

Update main post-24.04 (#7160 )

mc-nv
and
krishung5
authored

Commits on May 3, 2024

Remove meetup note now that the event has completed (#7179 )
Tabrizian
authored

Commits on May 8, 2024

Rename triton_tensorrtllm_worker -> trtllmExecutorWorker (#7194 )
krishung5
authored

Commits on May 9, 2024

Tests for Top Level Request Caching for Ensemble Models (#7074 )
lkomali
authored

Commits on May 10, 2024

Test cuda shared memory offset and byte size out of bounds(#7202 )
jbkyang-nvi
authored

Commits on May 13, 2024

Upgrade the golang version to 1.22.3 (#7208 )
tanmayv25
authored

Commits on May 14, 2024

Update 'Dockerfile' Python path to include DALI (#7216 )
mc-nv
authored

Commits on May 15, 2024

Remove the dependency on CUDA driver (#7224 )
krishung5
authored

Commits on May 16, 2024

Commits on May 17, 2024

Fix the L0_simple_go_client (#7239 )
tanmayv25
authored

Commits on May 18, 2024

Add section on ensemble model caching (#7234 )
rmccorm4
authored

Commits on May 20, 2024

Add testing for escaped log messages

nnshah1
and
rmccorm4
authored

Commits on May 21, 2024

Commits on May 22, 2024

Return an error if --load-model is specified without explicit model control mode (#7259 )
rmccorm4
authored

Commits on May 23, 2024

Exclude Jax example from Python 3.8 (#7260 )
krishung5
authored

Commits on May 24, 2024

Commits on May 28, 2024

Fix Python 3.11 env (#7274 )
krishung5
authored

Commits on May 29, 2024

Commits on Jun 3, 2024

Commits on Jun 4, 2024

fix: Fix L0_input_validation--base (#7304 )
yinggeh
authored

Commits on Jun 18, 2024

fix: Fix version for setuptools and grpcio-tools. Remove cudnn 8 installation (#7331 )
krishung5
authored

Commits on Jun 20, 2024

ci: Add INT64 Datatype Support for Shape Tensors in TensorRT Backend (#7329 )
pskiran1
authored

Commits on Jun 26, 2024

Update 15-container-copyright.txt (#7375 )
Tabrizian
authored

Commits on Jun 28, 2024

Update main post -24.06 (#7380 )
mc-nv
authored

Commits on Jul 3, 2024

test: Add input byte size tests using C APIs (#7372 )
yinggeh
authored

Commits on Jul 8, 2024

Commits on Jul 10, 2024

Commits on Jul 12, 2024

Commits on Jul 25, 2024

Commits on Jul 27, 2024

ci: Fix shape and reformat free tensor handling in the input byte size check (#7444 )
pskiran1
authored

Commits on Jul 29, 2024

Commits on Jul 30, 2024

test: Add BF16 test for python backend (#7483 )
rmccorm4
authored

Commits on Jul 31, 2024

Commits on Aug 2, 2024

Including 'tritonserver.lib' into final package (#7491 )
mc-nv
authored

Commits on Aug 5, 2024

build: Add default value for argument 'TRITON_REPO_ORGANIZATION' from sdk Dockerfile (#7437 )
zhanga5
authored

Commits on Aug 6, 2024

chore:Purge PA from Client Repo (#7488 )
fpetrini15
authored

Commits on Aug 7, 2024

Commits on Aug 8, 2024

Add vLLM x Triton user meetup announcement (#7509 )
harryskim
authored

Commits on Aug 10, 2024

Fix benchmarking tests (#7461 )
pskiran1
authored

Commits on Aug 17, 2024

test: Add python backend tests for the new histogram metric (#7540 )
yinggeh
authored

Commits on Aug 20, 2024

Commits on Aug 22, 2024

ci: Raise Documentation Generation Errors (#7559 )
fpetrini15
authored

Commits on Aug 23, 2024

docs: Add tensorrtllm_backend into doc generation (#7563 )
krishung5
authored

Commits on Aug 30, 2024

feat: Python Deployment of Triton Inference Server (#7501 )

authored

Commits on Sep 3, 2024

fix: Adding copyright info (#7591 )
KrishnanPrash
authored

Commits on Sep 4, 2024

test: Refactor core input size checks (#7592 )
yinggeh
authored

Commits on Sep 7, 2024

Don't Build tritonfrontend for Windows. (#7599 )
fpetrini15
authored

Commits on Sep 11, 2024

Commits on Sep 17, 2024

Fix: Add mutex lock for state completion check in gRPC streaming to prevent race condition (#7617 )
pskiran1
authored

Commits on Sep 19, 2024

Update fetch_models.sh (#7621 )
vd-nv
authored

Commits on Sep 20, 2024

ci: Set stability factor to a higher value (#7634 )
lkomali
authored

Commits on Oct 1, 2024

[docs] Removed vLLM meetup announcement (#7673 )
oandreeva-nv
authored

Commits on Oct 3, 2024

[doc]Adjusted formatting of the warning (#7675 )
oandreeva-nv
authored

Commits on Oct 8, 2024

Commits on Oct 9, 2024

build: Adding tritonfrontend to build.py (#7681 )

KrishnanPrash
and
rmccorm4
authored

Commits on Oct 11, 2024

feat: OpenAI Compatible Frontend (#7561 )
rmccorm4
authored

Commits on Oct 12, 2024

docs: Add beta note to OpenAI compatible API (#7695 )
rmccorm4
authored

Commits on Oct 17, 2024

docs: Clarify meanings of ensemble key and value (#7711 )
kthui
authored

Commits on Oct 19, 2024

Commits on Oct 21, 2024

Revert "Change compute capablity min value (#7708 )" (#7721 )
mc-nv
authored

Commits on Oct 24, 2024

test: Add small delay to L0_lifecycle test_load_new_model_version after each model reload (#7735 )
kthui
authored

Commits on Oct 29, 2024

Removing caching on windows. (#7717 )
mc-nv
authored

Commits on Oct 31, 2024

Commits on Nov 5, 2024

Working Python Workflow
KrishnanPrash
committed

Commits on Nov 7, 2024

Commits on Nov 13, 2024

Update ONNX version for generated models (#7785 )
mc-nv
authored

Commits on Nov 14, 2024

test: RHEL Filesystem Tests (#7788 )
fpetrini15
authored

Commits on Nov 15, 2024

Update model generation scenario (#7793 ) (#7797 )
mc-nv
authored

Commits on Nov 19, 2024

fix: Fix L0_input_validation (#7800 )
pskiran1
authored

Commits on Nov 20, 2024

Commits on Nov 21, 2024

Commits on Nov 22, 2024

fix: Resolve integer overflow in Load API file decoding (#7787 )

pskiran1
and
GuanLuo
authored

Commits on Nov 25, 2024

feat: Enable deferred unregistering of shared memory regions after inference (#7743 )
pskiran1
authored

Commits on Nov 26, 2024

Commits on Nov 28, 2024

build: Update OpenVINO model generation script with new API (#7811 )
yinggeh
authored

Commits on Dec 2, 2024

Commits on Dec 4, 2024

fix: L0_sequence_batcher_cudashm (#7852 )
oandreeva-nv
authored

Commits on Dec 9, 2024

fix: gRPC segfault due to Low Request Cancellation Timeout (#7840 )
yinggeh
authored

Commits on Dec 10, 2024

ci: RHEL8 L0_backend_python Support (#7859 )

fpetrini15
and
nv-kmcgill53
authored

Commits on Dec 12, 2024

Update generated Dockerfile (#7876 )
mc-nv
authored

Commits on Dec 16, 2024

fix: Handle dict type for content field in Chat Completions endpoint (#7849 )
dongs0104
authored

Commits on Dec 17, 2024

ci: Fix Windows CI Errors (#7837 )

fpetrini15
and
rmccorm4
authored

Commits on Dec 18, 2024

docs: Re-structure User Guides for Discoverability (#7807 )

authored

Commits on Dec 19, 2024

Commits on Dec 20, 2024