Disable donated_buffer for all ops's backward benchmarking #104
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
FindHao
had a problem deploying
to
docker-s3-upload
GitHub Actions
Failure
FindHao
had a problem deploying
to
docker-s3-upload
GitHub Actions
Failure
xuzhao9
reviewed
Dec 7, 2024
tritonbench/utils/triton_op.py
Outdated
| @@ -39,6 +39,8 @@ | |||
| tqdm = None | |||
|
|
|||
| logger = logging.getLogger(__name__) | |||
| # TODO: remove this once we have a better way to handle backward benchmarking | |||
| torch._functorch.config.donated_buffer = False | |||
There was a problem hiding this comment.
This seems an overkill to me. Can we disable it only in backward and forward_backward?
FindHao
had a problem deploying
to
docker-s3-upload
GitHub Actions
Failure
FindHao
had a problem deploying
to
docker-s3-upload
GitHub Actions
Failure
|
@FindHao has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
xuzhao9
approved these changes
Dec 7, 2024
FindHao
had a problem deploying
to
docker-s3-upload
GitHub Actions
Failure
FindHao
had a problem deploying
to
docker-s3-upload
GitHub Actions
Failure
|
@FindHao has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
FindHao
force-pushed
the
findhao/disable_donated_buffer
branch
from
5217014 to
5aa17b6
Compare
FindHao
had a problem deploying
to
docker-s3-upload
GitHub Actions
Failure
FindHao
had a problem deploying
to
docker-s3-upload
GitHub Actions
Failure
|
It looks the ci errors are true, and the config is not compatible with flash_attention. I'm going to set this config for |
…_entropy, geglu, swiglu, and layernorm
FindHao
had a problem deploying
to
docker-s3-upload
GitHub Actions
Failure
FindHao
had a problem deploying
to
docker-s3-upload
GitHub Actions
Failure
|
@FindHao has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
FindHao
had a problem deploying
to
docker-s3-upload
GitHub Actions
Failure
FindHao
had a problem deploying
to
docker-s3-upload
GitHub Actions
Failure
|
@FindHao has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
facebook-github-bot
pushed a commit
that referenced
this pull request
Dec 12, 2024
Summary: The previous PR #104 causes the following issue. ``` % python run.py --op geglu --mode fwd --precision fp32 --metrics latency,speedup --csv --cudagraph 0%| | 0/4 [00:03<?, ?it/s] Caught exception, terminating early with partial results Traceback (most recent call last): File "/scratch/yhao/pta/tritonbench/tritonbench/utils/triton_op.py", line 782, in run y_vals: Dict[str, BenchmarkOperatorMetrics] = functools.reduce( File "/scratch/yhao/pta/tritonbench/tritonbench/utils/triton_op.py", line 770, in _reduce_benchmarks acc[bm_name] = self._do_bench( File "/scratch/yhao/pta/tritonbench/tritonbench/utils/triton_op.py", line 981, in _do_bench fn = self._get_bm_func(fn_name) File "/scratch/yhao/pta/tritonbench/tritonbench/utils/triton_op.py", line 667, in _get_bm_func fwd_fn = fwd_fn_lambda(*self.example_inputs) File "/scratch/yhao/pta/tritonbench/tritonbench/utils/triton_op.py", line 481, in _inner return function(self, *args, **kwargs) File "/scratch/yhao/pta/tritonbench/tritonbench/operators/geglu/operator.py", line 69, in inductor_geglu compiled = torch.compile(self.baseline_model) UnboundLocalError: local variable 'torch' referenced before assignment (B, T, H) ``` we should use `from torch._functorch import config` rather than `import torch._functorch.config` Pull Request resolved: #113 Reviewed By: adamomainz Differential Revision: D67110110 Pulled By: FindHao fbshipit-source-id: e5143b06d0e62fb2a7b83464e23126e73a52ee10
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
It is still a temporary fix for backward benchmarking. Related discussion #40