Ruff configuration from pyproject.toml is ignored #54

beajifu · 2023-10-13T14:07:36Z

Hi,

I'm trying to use the pre-commit hook for Ruff, but it ignores my settings from the pyproject.toml. I want Ruff to ignore the line-length but if I commit a file with lines that exceed this limit it is complaining about it. If I use a ruff.toml to configure Ruff it works as expected.

I use version 0.0.292.

My settings in .pre-commit-config.yaml:

# See https://pre-commit.com for more information
# See https://pre-commit.com/hooks.html for more hooks
repos:
-   repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v3.2.0
    hooks:
    -   id: end-of-file-fixer
    -   id: check-added-large-files
-   repo: https://github.com/charliermarsh/ruff-pre-commit
    rev: v0.0.292
    hooks:
    -   id: ruff

Ruff configuration part in my pyproject.toml (same is used in the ruff.toml)

[tool.ruff]
select = ["E"]

# Never enforce `E501` (line length violations).
ignore = ["E501"]

The text was updated successfully, but these errors were encountered:

charliermarsh · 2023-10-13T19:35:51Z

Hmm, a bit of a tough one to debug. When you switch between pyproject.toml and ruff.toml, are the files located in the same directory?

beajifu · 2023-10-14T20:21:06Z

Yes they are in the same directory.

I tried it on another laptop with a small example project and this problem did not occur. I try to find a way to reproduce the problem and give you an update.

beajifu · 2023-10-15T11:49:59Z

I could reproduce the problem and find a solution. If I use a project with already added code and a pyproject.toml to the git repo and then

add pre-commit to the virtual environment,
create a .pre-commit-config.yaml,
install the git hooks with pre-commit install
add the ruff configuration (ignore line length) to the pyproject.toml
try to only commit the python file that contains too long lines -> ruff complains about too long lines

When I first commit the .pre-commit-config.yaml and the pyproject.toml and after that commit the file with the long line then ruff ignores it as expected. Is this behavior expected?

klow68 · 2023-10-19T08:24:38Z

same issue with v0.1.0 but committing the config doesn't change anything

found the issue : the configuration needs to be added in the toml and we need to select the rule to use even for line-length

repos:
  - repo: https://github.com/astral-sh/ruff-pre-commit
    rev: v0.1.0
    hooks:
      - id: ruff
        args: [--select=C901, --select=E501]

by default line-length is not checked, seems strange no ?

charliermarsh · 2023-10-19T16:08:28Z

@klow68 - We removed E501 from the default configuration in v0.1.0 (https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md#breaking-changes), so I think that's working as intended.

dhruvmanila · 2023-10-30T02:34:38Z

@beajifu I'm unable to reproduce with the steps mentioned although I'm using F401 as an example rule.

Gist: https://gist.github.com/dhruvmanila/8c796087330cb22699f3b20e929fde45

.
├── .pre-commit-config.yaml
├── pyproject.toml
└── test.py

Steps

Install pre-commit and hooks
Comment out the ignore in config file

$ pre-commit run
ruff.....................................................................Failed
- hook id: ruff
- exit code: 1

test.py:1:8: F401 [*] `os` imported but unused
Found 1 error.
[*] 1 potentially fixable with the --fix option.

Uncomment the ignore in config file

$ pre-commit run
ruff.....................................................................Passed

rpop0 · 2023-11-21T15:03:39Z

My configuration in pyproject.toml also seems to be having an issue.

Running ruff without any config: 3 errors

Running ruff with 3 select = ["ALL"] in pyproject.toml: 389 errors

Running ruff through pre-commit: 89 errors.

It seems like running ruff through pre-commit is using more rules than default but ignoring the rules in pyproject.toml

rytilahti · 2023-11-21T16:23:07Z

Chiming in just in case it may help someone. In my case, clearing the precommit cache ($HOME/.cache/pre-commit/) fixed a similarly sounding issue some time ago. Alas, I didn't keep the cache so I couldn't debug it further.

caerulescens · 2024-01-12T03:47:20Z

This still seems to be an issue; using v0.1.7

charliermarsh · 2024-01-12T04:25:34Z

Would really appreciate if you could share some more information -- we weren't able to reproduce it above and it doesn't seem to be a common issue, so we need some help. What behavior are you seeing? What commands are you running? How is your project structured? Have you tried clearing the cache, as above? Thank you in advance!

caerulescens · 2024-01-12T12:25:10Z

It seems I have more than one project reliably producing this bug (even after clearing caches), and I constructed a new project as a minimal example for @charliermarsh to reproduce the bug reliably. The new project I made ended up reading the pyproject.toml correctly, and I checked by causing N999: Invalid module name and then ignoring the rule: ignore = ["N999"] on a separate run. The bugged and new project that I compared both use ruff-pre-commit v0.1.7 and were invoked the same way. I don't think it's related to the version, but if I spend some more time I can isolate the issue.

rafalkrupinski · 2024-01-26T23:43:02Z

[tool.ruff]
select = ["E"]

# Never enforce `E501` (line length violations).
ignore = ["E501"]

don't these go in tool.ruff.lint?

caerulescens · 2024-01-27T00:13:12Z

@rafalkrupinski I double checked docs, and you're right. I have been busy with other tasks, but I saved this thread for later. I'll retest, and repost.

charliermarsh · 2024-01-27T00:15:05Z

@caerulescens - Both are accepted right now -- putting ignore under [tool.ruff] and [tool.ruff.lint] are interchangeable. (We're moving towards the latter, but they both work as-is.)

caerulescens · 2024-01-27T00:49:13Z

I realized the same thing just now; I'll retest

caerulescens · 2024-01-27T01:01:32Z

I re-verified that I can [still] produce this bug with projects I'm working on; the [tool.ruff] section is being used. To isolate this bug, I'll probably copy/paste the repository locally, and I'll just start removing things until I can't reproduce the bug anymore, and then I'll post the results here. Earlier I tried to construct the minimum example from scratch, and I wasn't successful.

Quatters · 2024-01-30T10:15:11Z

@charliermarsh just created repo in which pre-commit hook ignores pyproject.toml config unlike the ruff itself. Hope this would help.
https://github.com/Quatters/ruff-pre-commit-bug-demo

dhruvmanila · 2024-01-30T10:41:46Z

@Quatters Hi, thanks for providing the repository for the problem you're facing. But, I don't think it's a bug with Ruff or pre-commit, I've described in detail about what's happening here: astral-sh/ruff#9696 (comment).

One solution would be to use pass_filenames: false in the pre-commit config so that the file discovery is done by Ruff taking into account the includes and excludes configured by the user in their pyproject.toml.

repos:
  - repo: https://github.com/astral-sh/ruff-pre-commit
    rev: v0.1.15
    hooks:
      - id: ruff
        pass_filenames: false

This solves your problem.

That said, maybe Ruff's pre-commit hook should do that by default?

Quatters · 2024-01-31T01:00:04Z

@dhruvmanila thank you for the quick response. Yes, providing pass_filenames: false helped in my case.

That said, maybe Ruff's pre-commit hook should do that by default?

I'm not sure about doing that by default (will there be any side effects/breaking changes?). Anyway, I think information about pass_filenames would be useful in README.

rafalkrupinski · 2024-01-31T20:05:47Z

One solution would be to use pass_filenames: false in the pre-commit config so that the file discovery is done by Ruff taking into account the includes and excludes configured by the user in their pyproject.toml.
That said, maybe Ruff's pre-commit hook should do that by default?

Wouldn't that make ruff to check all files every time? I think pre-commit users expect it to only check changed files, even if it is fast.

Even if pre-commit passes filenames to ruff check, I still don't see why it wouldn't reed the configuration...

dougthor42 · 2024-01-31T22:28:18Z

Agreed, running with pass_filenames: False is not a long term solution. Pre-commit needs to be the decider on what files get passed. This is one way of slowly fixing a large project - only touched files get linted/formatted.

It does help narrow down the issue though, good find!

caerulescens · 2024-02-05T03:03:30Z

@dhruvmanila That is not a usable work around for the reasons that @dougthor42 mentions.

@charliermarsh I played around with this issue tonight, and I have discovered more behavior surrounding this issue. I did something similar to the squeeze theorem but with code; I constructed a minimal project that doesn't have the issue, and I have a fully featured project with the issue (for the same rule). I've been removing parts of the larger project until I identify the issue.

pyproject.toml observations:

If [tool.ruff.lint] is present, then all settings in [tool.ruff] are overridden; I'd imagine this is expected as [tool.ruff.lint] is set to overtake [tool.ruff]. Meaning, ignore = [] in [tool.ruff.lint] overrides all specified rules for ignore in [tool.ruff].
I have no issue with ignoring D rules in [tool.ruff] or [tool.ruff.lint], but N999 is getting ignored regardless. Meaning, I have code that will invoke an error for D100, D104, and N999.

[tool.ruff]
line-length = 88  # Not relevant
target-version = "py310"  # Not relevant
src = ["src", "tests", "docs"]  # Not relevant
select = ["F", "E", "W", "C90", "S", "I", "N", "D", "UP"]
ignore = ["D100", "D104", "N999"]

[tool.ruff.pydocstyle]
convention = "pep257"  # Not relevant

Will produce:

{{ cookiecutter.repository_name }}/src/{{ cookiecutter.package_name }}/__init__.py:1:1: N999 Invalid module name: '{{ cookiecutter.package_name }}'
Found 1 error.

Then, removing rule D100 from ignore in [tool.ruff] or [tool.ruff.lint] will produce:

docs/conf.py:1:1: D100 Missing docstring in public module
tests/conftest.py:1:1: D100 Missing docstring in public module
tests/test_cookiecutter_terragrunt.py:1:1: D100 Missing docstring in public module
{{ cookiecutter.repository_name }}/src/{{ cookiecutter.package_name }}/__init__.py:1:1: N999 Invalid module name: '{{ cookiecutter.package_name }}'
Found 4 errors.

This means that [tool.ruff] and [tool.ruff.lint] within pyproject.toml is getting read by pre-commit; ~~this seems to be a specific issue that can be replicated with N999 but not maybe not other rules.~~ This also means that [tool.ruff] and [tool.ruff.lint] suffer from the same problem, so the code that's causing the issue is shared by [tool.ruff] and [tool.ruff.lint], which probably doesn't narrow down much.
3. I have a minimal project that can correctly ignore N999, yet it has the same pyproject.toml settings seen in (2).

caerulescens · 2024-02-05T03:06:45Z

@charliermarsh I think I solved it! The problem occurs when there's another pyproject.toml within the project. The problem within the larger project was solved when I removed the second pyproject.toml within {{ cookiecutter.repository_name }}, which is used by cookiecutter. It seems that ruff prioritized [tool.ruff] in {{ cookiecutter.repository_name }}/pyproject.toml over the top-level pyproject.toml. This explains why this issue could be difficult to reproduce; it requires having multiple configurations for ruff present. I had the same D rules ignored in {{ cookiecutter.repository_name }}/pyproject.toml as the top-level pyproject.toml, which was misleading me during the debugging process.

This issue can be closed as ruff does read the pyproject.toml correctly. Maybe it would be helpful to note this behavior in the documentation if it's not already there? I think it could be useful to raise a warning when multiple configurations are found, and indicate that it could lead to unexpected behavior.

rpop0 · 2024-02-05T08:05:07Z

@charliermarsh I think I solved it! The problem occurs when there's another pyproject.toml within the project. The problem within the larger project was solved when I removed the second pyproject.toml within {{ cookiecutter.repository_name }}, which is used by cookiecutter. It seems that ruff prioritized [tool.ruff] in {{ cookiecutter.repository_name }}/pyproject.toml over the top-level pyproject.toml. This explains why this issue could be difficult to reproduce; it requires having multiple configurations for ruff present. I had the same D rules ignored in {{ cookiecutter.repository_name }}/pyproject.toml as the top-level pyproject.toml, which was misleading me during the debugging process.

This issue can be closed as ruff does read the pyproject.toml correctly. Maybe it would be helpful to note this behavior in the documentation if it's not already there? I think it could be useful to raise a warning when multiple configurations are found, and indicate that it could lead to unexpected behavior.

This happens to me and I have a single pyproject.toml file

caerulescens · 2024-02-05T11:36:03Z

@rpop0 Could you elaborate further? This issue doesn't seem to be a problem; do you have other types of configurations present like ruff.toml? I identified my situation as user error, and I confirmed the precedence that ruff uses for reading settings from the pyproject.toml.

rpop0 · 2024-02-05T11:38:06Z

@rpop0 Could you elaborate further? This issue doesn't seem to be a problem; do you have other types of configurations present like ruff.toml? I identified my situation as user error, and I confirmed the precedence that ruff uses for reading settings from the pyproject.toml.

No other types of configurations besides the single pyproject.toml file.

caerulescens · 2024-02-05T11:38:43Z

@rpop0 How are you reproducing this bug? Could you post a minimal example repository for us or maybe zip a folder and attach here?

caerulescens · 2024-02-05T11:40:19Z

At the very least, the title for this issue is incorrect because [tool.ruff] and [tool.ruff.lint] within the pyproject.toml is getting read by pre-commit.

wiseyoungbuck · 2024-03-02T18:13:33Z

I have been having the same issue where the ruff pre-commit hook was ignoring the line-length setting.
I only had a single pyproject.toml file and I had the same behavior with a single ruff.toml file. The pass_filenames: false solution would allow the formatter to work properly, but then it would run on all files instead of only running on the newly committed changes.

I am using VSCode and the Ruff VSCode extension. Console commands in VSCode were working fine, but pre-commit would undo the formatting done by the VSCode extension.

When I disabled the VSCode extension, the problematic behavior went away and the precommit behavior starting working as expected. I was also able to reenable the Ruff VSCode extension and the issue has not yet returned. I hope this is helpful.

ezeparziale · 2024-05-04T21:47:01Z

I have the same problem that ruff-pre-commit does not load ruff config from pyproject.toml.

I resolved it by adding the config in the args.

  - repo: https://github.com/astral-sh/ruff-pre-commit
    # Ruff version.
    rev: v0.4.3
    hooks:
      # Run the linter.
      - id: ruff
        args: ["--select", "E,W,F,I,C,B,UP", "--ignore", "E203,B008,C901"]
      # Run the formatter.
      - id: ruff-format

mazzma12 · 2024-05-07T08:00:56Z

I have the same problem that ruff-pre-commit does not load ruff config from pyproject.toml.

I resolved it by adding the config in the args.

  - repo: https://github.com/astral-sh/ruff-pre-commit
    # Ruff version.
    rev: v0.4.3
    hooks:
      # Run the linter.
      - id: ruff
        args: ["--select", "E,W,F,I,C,B,UP", "--ignore", "E203,B008,C901"]
      # Run the formatter.
      - id: ruff-format

Same here, but more explicitly for the --exclude config which is not accounted for

artefactop · 2024-06-23T12:15:40Z

I had the same problem and I fixed it with this config, passing the config file to the args explicitly.

repos:
- repo: https://github.com/astral-sh/ruff-pre-commit
  # Ruff version.
  rev: v0.4.5
  hooks:
    # Run the linter.
    - id: ruff
      args: [ --fix, --config=pyproject.toml ]
    # Run the formatter.
    - id: ruff-format

NIvo172 · 2024-07-12T08:30:01Z

If you think your config is ignored. Add --verbose to the pre-commit hook.

- repo: https://github.com/astral-sh/ruff-pre-commit
  # Ruff: Formatter and Linter.
  rev: v0.5.1
  hooks:
    - id: ruff-format
      args: [--verbose, ]
    - id: ruff
      args: [--verbose, --fix,]

Then you can run pre-commit run --verbose --all-files

The output for me is:

ruff-format..............................................................Passed
- hook id: ruff-format
- duration: 0.08s

[ruff::resolve][DEBUG] Using configuration file (via parent) at: SOMEPATH/pyproject_template\pyproject.toml

ezeparziale · 2024-07-14T13:38:59Z

I upgraded from 0.4.5 to 0.5.1, and Ruff detects pyproject.toml automatically.

# See https://pre-commit.com for more information
default_language_version:
  python: python3.12
repos:
  - repo: https://github.com/astral-sh/ruff-pre-commit
    # Ruff version.
    rev: v0.5.1
    hooks:
      # Run the linter.
      - id: ruff
        args: [--fix]
      # Run the formatter.
      - id: ruff-format

Blumenkind111 · 2024-07-31T19:15:21Z

It works now for me too, but one thing I stumbled upon was when updating some ruff rules or exceptions and not commiting them with the python files (for whatever reason), the hook uses the last commited version of pyproject.toml, which does not have the new rule (or exclusion) and thus produces other results when calling ruff check from console.

NIvo172 · 2024-08-01T07:14:28Z

It works now for me too, but one thing I stumbled upon was when updating some ruff rules or exceptions and not commiting them with the python files (for whatever reason), the hook uses the last commited version of pyproject.toml, which does not have the new rule (or exclusion) and thus produces other results when calling ruff check from console.

@Blumenkind111 This is expected behavior of pre-commit. Pre-Commit Issue #818

exhuma · 2024-10-10T12:47:37Z

I'm currently also running into this.

My issue was that the exclude setting to ignore some folders was under the [tool.ruff.lint] section. Following the comment below made me think that was the correct location:

@caerulescens - Both are accepted right now -- putting ignore under [tool.ruff] and [tool.ruff.lint] are interchangeable. (We're moving towards the latter, but they both work as-is.)

Moving it back to [tool.ruff] fixed it.

tazr · 2024-10-22T20:22:16Z

In case that might help someone. In my case I noticed that the pyproject.toml was read (syntax errors there were found by ruff) but that ignore directives were... ignored. I found that was because I was using a --select directive in the .pre-commit-config.yaml file. For some reason this seems to reset the options put in the pyproject.toml. (This might be a feature so as to enforce some checks but I am unaware of a documentation supporting that.)

In order to be able to have ignore rules defined in one or multiple pyproject.toml I had to move select directives from the .pre-commit-config.yaml to those files.

denes44 · 2024-11-09T15:16:13Z

@tazr For that, I believe you should use --extend-select instead of --select

charliermarsh mentioned this issue Jan 14, 2024

import sorting? #64

Open

NIvo172 mentioned this issue Jul 11, 2024

Don't add space before new function on top of cell astral-sh/ruff#9696

Closed

rexledesma mentioned this issue Aug 15, 2024

[chore] unify local and pre-commit ruff configuration dagster-io/dagster#23501

Merged

Ruff configuration from pyproject.toml is ignored #54

Ruff configuration from pyproject.toml is ignored #54

Comments

beajifu commented Oct 13, 2023

charliermarsh commented Oct 13, 2023

beajifu commented Oct 14, 2023

beajifu commented Oct 15, 2023

klow68 commented Oct 19, 2023 • edited Loading

charliermarsh commented Oct 19, 2023

dhruvmanila commented Oct 30, 2023 • edited Loading

Steps

rpop0 commented Nov 21, 2023 • edited Loading

rytilahti commented Nov 21, 2023

caerulescens commented Jan 12, 2024 • edited Loading

charliermarsh commented Jan 12, 2024

caerulescens commented Jan 12, 2024 • edited Loading

rafalkrupinski commented Jan 26, 2024 • edited Loading

caerulescens commented Jan 27, 2024

charliermarsh commented Jan 27, 2024

caerulescens commented Jan 27, 2024 • edited Loading

caerulescens commented Jan 27, 2024 • edited Loading

Quatters commented Jan 30, 2024 • edited Loading

dhruvmanila commented Jan 30, 2024

Quatters commented Jan 31, 2024

rafalkrupinski commented Jan 31, 2024

dougthor42 commented Jan 31, 2024

caerulescens commented Feb 5, 2024 • edited Loading

caerulescens commented Feb 5, 2024 • edited Loading

rpop0 commented Feb 5, 2024

caerulescens commented Feb 5, 2024 • edited Loading

rpop0 commented Feb 5, 2024

caerulescens commented Feb 5, 2024 • edited Loading

caerulescens commented Feb 5, 2024 • edited Loading

wiseyoungbuck commented Mar 2, 2024

ezeparziale commented May 4, 2024

mazzma12 commented May 7, 2024

artefactop commented Jun 23, 2024

NIvo172 commented Jul 12, 2024 • edited Loading

ezeparziale commented Jul 14, 2024

Blumenkind111 commented Jul 31, 2024

NIvo172 commented Aug 1, 2024 • edited Loading

exhuma commented Oct 10, 2024

tazr commented Oct 22, 2024

denes44 commented Nov 9, 2024

klow68 commented Oct 19, 2023 •

edited

Loading

dhruvmanila commented Oct 30, 2023 •

edited

Loading

rpop0 commented Nov 21, 2023 •

edited

Loading

caerulescens commented Jan 12, 2024 •

edited

Loading

caerulescens commented Jan 12, 2024 •

edited

Loading

rafalkrupinski commented Jan 26, 2024 •

edited

Loading

caerulescens commented Jan 27, 2024 •

edited

Loading

caerulescens commented Jan 27, 2024 •

edited

Loading

Quatters commented Jan 30, 2024 •

edited

Loading

caerulescens commented Feb 5, 2024 •

edited

Loading

caerulescens commented Feb 5, 2024 •

edited

Loading

caerulescens commented Feb 5, 2024 •

edited

Loading

caerulescens commented Feb 5, 2024 •

edited

Loading

caerulescens commented Feb 5, 2024 •

edited

Loading

NIvo172 commented Jul 12, 2024 •

edited

Loading

NIvo172 commented Aug 1, 2024 •

edited

Loading