C should not be assignable to Self@C unless C is final #706

[edmondop]

This PR addresses #706 by making the return-type check verify, in case the return type is Self, that the class is defined as final

[meta-cla]

Hi @edmondop!

Thank you for your pull request and welcome to our community.

Action Required

In order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you.

Process

In order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA.

Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with CLA signed. The tagging process may take up to 1 hour after signing. Please give it that time before contacting us about it.

If you have received this in error or have any questions, please contact us at [email protected]. Thanks!

[meta-cla]

Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Meta Open Source project. Thanks!

[stroxler]

Hi @edmondop, thanks for the PR!

This looks plausible to me, but there are several failing tests and it would be nice to see the behavior changes in the diff view in case this is interacting badly with other parts of Pyrefly (in particular, I see some incompatible-override errors popping up and it would be nice to make sure those make sense).

cc @rchen152 and @samwgoldman who are probably our experts on the Self type (and for that matter function subtyping as well!)

[edmondop]

Thanks, will look into the failing tests. I didn’t catch them locally because I think they were all passing until the new one was failing and once I fixed the new one, I forgot to run again the others. Let me look into the failing tests before anyone spend time reviewing broken code

…

On Sat, Sep 20, 2025 at 8:31 PM Steven Troxler ***@***.***> wrote: ***@***.**** commented on this pull request. Hi @edmondop <https://github.com/edmondop>, thanks for the PR! This looks plausible to me, but there are several failing tests and it would be nice to see the behavior changes in the diff view in case this is interacting badly with other parts of Pyrefly (in particular, I see some incompatible-override errors popping up and it would be nice to make sure those make sense). cc @rchen152 <https://github.com/rchen152> and @samwgoldman <https://github.com/samwgoldman> who are probably our experts on the Self type (and for that matter function subtyping as well!) — Reply to this email directly, view it on GitHub <#1117 (review)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAGGOKIYFZYYSJ6654MZCCL3TXWWNAVCNFSM6AAAAACHB4ZOT2VHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZTENBZGMZDEOBYGY> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[edmondop]

I confirm the change has broken some existing funtionality, and some tests that should be passing are not. I haven't investigated the other tests, but this one is a good example:

testcase!(
    test_call_instance_method_from_classmethod,
    r#"
class A:
    def f(self):
        pass

class B(A):
    @classmethod
    def g(cls):
        super().f(cls())
    "#,
);

I need to dive deeper into subset.rs and understand if I have the context, I think the difference in the good case is that Self is in the return type, while here is an argument to f. According to Liskov Substitution principle, this test case should pass for any subclass of B, so the fact that Bm is not final is not a problem

[edmondop]

I haven't being able to find a solution that affect only subset.rs that wouldn't break previous tests, because within solver it's unclear whether got/want are in parameter or return type position, while in the solve.rs I extended the return type handling code

[asukaminato0721]

how about https://rust-lang.github.io/rfcs/2497-if-let-chains.html

[edmondop]

Thanks, I thought I had run cargo clippy. Fixed now

[stroxler]

Yeah this is tricky and tied into a few very difficult questions:

• what the Self type actually means (PEP 673 actually specified it as a method level type parameter, but I don't think most type checkers treat it that way anymore)
• whether it is allowed to appear in parameter position, except by being bound as the receiver object (this is unsound, but I think most type checkers allow it anyway because best-effort unsound analysis is sometimes helpful for stating intent and catching a subset of bugs)

If we actually solved Self as a bound type variable in the call as PEP 673 specifies, I'm pretty certain that the error would go away but at the moment our handling of Self is entirely adhoc and I don't think we ever actually solve it as a Quantified.

The adhoc handling has benefits because it allows us to do things like handle an attribute specified as Self (which is again unsound if the attribute is mutable, but we allow this), so I'm unsure how easy it would be to change the behavior and match PEP 673 better. We might be able to detect the direct use of an unbound method and convert it to a ForAll type where Self behaves like a normal function-level type variable.

Let's see if @rchen152 and @samwgoldman have thoughts

[samwgoldman]

Thanks for contributing! I don't think the fix is in the right place. This diff adds some logic to the code handling returns with explicit annotations, but assignability happens in more places, like x: Self = C() within a method in C. It would also apply to the cases in returns that are not handled by this change, async and generated functions.

The right place for this fix is to remove the invalid rules in subset.rs, here: https://github.com/facebook/pyrefly/blob/main/pyrefly/lib/solver/subset.rs#L1005-L1013

These rules say it is OK to assign C to Self@C, which is exactly false, so the right thing to do is remove them.

However, there is a snag -- we rely on this rule in a few places. Usually this happens when we fail to preserve "Self"-ness. I recently fixed a couple examples of this, in 968a99c and in fc7592c.

I know of at least one other place we get this wrong -- calling a constructor on the call target type[Self] [Sandbox].

[samwgoldman]

I think the original issue is mislabeled as "good first issue" -- but you are welcome to keep going if you'd like. Try removing the invalid rules first and see what tests fail. Hopefully you will see an example like the ctor call on type[Self], but maybe some others. Those issues would need to be fixed first, preferably in separate PRs.

[edmondop]

Thanks, I tried to originally make that change and it was causing several test to fail. I was trying to restrict my condition in the subset.rs based on the location of the type (return type) but I wasn't able to find a way through the type_order to access this information. Was I looking in the wrong place?

[samwgoldman]

I see -- in subset.rs it would not be possible to restrict to just return types, but I don't think such a restriction is worthwhile, since the bug exists everywhere. The fact that other tests start to fail is likely because of other latent issues that need to be fixed first, per my comment above. For example, the test_call_instance_method_from_classmethod fails because calling cls() should return Self@C but currently incorrectly returns C -- constructing type[Self] should return Self.

[edmondop]

I think I found the root cause of why I wasn't able to solve the problem in subset.rs. Doing the check on final would fix the problems indicated in the issue #706 but would introduce test failures somewhere else.

The problem was that in call.rs

     Type::Type(box Type::ClassType(cls)) | Type::Type(box Type::SelfType(cls)) => {
                Some(CallTarget::Class(cls))
            }

this handling means that later in the call_infer, the type[Self] becomes a type[ConcreteClass],

In practice this mean that would end up being processed on a different branch of subset.rs rather than the one we expect

class B:
    @classmethod
    def factory(cls) -> Self:
        return cls()

The solution seems to be to introduce a new enum type for the CallTarget of type Self, and produce a Type::SelfType later for subset.rs. This ensures that subset.rs pattern matching receive the right type. (I haven't figure out how to handle type parameters yet)

[edmondop]

maybe this should be factor out so we can share it between ClassTarget::Class and ClassTarget::SelfClass since this code is identical