Structural types with "primitive" methods throw `NoSuchMethodException`

[xuwei-k]

Compiler version

• 3.8.0-RC4
• 3.7.4
• 3.3.7

Minimized code

//> using scala 3.8.0-RC4

import scala.reflect.Selectable.reflectiveSelectable

def x(y: Any{def +(z: Int): Int}): Int = y + 5
 
@main def test = print(x(2))

Output

java.lang.NoSuchMethodException: java.lang.Integer.$plus(int)
	at java.base/java.lang.Class.getMethod(Class.java:2397)
	at scala.reflect.Selectable.applyDynamic(Selectable.scala:40)
	at scala.reflect.Selectable.applyDynamic$(Selectable.scala:13)
	at scala.reflect.Selectable$DefaultSelectable.applyDynamic(Selectable.scala:53)
	at A$package$.x(A.scala:5)
	at A$package$.test(A.scala:7)
	at test.main(A.scala:7)

Expectation

compile error or success. not runtime error

Note

• Structural typing, exception bug#192
• scala/scala@5e17585

[prolativ]

Seems like a duplicate of #16995

[hamzaremmal]

Seems like a duplicate of #16995

It doesn't seem to be a duplicate to me. Why already close it? One has a clear solution (#16995) while this one has no clear solution that is valid and doesn't hurt performance (#24859).

[bishabosha]

This is specific to any of Scala stdlib fake methods i assume? Here is String.+

scala> import scala.reflect.Selectable.reflectiveSelectable
     |
     | def x(y: Any{def +(z: Any): String}): String = y + " hello"
def x(y: Any{def +(z: Any): String}): String

scala> x("wow,")
java.lang.NoSuchMethodException: java.lang.String.$plus(java.lang.Object)
  at java.base/java.lang.Class.getMethod(Class.java:2108)
  at scala.reflect.Selectable.applyDynamic(Selectable.scala:38)

Array.apply

scala> import scala.reflect.Selectable.reflectiveSelectable
     |
     | def x(y: Any{def apply(i: Int): String}): String = y.apply(0)
def x(y: Any{def apply(i: Int): String}): String

scala> x(Array("abc"))
java.lang.NoSuchMethodException: [Ljava.lang.String;.apply(int)

[hamzaremmal]

Yes, every type where the representation in the JVM is different from the Scala one. Either via boxing: Int -> java.lang.Integer in the original example. Or via primitive types: scala.Array and operators + of String in @bishabosha's examples. This is the core issue; the difference in the representation at compiletime and at runtime. This list, obviously, is not exhaustive but it gives the idea of why I believe there is no clear solution that doesn't hurt performance (since we have to check these edge cases in the implementation of reflectiveSelectable).

This is different from #16995 where the fix there is to simply erase the classOf parameters too correctly.

[sjrd]

Since it hasn't been pointed out yet: Scala 2 handles this correctly:

$ cs launch scala:2.13.18
Welcome to Scala 2.13.18 (OpenJDK 64-Bit Server VM, Java 1.8.0_472).
Type in expressions for evaluation. Or try :help.

scala> import scala.language.reflectiveCalls
import scala.language.reflectiveCalls

scala> def x(y: Any{def +(z: Int): Int}): Int = y + 5
def x(y: Any{def +(z: Int): Int}): Int

scala> print(x(2))
7

That's what these are for:
https://github.com/scala/scala/blob/8073bce6dd787d96b627a05438ca9199961358b4/src/library/scala/runtime/BoxesRunTime.java#L80

[hamzaremmal]

@sjrd but reflectiveCalls is not the same as reflectiveSelectable, right? I don't know how the former is implemented but I know that the second one is implemented at the user level:

scala3/library/src/scala/reflect/Selectable.scala

Lines 13 to 43 in 7ddf329

	trait Selectable extends scala.Selectable:
	/** The value from which structural members are selected.
	* By default this is the Selectable instance itself, but it can
	* be overridden.
	*/
	protected def selectedValue: Any = this
	// The Scala.js codegen relies on this method being final for correctness
	/** Selects member with given name. */
	final def selectDynamic(name: String): Any =
	val rcls = selectedValue.getClass
	try
	val fld = rcls.getField(NameTransformer.encode(name)).nn
	ensureAccessible(fld)
	fld.get(selectedValue)
	catch case ex: NoSuchFieldException =>
	applyDynamic(name)()
	// The Scala.js codegen relies on this method being final for correctness
	/** Selects method and applies to arguments.
	* @param name The name of the selected method
	* @param paramTypes The class tags of the selected method's formal parameter types
	* @param args The arguments to pass to the selected method
	*/
	final def applyDynamic(name: String, paramTypes: Class[?]*)(args: Any*): Any =
	val rcls = selectedValue.getClass
	val mth = rcls.getMethod(NameTransformer.encode(name), paramTypes*).nn
	ensureAccessible(mth)
	mth.invoke(selectedValue, args.asInstanceOf[Seq[AnyRef]]*)

The fix I have in mind is to add support here and special case here, which it will obviously be problematic from a performance point of view.

[bishabosha]

Scala 2 compiler also has a phase that transforms applyDynamic of the old reflectiveCalls to invoke a static cache that is stored in the calling class, - this also aparently handles the array apply and other primitive classes with an explicit branch:
(see https://github.com/scala/scala/blob/8073bce6dd787d96b627a05438ca9199961358b4/src/compiler/scala/tools/nsc/transform/CleanUp.scala#L80)

scala> def x(y: Any{def apply(i: Int): String}): String = y.apply(0)
<synthetic> <static> def reflMethod$Method1(x$1: Class): reflect.Method = {
  var methodCache1: runtime.StructuralCallSite = <apply-dynamic>(apply#<none>, (bootstrap, classOf[]));
  var method1: reflect.Method = methodCache1.find(x$1);
  if (method1.ne(null))
    return method1
  else
    {
      method1 = scala.runtime.ScalaRunTime.ensureAccessible(x$1.getMethod("apply", methodCache1.parameterTypes()));
      methodCache1.add(x$1, method1);
      return method1
    }
};
def x(y: Object): String = {
  val qual1: Object = y;
  if (qual1.getClass().isArray())
    scala.runtime.ScalaRunTime.array_apply(qual1, scala.Int.unbox(scala.Int.box(0))).$asInstanceOf[String]()
  else
    try {
      iw.this.reflMethod$Method1(qual1.getClass()).invoke(qual1, Array[Object]{scala.Int.box(0)})
    } catch {
      case (1 @ (_: reflect.InvocationTargetException)) => throw 1.getCause()
    }.$asInstanceOf[String]()

[hamzaremmal]

This is the output tree of Scala 2.13.16 for @sjrd's example:

package <empty> {
  object Foo extends Object {
    <synthetic> <static> def reflMethod$Method1(x$1: Class): reflect.Method = {
      var methodCache1: runtime.StructuralCallSite = <apply-dynamic>(apply#<none>, (bootstrap, classOf[]));
      var method1: reflect.Method = methodCache1.find(x$1);
      if (method1.ne(null))
        return method1
      else
        {
          method1 = scala.runtime.ScalaRunTime.ensureAccessible(x$1.getMethod("$plus", methodCache1.parameterTypes()));
          methodCache1.add(x$1, method1);
          return method1
        }
    };
    def x(y: Object): Int = scala.Int.unbox({
      val qual1: Object = y;
      if (scala.runtime.BoxesRunTime.isBoxedNumber(qual1))
        scala.runtime.BoxesRunTime.add(qual1, scala.Int.box(5)).$asInstanceOf[Integer]()
      else
        try {
  Foo.reflMethod$Method1(qual1.getClass()).invoke(qual1, Array[Object]{scala.Int.box(5)})
} catch {
  case (1 @ (_: reflect.InvocationTargetException)) => throw 1.getCause()
}.$asInstanceOf[Integer]()
    });
    def test(): Unit = scala.Predef.print(scala.Int.box(Foo.this.x(scala.Int.box(2))));
    def <init>(): Foo.type = {
      Foo.super.<init>();
      ()
    }
  }
}

The difference between Scala 2 and Scala 3 is that Scala 2 uses the knowledge and has assumptions about what could go wrong and how to handle them and it only handles them when needed while Scala 3, we have a more general definition in the stdlib that has no knowledge about what is being called and what could go wrong.