coulomb-runtime for scala3

[erikerlandson]

This PR is to develop unit handling at runtime for scala3 coulomb, using the new staging compiler and phased metaprogramming.
https://docs.scala-lang.org/scala3/reference/metaprogramming/staging.html

breadcrumb xref for running staging compiler in sbt repl: scala/scala3#7647

[erikerlandson]

I'm reframing this as coulomb-runtime instead of coulomb-parser because there are many possible runtime formats for units, each with their own parser. The core of this library capability is more about being able to lift unit types only known at runtime into the more standard statically available types. I may not implement any specific parsers in this library, as they are more specific to individual runtime formats (avro, other json types, etc)

[erikerlandson]

A couple nice links on scala 3 metaprogramming
https://softwaremill.com/scala-3-macros-tips-and-tricks/
https://docs.scala-lang.org/scala3/reference/metaprogramming/staging.html
https://docs.scala-lang.org/scala3/reference/metaprogramming/macros.html

[erikerlandson]

It's working.

scala> import scala.quoted.*, coulomb.runtime.*, coulomb.runtime.syntax.*, coulomb.*, coulomb.syntax.*, coulomb.rational.Rational, coulomb.policy.standard.given
                                                                                                                                    
scala> given staging.Compiler = staging.Compiler.make(classOf[staging.Compiler].getClassLoader)
lazy val given_Compiler: quoted.staging.Compiler
                                                                                                                                    
scala> 1d.withUnitRuntime[coulomb.units.us.Yard](UnitAST.of[coulomb.units.si.Meter])
val res0: coulomb.Quantity[Double, coulomb.units.us.Yard] = 1.0936132983377078

[erikerlandson]

@armanbilge My initial purpose here was to support the scala-2 "parsing" on scala-3, which this does, but I'm also starting to think it might allow me to support your earlier request for "runtime Quantities" without creating an entire new parallel universe of runtime defs.

You can see that my pair (v, RuntimeUnit) could become case class RuntimeQuantity[V](v: V, unit: RuntimeUnit), and the function kernelRuntime shows that we can bridge the realm between runtime conversions but using the static implicit unit definitions we already have.

There would be some work to do on what the most elegant design is, but this now makes it seem feasible.

scala> import scala.quoted.*, coulomb.runtime.*, coulomb.runtime.syntax.*, coulomb.*, coulomb.syntax.*, coulomb.rational.Rational, coulomb.policy.standard.given, coulomb.units.si.{*, given}, coulomb.units.time.{*, given}, coulomb.units.us.{*, given}
                                                                                                                                    
scala> given staging.Compiler = staging.Compiler.make(classOf[staging.Compiler].getClassLoader)
lazy val given_Compiler: quoted.staging.Compiler
                                                                                                                                    
scala> val rq = (1d, RuntimeUnit.of[Meter / Second])
val rq: (Double, coulomb.runtime.RuntimeUnit) = (1.0,Div(UnitType(coulomb.units.si$.Meter),UnitType(coulomb.units.si$.Second)))
                                                                                                                                    
scala> rq.toQuantity[Yard / Minute]
val res0: 
  Either[String, 
    coulomb.Quantity[Double, coulomb.units.us.Yard / coulomb.units.time.Minute]
  ] = Right(65.61679790026247)

[erikerlandson]

So in a hypothetical world with RuntimeQuantity, the implementation of things like loading unit-aware data from external formats such as pureconfig, avro, etc, become routines that return RuntimeQuantity, and the process of lifting to statically typed Quantity is just a call to toQuantity

[armanbilge]

but I'm also starting to think it might allow me to support your earlier request for "runtime Quantities" without creating an entire new parallel universe of runtime defs.

This is a very interesting idea! I think you are right—it lets you bridge that gap.

The only catch is I'm guessing that runtime staging is not support on Scala.js or Scala Native. But it's worth experimenting with on the JVM.

[erikerlandson]

@armanbilge it looks like you are right - I added a unit test, which runs in JVM but doesn't in either JS or Native. That's a damn shame, because I'm really starting to like how it is shaping up.

Tangent: is there a corresponding ThisBuild / endYear := Some(2023), the variations I tried don't exist.

[erikerlandson]

@armanbilge I might experiment with other approaches. One thing that is nice about this is that it appears to work without me explicitly generating any maps from types to corresponding static unit defs. The calls to Implicits.search are working without that.

But I could go back to experimenting with constructing such maps, which might allow similar resolutions at runtime. It would be slightly annoying because I'd need to duplicate the logic of cansig in a runtime function but it would only need to be done once and it could still leverage existing static unit defs. You'd just have to generate some kind of mapper at compile time and explicitly give it which defs you wanted.

[erikerlandson]

This is so clean, I feel unsatisfied with anything else 😂

[armanbilge]

Tangent: is there a corresponding ThisBuild / endYear := Some(2023), the variations I tried don't exist.

Yep, see: sbt/sbt-header#282. You might need to add the latest version of sbt-header to your project/plugins.sbt.

---

@armanbilge I might experiment with other approaches. One thing that is nice about this is that it appears to work without me explicitly generating any maps from types to corresponding static unit defs.

Yeah honestly this is one of the best applications I've seen of staging. It maybe even challenges the premise from that post I think I shared a while back, about the inevitability of defining things in both the runtime and compile time layer.

So I think this is worth thinking more about, and talking about :) Scala is not there yet, but maybe it will eventually be able to do something like this on all platforms.

[erikerlandson]

@armanbilge at least for coulomb, I feel like this unambiguously shows we can support both runtime and compile-time with the existing static unit defs. I have no idea how general that result is for other applications. I'm still a bit confused about how it's working here, but it definitely is.

As far as support for JS and Native, in theory it seems like it could. It knows at compile-time what the result-type of the runtime expression will be, so the staging compiler could run against the AST and leave behind the result in whatever JS or Native is expecting. I don't have much feel for whether the quoted expressions are expressible in JS or native, or if they need to be.

[erikerlandson]

@armanbilge I think the thing I'm most curious about is that Implicits.search is finding BaseUnit and DerivedUnit definitions at run-time, even though I am not importing them into scope in my runtime code. It almost seems like it violates the semantic that one has to import given definitions to use them. Not that I am complaining here, since it allows me to do this with magical simplicity.

[erikerlandson]

@armanbilge I think I have a solution that feels good. I defined a new concept called CoefficientRuntime, which owns the responsibility to identify coefficients at runtime. The current flavor uses the staging compiler, so if you are on the JVM it will work like it has been. However, if you want to work on JS or Native, I can define ways to create other flavors of CoefficientRuntime that may be a bit less elegant but will not require staging compilation. Most of the rest of the library will be written (using crt: CoefficientRuntime...) so as long as you declare your chosen flavor: given CoefficientRuntime = ... everything else will work the same.

[erikerlandson]

@armanbilge this should work on JS and Native:

Welcome to Scala 3.2.2 (17.0.5, Java OpenJDK 64-Bit Server VM).
Type in expressions for evaluation. Or try :help.
                                                                                                                                                                                                             
scala> import scala.quoted.*, coulomb.runtime.*, coulomb.runtime.syntax.*, coulomb.*, coulomb.syntax.*, coulomb.rational.Rational, coulomb.policy.standard.given, coulomb.units.si.{*, given}, coulomb.units.time.{*, given}, coulomb.units.us.{*, given}, coulomb.units.si.prefixes.{*, given}
                                                                                                                                                                                                             
scala> import coulomb.runtime.conversion.runtimes.mapping.*
                                                                                                                                                                                                             
scala> import MappingCoefficientRuntime.{TNil, &:}
                                                                                                                                                                                                             
scala> MappingCoefficientRuntime.of[Kilogram &: Meter &: Second &: Kilo &: Yard &: TNil]
val res0: coulomb.runtime.conversion.runtimes.mapping.MappingCoefficientRuntime = coulomb.runtime.conversion.runtimes.mapping.MappingCoefficientRuntime@78696450
                                                                                                                                                                                                             
scala> res0.coefficientRational(RuntimeUnit.of[Kilo * Meter], RuntimeUnit.of[Meter])
val res1: Either[String, coulomb.rational.Rational] = Right(1/1000)
                                                                                                                                                                                                             
scala> res0.coefficientRational(RuntimeUnit.of[Kilo * Meter], RuntimeUnit.of[Second])
val res2: Either[String, coulomb.rational.Rational] = Left(non-convertible units: Kilo*Meter, Second)
                                                                                                                                                                                                             
scala> 

[erikerlandson]

As a convenience, I now support module names, so it will automatically ingest all defined units in a given module:

MappingCoefficientRuntime.of["coulomb.units.si" &: "coulomb.units.si.prefixes" &: TNil]

[erikerlandson]

@armanbilge

coulomb/runtime/src/test/scala/coulomb/runtimequantity.scala

Line 56 in c4e67fd

test("addition strict") {

[erikerlandson]

From here on out, I think I'm going to want to co-implement this with coulomb-pureconfig since I'll want to make sure that the design of RuntimeQuantity provides what is needed to support pureconfig integration.

xref: pureconfig/pureconfig#1330 (comment)

[erikerlandson]

It would be nice to support polymorphic inputs with pureconfig, or equivalently an "or" combinator for pureconfig readers (and writers).

xref: pureconfig/pureconfig#1472

[erikerlandson]

expression parsing for cats-parse
typelevel/cats-parse#449
https://github.com/j-mie6/design-patterns-for-parser-combinators#readme