make lookup operator honor refined return type of "get()"

[CeylonMigrationBot]

[@jvasileff] In "6.8.7. Correspondence, subrange, and stream operators", the lookup operator is defined as yielding the result of lhs.get(index) with type Y? for lhs Correspondence<X,Y>. But the actual return type of lhs.get(index) may be a subtype of Y?.

It would be nice if the lookup operator honored the actual return type of get rather than assuming an optional type.

Usage example:

object multimap satisfies Correspondence<String, [Integer*]> {
    shared actual Boolean defines(String key) => nothing;
    shared actual [Integer*] get(String key) => [];
}
[Integer*] result1 = multimap[""]; // error Integer[]? is not assignable to Integer[]
[Integer*] result2 = multimap.get(""); // ok

[Migrated from ceylon/ceylon-spec#1411]

[CeylonMigrationBot]

[@gdejohn]
👍

I've had this same thought.

[xkr47]

Bumped into this too. 👍 I created a map that automatically creates items if they did not exist in the map before, and thus the return value is never null..

[gavinking]

If only I would have had the foresight to define Correspondence and Map/List like this:

shared interface Correspondence<in Key, out Item=Anything>
        given Key satisfies Object {
    shared formal Item get(Key key);

    ...
}

Or to have defined Map like this:

shared interface Map<out Key=Object, out Item=Anything>
        satisfies Collection<Key->Item> &
                  Correspondence<Object,Item?>
        given Key satisfies Object { ... }

shared interface List<out Element=Anything>
        satisfies Collection<Element> &
                  Correspondence<Integer,Element?> &
                  Ranged<Integer,Element,List<Element>> {

I screwed up here, it seems to me :-(

[xkr47]

Does the x[y] operator have to be tied to the Correspondence interface? It can't just alias to x.get(y) ?

[gavinking]

On the other hand, there's a somewhat strong argument that for a "total" correspondence, the correct type to use is X(Y), not Correspondence<X,Y>, and then you get the operator x(y) instead of x[y]. The operations of Correspondence assume that we are dealing with a partial function.

[gavinking]

Does the x[y] operator have to be tied to the Correspondence interface? It can't just alias to x.get(y) ?

Well, that's the way all other operators in Ceylon are defined.

[gavinking]

We could, in principle, the next time we break BC, make the following change:

shared interface Correspondence<in Key, out Item=Anything, out Absent=Null>
        given Key satisfies Object {
    shared formal Item | Absent get(Key key);

    ...
}

[xkr47]

is that better/less breaking than the earlier suggestion you had?

[gavinking]

Less breaking.

[lucaswerkmeister]

for a "total" correspondence, the correct type to use is X(Y)

That would require #3950 to be useful.

[phlopsi]

For the Correspondence interface, I'd keep Null as the type to represent absence and instead restrict Item to be of type Object, i.e. changing the interface to:

shared interface Correspondence<in Key, out Item=Object>
    given Key satisfies Object
    given Item satisfies Object

If one tries to write a generic function for storing arbritary values, Absent could be included in the Item type, too. In that case, the return type may still be ambiguous. What would make more sense is to stick with Item?, but let Item satisfy Object. For example, if Null needs to be stored, one could define Item as Supplier<Anything> to differentiate between the null returned for "no item found" and the null, that is stored as an item.

Alternatively, although I don't know if it's possible, Absent would make sense, if it's possible to tell the compiler something like given Item not satisfies Absent, i.e. Item may not include the type of Absent in order to rule out ambiguity when calling Correspondence.get.

[jvasileff]

@phlopsi There's nothing stopping you from doing that today. For example:

alias Maybe<Element> => [Element] | [];
Map<String, Maybe<Integer>> m = map { "a"->[], "b"->[1] };

But I haven't seen that done in Ceylon code, and I'm not sure it would be a good idea to force people to use that pattern.

Taking a step back, we know that optionals can be represented by monads (Maybe<Element>) or union types (Element | Null). They have the following respective downsides:

• With maybes, you can wind up with deeply nested types like Maybe<Maybe<Maybe<Element>>> that carry more information that you may want, if all you need is Maybe<Element> (e.g. listOfMaybeStrings.first would return Maybe<Maybe<String>>)
• With union types, Element | Null | Null | Null always collapses to Element | Null, resulting in an ambiguity when you do care about the intermediate steps.

Which one is better varies by use case. (Although, it's worth noting that unions have the additional advantage that Element is assignable to Element | Null.)

Now, your suggestion is have two standard ways to represent optionals in order to eliminate the above tradeoff. But I doubt that would work well in practice:

• there would be two standard/common ways to do things like exists, else, and ?. for optionals which would make both reading and writing code more difficult
• inevitably you'd run into confusing mixes of optional types at various depths, like Maybe<Maybe<String?>?>

A very "simple" type that I think would be confusing to work with:

Map<String, Maybe<Map<String, Maybe<String>>>>

as opposed to the more Ceylonic:

Map<String, Map<String, String?>?>

So, I think it's better for a language to stick with one approach or the other, which for Ceylon has been to use union types.

[phlopsi]

@jvasileff Deep-nesting has nothing to do with this. It's exactly 1 wrapper, if and only if the Absent type (currently fixed to Null) would be included in the Item type.

If the language would provide a way to model the inverse of given A satisfies B, that would be a huge improvement, because the strong point of this language is, that it is explicit and wherever possible, uses syntax errors instead of runtime errors, to detect errors as early as possible, namely during typechecking/compilation. Having a part of the language be ambiguous by design and prone to semantic errors doesn't look ceylonic to me, at all, if there is an obvious way to model them as syntactic errors, instead, which is the case for this issue.

The only other way to not be ambiguous is to reserve a type, that may not be included in the Item type. Since Null is the only type, that is not an Object and Object is implicitly satisfied by everything except Null, restricting Item to being an Object and reserving Null as a pure Absent type, is the only logical choice I'm seeing, unless I'm missing important information.