From ebf59fe74e5c819f16ba572514a65ab412c42051 Mon Sep 17 00:00:00 2001 From: Zachary Sunberg Date: Thu, 28 Jul 2022 22:46:32 -0700 Subject: [PATCH 01/14] wrote main ideas in README --- README.md | 79 +++++++++++++++++++++++++++++++++++++++++++------------ 1 file changed, 62 insertions(+), 17 deletions(-) diff --git a/README.md b/README.md index 1328f9e..3545b87 100644 --- a/README.md +++ b/README.md @@ -5,25 +5,72 @@ [![Code Style: Blue](https://img.shields.io/badge/code%20style-blue-4495d1.svg)](https://github.com/invenia/BlueStyle) [![PkgEval](https://JuliaCI.github.io/NanosoldierReports/pkgeval_badges/C/CommonRLSpaces.svg)](https://JuliaCI.github.io/NanosoldierReports/pkgeval_badges/report.html) +## Introduction + +A space is simply a set of objects. In a reinforcement learning context, spaces define the sets of possible states, actions, and observations. + +In Julia, spaces can be represented by a variety of objects. For instance, a small discrete action set might be represented with `["up", "left", "down", "right"]`, or an interval of real numbers might be represented with an object from the `IntervalSets` package. In general, the space defined by any Julia object is the set of objects `x` for which `x in space` returns `true`. + +In addition to establishing the definition above, this package provides three useful tools: +1. Traits to communicate about the properties of spaces, e.g. whether they are continuous or discrete, how many dimensions they have, and how to interact with them. +2. Functions such as `product` for constructing more complex spaces +3. Constructors to for spaces whose elements are arrays, such as `ArraySpace` and `Box`. + +## Concepts and Interface + +### Interface for all spaces + +Since a space is simply a set of objects, a wide variety of common Julia types including `Vector`, `Set`, `Tuple`, and `Dict`1can represent a space. +Because of this inclusive definition, there is a very minimal interface that all spaces are expected to implement. Specifically, it consists of +- `in(x, space)`, which tests whether `x` is a member of the set `space` (this can also be called with the `x in space` syntax). +- `rand(space)`, which returns a valid member of the set2. +- `eltype(space)`, which returns the type of the elements in the space. + +In addition, the `SpaceStyle` trait is always defined. Calling `SpaceStyle(space)` will return either a `FiniteSpaceStyle`, `ContinuousSpaceStyle`, `HybridSpaceStyle`, or an `UnknownSpaceStyle` object. + +### Finite discrete spaces + +Spaces with a finite number of elements have `FiniteSpaceStyle`. These spaces are guaranteed to be iterable, implementing Julia's [iteration interface](https://docs.julialang.org/en/v1/manual/interfaces/). In particular `collect(space)` will return all elements in an array. + +### Continuous spaces + +Continuous spaces represent sets that have an uncountable number of elements they have a `SpaceStyle` of type `ContinuousSpaceStyle`. CommonRLSpaces does not adopt a rigorous mathematical definition of a continuous set, but, roughly, elements in the interior of a continuous space have other elements arbitrarily close to them. +Continuous spaces have two additional interface functions: +- `bounds(space)` returns upper and lower bounds in a tuple. For example, if `space` is a unit circle, `bounds(space)` will return `([-1.0, -1.0], [1.0, 1.0])`. This allows agents to choose policies that appropriately cover the space e.g. a normal distribution with a mean of `mean(bounds(space))` and a standard deviation of half the distance between the bounds. +- [I have a few other ideas, but we can discuss later] + +### Hybrid spaces + +[need to figure this out] + +### Spaces of arrays + +[need to figure this out, but I think `elsize(space)` should return the size of the arrays in the space] + +--- + +1Note: the elements of a space represented by a `Dict` are key-value `Pair`s. +2[TODO: should we make any guarantees about whether `rand(space)` is drawn from a uniform distribution?] + ## Usage ### Construction |Category|Style|Example| |:---|:----|:-----| -|Enumerable discrete space| `DiscreteSpaceStyle{()}()` | `Space((:cat, :dog))`, `Space(0:1)`, `Space(1:2)`, `Space(Bool)`| -|Multi-dimensional discrete space| `DiscreteSpaceStyle{(3,4)}()` | `Space((:cat, :dog), 3, 4)`, `Space(0:1, 3, 4)`, `Space(1:2, 3, 4)`, `Space(Bool, 3, 4)`| -|Multi-dimensional variable discrete space| `DiscreteSpaceStyle{(2,)}()` | `Space(SVector((:cat, :dog), (:litchi, :longan, :mango))`, `Space([-1:1, (false, true)])`| -|Continuous space| `ContinuousSpaceStyle{()}()` | `Space(-1.2..3.3)`, `Space(Float32)`| -|Multi-dimensional continuous space| `ContinuousSpaceStyle{(3,4)}()` | `Space(-1.2..3.3, 3, 4)`, `Space(Float32, 3, 4)`| +|Enumerable discrete space| `FiniteSpaceStyle{()}()` | `(:cat, :dog)`, `0:1`, `["a","b","c"]` | +|One dimensional continuous space| `ContinuousSpaceStyle{()}()` | `-1.2..3.3`, `Interval(1.0, 2.0)` | +|Multi-dimensional discrete space| `FiniteSpaceStyle{(3,4)}()` | `ArraySpace((:cat, :dog), 3, 4)`, `ArraySpace(0:1, 3, 4)`, `ArraySpace(1:2, 3, 4)`, `ArraySpace(Bool, 3, 4)`| +|Multi-dimensional variable discrete space| `FiniteSpaceStyle{(2,)}()` | `product((:cat, :dog), (:litchi, :longan, :mango))`, `product(-1:1, (false, true))`| +|Multi-dimensional continuous space| `ContinuousSpaceStyle{(3,4)}()` | `Box([-1.0, -2.0], [2.0, 4.0])`, `product(-1.2..3.3, -4.6..5.0)`, `ArraySpace(-1.2..3.3, 3, 4)`, `ArraySpace(Float32, 3, 4)` | +|Multi-dimensional hybrid space| `HybridSpaceStyle{(2,),()}()` | `product(-1.2..3.3, -4.6..5.0, [:cat, :dog])`, `product(Box([-1.0, -2.0], [2.0, 4.0]), [1,2,3])`| ### API ```julia julia> using CommonRLSpaces -julia> s = Space((:litchi, :longan, :mango)) -Space{Tuple{Symbol, Symbol, Symbol}}((:litchi, :longan, :mango)) +julia> s = (:litchi, :longan, :mango) julia> rand(s) :litchi @@ -31,13 +78,12 @@ julia> rand(s) julia> rand(s) in s true -julia> size(s) -() +julia> length(s) +3 ``` ```julia -julia> s = Space(UInt8, 2,3) -Space{Matrix{UnitRange{UInt8}}}(UnitRange{UInt8}[0x00:0xff 0x00:0xff 0x00:0xff; 0x00:0xff 0x00:0xff 0x00:0xff]) +julia> s = ArraySpace(UInt8, 2,3) julia> rand(s) 2×3 Matrix{UInt8}: @@ -48,15 +94,14 @@ julia> rand(s) in s true julia> SpaceStyle(s) -DiscreteSpaceStyle{(2, 3)}() +FiniteSpaceStyle{(2, 3)}() -julia> size(s) +julia> elsize(s) (2, 3) ``` ```julia -julia> s = Space(SVector(-1..1, 0..1)) -Space{SVector{2, ClosedInterval{Int64}}}(ClosedInterval{Int64}[-1..1, 0..1]) +julia> s = product(-1..1, 0..1) julia> rand(s) 2-element SVector{2, Float64} with indices SOneTo(2): @@ -69,6 +114,6 @@ true julia> SpaceStyle(s) ContinuousSpaceStyle{(2,)}() -julia> size(s) +julia> elsize(s) (2,) -``` \ No newline at end of file +``` From f1ad3728ee082e73b75f7f78015bd44a1ce7ac3e Mon Sep 17 00:00:00 2001 From: Zachary Sunberg Date: Thu, 28 Jul 2022 23:03:27 -0700 Subject: [PATCH 02/14] fixed typo --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 3545b87..8fda583 100644 --- a/README.md +++ b/README.md @@ -62,7 +62,7 @@ Continuous spaces have two additional interface functions: |One dimensional continuous space| `ContinuousSpaceStyle{()}()` | `-1.2..3.3`, `Interval(1.0, 2.0)` | |Multi-dimensional discrete space| `FiniteSpaceStyle{(3,4)}()` | `ArraySpace((:cat, :dog), 3, 4)`, `ArraySpace(0:1, 3, 4)`, `ArraySpace(1:2, 3, 4)`, `ArraySpace(Bool, 3, 4)`| |Multi-dimensional variable discrete space| `FiniteSpaceStyle{(2,)}()` | `product((:cat, :dog), (:litchi, :longan, :mango))`, `product(-1:1, (false, true))`| -|Multi-dimensional continuous space| `ContinuousSpaceStyle{(3,4)}()` | `Box([-1.0, -2.0], [2.0, 4.0])`, `product(-1.2..3.3, -4.6..5.0)`, `ArraySpace(-1.2..3.3, 3, 4)`, `ArraySpace(Float32, 3, 4)` | +|Multi-dimensional continuous space| `ContinuousSpaceStyle{(2,)}()` or `ContinuousSpaceStyle{(3,4)}()` | `Box([-1.0, -2.0], [2.0, 4.0])`, `product(-1.2..3.3, -4.6..5.0)`, `ArraySpace(-1.2..3.3, 3, 4)`, `ArraySpace(Float32, 3, 4)` | |Multi-dimensional hybrid space| `HybridSpaceStyle{(2,),()}()` | `product(-1.2..3.3, -4.6..5.0, [:cat, :dog])`, `product(Box([-1.0, -2.0], [2.0, 4.0]), [1,2,3])`| ### API From 54eaca2b4eb8500b0f7e9ae5b2982b9d35c98091 Mon Sep 17 00:00:00 2001 From: Zachary Sunberg Date: Fri, 29 Jul 2022 10:29:44 -0700 Subject: [PATCH 03/14] added basic functionality --- README.md | 4 ++-- src/CommonRLSpaces.jl | 12 +++++++--- src/basic.jl | 54 ++++++++++++++++++------------------------- test/basic.jl | 10 ++++++++ test/runtests.jl | 52 ++--------------------------------------- 5 files changed, 46 insertions(+), 86 deletions(-) create mode 100644 test/basic.jl diff --git a/README.md b/README.md index 8fda583..a730588 100644 --- a/README.md +++ b/README.md @@ -34,10 +34,10 @@ Spaces with a finite number of elements have `FiniteSpaceStyle`. These spaces ar ### Continuous spaces -Continuous spaces represent sets that have an uncountable number of elements they have a `SpaceStyle` of type `ContinuousSpaceStyle`. CommonRLSpaces does not adopt a rigorous mathematical definition of a continuous set, but, roughly, elements in the interior of a continuous space have other elements arbitrarily close to them. +Continuous spaces represent sets that have an uncountable number of elements they have a `SpaceStyle` of type `ContinuousSpaceStyle`. CommonRLSpaces does not adopt a rigorous mathematical definition of a continuous set, but, roughly, elements in the interior of a continuous space have other elements very close to them. Continuous spaces have two additional interface functions: - `bounds(space)` returns upper and lower bounds in a tuple. For example, if `space` is a unit circle, `bounds(space)` will return `([-1.0, -1.0], [1.0, 1.0])`. This allows agents to choose policies that appropriately cover the space e.g. a normal distribution with a mean of `mean(bounds(space))` and a standard deviation of half the distance between the bounds. -- [I have a few other ideas, but we can discuss later] +- `clamp(x, space)` returns an element of `space` that is near `x`. i.e. if `space` is a unit circle, `clamp([2.0, 0.0], space)` might return `[1.0, 0.0]`. This allows for a convenient way for an agent to find a valid action if they sample actions from a distribution that doesn't match the space exactly (e.g. a normal distribution). ### Hybrid spaces diff --git a/src/CommonRLSpaces.jl b/src/CommonRLSpaces.jl index e48970a..3b76218 100644 --- a/src/CommonRLSpaces.jl +++ b/src/CommonRLSpaces.jl @@ -2,11 +2,17 @@ module CommonRLSpaces using Reexport -@reexport using FillArrays @reexport using IntervalSets -@reexport using StaticArrays -@reexport import Base: OneTo +using StaticArrays +using FillArrays + +export + SpaceStyle, + AbstractSpaceStyle, + FiniteSpaceStyle, + ContinuousSpaceStyle, + UnknownSpaceStyle include("basic.jl") end diff --git a/src/basic.jl b/src/basic.jl index af6d747..5d0fcfc 100644 --- a/src/basic.jl +++ b/src/basic.jl @@ -1,43 +1,34 @@ -export Space, SpaceStyle, DiscreteSpaceStyle, ContinuousSpaceStyle, TupleSpace, NamedSpace, DictSpace - using Random -struct Space{T} - s::T -end - -Space(s::Type{T}) where {T<:Integer} = Space(typemin(T):typemax(T)) -Space(s::Type{T}) where {T<:AbstractFloat} = Space(typemin(T) .. typemax(T)) +##### -Space(x, dims::Int...) = Space(Fill(x, dims)) -Space(x::Type{T}, dim::Int, extra_dims::Int...) where {T<:Integer} = Space(Fill(typemin(x):typemax(T), dim, extra_dims...)) -Space(x::Type{T}, dim::Int, extra_dims::Int...) where {T<:AbstractFloat} = Space(Fill(typemin(x) .. typemax(T), dim, extra_dims...)) -Space(x::Type{T}, dim::Int, extra_dims::Int...) where {T} = Space(Fill(T, dim, extra_dims...)) +abstract type AbstractSpaceStyle end -Base.size(s::Space) = size(SpaceStyle(s)) -Base.length(s::Space) = length(SpaceStyle(s), s) -Base.getindex(s::Space, i...) = getindex(SpaceStyle(s), s, i...) -Base.:(==)(s1::Space, s2::Space) = s1.s == s2.s +struct FiniteSpaceStyle <: AbstractSpaceStyle end +struct ContinuousSpaceStyle <: AbstractSpaceStyle end +struct UnknownSpaceStyle <: AbstractSpaceStyle end -##### +SpaceStyle(space::Any) = UnknownSpaceStyle() -abstract type AbstractSpaceStyle{S} end +SpaceStyle(::Tuple) = FiniteSpaceStyle() +SpaceStyle(::NamedTuple) = FiniteSpaceStyle() -struct DiscreteSpaceStyle{S} <: AbstractSpaceStyle{S} end -struct ContinuousSpaceStyle{S} <: AbstractSpaceStyle{S} end +function SpaceStyle(x::Union{AbstractArray,AbstractDict,AbstractSet}) + if Base.IteratorSize(x) isa Union{Base.HasLength, Base.HasShape} && length(x) < Inf + return FiniteSpaceStyle() + else + return UnknownSpaceStyle() + end +end -SpaceStyle(::Space{<:Tuple}) = DiscreteSpaceStyle{()}() -SpaceStyle(::Space{<:AbstractVector{<:Number}}) = DiscreteSpaceStyle{()}() -SpaceStyle(::Space{<:AbstractInterval}) = ContinuousSpaceStyle{()}() +SpaceStyle(::AbstractInterval) = ContinuousSpaceStyle() -SpaceStyle(s::Space{<:AbstractArray{<:Tuple}}) = DiscreteSpaceStyle{size(s.s)}() -SpaceStyle(s::Space{<:AbstractArray{<:AbstractRange}}) = DiscreteSpaceStyle{size(s.s)}() -SpaceStyle(s::Space{<:AbstractArray{<:AbstractInterval}}) = ContinuousSpaceStyle{size(s.s)}() +#= Base.size(::AbstractSpaceStyle{S}) where {S} = S -Base.length(::DiscreteSpaceStyle{()}, s) = length(s.s) -Base.getindex(::DiscreteSpaceStyle{()}, s, i...) = getindex(s.s, i...) -Base.length(::DiscreteSpaceStyle, s) = mapreduce(length, *, s.s) +Base.length(::FiniteSpaceStyle{()}, s) = length(s.s) +Base.getindex(::FiniteSpaceStyle{()}, s, i...) = getindex(s.s, i...) +Base.length(::FiniteSpaceStyle, s) = mapreduce(length, *, s.s) ##### @@ -79,7 +70,7 @@ function Random.rand(rng::AbstractRNG, s::Interval{:closed,:closed,T}) where {T} end Base.iterate(s::Space, args...) = iterate(SpaceStyle(s), s, args...) -Base.iterate(::DiscreteSpaceStyle{()}, s::Space, args...) = iterate(s.s, args...) +Base.iterate(::FiniteSpaceStyle{()}, s::Space, args...) = iterate(s.s, args...) ##### @@ -94,4 +85,5 @@ Random.rand(rng::AbstractRNG, s::DictSpace) = Dict(k => rand(rng, s[k]) for k in Base.in(xs::Tuple, ts::TupleSpace) = length(xs) == length(ts) && all(((x, s),) -> x in s, zip(xs, ts)) Base.in(xs::AbstractVector, ts::VectorSpace) = length(xs) == length(ts) && all(((x, s),) -> x in s, zip(xs, ts)) Base.in(xs::NamedTuple{names}, ns::NamedTuple{names,<:TupleSpace}) where {names} = all(((x, s),) -> x in s, zip(xs, ns)) -Base.in(xs::Dict, ds::DictSpace) = length(xs) == length(ds) && all(k -> haskey(ds, k) && xs[k] in ds[k], keys(xs)) \ No newline at end of file +Base.in(xs::Dict, ds::DictSpace) = length(xs) == length(ds) && all(k -> haskey(ds, k) && xs[k] in ds[k], keys(xs)) +=# diff --git a/test/basic.jl b/test/basic.jl new file mode 100644 index 0000000..370dac9 --- /dev/null +++ b/test/basic.jl @@ -0,0 +1,10 @@ +@testset "Styles of types outside of this package" begin + struct TestSpace1 end + @test SpaceStyle(TestSpace1()) == UnknownSpaceStyle() + @test SpaceStyle((1,2)) == FiniteSpaceStyle() + @test SpaceStyle((a=1, b=2)) == FiniteSpaceStyle() + @test SpaceStyle([1,2]) == FiniteSpaceStyle() + @test SpaceStyle(Dict(:a=>1)) == FiniteSpaceStyle() + @test SpaceStyle(Set([1,2])) == FiniteSpaceStyle() + @test SpaceStyle(1..2) == ContinuousSpaceStyle() +end diff --git a/test/runtests.jl b/test/runtests.jl index bc921e5..2780cb5 100644 --- a/test/runtests.jl +++ b/test/runtests.jl @@ -2,54 +2,6 @@ using CommonRLSpaces using Test @testset "CommonRLSpaces.jl" begin - @testset "Spaces" begin - s1 = Space((:cat, :dog)) - @test :cat in s1 - @test !(nothing in s1) - - s2 = Space(0:1) - @test 0 in s2 - @test !(0.5 in s2) - - s3 = Space(Bool) - @test false in s3 - @test true in s3 - - s4 = Space(Float64) - @test rand() in s4 - @test 0 in s4 - - s5 = Space(Float64, 3, 4) - @test rand(3, 4) in s5 - - s6 = Space(SVector((:cat, :dog), (:litchi, :longan, :mango))) - @test SVector(:dog, :litchi) in s6 - - s7 = Space([-1 .. 1, 2 .. 3]) - @test [0, 2] in s7 - @test !([-5, 5] in s7) - - for _ in 1:100 - for s in [s1, s2, s3, s4, s5, s6, s7] - @test rand(s) in s - end - end - end - - @testset "Meta Spaces" begin - s1 = (Space(1:2), Space(Float64, 2, 3)) - @test (1, rand(2, 3)) in s1 - - s2 = (a=Space(1:2), b=Space(Float64, 2, 3)) - @test (a=1, b=rand(2, 3)) in s2 - - s3 = Dict(:a => Space(1:2), :b => Space(Float64, 2, 3)) - @test Dict(:a => 1, :b => rand(2, 3)) in s3 - - for _ in 1:100 - for s in [s1, s2, s3] - rand(s) in s - end - end - end + include("basic.jl") + # include("array.jl") end From 5ff9fb0f6bc5db622eb7d3e136645bd115b29b55 Mon Sep 17 00:00:00 2001 From: Zachary Sunberg Date: Fri, 29 Jul 2022 11:58:13 -0700 Subject: [PATCH 04/14] added Box --- src/CommonRLSpaces.jl | 10 +++++++++- src/array.jl | 43 +++++++++++++++++++++++++++++++++++++++++++ src/basic.jl | 3 +++ test/array.jl | 42 ++++++++++++++++++++++++++++++++++++++++++ test/runtests.jl | 4 +++- 5 files changed, 100 insertions(+), 2 deletions(-) create mode 100644 src/array.jl create mode 100644 test/array.jl diff --git a/src/CommonRLSpaces.jl b/src/CommonRLSpaces.jl index 3b76218..b7d5666 100644 --- a/src/CommonRLSpaces.jl +++ b/src/CommonRLSpaces.jl @@ -12,7 +12,15 @@ export AbstractSpaceStyle, FiniteSpaceStyle, ContinuousSpaceStyle, - UnknownSpaceStyle + UnknownSpaceStyle, + bounds, + elsize + include("basic.jl") +export + Box + +include("array.jl") + end diff --git a/src/array.jl b/src/array.jl new file mode 100644 index 0000000..0211dcc --- /dev/null +++ b/src/array.jl @@ -0,0 +1,43 @@ +abstract type AbstractArraySpace end + +struct Box{A<:AbstractArray} <: AbstractArraySpace + lower::A + upper::A + + Box{A}(lower, upper) where {A<:AbstractArray} = new(lower, upper) +end + +function Box(lower, upper; convert_to_static=true) + @assert size(lower) == size(upper) + sz = size(lower) + continuous_lower = convert(AbstractArray{similar_continuous_type(eltype(lower))}, lower) + continuous_upper = convert(AbstractArray{similar_continuous_type(eltype(upper))}, upper) + if convert_to_static + final_lower = SArray{Tuple{sz...}}(continuous_lower) + final_upper = SArray{Tuple{sz...}}(continuous_upper) + else + final_lower, final_upper = promote(continuous_lower, continuous_upper) + end + return Box{typeof(final_lower)}(final_lower, final_upper) +end + +SpaceStyle(::Box) = ContinuousSpaceStyle() + +function Base.rand(rng::AbstractRNG, sp::Random.SamplerTrivial{<:Box}) + box = sp[] + return box.lower + rand_similar(rng, box.lower) .* (box.upper-box.lower) +end + +rand_similar(rng::AbstractRNG, a::StaticArray) = rand(rng, typeof(a)) +rand_similar(rng::AbstractRNG, a::AbstractArray) = rand(rng, eltype(a), size(a)...) + +similar_continuous_type(T::Type{<:AbstractFloat}) = T +similar_continuous_type(T::Type{<:Number}) = Float64 + +Base.in(x::AbstractArray, b::Box) = all(b.lower .<= x .<= b.upper) + +Base.eltype(::Box{A}) where A = A +elsize(b::Box) = size(b.lower) + +bounds(b::Box) = (b.lower, b.upper) +Base.clamp(x::AbstractArray, b::Box) = clamp(x, b.lower, b.upper) diff --git a/src/basic.jl b/src/basic.jl index 5d0fcfc..30a3ec3 100644 --- a/src/basic.jl +++ b/src/basic.jl @@ -23,6 +23,9 @@ end SpaceStyle(::AbstractInterval) = ContinuousSpaceStyle() +function elsize end # note: different than Base.elsize + +function bounds end #= Base.size(::AbstractSpaceStyle{S}) where {S} = S diff --git a/test/array.jl b/test/array.jl new file mode 100644 index 0000000..b3ad88d --- /dev/null +++ b/test/array.jl @@ -0,0 +1,42 @@ +@testset "Box" begin + @testset "Box with static" begin + lower = SA[-1.0, -2.0] + upper = SA[1.0, 2.0] + b = Box(lower, upper) + @test @inferred SpaceStyle(b) == ContinuousSpaceStyle() + @test eltype(b) <: AbstractArray{Float64} + @test eltype(b) <: StaticArray + @test @inferred typeof(rand(b)) == eltype(b) + @test @inferred [0.0, 0.0] in b + @test @inferred rand(b) in b + @test @inferred bounds(b) == (lower, upper) + @test @inferred clamp(SA[3.0, 4.0], b) in b + @test @inferred elsize(b) == (2,) + end + + @testset "Box with builtin" begin + lower = [-1.0, -2.0] + upper = [1.0, 2.0] + b = Box(lower, upper; convert_to_static=false) + @test @inferred SpaceStyle(b) == ContinuousSpaceStyle() + @test @inferred eltype(b) == Vector{Float64} + @test @inferred [0.0, 0.0] in b + @test @inferred rand(b) in b + @test @inferred bounds(b) == (lower, upper) + @test @inferred clamp([3.0, 4.0], b) in b + @test @inferred elsize(b) == (2,) + end + + @testset "Box with integer" begin + lower = [-1, -2] + upper = [1, 2] + b = Box(lower, upper) + @test @inferred SpaceStyle(b) == ContinuousSpaceStyle() + @test eltype(b) <: AbstractVector{Float64} + @test @inferred [0.0, 0.0] in b + @test @inferred rand(b) in b + @test @inferred bounds(b) == (lower, upper) + @test @inferred clamp([3.0, 4.0], b) in b + @test @inferred elsize(b) == (2,) + end +end diff --git a/test/runtests.jl b/test/runtests.jl index 2780cb5..ba33bd7 100644 --- a/test/runtests.jl +++ b/test/runtests.jl @@ -1,7 +1,9 @@ using CommonRLSpaces using Test +using StaticArrays + @testset "CommonRLSpaces.jl" begin include("basic.jl") - # include("array.jl") + include("array.jl") end From 0c63bcd827e415277e134c3b445a5a213e32c8a5 Mon Sep 17 00:00:00 2001 From: Zachary Sunberg Date: Fri, 29 Jul 2022 17:36:02 -0700 Subject: [PATCH 05/14] added ArraySpace constructor --- src/array.jl | 37 +++++++++++++++++++++++------ src/basic.jl | 65 ++------------------------------------------------- test/array.jl | 19 ++++++++++++--- test/basic.jl | 5 ++++ 4 files changed, 53 insertions(+), 73 deletions(-) diff --git a/src/array.jl b/src/array.jl index 0211dcc..f92a9e1 100644 --- a/src/array.jl +++ b/src/array.jl @@ -1,4 +1,5 @@ abstract type AbstractArraySpace end +# Maybe AbstractArraySpace should have an eltype parameter so that you could call convert(AbstractArraySpace{Float32}, space) struct Box{A<:AbstractArray} <: AbstractArraySpace lower::A @@ -7,11 +8,11 @@ struct Box{A<:AbstractArray} <: AbstractArraySpace Box{A}(lower, upper) where {A<:AbstractArray} = new(lower, upper) end -function Box(lower, upper; convert_to_static=true) +function Box(lower, upper; convert_to_static::Bool=false) @assert size(lower) == size(upper) sz = size(lower) - continuous_lower = convert(AbstractArray{similar_continuous_type(eltype(lower))}, lower) - continuous_upper = convert(AbstractArray{similar_continuous_type(eltype(upper))}, upper) + continuous_lower = convert(AbstractArray{float(eltype(lower))}, lower) + continuous_upper = convert(AbstractArray{float(eltype(upper))}, upper) if convert_to_static final_lower = SArray{Tuple{sz...}}(continuous_lower) final_upper = SArray{Tuple{sz...}}(continuous_upper) @@ -21,6 +22,9 @@ function Box(lower, upper; convert_to_static=true) return Box{typeof(final_lower)}(final_lower, final_upper) end +# By default, convert builtin arrays to static +Box(lower::Array, upper::Array) = Box(lower, upper; convert_to_static=true) + SpaceStyle(::Box) = ContinuousSpaceStyle() function Base.rand(rng::AbstractRNG, sp::Random.SamplerTrivial{<:Box}) @@ -31,13 +35,32 @@ end rand_similar(rng::AbstractRNG, a::StaticArray) = rand(rng, typeof(a)) rand_similar(rng::AbstractRNG, a::AbstractArray) = rand(rng, eltype(a), size(a)...) -similar_continuous_type(T::Type{<:AbstractFloat}) = T -similar_continuous_type(T::Type{<:Number}) = Float64 - Base.in(x::AbstractArray, b::Box) = all(b.lower .<= x .<= b.upper) Base.eltype(::Box{A}) where A = A elsize(b::Box) = size(b.lower) bounds(b::Box) = (b.lower, b.upper) -Base.clamp(x::AbstractArray, b::Box) = clamp(x, b.lower, b.upper) +Base.clamp(x::AbstractArray, b::Box) = clamp.(x, b.lower, b.upper) + + +struct RepeatedSpace{B, S<:Tuple} <: AbstractArraySpace + base_space::B + elsize::S +end + +ArraySpace(base_space, size...) = RepeatedSpace(base_space, size) + +SpaceStyle(s::RepeatedSpace) = SpaceStyle(s.base_space) + +Base.rand(rng::AbstractRNG, sp::Random.SamplerTrivial{<:RepeatedSpace}) = rand(rng, sp[].base_space, sp[].elsize...) + +Base.in(x::AbstractArray, s::RepeatedSpace) = all(entry in s.base for entry in s) +Base.eltype(s::RepeatedSpace) = AbstractArray{eltype(s.base_space), length(s.elsize)} +Base.eltype(s::RepeatedSpace{<:AbstractInterval}) = AbstractArray{Random.gentype(s.base_space), length(s.elsize)} +Base.elsize(s::RepeatedSpace) = s.elsize + +function bounds(s::RepeatedSpace) + bs = bounds(s.base_space) + return (Fill(first(bs), s.elsize...), Fill(last(bs), s.elsize...)) +end diff --git a/src/basic.jl b/src/basic.jl index 30a3ec3..006b23d 100644 --- a/src/basic.jl +++ b/src/basic.jl @@ -27,66 +27,5 @@ function elsize end # note: different than Base.elsize function bounds end -#= -Base.size(::AbstractSpaceStyle{S}) where {S} = S -Base.length(::FiniteSpaceStyle{()}, s) = length(s.s) -Base.getindex(::FiniteSpaceStyle{()}, s, i...) = getindex(s.s, i...) -Base.length(::FiniteSpaceStyle, s) = mapreduce(length, *, s.s) - -##### - -Random.rand(rng::Random.AbstractRNG, s::Space) = rand(rng, s.s) - -Random.rand( - rng::Random.AbstractRNG, - s::Union{ - <:Space{<:AbstractArray{<:Tuple}}, - <:Space{<:AbstractArray{<:AbstractRange}}, - <:Space{<:AbstractArray{<:AbstractInterval}} - } -) = map(x -> rand(rng, x), s.s) - -Base.in(x, s::Space) = x in s.s -Base.in(x, s::Space{<:Type}) = x isa s.s - -Base.in( - x, - s::Union{ - <:Space{<:AbstractArray{<:Tuple}}, - <:Space{<:AbstractArray{<:AbstractRange}}, - <:Space{<:AbstractArray{<:AbstractInterval}} - } -) = size(x) == size(s) && all(x -> x[1] in x[2], zip(x, s.s)) - -function Random.rand(rng::AbstractRNG, s::Interval{:closed,:closed,T}) where {T} - if s == typemin(T) .. typemax(T) - rand(T) - else - r = rand(rng) - - if r == 0.0 - r = rand(Bool) - end - - r * (s.right - s.left) + s.left - end -end - -Base.iterate(s::Space, args...) = iterate(SpaceStyle(s), s, args...) -Base.iterate(::FiniteSpaceStyle{()}, s::Space, args...) = iterate(s.s, args...) - -##### - -const TupleSpace = Tuple{Vararg{Space}} -const NamedSpace = NamedTuple{<:Any,<:TupleSpace} -const VectorSpace = Vector{<:Space} -const DictSpace = Dict{<:Any,<:Space} - -Random.rand(rng::AbstractRNG, s::Union{TupleSpace,NamedSpace,VectorSpace}) = map(x -> rand(rng, x), s) -Random.rand(rng::AbstractRNG, s::DictSpace) = Dict(k => rand(rng, s[k]) for k in keys(s)) - -Base.in(xs::Tuple, ts::TupleSpace) = length(xs) == length(ts) && all(((x, s),) -> x in s, zip(xs, ts)) -Base.in(xs::AbstractVector, ts::VectorSpace) = length(xs) == length(ts) && all(((x, s),) -> x in s, zip(xs, ts)) -Base.in(xs::NamedTuple{names}, ns::NamedTuple{names,<:TupleSpace}) where {names} = all(((x, s),) -> x in s, zip(xs, ns)) -Base.in(xs::Dict, ds::DictSpace) = length(xs) == length(ds) && all(k -> haskey(ds, k) && xs[k] in ds[k], keys(xs)) -=# +bounds(i::AbstractInterval) = (infimum(i), supremum(i)) +Base.clamp(x, i::AbstractInterval) = IntevalSets.clamp(x, i) diff --git a/test/array.jl b/test/array.jl index b3ad88d..71f9b80 100644 --- a/test/array.jl +++ b/test/array.jl @@ -1,5 +1,5 @@ @testset "Box" begin - @testset "Box with static" begin + @testset "Box with StaticVector" begin lower = SA[-1.0, -2.0] upper = SA[1.0, 2.0] b = Box(lower, upper) @@ -14,7 +14,7 @@ @test @inferred elsize(b) == (2,) end - @testset "Box with builtin" begin + @testset "Box with Vector{Float64}" begin lower = [-1.0, -2.0] upper = [1.0, 2.0] b = Box(lower, upper; convert_to_static=false) @@ -27,7 +27,7 @@ @test @inferred elsize(b) == (2,) end - @testset "Box with integer" begin + @testset "Box with Vector{Int}" begin lower = [-1, -2] upper = [1, 2] b = Box(lower, upper) @@ -39,4 +39,17 @@ @test @inferred clamp([3.0, 4.0], b) in b @test @inferred elsize(b) == (2,) end + + @testset "Box with Matrix" begin + lower = -[1 2; 3 4] + upper = [1 2; 3 4] + b = Box(lower, upper) + @test @inferred SpaceStyle(b) == ContinuousSpaceStyle() + @test eltype(b) <: AbstractMatrix{Float64} + @test @inferred zeros(2,2) in b + @test @inferred rand(b) in b + @test @inferred bounds(b) == (lower, upper) + @test @inferred clamp([3 -4; 5 -6], b) in b + @test @inferred elsize(b) == (2,2) + end end diff --git a/test/basic.jl b/test/basic.jl index 370dac9..5a47696 100644 --- a/test/basic.jl +++ b/test/basic.jl @@ -8,3 +8,8 @@ @test SpaceStyle(Set([1,2])) == FiniteSpaceStyle() @test SpaceStyle(1..2) == ContinuousSpaceStyle() end + +@testset "Bounds and clamp for IntervalSets" begin + @test bounds(1..2) == (1,2) + @test clamp(0.1, 1..2) == 1 +end From 9cb43f8b4c847c2dca1da827152b0eaed626e385 Mon Sep 17 00:00:00 2001 From: Zachary Sunberg Date: Sat, 30 Jul 2022 11:19:16 -0700 Subject: [PATCH 06/14] ArraySpace tests mostly pass --- src/CommonRLSpaces.jl | 3 ++- src/array.jl | 6 ++++-- src/basic.jl | 2 +- test/array.jl | 24 ++++++++++++++++++++++++ 4 files changed, 31 insertions(+), 4 deletions(-) diff --git a/src/CommonRLSpaces.jl b/src/CommonRLSpaces.jl index b7d5666..1be85ab 100644 --- a/src/CommonRLSpaces.jl +++ b/src/CommonRLSpaces.jl @@ -19,7 +19,8 @@ export include("basic.jl") export - Box + Box, + ArraySpace include("array.jl") diff --git a/src/array.jl b/src/array.jl index f92a9e1..54aad97 100644 --- a/src/array.jl +++ b/src/array.jl @@ -55,12 +55,14 @@ SpaceStyle(s::RepeatedSpace) = SpaceStyle(s.base_space) Base.rand(rng::AbstractRNG, sp::Random.SamplerTrivial{<:RepeatedSpace}) = rand(rng, sp[].base_space, sp[].elsize...) -Base.in(x::AbstractArray, s::RepeatedSpace) = all(entry in s.base for entry in s) +Base.in(x::AbstractArray, s::RepeatedSpace) = all(entry in s.base_space for entry in x) Base.eltype(s::RepeatedSpace) = AbstractArray{eltype(s.base_space), length(s.elsize)} Base.eltype(s::RepeatedSpace{<:AbstractInterval}) = AbstractArray{Random.gentype(s.base_space), length(s.elsize)} -Base.elsize(s::RepeatedSpace) = s.elsize +elsize(s::RepeatedSpace) = s.elsize function bounds(s::RepeatedSpace) bs = bounds(s.base_space) return (Fill(first(bs), s.elsize...), Fill(last(bs), s.elsize...)) end + +Base.clamp(x::AbstractArray, s::RepeatedSpace) = map(entry -> clamp(entry, s.base_space), x) diff --git a/src/basic.jl b/src/basic.jl index 006b23d..13dbb92 100644 --- a/src/basic.jl +++ b/src/basic.jl @@ -13,7 +13,7 @@ SpaceStyle(space::Any) = UnknownSpaceStyle() SpaceStyle(::Tuple) = FiniteSpaceStyle() SpaceStyle(::NamedTuple) = FiniteSpaceStyle() -function SpaceStyle(x::Union{AbstractArray,AbstractDict,AbstractSet}) +function SpaceStyle(x::Union{AbstractArray,AbstractDict,AbstractSet,AbstractRange}) if Base.IteratorSize(x) isa Union{Base.HasLength, Base.HasShape} && length(x) < Inf return FiniteSpaceStyle() else diff --git a/test/array.jl b/test/array.jl index 71f9b80..bdd1d74 100644 --- a/test/array.jl +++ b/test/array.jl @@ -52,4 +52,28 @@ @test @inferred clamp([3 -4; 5 -6], b) in b @test @inferred elsize(b) == (2,2) end + + @testset "ArraySpace with Range" begin + s = ArraySpace(1:5, 3, 4) + @test @inferred SpaceStyle(s) == FiniteSpaceStyle() + @test eltype(s) <: AbstractMatrix{eltype(1:5)} + @test @inferred ones(Int, 3, 4) in s + @test @inferred rand(s) in s + @test rand(s) isa Matrix{Int} + # @test @inferred bounds(s) == (ones(Int, 3, 4), 5*ones(Int, 3, 4)) # note: not actually required by interface since this is FiniteSpaceStyle + @test_broken collect(s) isa Vector{Matrix{Int}} + @test @inferred elsize(s) == (3,4) + end + + @testset "ArraySpace with IntervalSet" begin + s = ArraySpace(1..5, 3, 4) + @test @inferred SpaceStyle(s) == ContinuousSpaceStyle() + @test eltype(s) <: AbstractMatrix{Float64} + @test @inferred ones(Float64, 3, 4) in s + @test @inferred rand(s) in s + @test rand(s) isa Matrix{Float64} + @test @inferred bounds(s) == (ones(3, 4), 5*ones(3, 4)) + @test @inferred clamp(zeros(3,4), s) == ones(3,4) + @test @inferred elsize(s) == (3,4) + end end From 8a501c8fe43eddf1c1d39f1fc2a00b141715bbbd Mon Sep 17 00:00:00 2001 From: Zachary Sunberg Date: Sat, 30 Jul 2022 15:03:20 -0700 Subject: [PATCH 07/14] added products of boxes and friends --- src/CommonRLSpaces.jl | 5 +++++ src/array.jl | 1 + src/product.jl | 23 +++++++++++++++++++++++ test/array.jl | 9 +++++++++ test/product.jl | 15 +++++++++++++++ test/runtests.jl | 1 + 6 files changed, 54 insertions(+) create mode 100644 src/product.jl create mode 100644 test/product.jl diff --git a/src/CommonRLSpaces.jl b/src/CommonRLSpaces.jl index 1be85ab..ae67b39 100644 --- a/src/CommonRLSpaces.jl +++ b/src/CommonRLSpaces.jl @@ -24,4 +24,9 @@ export include("array.jl") +export + product + +include("product.jl") + end diff --git a/src/array.jl b/src/array.jl index 54aad97..dc4e6b6 100644 --- a/src/array.jl +++ b/src/array.jl @@ -43,6 +43,7 @@ elsize(b::Box) = size(b.lower) bounds(b::Box) = (b.lower, b.upper) Base.clamp(x::AbstractArray, b::Box) = clamp.(x, b.lower, b.upper) +Base.convert(t::Type{<:Box}, i::ClosedInterval) = t(SA[minimum(i)], SA[maximum(i)]) struct RepeatedSpace{B, S<:Tuple} <: AbstractArraySpace base_space::B diff --git a/src/product.jl b/src/product.jl new file mode 100644 index 0000000..ad1fcc6 --- /dev/null +++ b/src/product.jl @@ -0,0 +1,23 @@ +product(i1::ClosedInterval, i2::ClosedInterval) = Box(SA[minimum(i1), minimum(i2)], SA[maximum(i1), maximum(i2)]) + +product(b::Box, i::ClosedInterval) = product(b, convert(Box, i)) +product(i::ClosedInterval, b::Box) = product(convert(Box, i), b) +product(b1::Box{<:AbstractVector}, b2::Box{<:AbstractVector}) = Box(vcat(b1.lower, b2.lower), vcat(b1.upper, b2.upper)) +function product(b1::Box, b2::Box) + if size(b1.lower, 2) == size(b2.lower, 2) # same number of columns + return Box(vcat(b1.lower, b2.lower), vcat(b1.upper, b2.upper)) + else + return GenericrSpaceProduct((b1, b2)) + end +end + +# handle case of 3 or more +product(s1, s2, s3, args...) = product(product(s1, s2), s3, args...) + +struct GenericrSpaceProduct{T<:Tuple} + members::T +end + +# handle any case not covered above +product(s1, s2) = GenericrSpaceProduct((s1, s2)) +product(s1::GenericrSpaceProduct, s2) = GenericrSpaceProduct((s1.members..., s2)) diff --git a/test/array.jl b/test/array.jl index bdd1d74..30e3fec 100644 --- a/test/array.jl +++ b/test/array.jl @@ -53,6 +53,15 @@ @test @inferred elsize(b) == (2,2) end + @testset "Box comparison" begin + @test Box([1,2], [3,4]) == Box([1,2], [3,4]) + @test Box([1,2], [3,4]) != Box([1,3], [3,4]) + end + + @testset "Interval to box conversion" begin + @test convert(Box, 1..2) == Box([1], [2]) + end + @testset "ArraySpace with Range" begin s = ArraySpace(1:5, 3, 4) @test @inferred SpaceStyle(s) == FiniteSpaceStyle() diff --git a/test/product.jl b/test/product.jl new file mode 100644 index 0000000..e8ecf6b --- /dev/null +++ b/test/product.jl @@ -0,0 +1,15 @@ +@testset "Product of boxes" begin + @test product(Box([1], [2]), Box([3], [4])) == Box([1,3], [2,4]) + @test product(Box(ones(2,1), 2*ones(2,1)), Box([3], [4])) == Box(@SMatrix([1;1;3]), @SMatrix([2;2;4])) +end + +@testset "Product of intervals" begin + @test @inferred product(1..2, 3..4) == Box([1,3], [2,4]) + @test @inferred product(1..2, 3..4, 5..6) == Box([1,3,5], [2,4,6]) + @test @inferred product(1..2, 3..4, 5..6, 7..8) == Box([1,3,5,7], [2,4,6,8]) +end + +@testset "Product of Box and interval" begin + @test @inferred product(Box([1,3], [2,4]), 5..6) == Box([1,3,5], [2,4,6]) + @test @inferred product(5..6, Box([1,3], [2,4])) == Box([5,1,3], [6,2,4]) +end diff --git a/test/runtests.jl b/test/runtests.jl index ba33bd7..7cc5f6d 100644 --- a/test/runtests.jl +++ b/test/runtests.jl @@ -6,4 +6,5 @@ using StaticArrays @testset "CommonRLSpaces.jl" begin include("basic.jl") include("array.jl") + include("product.jl") end From 61924cfa7f805f50dc960106e95f5f497ebd1deb Mon Sep 17 00:00:00 2001 From: Zachary Sunberg Date: Mon, 1 Aug 2022 23:20:34 -0600 Subject: [PATCH 08/14] added some words in the README about products --- README.md | 10 +++++++++- src/array.jl | 9 ++++++++- src/basic.jl | 4 ++++ src/product.jl | 7 +++---- 4 files changed, 24 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index a730588..a9556af 100644 --- a/README.md +++ b/README.md @@ -41,12 +41,20 @@ Continuous spaces have two additional interface functions: ### Hybrid spaces -[need to figure this out] +The interface for hybrid continuous-discrete spaces is currently planned, but not yet defined. If the space style is not `FiniteSpaceStyle` or `ContinuousSpaceStyle`, it is `UnknownSpaceStyle`. ### Spaces of arrays [need to figure this out, but I think `elsize(space)` should return the size of the arrays in the space] +### Cartesian products of spaces + +The Cartesian product of two spaces `a` and `b` can be constructed with `c = product(a, b)`. + +The exact form of the resulting space is an implementation detail. The only guarantees are (1) that there will be one unique element of `c` for every combination of one object from `a` and one object from `b` and (2) that the resulting collection conforms to the interface above. + +The `TupleProductSpace` constructor provides a specialized Cartesian product where each element is a tuple, i.e. `TupleProductSpace(a, b)` have type `Tuple{eltype(a), eltype(b)}`. + --- 1Note: the elements of a space represented by a `Dict` are key-value `Pair`s. diff --git a/src/array.jl b/src/array.jl index dc4e6b6..3c7d028 100644 --- a/src/array.jl +++ b/src/array.jl @@ -1,7 +1,14 @@ abstract type AbstractArraySpace end # Maybe AbstractArraySpace should have an eltype parameter so that you could call convert(AbstractArraySpace{Float32}, space) -struct Box{A<:AbstractArray} <: AbstractArraySpace +""" + Box(lower, upper) + +A Box represents a space of real-valued arrays bounded element-wise above by `upper` and below by `lower`, e.g. `Box([-1, -2], [3, 4]` represents the two-dimensional vector space that is the Cartesian product of the two closed sets: ``[-1, 3] \times [-2, 4]``. + +The elements of a Box are always `AbstractArray`s with `AbstractFloat` elements. `Box`es always have `ContinuousSpaceStyle`, and products of `Box`es with `Box`es or `ClosedInterval`s are `Box`es when the dimensions are compatible. +""" +struct Box{A<:AbstractArray{<:AbstractFloat}} <: AbstractArraySpace lower::A upper::A diff --git a/src/basic.jl b/src/basic.jl index 13dbb92..01af6c5 100644 --- a/src/basic.jl +++ b/src/basic.jl @@ -23,6 +23,10 @@ end SpaceStyle(::AbstractInterval) = ContinuousSpaceStyle() +promote_spacestyle(::FiniteSpaceStyle, ::FiniteSpaceStyle) = FiniteSpaceStyle() +promote_spacestyle(::ContinuousSpaceStyle, ::ContinuousSpaceStyle) = ContinuousSpaceStyle() +promote_spacestyle(_, _) = UnknownSpaceStyle() + function elsize end # note: different than Base.elsize function bounds end diff --git a/src/product.jl b/src/product.jl index ad1fcc6..abdada9 100644 --- a/src/product.jl +++ b/src/product.jl @@ -12,12 +12,11 @@ function product(b1::Box, b2::Box) end # handle case of 3 or more -product(s1, s2, s3, args...) = product(product(s1, s2), s3, args...) +product(s1, s2, s3, args...) = foldl(product, (s1, s2, s3, args...)) -struct GenericrSpaceProduct{T<:Tuple} +struct TupleProductSpace{T<:Tuple} members::T end # handle any case not covered above -product(s1, s2) = GenericrSpaceProduct((s1, s2)) -product(s1::GenericrSpaceProduct, s2) = GenericrSpaceProduct((s1.members..., s2)) +product(s1, s2) = TupleProductSpace((s1, s2)) From 02605b58a461c2cab86fcde375d841b7c0e22a55 Mon Sep 17 00:00:00 2001 From: Zachary Sunberg Date: Mon, 1 Aug 2022 23:35:21 -0600 Subject: [PATCH 09/14] slight rewording --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index a9556af..b5f08a6 100644 --- a/README.md +++ b/README.md @@ -51,9 +51,9 @@ The interface for hybrid continuous-discrete spaces is currently planned, but no The Cartesian product of two spaces `a` and `b` can be constructed with `c = product(a, b)`. -The exact form of the resulting space is an implementation detail. The only guarantees are (1) that there will be one unique element of `c` for every combination of one object from `a` and one object from `b` and (2) that the resulting collection conforms to the interface above. +The exact form of the resulting space is unspecified and should be considered an implementation detail. The only guarantees are (1) that there will be one unique element of `c` for every combination of one object from `a` and one object from `b` and (2) that the resulting collection conforms to the interface above. -The `TupleProductSpace` constructor provides a specialized Cartesian product where each element is a tuple, i.e. `TupleProductSpace(a, b)` have type `Tuple{eltype(a), eltype(b)}`. +The `TupleProductSpace` constructor provides a specialized Cartesian product where each element is a tuple, i.e. `TupleProductSpace(a, b)` has elements of type `Tuple{eltype(a), eltype(b)}`. --- From e83a7322374d91af312b3b7db1a16d21f55ea515 Mon Sep 17 00:00:00 2001 From: Zachary Sunberg Date: Tue, 2 Aug 2022 23:48:19 -0600 Subject: [PATCH 10/14] added tuple fallback --- README.md | 4 ++-- src/basic.jl | 3 +++ src/product.jl | 30 ++++++++++++++++++++++++------ 3 files changed, 29 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index b5f08a6..0109b5f 100644 --- a/README.md +++ b/README.md @@ -51,9 +51,9 @@ The interface for hybrid continuous-discrete spaces is currently planned, but no The Cartesian product of two spaces `a` and `b` can be constructed with `c = product(a, b)`. -The exact form of the resulting space is unspecified and should be considered an implementation detail. The only guarantees are (1) that there will be one unique element of `c` for every combination of one object from `a` and one object from `b` and (2) that the resulting collection conforms to the interface above. +The exact form of the resulting space is unspecified and should be considered an implementation detail. The only guarantees are (1) that there will be one unique element of `c` for every combination of one object from `a` and one object from `b` and (2) that the resulting space conforms to the interface above. -The `TupleProductSpace` constructor provides a specialized Cartesian product where each element is a tuple, i.e. `TupleProductSpace(a, b)` has elements of type `Tuple{eltype(a), eltype(b)}`. +The `TupleSpaceProduct` constructor provides a specialized Cartesian product where each element is a tuple, i.e. `TupleSpaceProduct(a, b)` has elements of type `Tuple{eltype(a), eltype(b)}`. --- diff --git a/src/basic.jl b/src/basic.jl index 01af6c5..1e52627 100644 --- a/src/basic.jl +++ b/src/basic.jl @@ -27,6 +27,9 @@ promote_spacestyle(::FiniteSpaceStyle, ::FiniteSpaceStyle) = FiniteSpaceStyle() promote_spacestyle(::ContinuousSpaceStyle, ::ContinuousSpaceStyle) = ContinuousSpaceStyle() promote_spacestyle(_, _) = UnknownSpaceStyle() +# handle case of 3 or more +promote_spacestyle(s1, s2, s3, others...) = foldl(promote_spacestyle, (s1, s2, s3, args...)) + function elsize end # note: different than Base.elsize function bounds end diff --git a/src/product.jl b/src/product.jl index abdada9..85cd3b9 100644 --- a/src/product.jl +++ b/src/product.jl @@ -7,16 +7,34 @@ function product(b1::Box, b2::Box) if size(b1.lower, 2) == size(b2.lower, 2) # same number of columns return Box(vcat(b1.lower, b2.lower), vcat(b1.upper, b2.upper)) else - return GenericrSpaceProduct((b1, b2)) + return TupleSpaceProduct((b1, b2)) end end # handle case of 3 or more -product(s1, s2, s3, args...) = foldl(product, (s1, s2, s3, args...)) +product(s1, s2, s3, args...) = foldl(product, (s1, s2, s3, args...)) # not totally sure if this should be foldl or foldr -struct TupleProductSpace{T<:Tuple} - members::T +struct TupleSpaceProduct{T<:Tuple} + ss::T end -# handle any case not covered above -product(s1, s2) = TupleProductSpace((s1, s2)) +TupleSpaceProduct(s1, s2, others...) = TupleSpaceProduct((s1, s2, others...)) + +subspaces(s::TupleSpaceProduct) = s.ss + +product(s1::TupleSpaceProduct, s2::TupleSpaceProduct) = TupleSpaceProduct(subspaces(s1)..., subspaces(s2)...) + +# handle any case not covered elsewhere by making a TupleSpaceProduct +# if one of the members is already a TupleSpaceProduct, we add put them together in a new "flat" TupleSpaceProduct +# note: if we had defined product(s1::TupleSpaceProduct, s2) it might be annoying because product(s1, s2::AnotherSpace) would be ambiguous with it +function product(s1, s2) + if s1 isa TupleSpaceProduct + return TupleSpaceProduct(subspaces(s1)..., s2) + elseif s2 isa TupleSpaceProduct + return TupleSpaceProduct(s1, subspaces(s2)...) + else + return TupleSpaceProduct(s1, s2) + end +end + +SpaceStyle(s::TupleSpaceProduct) = promote_spacestyle(subspaces(s)...) From 361d9994c50670a7db9f43fc068a3e59943a74e6 Mon Sep 17 00:00:00 2001 From: Zachary Sunberg Date: Wed, 3 Aug 2022 22:34:24 -0600 Subject: [PATCH 11/14] finished implementing TupleProduct --- src/CommonRLSpaces.jl | 3 ++- src/product.jl | 42 +++++++++++++++++++++++++++++------------- test/product.jl | 26 ++++++++++++++++++++++++++ 3 files changed, 57 insertions(+), 14 deletions(-) diff --git a/src/CommonRLSpaces.jl b/src/CommonRLSpaces.jl index ae67b39..e66fc2a 100644 --- a/src/CommonRLSpaces.jl +++ b/src/CommonRLSpaces.jl @@ -25,7 +25,8 @@ export include("array.jl") export - product + product, + TupleProduct include("product.jl") diff --git a/src/product.jl b/src/product.jl index 85cd3b9..74ae9c9 100644 --- a/src/product.jl +++ b/src/product.jl @@ -14,27 +14,43 @@ end # handle case of 3 or more product(s1, s2, s3, args...) = foldl(product, (s1, s2, s3, args...)) # not totally sure if this should be foldl or foldr -struct TupleSpaceProduct{T<:Tuple} +struct TupleProduct{T<:Tuple} ss::T end -TupleSpaceProduct(s1, s2, others...) = TupleSpaceProduct((s1, s2, others...)) +TupleProduct(s1, s2, others...) = TupleProduct((s1, s2, others...)) -subspaces(s::TupleSpaceProduct) = s.ss +subspaces(s::TupleProduct) = s.ss -product(s1::TupleSpaceProduct, s2::TupleSpaceProduct) = TupleSpaceProduct(subspaces(s1)..., subspaces(s2)...) +product(s1::TupleProduct, s2::TupleProduct) = TupleProduct(subspaces(s1)..., subspaces(s2)...) -# handle any case not covered elsewhere by making a TupleSpaceProduct -# if one of the members is already a TupleSpaceProduct, we add put them together in a new "flat" TupleSpaceProduct -# note: if we had defined product(s1::TupleSpaceProduct, s2) it might be annoying because product(s1, s2::AnotherSpace) would be ambiguous with it +# handle any case not covered elsewhere by making a TupleProduct +# if one of the members is already a TupleProduct, we add put them together in a new "flat" TupleProduct +# note: if we had defined product(s1::TupleProduct, s2) it might be annoying because product(s1, s2::AnotherProduct) would be ambiguous with it function product(s1, s2) - if s1 isa TupleSpaceProduct - return TupleSpaceProduct(subspaces(s1)..., s2) - elseif s2 isa TupleSpaceProduct - return TupleSpaceProduct(s1, subspaces(s2)...) + if s1 isa TupleProduct + return TupleProduct(subspaces(s1)..., s2) + elseif s2 isa TupleProduct + return TupleProduct(s1, subspaces(s2)...) else - return TupleSpaceProduct(s1, s2) + return TupleProduct(s1, s2) end end -SpaceStyle(s::TupleSpaceProduct) = promote_spacestyle(subspaces(s)...) +SpaceStyle(s::TupleProduct) = promote_spacestyle(map(SpaceStyle, subspaces(s))...) + +Base.rand(rng::AbstractRNG, sp::Random.SamplerTrivial{<:TupleProduct}) = map(s->rand(rng, s), subspaces(sp[])) +function Base.in(element, space::TupleProduct) + @assert length(element) == length(subspaces(space)) + return all(element[i] in s for (i, s) in enumerate(subspaces(space))) +end +Base.eltype(space::TupleProduct) = Tuple{map(eltype, subspaces(space))...} + +Base.length(space::TupleProduct) = mapreduce(length, *, subspaces(space)) +Base.iterate(space, args...) = iterate(Iterators.product(subspaces(space)...), args...) + +function bounds(s::TupleProduct) + bds = map(bounds, subspaces(s)) + return (first.(bds), last.(bds)) +end +Base.clamp(x, s::TupleProduct) = map(clamp, x, subspaces(s)) diff --git a/test/product.jl b/test/product.jl index e8ecf6b..0c89c5f 100644 --- a/test/product.jl +++ b/test/product.jl @@ -13,3 +13,29 @@ end @test @inferred product(Box([1,3], [2,4]), 5..6) == Box([1,3,5], [2,4,6]) @test @inferred product(5..6, Box([1,3], [2,4])) == Box([5,1,3], [6,2,4]) end + +@testset "TupleProduct discrete" begin + tp = TupleProduct([1,2], [3,4]) + @test @inferred rand(tp) in tp + @test (1,3) in tp + @test !((1,2) in tp) + @test @inferred eltype(tp) == Tuple{Int, Int} + @test @inferred SpaceStyle(tp) == FiniteSpaceStyle() + elems = @inferred collect(tp) + @test @inferred all(e in tp for e in elems) + @test @inferred all(e in elems for e in tp) +end + +@testset "TupleProduct continuous" begin + tp = TupleProduct(1..2, 3..4) + @test @inferred rand(tp) in tp + @test (1,3) in tp + @test !((1,2) in tp) + @test_broken eltype(tp) == Tuple{Float64, Float64} + @test @inferred SpaceStyle(tp) == ContinuousSpaceStyle() + @test @inferred bounds(tp) == ((1,3), (2,4)) + @test @inferred bounds(TupleProduct(1..2, 3..4, 5..6)) == ((1,3,5), (2,4,6)) + @test @inferred clamp((0,0), tp) == (1, 3) +end + + From c342b84a94b3b72cab827f76127bbde55cff6b26 Mon Sep 17 00:00:00 2001 From: Zachary Sunberg Date: Wed, 3 Aug 2022 22:42:49 -0600 Subject: [PATCH 12/14] fixed Box docstring --- src/array.jl | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/array.jl b/src/array.jl index 3c7d028..d906b6a 100644 --- a/src/array.jl +++ b/src/array.jl @@ -4,7 +4,7 @@ abstract type AbstractArraySpace end """ Box(lower, upper) -A Box represents a space of real-valued arrays bounded element-wise above by `upper` and below by `lower`, e.g. `Box([-1, -2], [3, 4]` represents the two-dimensional vector space that is the Cartesian product of the two closed sets: ``[-1, 3] \times [-2, 4]``. +A Box represents a space of real-valued arrays bounded element-wise above by `upper` and below by `lower`, e.g. `Box([-1, -2], [3, 4]` represents the two-dimensional vector space that is the Cartesian product of the two closed sets: ``[-1, 3] \\times [-2, 4]``. The elements of a Box are always `AbstractArray`s with `AbstractFloat` elements. `Box`es always have `ContinuousSpaceStyle`, and products of `Box`es with `Box`es or `ClosedInterval`s are `Box`es when the dimensions are compatible. """ From e48e17bee171410ead12943b8a515f656499296f Mon Sep 17 00:00:00 2001 From: Zachary Sunberg Date: Thu, 4 Aug 2022 23:41:33 -0600 Subject: [PATCH 13/14] added docstrings, updated readme --- README.md | 28 +++++++++++++++++++--------- src/CommonRLSpaces.jl | 2 ++ src/array.jl | 5 +++++ src/basic.jl | 34 ++++++++++++++++++++++++++++++---- src/product.jl | 8 ++++++++ 5 files changed, 64 insertions(+), 13 deletions(-) diff --git a/README.md b/README.md index 0109b5f..1d5530e 100644 --- a/README.md +++ b/README.md @@ -91,18 +91,19 @@ julia> length(s) ``` ```julia -julia> s = ArraySpace(UInt8, 2,3) +julia> s = ArraySpace(1:5, 2,3) +CommonRLSpaces.RepeatedSpace{UnitRange{Int64}, Tuple{Int64, Int64}}(1:5, (2, 3)) julia> rand(s) -2×3 Matrix{UInt8}: - 0x7b 0x38 0xf3 - 0x6a 0xe1 0x28 +2×3 Matrix{Int64}: + 4 1 1 + 3 2 2 julia> rand(s) in s true julia> SpaceStyle(s) -FiniteSpaceStyle{(2, 3)}() +FiniteSpaceStyle() julia> elsize(s) (2, 3) @@ -110,18 +111,27 @@ julia> elsize(s) ```julia julia> s = product(-1..1, 0..1) +Box{StaticArraysCore.SVector{2, Float64}}([-1.0, 0.0], [1.0, 1.0]) julia> rand(s) -2-element SVector{2, Float64} with indices SOneTo(2): - 0.5563101538643473 - 0.9227368869418011 +2-element StaticArraysCore.SVector{2, Float64} with indices SOneTo(2): + 0.03049072910834738 + 0.6295234114874269 julia> rand(s) in s true julia> SpaceStyle(s) -ContinuousSpaceStyle{(2,)}() +ContinuousSpaceStyle() julia> elsize(s) (2,) + +julia> bounds(s) +([-1.0, 0.0], [1.0, 1.0]) + +julia> clamp([5, 5], s) +2-element StaticArraysCore.SizedVector{2, Float64, Vector{Float64}} with indices SOneTo(2): + 1.0 + 1.0 ``` diff --git a/src/CommonRLSpaces.jl b/src/CommonRLSpaces.jl index e66fc2a..7b68225 100644 --- a/src/CommonRLSpaces.jl +++ b/src/CommonRLSpaces.jl @@ -6,6 +6,8 @@ using Reexport using StaticArrays using FillArrays +using Random +import Base: clamp export SpaceStyle, diff --git a/src/array.jl b/src/array.jl index d906b6a..7f764e5 100644 --- a/src/array.jl +++ b/src/array.jl @@ -57,6 +57,11 @@ struct RepeatedSpace{B, S<:Tuple} <: AbstractArraySpace elsize::S end +""" + ArraySpace(base_space, size...) + +Create a space of Arrays with shape `size`, where each element of the array is drawn from `base_space`. +""" ArraySpace(base_space, size...) = RepeatedSpace(base_space, size) SpaceStyle(s::RepeatedSpace) = SpaceStyle(s.base_space) diff --git a/src/basic.jl b/src/basic.jl index 1e52627..3bf5314 100644 --- a/src/basic.jl +++ b/src/basic.jl @@ -1,13 +1,14 @@ -using Random - -##### - abstract type AbstractSpaceStyle end struct FiniteSpaceStyle <: AbstractSpaceStyle end struct ContinuousSpaceStyle <: AbstractSpaceStyle end struct UnknownSpaceStyle <: AbstractSpaceStyle end +""" + SpaceStyle(space) + +Holy-style trait that describes whether the space is continuous, finite discrete, or an unknown type. See CommonRLInterface for a more detailed description of the styles. +""" SpaceStyle(space::Any) = UnknownSpaceStyle() SpaceStyle(::Tuple) = FiniteSpaceStyle() @@ -30,9 +31,34 @@ promote_spacestyle(_, _) = UnknownSpaceStyle() # handle case of 3 or more promote_spacestyle(s1, s2, s3, others...) = foldl(promote_spacestyle, (s1, s2, s3, args...)) +"Return the size of the objects in a space. This is guaranteed to be defined if the objects in the space are arrays, but otherwise it may not be defined." function elsize end # note: different than Base.elsize +""" + bounds(space) + +Return a `Tuple` containing lower and upper bounds for the elements in a space. + +For example, if `space` is a unit circle, `bounds(space)` will return `([-1.0, -1.0], [1.0, 1.0])`. This allows agents to choose policies that appropriately cover the space e.g. a normal distribution with a mean of `mean(bounds(space))` and a standard deviation of half the distance between the bounds. + +`bounds` should be defined for ContinuousSpaceStyle spaces. + +# Example +```juliadoctest +julia> bounds(1..2) +(1, 2) +``` +""" function bounds end +""" + clamp(x, space) + +Return an element of `space` that is near `x`. + +For example, if `space` is a unit circle, `clamp([2.0, 0.0], space)` might return `[1.0, 0.0]`. This allows for a convenient way for an agent to find a valid action if they sample actions from a distribution that doesn't match the space exactly (e.g. a normal distribution). +""" +function clamp end + bounds(i::AbstractInterval) = (infimum(i), supremum(i)) Base.clamp(x, i::AbstractInterval) = IntevalSets.clamp(x, i) diff --git a/src/product.jl b/src/product.jl index 74ae9c9..f8e5506 100644 --- a/src/product.jl +++ b/src/product.jl @@ -18,8 +18,16 @@ struct TupleProduct{T<:Tuple} ss::T end +""" + TupleProduct(space1, space2, ...) + +Create a space representing the Cartesian product of the argument. Each element is a `Tuple` containing one element from each of the constituent spaces. + +Use `subspaces` to access a `Tuple` containing the constituent spaces. +""" TupleProduct(s1, s2, others...) = TupleProduct((s1, s2, others...)) +"Return a `Tuple` containing the spaces used to create a `TupleProduct`" subspaces(s::TupleProduct) = s.ss product(s1::TupleProduct, s2::TupleProduct) = TupleProduct(subspaces(s1)..., subspaces(s2)...) From 9ee91b54efe692f25b1a23499c20caa9fca51dbd Mon Sep 17 00:00:00 2001 From: Jun Tian Date: Fri, 5 Aug 2022 14:32:24 +0800 Subject: [PATCH 14/14] add minor modifications --- README.md | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 1d5530e..707b335 100644 --- a/README.md +++ b/README.md @@ -9,10 +9,11 @@ A space is simply a set of objects. In a reinforcement learning context, spaces define the sets of possible states, actions, and observations. -In Julia, spaces can be represented by a variety of objects. For instance, a small discrete action set might be represented with `["up", "left", "down", "right"]`, or an interval of real numbers might be represented with an object from the `IntervalSets` package. In general, the space defined by any Julia object is the set of objects `x` for which `x in space` returns `true`. +In Julia, spaces can be represented by a variety of objects. For instance, a small discrete action set might be represented with `["up", "left", "down", "right"]`, or an interval of real numbers might be represented with an object from the [`IntervalSets`](https://github.com/JuliaMath/IntervalSets.jl) package. In general, the space defined by any Julia object is the set of objects `x` for which `x in space` returns `true`. In addition to establishing the definition above, this package provides three useful tools: -1. Traits to communicate about the properties of spaces, e.g. whether they are continuous or discrete, how many dimensions they have, and how to interact with them. + +1. Traits to communicate about the properties of spaces, e.g. whether they are continuous or discrete, how many subspaces they have, and how to interact with them. 2. Functions such as `product` for constructing more complex spaces 3. Constructors to for spaces whose elements are arrays, such as `ArraySpace` and `Box`. @@ -35,9 +36,12 @@ Spaces with a finite number of elements have `FiniteSpaceStyle`. These spaces ar ### Continuous spaces Continuous spaces represent sets that have an uncountable number of elements they have a `SpaceStyle` of type `ContinuousSpaceStyle`. CommonRLSpaces does not adopt a rigorous mathematical definition of a continuous set, but, roughly, elements in the interior of a continuous space have other elements very close to them. -Continuous spaces have two additional interface functions: + +Continuous spaces have some additional interface functions: + - `bounds(space)` returns upper and lower bounds in a tuple. For example, if `space` is a unit circle, `bounds(space)` will return `([-1.0, -1.0], [1.0, 1.0])`. This allows agents to choose policies that appropriately cover the space e.g. a normal distribution with a mean of `mean(bounds(space))` and a standard deviation of half the distance between the bounds. - `clamp(x, space)` returns an element of `space` that is near `x`. i.e. if `space` is a unit circle, `clamp([2.0, 0.0], space)` might return `[1.0, 0.0]`. This allows for a convenient way for an agent to find a valid action if they sample actions from a distribution that doesn't match the space exactly (e.g. a normal distribution). +- `clamp!(x, space)`, similar to `clamp`, but clamps `x` in place. ### Hybrid spaces