Skip to content

Commit

Permalink
Rename package
Browse files Browse the repository at this point in the history
  • Loading branch information
Zinoex committed Jan 5, 2024
1 parent ab54eaf commit 6bb3e78
Show file tree
Hide file tree
Showing 25 changed files with 92 additions and 88 deletions.
4 changes: 2 additions & 2 deletions Project.toml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name = "IMDP"
name = "IntervalMDP"
uuid = "051c988a-e73c-45a4-90ec-875cac0402c7"
authors = ["Frederik Baymler Mathiesen <[email protected]> and contributors"]
version = "0.1.0"
Expand All @@ -15,7 +15,7 @@ CUDA = "052768ef-5323-5732-b1bb-66c8b64840ba"
LLVM = "929cbde3-209d-540e-8aea-75f648917ca0"

[extensions]
IMDPCudaExt = ["Adapt", "CUDA", "LLVM"]
IntervalMDPCudaExt = ["Adapt", "CUDA", "LLVM"]

[compat]
CUDA = "5.1.0"
Expand Down
20 changes: 11 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# IMDP.jl - Interval Markov Decision Processes
# IntervalMDP.jl - Interval Markov Decision Processes

[![Documentation](https://img.shields.io/badge/docs-latest-blue.svg)](https://www.baymler.com/IMDP.jl/dev/usage/)
[![Build Status](https://github.com/zinoex/IMDP.jl/actions/workflows/CI.yml/badge.svg?branch=main)](https://github.com/zinoex/IMDP.jl/actions/workflows/CI.yml?query=branch%3Amain)
[![Codecov](https://codecov.io/gh/Zinoex/IMDP.jl/graph/badge.svg?token=K62S0148BK)](https://codecov.io/gh/Zinoex/IMDP.jl)
[![Documentation](https://img.shields.io/badge/docs-latest-blue.svg)](https://www.baymler.com/IntervalMDP.jl/dev/usage/)
[![Build Status](https://github.com/zinoex/IntervalMDP.jl/actions/workflows/CI.yml/badge.svg?branch=main)](https://github.com/zinoex/IntervalMDP.jl/actions/workflows/CI.yml?query=branch%3Amain)
[![Codecov](https://codecov.io/gh/Zinoex/IntervalMDP.jl/graph/badge.svg?token=K62S0148BK)](https://codecov.io/gh/Zinoex/IntervalMDP.jl)

IMDP.jl is a Julia package for modeling and certifying Interval Markov Decision Processes (IMDPs) via Value Iteration.
IntervalMDP.jl is a Julia package for modeling and certifying Interval Markov Decision Processes (IMDPs) via Value Iteration.

IMDPs are a generalization of Markov Decision Processes (MDPs) where the transition probabilities
are represented by intervals instead of point values, to model uncertainty. IMDPs are also frequently
Expand All @@ -26,10 +26,10 @@ certificate using CUDA hardware.

This package requires Julia v1.9 or later. Refer to the [official documentation](https://julialang.org/downloads/) on how to install it for your system.

To install `IMDP.jl`, use the following command inside Julia's REPL:
To install `IntervalMDP.jl`, use the following command inside Julia's REPL:

```julia
julia> import Pkg; Pkg.add("IMDP")
julia> import Pkg; Pkg.add("IntervalMDP")
```

If you want to use the CUDA extension, you also need to install `CUDA.jl`:
Expand All @@ -41,6 +41,8 @@ julia> import Pkg; Pkg.add("CUDA")
Here is an example of how to use the package to solve a finite horizon reachability problem for an Interval Markov Chain (IMC) with 3 states and 1 initial state.
The goal is to compute the maximum pessimistic probability of reaching state 3 within 10 time steps.
```julia
using IntervalMDP

# IMC
prob = IntervalProbabilities(;
lower = [
Expand All @@ -67,9 +69,9 @@ problem = Problem(imc, spec)
V, k, residual = value_iteration(problem)
```

See [Usage](https://www.baymler.com/IMDP.jl/dev/usage/) for more information about different specifications, using sparse matrices, and CUDA.
See [Usage](https://www.baymler.com/IntervalMDP.jl/dev/usage/) for more information about different specifications, using sparse matrices, and CUDA.

## Copyright notice
Technische Universiteit Delft hereby disclaims all copyright interest in the program “IMDP.jl” (GPU-accelerated value iteration for Interval Markov Decision Processes) written by the Frederik Baymler Mathiesen. Fred van Keulen, Dean of Mechanical Engineering.
Technische Universiteit Delft hereby disclaims all copyright interest in the program “IntervalMDP.jl” (GPU-accelerated value iteration for Interval Markov Decision Processes) written by the Frederik Baymler Mathiesen. Fred van Keulen, Dean of Mechanical Engineering.

© 2024, Frederik Baymler Mathiesen, HERALD Lab, TU Delft
2 changes: 1 addition & 1 deletion docs/Project.toml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
[deps]
Documenter = "e30172f5-a6a5-5a46-863b-614d45cd2de4"
IMDP = "051c988a-e73c-45a4-90ec-875cac0402c7"
IntervalMDP = "051c988a-e73c-45a4-90ec-875cac0402c7"
Revise = "295af30f-e4ad-537b-8983-00126c2a3abe"
SparseArrays = "2f01184e-e22b-5df5-ae63-d93ebab69eaf"
12 changes: 6 additions & 6 deletions docs/make.jl
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
using IMDP
using IntervalMDP
using Documenter

push!(LOAD_PATH, "../src/")
DocMeta.setdocmeta!(IMDP, :DocTestSetup, :(using IMDP); recursive = true)
DocMeta.setdocmeta!(IntervalMDP, :DocTestSetup, :(using IntervalMDP); recursive = true)

makedocs(;
modules = [IMDP],
modules = [IntervalMDP],
authors = "Frederik Baymler Mathiesen <[email protected]> and contributors",
sitename = "IMDP.jl",
sitename = "IntervalMDP.jl",
format = Documenter.HTML(;
prettyurls = get(ENV, "CI", "false") == "true",
canonical = "https://www.baymler.com/IMDP.jl",
canonical = "https://www.baymler.com/IntervalMDP.jl",
edit_link = "main",
assets = String[],
),
Expand All @@ -31,4 +31,4 @@ makedocs(;
doctest = false,
)

deploydocs(; repo = "github.com/Zinoex/IMDP.jl", devbranch = "main")
deploydocs(; repo = "github.com/Zinoex/IntervalMDP.jl", devbranch = "main")
2 changes: 1 addition & 1 deletion docs/src/data.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,4 @@

## bmdp-tool

## IMDP.jl
## IntervalMDP.jl
12 changes: 6 additions & 6 deletions docs/src/index.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
```@meta
CurrentModule = IMDP
CurrentModule = IntervalMDP
```

# IMDP
[IMDP.jl](https://github.com/zinoex/IMDP.jl) is a [Julia](https://julialang.org/) package for modeling
# IntervalMDP
[IntervalMDP.jl](https://github.com/zinoex/IntervalMDP.jl) is a [Julia](https://julialang.org/) package for modeling
and certifying Interval Markov Decision Processes (IMDPs) via Value Iteration.

IMDPs are a generalization of Markov Decision Processes (MDPs) where the transition probabilities
Expand All @@ -21,7 +21,7 @@ introduces for enabling better use of the available hardware and higher performa
- Dense and sparse matrix support
- Parametric probability types for customizable precision
- Multithreaded CPU and CUDA-accelerated value iteration
- Data loading and writing in formats by various tools (PRISM, bmdp-tool, IMDP.jl)
- Data loading and writing in formats by various tools (PRISM, bmdp-tool, IntervalMDP.jl)

!!! info
Until now, all state-of-the-art tools for IMDPs have been standalone programs.
Expand All @@ -31,10 +31,10 @@ introduces for enabling better use of the available hardware and higher performa

This package requires Julia v1.9 or later. Refer to the [official documentation](https://julialang.org/downloads/) on how to install it for your system.

To install `IMDP.jl`, use the following command inside Julia's REPL:
To install `IntervalMDP.jl`, use the following command inside Julia's REPL:

```julia
julia> import Pkg; Pkg.add("IMDP")
julia> import Pkg; Pkg.add("IntervalMDP")
```

If you want to use the CUDA extension, you also need to install `CUDA.jl`:
Expand Down
4 changes: 2 additions & 2 deletions docs/src/reference/data.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Data formats

```@meta
CurrentModule = IMDP.Data
CurrentModule = IntervalMDP.Data
```

## PRISM
Expand All @@ -16,7 +16,7 @@ read_bmdp_tool_file
write_bmdp_tool_file
```

## IMDP.jl
## IntervalMDP.jl
```@docs
read_imdp_jl
read_imdp_jl_model
Expand Down
8 changes: 5 additions & 3 deletions docs/src/usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@ Both systems consist of states, a designated initial state, and a transition mat
An example of how to construct either is the following:

```julia
using IntervalMDP

# IMC
prob = IntervalProbabilities(;
lower = [
Expand Down Expand Up @@ -209,16 +211,16 @@ prob = IntervalProbabilities(;
),
)

mc = IMDP.cu(IntervalMarkovChain(prob, 1))
mc = IntervalMDP.cu(IntervalMarkovChain(prob, 1))

# Transfer transition matrices separately
prob = IntervalProbabilities(;
lower = IMDP.cu(sparse_hcat(
lower = IntervalMDP.cu(sparse_hcat(
SparseVector(3, [2, 3], [0.1, 0.2]),
SparseVector(3, [1, 2, 3], [0.5, 0.3, 0.1]),
SparseVector(3, [3], [1.0]),
)),
upper = IMDP.cu(sparse_hcat(
upper = IntervalMDP.cu(sparse_hcat(
SparseVector(3, [1, 2, 3], [0.5, 0.6, 0.7]),
SparseVector(3, [1, 2, 3], [0.7, 0.5, 0.3]),
SparseVector(3, [3], [1.0]),
Expand Down
16 changes: 8 additions & 8 deletions ext/IMDPCudaExt.jl → ext/IntervalMDPCudaExt.jl
Original file line number Diff line number Diff line change
@@ -1,34 +1,34 @@
module IMDPCudaExt
module IntervalMDPCudaExt

import LLVM
using LLVM.Interop: assume

using CUDA, CUDA.CUSPARSE, Adapt, SparseArrays

using IMDP, LinearAlgebra
using IntervalMDP, LinearAlgebra

Adapt.@adapt_structure IntervalProbabilities

# Opinionated conversion to GPU with Float64 values and Int32 indices
IMDP.cu(model) = adapt(IMDP.CuModelAdaptor{Float64, Int32}, model)
IntervalMDP.cu(model) = adapt(IntervalMDP.CuModelAdaptor{Float64, Int32}, model)

function Adapt.adapt_structure(T::Type{<:IMDP.CuModelAdaptor}, mc::IntervalMarkovChain)
function Adapt.adapt_structure(T::Type{<:IntervalMDP.CuModelAdaptor}, mc::IntervalMarkovChain)
return IntervalMarkovChain(
adapt(T, transition_prob(mc)),
adapt(CuArray{IMDP.indtype(T)}, initial_states(mc)),
adapt(CuArray{IntervalMDP.indtype(T)}, initial_states(mc)),
num_states(mc),
)
end

function Adapt.adapt_structure(
T::Type{<:IMDP.CuModelAdaptor},
T::Type{<:IntervalMDP.CuModelAdaptor},
mdp::IntervalMarkovDecisionProcess,
)
return IntervalMarkovDecisionProcess(
adapt(T, transition_prob(mdp)),
adapt(CuArray{IMDP.indtype(T)}, IMDP.stateptr(mdp)),
adapt(CuArray{IntervalMDP.indtype(T)}, IntervalMDP.stateptr(mdp)),
actions(mdp),
adapt(CuArray{IMDP.indtype(T)}, initial_states(mdp)),
adapt(CuArray{IntervalMDP.indtype(T)}, initial_states(mdp)),
num_states(mdp),
)
end
Expand Down
6 changes: 3 additions & 3 deletions ext/cuda/array.jl
Original file line number Diff line number Diff line change
Expand Up @@ -145,14 +145,14 @@ CUDA.CUSPARSE.CuSparseMatrixCSC{Tv, Ti}(M::SparseMatrixCSC) where {Tv, Ti} =
)

Adapt.adapt_storage(
::Type{IMDP.CuModelAdaptor{Tv, Ti}},
::Type{IntervalMDP.CuModelAdaptor{Tv, Ti}},
M::SparseArrays.FixedSparseCSC,
) where {Tv, Ti} = CuSparseMatrixCSC{Tv, Ti}(M)

Adapt.adapt_storage(
::Type{IMDP.CuModelAdaptor{Tv, Ti}},
::Type{IntervalMDP.CuModelAdaptor{Tv, Ti}},
M::SparseMatrixCSC,
) where {Tv, Ti} = CuSparseMatrixCSC{Tv, Ti}(M)

Adapt.adapt_storage(::Type{IMDP.CuModelAdaptor{Tv, Ti}}, x::AbstractArray) where {Tv, Ti} =
Adapt.adapt_storage(::Type{IntervalMDP.CuModelAdaptor{Tv, Ti}}, x::AbstractArray) where {Tv, Ti} =
adapt(CuArray{Tv}, x)
10 changes: 5 additions & 5 deletions ext/cuda/interval_probabilities.jl
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@

function IMDP.compute_gap(lower::M, upper::M) where {Tv, Ti, M <: CuSparseMatrixCSC{Tv, Ti}}
function IntervalMDP.compute_gap(lower::M, upper::M) where {Tv, Ti, M <: CuSparseMatrixCSC{Tv, Ti}}
# lower = CuSparseMatrixCOO(lower)

# FIXME: This is an ugly, non-robust hack.
upper = SparseMatrixCSC(upper)
lower = SparseMatrixCSC(lower)
lower, gap = IMDP.compute_gap(lower, upper)
return adapt(IMDP.CuModelAdaptor{Tv, Ti}, lower),
adapt(IMDP.CuModelAdaptor{Tv, Ti}, gap)
lower, gap = IntervalMDP.compute_gap(lower, upper)
return adapt(IntervalMDP.CuModelAdaptor{Tv, Ti}, lower),
adapt(IntervalMDP.CuModelAdaptor{Tv, Ti}, gap)
end

function IMDP.interval_prob_hcat(
function IntervalMDP.interval_prob_hcat(
T,
transition_probs::Vector{
<:IntervalProbabilities{Tv, <:AbstractVector{Tv}, <:CuSparseMatrixCSC{Tv, Ti}},
Expand Down
8 changes: 4 additions & 4 deletions ext/cuda/ordering.jl
Original file line number Diff line number Diff line change
Expand Up @@ -21,10 +21,10 @@ struct CuSparseDeviceOrdering{T, A} <: AbstractStateOrdering{T}
end

# Permutations are specific to each state
IMDP.perm(order::CuSparseOrdering, state) = order.subsets[state]
IMDP.perm(order::CuSparseDeviceOrdering, state) = order.subsets[state]
IntervalMDP.perm(order::CuSparseOrdering, state) = order.subsets[state]
IntervalMDP.perm(order::CuSparseDeviceOrdering, state) = order.subsets[state]

function IMDP.sort_states!(order::CuSparseOrdering, V; max = true)
function IntervalMDP.sort_states!(order::CuSparseOrdering, V; max = true)
sort_subsets!(order, V; max = max)

return order
Expand Down Expand Up @@ -185,7 +185,7 @@ end
return sync_threads()
end

function IMDP.construct_ordering(T, p::CuSparseMatrixCSC)
function IntervalMDP.construct_ordering(T, p::CuSparseMatrixCSC)
# Assume that input/start state is on the columns and output/target state is on the rows
vecptr = CuVector{T}(p.colPtr)
perm = CuVector{T}(SparseArrays.rowvals(p))
Expand Down
2 changes: 1 addition & 1 deletion ext/cuda/probability_assignment.jl
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@

function IMDP.probability_assignment!(
function IntervalMDP.probability_assignment!(
p::CuSparseMatrixCSC{Tv, Ti},
prob::IntervalProbabilities{Tv},
ordering::CuSparseOrdering{Ti},
Expand Down
16 changes: 8 additions & 8 deletions ext/cuda/value_iteration.jl
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@

function IMDP.construct_value_function(
function IntervalMDP.construct_value_function(
::MR,
num_states,
) where {R, MR <: CuSparseMatrixCSC{R}}
V = CUDA.zeros(R, num_states)
return V
end

function IMDP.construct_nonterminal(
function IntervalMDP.construct_nonterminal(
mc::IntervalMarkovChain{<:IntervalProbabilities{R, VR, MR}},
terminal::AbstractVector{Ti},
) where {R, VR <: AbstractVector{R}, MR <: CuSparseMatrixCSC{R}, Ti}
Expand All @@ -17,11 +17,11 @@ function IMDP.construct_nonterminal(
return nonterminal
end

function IMDP.construct_nonterminal(
function IntervalMDP.construct_nonterminal(
mdp::IntervalMarkovDecisionProcess{<:IntervalProbabilities{R, VR, MR}},
terminal::AbstractVector{Ti},
) where {R, VR <: AbstractVector{R}, MR <: CuSparseMatrixCSC{R}, Ti}
sptr = Vector(IMDP.stateptr(mdp))
sptr = Vector(IntervalMDP.stateptr(mdp))

nonterminal = convert.(Ti, setdiff(collect(1:num_states(mdp)), terminal))
nonterminal_actions = mapreduce(i -> sptr[i]:(sptr[i + 1] - 1), vcat, nonterminal)
Expand All @@ -32,11 +32,11 @@ function IMDP.construct_nonterminal(
return nonterminal, nonterminal_actions
end

function IMDP.step_imc!(
function IntervalMDP.step_imc!(
ordering,
p,
prob::IntervalProbabilities{R, VR, MR},
value_function::IMDP.IMCValueFunction;
value_function::IntervalMDP.IMCValueFunction;
upper_bound,
discount = 1.0,
) where {R, VR <: AbstractVector{R}, MR <: CuSparseMatrixCSC{R}}
Expand All @@ -51,13 +51,13 @@ function IMDP.step_imc!(
return value_function
end

function IMDP.step_imdp!(
function IntervalMDP.step_imdp!(
ordering,
p,
prob::IntervalProbabilities{R, VR, MR},
stateptr,
maxactions,
value_function::IMDP.IMDPValueFunction;
value_function::IntervalMDP.IMDPValueFunction;
maximize,
upper_bound,
discount = 1.0,
Expand Down
2 changes: 1 addition & 1 deletion src/Data/Data.jl
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
module Data
using IMDP, SparseArrays
using IntervalMDP, SparseArrays
using NCDatasets, JSON

include("bmdp-tool.jl")
Expand Down
2 changes: 1 addition & 1 deletion src/Data/bmdp-tool.jl
Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,7 @@ function write_bmdp_tool_file(
prob = transition_prob(mdp)
l, g = lower(prob), gap(prob)
num_columns = num_source(prob)
sptr = IMDP.stateptr(mdp)
sptr = IntervalMDP.stateptr(mdp)
act = actions(mdp)

number_states = num_states(mdp)
Expand Down
Loading

0 comments on commit 6bb3e78

Please sign in to comment.