Skip to content

Commit b4b08b0

Browse files
Aero-SpecAero-Spec
authored
Update algorithm.md - Made Tim's Requested Changes
1 parent 4678dc0 commit b4b08b0

File tree

1 file changed

+25
-111
lines changed

1 file changed

+25
-111
lines changed

‎docs/src/algorithm.md

+25-111
Original file line numberDiff line numberDiff line change
@@ -1,45 +1,50 @@
11
# Algorithm
22

3-
The divided rectangles algorithm, or DIRECT (for DIvided RECTangles), incrementally refines a retangular partition of the design space. The refinement is driven a heuristic that involves reasoning about potential Lipschitz constants.
3+
The divided rectangles algorithm, or DIRECT (for DIvided RECTangles), incrementally refines a retangular partition of the design space. The refinement is driven by a heuristic that involves reasoning about potential Lipschitz constants.
4+
5+
The strength of the DIRECT algorithm lies in its ability to systematically explore the entire search space while focusing on the most promising areas. This systematic coverage helps the algorithm escape local minima, making it particularly effective for objective functions with multiple local minima.
6+
7+
Additionally, by not requiring the Lipschitz constant, the DIRECT algorithm is adaptable to various optimization problems, including those where the smoothness of the objective function is not well understood.
48

9+
---
10+
11+
- The figure below shows the DIRECT method after 16 iterations on the Branin function. The cells are much denser around the minima of the Branin function because the DIRECT method is designed to increase its resolution in promising regions..
12+
13+
![page_11](https://github.com/user-attachments/assets/b833bedd-41aa-40c5-a27f-26188a171797)
514
---
615
### Key Concepts of the DIRECT Algorithm
716

8-
1. **Division of Search Space**:
17+
1. **Search Space**:
918

10-
- The algorithm begins by treating the entire feasible region as a single hyper-rectangle.
19+
- The algorithm minimizes an objective function f(x) over a hyper-rectangular search space.
1120
- The search space is normalized to the unit hypercube to avoid oversensitivity to dimensions with larger domains. If minimizing $f(x)$ in the interval between lower and upper ranges $a$ and $b$, DIRECT will instead minimize:
1221

1322
```math
1423
g(\mathbf{x}) = f(\mathbf{x} \odot (\mathbf{b} - \mathbf{a}) + \mathbf{a})
1524
```
1625

17-
After finding the minimum $x^*$ of $g$, the minimum of $f$ is
26+
After finding the minimum $x^*$ of $g$, The minimizer of $f$ is
1827

1928
```math
2029
\mathbf{x}^* \odot (\mathbf{b} - \mathbf{a}) + \mathbf{a}
2130
```
2231
---
2332

24-
- The figure below shows DIRECT method after 16 iterations on the Branin function. The cells are much denser around the minima of the Branin function because the DIRECT method is designed to increase resolution in promising regions.
25-
26-
![page_11](https://github.com/user-attachments/assets/b833bedd-41aa-40c5-a27f-26188a171797)
33+
2. **Function Evaluation**:
34+
- DIRECT partitions its search space into hyperrectangular intervals.
35+
- The objective function is evaluated at the center of each hyper-rectangle.
36+
- Each interval has a center $c^{(i)}$, an associated objective function value $f(c^{(i)})$, and a radius $r^{(i)}$. The radius is the distance from the center to a vertex."
2737

28-
---
38+
4. **Selection of Potentially Optimal Rectangles**:
39+
- In each iteration, the algorithm identifies potentially optimal rectangles. A rectangle is considered potentially optimal if it could contain the global minimum based on the evaluations performed so far.
2940

30-
2. **Function Evaluation**:
31-
- The function is evaluated at the center of each hyper-rectangle.
32-
- Each interval has a center $c^{(i)}$ and an associated objective function value $f(c^{(i)})$, as well as a radius $r^{(i)}$, which is the distance from the center to a vertex.
41+
### Lipschitz Lower Bound:
3342

34-
3. **Selection of Potentially Optimal Rectangles**:
35-
- After evaluation, the algorithm identifies potentially optimal rectangles. A rectangle is considered potentially optimal if it could contain the global minimum based on the evaluations performed so far.
43+
- The Lipschitz lower bound for an interval is a circular cone extending downward from its center $c^{(i)}$.
3644

37-
4. **Lipschitz Lower Bound**:
38-
- The Lipschitz lower bound for an interval is a circular cone extending downward from its center $c^{(i)}$
39-
40-
```math
41-
f(\mathbf{x}) \geq f(\mathbf{c}^{(i)}) - \ell \|\mathbf{x} - \mathbf{c}^{(i)}\|_2
42-
```
45+
```math
46+
f(\mathbf{x}) \geq f(\mathbf{c}^{(i)}) - \ell \|\mathbf{x} - \mathbf{c}^{(i)}\|_2
47+
```
4348
- This lower bound is constrained by the extents of the interval, and its lowest value is achieved at the vertices, which are all a distance $r^{(i)}$ from the center.
4449

4550
```math
@@ -78,7 +83,7 @@ f(c^{(i)}) - \ell r^{(i)}
7883
## Splitting Intervals
7984

8085
When splitting a region without equal side lengths, only the longest dimensions are split. Splitting proceeds on these dimensions in the same manner as with a hypercube. The width in a given dimension depends on how many times that dimension has been split. Since DIRECT always splits axis directions by thirds, a dimension
81-
that has been split d times will have a width of $3^−d$. If we have $n$ dimensions and track how many times each dimension of a given interval has been split in a vector $d$, then the radius of that interval is
86+
that has been split d times will have a width of $3^{−d}$. If we have $n$ dimensions and track how many times each dimension of a given interval has been split in a vector $d$, then the radius of that interval is
8287

8388
```math
8489
r = \left\|\left[ \frac{1}{2 \cdot 3^{-d_1}}, \dots, \frac{1}{2 \cdot 3^{-d_n}} \right]\right\|_2
@@ -92,94 +97,3 @@ r = \left\|\left[ \frac{1}{2 \cdot 3^{-d_1}}, \dots, \frac{1}{2 \cdot 3^{-d_n}}
9297

9398
![page_17](https://github.com/user-attachments/assets/99caea66-02b5-4371-90e2-69305c035ddf)
9499

95-
96-
---
97-
## Practical Implementations:
98-
99-
- Struct `DirectRectangle`:
100-
101-
```julia
102-
struct DirectRectangle
103-
c # center point
104-
y # center point value
105-
d # number of divisions per dimension
106-
r # the radius of the interval
107-
end
108-
```
109-
110-
- `direct` Function:
111-
112-
```julia
113-
function direct(f, a, b, k_max, r_min)
114-
g = x -> f(x .* (b - a) + a) # evaluate within unit hypercube
115-
n = length(a)
116-
c = fill(0.5, n)
117-
□s = [DirectRectangle(c, g(c), fill(0, n), sqrt(0.5^n))]
118-
c_best = c
119-
for k in 1 : k_max
120-
□s_split = get_split_intervals(□s, r_min)
121-
setdiff!(□s, □s_split)
122-
for □_split in □s_split
123-
append!(□s, split_interval(□_split, g))
124-
end
125-
c_best = □s[findmin(□.y for □ in □s)[2]].c
126-
end
127-
return c_best .* (b - a) + a # from unit hypercube
128-
end
129-
```
130-
- `is_ccw` Function:
131-
132-
```julia
133-
function is_ccw(a, b, c)
134-
return a.r * (b.y - c.y) - a.y * (b.r - c.r) + (b.r * c.y - b.y * c.r) < 1e-6
135-
end
136-
```
137-
- `get_split_intervals`Function
138-
139-
```julia
140-
function get_split_intervals(□s, r_min)
141-
hull = DirectRectangle[]
142-
sort!(□s, by = □ -> (□.r, □.y))
143-
for □ in □s
144-
if length(hull) >= 1 && □.r == hull[end].r
145-
continue # Repeated r values cannot be improvements
146-
end
147-
if length(hull) >= 1 && □.y ≤ hull[end].y
148-
pop!(hull) # Remove the last point if the new one is better
149-
end
150-
if length(hull) >= 2 && is_ccw(hull[end-1], hull[end], □)
151-
pop!(hull)
152-
end
153-
push!(hull, □)
154-
end
155-
filter!(□ -> □.r ≥ r_min, hull) # Only split intervals larger than the minimum radius
156-
return hull
157-
end
158-
```
159-
- `split_interval` Function:
160-
161-
```julia
162-
function split_interval(□, g)
163-
c, n, d_min, d = □.c, length(□.c), minimum(□.d), copy(□.d)
164-
dirs, δ = findall(d .== d_min), 3.0^(-d_min-1)
165-
Cs = [(c + δ*basis(i, n), c - δ*basis(i, n)) for i in dirs]
166-
Ys = [(g(C[1]), g(C[2])) for C in Cs]
167-
minvals = [min(Y[1], Y[2]) for Y in Ys]
168-
□s = DirectRectangle[]
169-
for j in sortperm(minvals)
170-
d[dirs[j]] += 1 # Increment the number of splits
171-
C, Y, r = Cs[j], Ys[j], norm(0.5 * 3.0.^(-d))
172-
push!(□s, DirectRectangle(C[1], Y[1], copy(d), r))
173-
push!(□s, DirectRectangle(C[2], Y[2], copy(d), r))
174-
end
175-
r = norm(0.5 * 3.0.^(-d))
176-
push!(□s, DirectRectangle(c, □.y, d, r))
177-
return □s
178-
end
179-
```
180-
---
181-
182-
### Strengths of the DIRECT Algorithm
183-
The strength of the DIRECT algorithm lies in its ability to systematically explore the entire search space while focusing on the most promising areas. This systematic coverage helps the algorithm escape local minima, making it particularly effective for objective functions with multiple local minima.
184-
By not requiring the Lipschitz constant, the DIRECT algorithm is adaptable to various optimization problems, including those where the smoothness of the objective function is not well understood.
185-

0 commit comments

Comments
 (0)