Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions Benchmarks/Benchmarks/Basic/BenchmarkRunner+Basic.swift
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,27 @@ let benchmarks: @Sendable () -> Void = {
benchmark.measurement(CustomMetrics.one, Int.random(in: 1 ... 1_000))
}

Benchmark("Amortized throughput benchmark using blackHoleMutating",
configuration: .init(metrics: .all + [CustomMetrics.two, CustomMetrics.one, CustomMetrics.three],
scalingFactor: .mega)) { benchmark in
var arg = 3.1415926e104 // constant value
for _ in benchmark.scaledIterations {
blackHoleMutating(&arg) // `x` assumed to be mutated, prevents loop-invariant code motion
blackHole(sqrt(arg))
}
}

Benchmark("Amortized latency benchmark using blackHoleMutating",
configuration: .init(metrics: .all + [CustomMetrics.two, CustomMetrics.one, CustomMetrics.three],
scalingFactor: .mega)) { benchmark in
var arg = 3.1415926e104 // constant value
blackHoleMutating(&x)
for _ in benchmark.scaledIterations {
arg = sqrt(arg)
}
blackHole(x)
}

Benchmark("All metrics",
configuration: .init(metrics: .all, skip: true)) { _ in
}
Expand Down
46 changes: 46 additions & 0 deletions Sources/Benchmark/Blackhole.swift
Original file line number Diff line number Diff line change
Expand Up @@ -36,3 +36,49 @@
public func identity<T>(_ value: T) -> T {
value
}

/// A more generalized variant of `blackHole` -- forces the compiler to assume that the argument is not only used, but also mutated.
/// Foils compiler optimizations like const-folding, loop-invariant code motion, and common-subexpression elimination.
/// For example, the `blackHole` does not always suffice for the following benchmark.
/// ```swift
/// Benchmark("Const-folded?",
/// configuration: .init(
/// metrics: [.wallClock, .mallocTotal],
/// scalingFactor: .mega
/// ) { benchmark in
/// let arguments = Arguments() // set up
/// benchmark.startMeasurement()
/// for _ in benchmark.scaledIterations {
/// blackHole(benchmarkee(arguments))
/// }
/// ```
/// If `benchmarkee` is a pure function, i.e. has no side-effects, the above code will get subjected to e.g. loop-invariant code motion.
/// ```swift
/// Benchmark("Const-folded?",
/// configuration: .init(
/// metrics: [.wallClock, .mallocTotal],
/// scalingFactor: .mega
/// ) { benchmark in
/// let arguments = Arguments() // set up
/// benchmark.startMeasurement()
/// let _result = benchmarkee(arguments)
/// for _ in benchmark.scaledIterations {
/// blackHole(_result) // no longer benchmarking `benchmarkee`!
/// }
/// ```
/// The correct way to implement this benchmark would then be
/// ```swift
/// Benchmark("Const-folded?",
/// configuration: .init(
/// metrics: [.wallClock, .mallocTotal],
/// scalingFactor: .mega
/// ) { benchmark in
/// var arguments = Arguments() // set up
/// benchmark.startMeasurement()
/// for _ in benchmark.scaledIterations {
/// blackHoleMutating(&arguments)
/// blackHole(benchmarkee(arguments))
/// }
/// ```
@_optimize(none)
public func blackHoleMutating(_: UnsafeMutableRawPointer) {}

Check warning on line 84 in Sources/Benchmark/Blackhole.swift

View check run for this annotation

Codecov / codecov/patch

Sources/Benchmark/Blackhole.swift#L84

Added line #L84 was not covered by tests
Loading