Skip to content

Commit d304d5f

Browse files
SteveDesmond-caSteveDesmond-ca
committed
Update results
Bump Microsoft.NET.Test.Sdk from 17.7.2 to 17.8.0 Bumps [Microsoft.NET.Test.Sdk](https://github.com/microsoft/vstest) from 17.7.2 to 17.8.0. - [Release notes](https://github.com/microsoft/vstest/releases) - [Changelog](https://github.com/microsoft/vstest/blob/main/docs/releases.md) - [Commits](microsoft/vstest@v17.7.2...v17.8.0) --- updated-dependencies: - dependency-name: Microsoft.NET.Test.Sdk dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]>
1 parent 852ea79 commit d304d5f

File tree

2 files changed

+69
-69
lines changed

2 files changed

+69
-69
lines changed

‎BenchmarkMockNet.Tests/BenchmarkMockNet.Tests.csproj

+1-1
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@
66
</PropertyGroup>
77

88
<ItemGroup>
9-
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.7.2" />
9+
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.8.0" />
1010
<PackageReference Include="xunit" Version="2.6.1" />
1111
<PackageReference Include="xunit.runner.visualstudio" Version="2.5.3" PrivateAssets="all" />
1212
<PackageReference Include="coverlet.collector" Version="6.0.0" PrivateAssets="all" />

‎Results.md

+68-68
Original file line numberDiff line numberDiff line change
@@ -1,111 +1,111 @@
11
# BenchmarkMockNet Results
22

3-
## Official Run: 07/26/2023
3+
## Official Run: 11/07/2023
44

55
| Framework | Version |
66
|-----------|---------|
7-
| FakeItEasy | 7.4.0 |
8-
| JustMock | 2023.2.719.137 |
9-
| Moq | 4.18.4 |
10-
| NSubstitute | 5.0.0 |
7+
| FakeItEasy | 8.0.0 |
8+
| JustMock | 2023.3.1011.155 |
9+
| Moq | 4.20.69 |
10+
| NSubstitute | 5.1.0 |
1111
| PCLMock | 5.1.3 |
12-
| Rocks | 7.1.2 |
12+
| Rocks | 7.1.4 |
1313

1414
### Construction
1515

1616
This test simply creates an `IThing` object to test using the given framework
1717

18-
| Method | Mean | Error | StdDev | Ratio | RatioSD | Gen0 | Gen1 | Allocated | Alloc Ratio |
19-
|------------- |-------------:|-------------:|-------------:|---------:|--------:|-------:|-------:|----------:|------------:|
20-
| Stub | 26.01 ns | 167.05 ns | 9.157 ns | 1.00 | 0.00 | - | - | 24 B | 1.00 |
21-
| FakeItEasy | 3,791.66 ns | 20,707.54 ns | 1,135.050 ns | 153.83 | 46.17 | 0.1500 | 0.0100 | 2962 B | 123.42 |
22-
| JustMockLite | 40,944.69 ns | 23,920.69 ns | 1,311.174 ns | 1,743.33 | 728.26 | 0.9100 | 0.0100 | 17033 B | 709.71 |
23-
| Moq | 3,642.32 ns | 15,868.14 ns | 869.786 ns | 149.35 | 46.79 | 0.1100 | - | 2088 B | 87.00 |
24-
| NSubstitute | 4,589.36 ns | 28,838.17 ns | 1,580.717 ns | 184.43 | 53.43 | 0.2700 | - | 5176 B | 215.67 |
25-
| PCLMock | 125.10 ns | 781.49 ns | 42.836 ns | 4.89 | 0.68 | - | - | 144 B | 6.00 |
26-
| Rocks | 118.88 ns | 798.64 ns | 43.776 ns | 4.59 | 0.46 | - | - | 144 B | 6.00 |
18+
| Method | Mean | Error | StdDev | Ratio | RatioSD | Gen0 | Gen1 | Allocated | Alloc Ratio |
19+
|------------- |-------------:|-------------:|------------:|---------:|--------:|-------:|-------:|----------:|------------:|
20+
| Stub | 31.34 ns | 211.41 ns | 11.59 ns | 1.00 | 0.00 | - | - | 24 B | 1.00 |
21+
| FakeItEasy | 4,979.65 ns | 30,856.32 ns | 1,691.34 ns | 167.74 | 52.16 | 0.1500 | 0.0100 | 2962 B | 123.42 |
22+
| JustMockLite | 48,256.19 ns | 35,224.70 ns | 1,930.78 ns | 1,719.42 | 739.33 | 0.9100 | 0.0100 | 17033 B | 709.71 |
23+
| Moq | 4,585.05 ns | 19,632.53 ns | 1,076.13 ns | 157.27 | 51.49 | 0.1100 | - | 2088 B | 87.00 |
24+
| NSubstitute | 5,473.06 ns | 29,137.72 ns | 1,597.14 ns | 185.43 | 56.33 | 0.2700 | - | 5176 B | 215.67 |
25+
| PCLMock | 155.87 ns | 1,080.57 ns | 59.23 ns | 5.03 | 0.70 | - | - | 144 B | 6.00 |
26+
| Rocks | 146.48 ns | 1,090.11 ns | 59.75 ns | 4.69 | 0.62 | - | - | 144 B | 6.00 |
2727

2828
### Return
2929

3030
A mock object is created, a method is setup to a return and `int` value, and that method is called
3131

32-
| Method | Mean | Error | StdDev | Ratio | RatioSD | Gen0 | Gen1 | Allocated | Alloc Ratio |
33-
|------------- |--------------:|-------------:|-------------:|----------:|---------:|-------:|-------:|----------:|------------:|
34-
| Stub | 23.44 ns | 163.39 ns | 8.956 ns | 1.00 | 0.00 | - | - | 24 B | 1.00 |
35-
| FakeItEasy | 7,408.36 ns | 31,718.42 ns | 1,738.594 ns | 343.86 | 122.95 | 0.2600 | 0.0100 | 4982 B | 207.58 |
36-
| JustMockLite | 232,518.92 ns | 97,332.39 ns | 5,335.117 ns | 11,243.18 | 5,237.54 | 1.8500 | 1.8400 | 34629 B | 1,442.88 |
37-
| Moq | 7,119.02 ns | 24,929.24 ns | 1,366.456 ns | 334.04 | 128.74 | 0.2100 | - | 3952 B | 164.67 |
38-
| NSubstitute | 8,359.82 ns | 30,508.35 ns | 1,672.265 ns | 390.95 | 146.51 | 0.4000 | 0.0100 | 7648 B | 318.67 |
39-
| PCLMock | 2,735.22 ns | 16,738.66 ns | 917.502 ns | 124.43 | 40.91 | 0.1000 | - | 1912 B | 79.67 |
40-
| Rocks | 627.43 ns | 1,372.57 ns | 75.235 ns | 29.60 | 11.45 | 0.0300 | - | 680 B | 28.33 |
32+
| Method | Mean | Error | StdDev | Ratio | RatioSD | Gen0 | Gen1 | Allocated | Alloc Ratio |
33+
|------------- |--------------:|--------------:|------------:|----------:|---------:|-------:|-------:|----------:|------------:|
34+
| Stub | 28.90 ns | 212.68 ns | 11.66 ns | 1.00 | 0.00 | - | - | 24 B | 1.00 |
35+
| FakeItEasy | 8,969.13 ns | 38,381.23 ns | 2,103.80 ns | 343.22 | 134.31 | 0.2600 | 0.0100 | 4982 B | 207.58 |
36+
| JustMockLite | 280,723.96 ns | 105,038.74 ns | 5,757.53 ns | 11,205.96 | 5,574.96 | 1.8500 | 1.8400 | 34629 B | 1,442.88 |
37+
| Moq | 7,620.79 ns | 26,993.24 ns | 1,479.59 ns | 294.42 | 121.71 | 0.2100 | - | 3952 B | 164.67 |
38+
| NSubstitute | 9,560.75 ns | 32,758.64 ns | 1,795.61 ns | 369.85 | 153.98 | 0.4000 | 0.0100 | 7648 B | 318.67 |
39+
| PCLMock | 3,186.99 ns | 15,562.04 ns | 853.01 ns | 120.68 | 43.98 | 0.1000 | - | 1912 B | 79.67 |
40+
| Rocks | 822.79 ns | 2,695.01 ns | 147.72 ns | 31.05 | 9.61 | 0.0300 | - | 680 B | 28.33 |
4141

4242
### EmptyReturn
4343

4444
A mock object is created, with no method setup, and an method returning an `int` is called
4545

46-
| Method | Mean | Error | StdDev | Ratio | RatioSD | Gen0 | Gen1 | Allocated | Alloc Ratio |
47-
|------------- |-------------:|-------------:|-------------:|---------:|---------:|-------:|-------:|----------:|------------:|
48-
| Stub | 24.07 ns | 170.59 ns | 9.351 ns | 1.00 | 0.00 | - | - | 24 B | 1.00 |
49-
| FakeItEasy | 4,966.52 ns | 22,831.31 ns | 1,251.461 ns | 224.73 | 81.10 | 0.2000 | 0.0100 | 3833 B | 159.71 |
50-
| JustMockLite | 51,175.45 ns | 33,067.03 ns | 1,812.516 ns | 2,417.90 | 1,137.33 | 1.1100 | 0.1200 | 20858 B | 869.08 |
51-
| Moq | 4,323.51 ns | 17,255.84 ns | 945.851 ns | 197.11 | 74.34 | 0.1300 | - | 2456 B | 102.33 |
52-
| NSubstitute | 6,335.88 ns | 30,583.95 ns | 1,676.409 ns | 287.00 | 106.47 | 0.3100 | - | 5872 B | 244.67 |
53-
| PCLMock | 2,821.60 ns | 15,994.90 ns | 876.734 ns | 124.79 | 37.02 | 0.1000 | - | 1912 B | 79.67 |
54-
| Rocks | 619.56 ns | 1,512.35 ns | 82.897 ns | 28.42 | 10.63 | 0.0300 | - | 680 B | 28.33 |
46+
| Method | Mean | Error | StdDev | Ratio | RatioSD | Gen0 | Gen1 | Allocated | Alloc Ratio |
47+
|------------- |-------------:|-------------:|------------:|---------:|---------:|-------:|-------:|----------:|------------:|
48+
| Stub | 28.71 ns | 204.15 ns | 11.19 ns | 1.00 | 0.00 | - | - | 24 B | 1.00 |
49+
| FakeItEasy | 5,890.26 ns | 29,446.18 ns | 1,614.04 ns | 223.10 | 80.85 | 0.2000 | 0.0100 | 3833 B | 159.71 |
50+
| JustMockLite | 60,504.16 ns | 41,721.72 ns | 2,286.91 ns | 2,398.37 | 1,130.18 | 1.1100 | 0.1200 | 20858 B | 869.08 |
51+
| Moq | 5,204.82 ns | 20,856.75 ns | 1,143.23 ns | 200.98 | 83.37 | 0.1300 | - | 2456 B | 102.33 |
52+
| NSubstitute | 7,862.23 ns | 31,442.13 ns | 1,723.45 ns | 299.91 | 110.04 | 0.3100 | - | 5872 B | 244.67 |
53+
| PCLMock | 3,256.73 ns | 13,948.61 ns | 764.57 ns | 124.13 | 45.95 | 0.1000 | - | 1912 B | 79.67 |
54+
| Rocks | 789.88 ns | 1,424.53 ns | 78.08 ns | 30.92 | 13.42 | 0.0300 | - | 680 B | 28.33 |
5555

5656
### EmptyMethod
5757

5858
A mock object is created, with no method setup, and a method with no return value is called
5959

60-
| Method | Mean | Error | StdDev | Ratio | RatioSD | Gen0 | Gen1 | Allocated | Alloc Ratio |
61-
|------------- |-------------:|-------------:|-------------:|---------:|---------:|-------:|-------:|----------:|------------:|
62-
| Stub | 23.31 ns | 162.18 ns | 8.890 ns | 1.00 | 0.00 | - | - | 24 B | 1.00 |
63-
| FakeItEasy | 4,747.24 ns | 23,120.39 ns | 1,267.307 ns | 221.09 | 79.83 | 0.2000 | 0.0100 | 3785 B | 157.71 |
64-
| JustMockLite | 50,786.94 ns | 35,234.99 ns | 1,931.349 ns | 2,466.02 | 1,137.15 | 1.1100 | 0.1300 | 20748 B | 864.50 |
65-
| Moq | 3,919.58 ns | 16,156.43 ns | 885.589 ns | 183.54 | 67.01 | 0.1300 | - | 2432 B | 101.33 |
66-
| NSubstitute | 6,622.10 ns | 29,619.79 ns | 1,623.561 ns | 308.77 | 109.90 | 0.3100 | - | 5872 B | 244.67 |
67-
| PCLMock | 2,819.66 ns | 15,792.48 ns | 865.639 ns | 129.57 | 42.44 | 0.1000 | - | 1872 B | 78.00 |
68-
| Rocks | 632.52 ns | 1,252.19 ns | 68.637 ns | 30.06 | 11.72 | 0.0300 | - | 672 B | 28.00 |
60+
| Method | Mean | Error | StdDev | Ratio | RatioSD | Gen0 | Gen1 | Allocated | Alloc Ratio |
61+
|------------- |-------------:|-------------:|------------:|---------:|---------:|-------:|-------:|----------:|------------:|
62+
| Stub | 28.84 ns | 206.64 ns | 11.33 ns | 1.00 | 0.00 | - | - | 24 B | 1.00 |
63+
| FakeItEasy | 6,006.92 ns | 27,867.68 ns | 1,527.52 ns | 226.54 | 78.82 | 0.2000 | 0.0100 | 3785 B | 157.71 |
64+
| JustMockLite | 60,569.59 ns | 41,286.68 ns | 2,263.06 ns | 2,397.36 | 1,142.45 | 1.1100 | 0.1300 | 20748 B | 864.50 |
65+
| Moq | 4,646.49 ns | 22,813.25 ns | 1,250.47 ns | 175.53 | 63.52 | 0.1300 | - | 2432 B | 101.33 |
66+
| NSubstitute | 6,980.76 ns | 31,131.66 ns | 1,706.43 ns | 266.23 | 103.37 | 0.3100 | - | 5872 B | 244.67 |
67+
| PCLMock | 3,090.80 ns | 15,101.93 ns | 827.79 ns | 116.08 | 39.15 | 0.1000 | - | 1872 B | 78.00 |
68+
| Rocks | 713.38 ns | 1,572.38 ns | 86.19 ns | 27.55 | 10.94 | 0.0300 | - | 672 B | 28.00 |
6969

7070
### OneParameter
7171

7272
A mock object is created, with no method setup, and a method with no return value requiring an `int` parameter is called
7373

74-
| Method | Mean | Error | StdDev | Ratio | RatioSD | Gen0 | Gen1 | Allocated | Alloc Ratio |
75-
|------------- |--------------:|-------------:|------------:|---------:|---------:|-------:|-------:|----------:|------------:|
76-
| Stub | 24.48 ns | 187.70 ns | 10.29 ns | 1.00 | 0.00 | - | - | 24 B | 1.00 |
77-
| FakeItEasy | 4,821.59 ns | 23,774.36 ns | 1,303.15 ns | 214.07 | 73.67 | 0.2000 | 0.0100 | 3922 B | 163.42 |
78-
| JustMockLite | 51,226.16 ns | 36,391.68 ns | 1,994.75 ns | 2,399.13 | 1,126.94 | 1.1300 | 0.1200 | 21220 B | 884.17 |
79-
| Moq | 4,068.67 ns | 17,542.87 ns | 961.58 ns | 181.91 | 64.61 | 0.1300 | - | 2464 B | 102.67 |
80-
| NSubstitute | 6,395.47 ns | 30,910.57 ns | 1,694.31 ns | 285.09 | 102.29 | 0.3100 | - | 5904 B | 246.00 |
81-
| PCLMock | 186,022.44 ns | 10,498.08 ns | 575.44 ns | 8,791.04 | 4,350.62 | 0.5600 | 0.5500 | 10584 B | 441.00 |
82-
| Rocks | 833.31 ns | 1,404.59 ns | 76.99 ns | 38.38 | 16.05 | 0.0400 | - | 792 B | 33.00 |
74+
| Method | Mean | Error | StdDev | Ratio | RatioSD | Gen0 | Gen1 | Allocated | Alloc Ratio |
75+
|------------- |--------------:|--------------:|------------:|---------:|---------:|-------:|-------:|----------:|------------:|
76+
| Stub | 28.64 ns | 202.43 ns | 11.10 ns | 1.00 | 0.00 | - | - | 24 B | 1.00 |
77+
| FakeItEasy | 6,006.30 ns | 29,564.27 ns | 1,620.52 ns | 228.04 | 82.64 | 0.2000 | 0.0100 | 3922 B | 163.42 |
78+
| JustMockLite | 60,026.49 ns | 42,608.40 ns | 2,335.51 ns | 2,380.53 | 1,110.35 | 1.1300 | 0.1200 | 21220 B | 884.17 |
79+
| Moq | 4,844.69 ns | 22,506.18 ns | 1,233.64 ns | 184.21 | 66.53 | 0.1300 | - | 2464 B | 102.67 |
80+
| NSubstitute | 7,445.89 ns | 32,069.41 ns | 1,757.83 ns | 283.07 | 99.70 | 0.3100 | - | 5904 B | 246.00 |
81+
| PCLMock | 234,568.49 ns | 114,493.45 ns | 6,275.77 ns | 9,373.17 | 4,580.23 | 0.5600 | 0.5500 | 10584 B | 441.00 |
82+
| Rocks | 936.82 ns | 5,065.72 ns | 277.67 ns | 33.92 | 4.93 | 0.0400 | - | 792 B | 33.00 |
8383

8484
### Callback
8585

8686
A mock object is created, a method is setup with a callback, and that method is called
8787

88-
| Method | Mean | Error | StdDev | Ratio | RatioSD | Gen0 | Gen1 | Allocated | Alloc Ratio |
89-
|------------- |--------------:|-------------:|-------------:|---------:|---------:|-------:|-------:|----------:|------------:|
90-
| Stub | 29.21 ns | 175.22 ns | 9.604 ns | 1.00 | 0.00 | - | - | 24 B | 1.00 |
91-
| FakeItEasy | 7,509.28 ns | 33,151.65 ns | 1,817.154 ns | 268.62 | 68.22 | 0.2700 | 0.0100 | 5167 B | 215.29 |
92-
| JustMockLite | 266,397.92 ns | 38,531.82 ns | 2,112.059 ns | 9,918.55 | 3,672.76 | 1.8300 | 1.8200 | 34286 B | 1,428.58 |
93-
| Moq | 6,984.42 ns | 25,295.49 ns | 1,386.531 ns | 251.79 | 67.96 | 0.2200 | - | 4112 B | 171.33 |
94-
| NSubstitute | 7,861.98 ns | 32,731.46 ns | 1,794.122 ns | 282.18 | 74.41 | 0.3600 | - | 6896 B | 287.33 |
95-
| PCLMock | 3,418.82 ns | 19,771.11 ns | 1,083.721 ns | 120.76 | 29.48 | 0.1100 | - | 2080 B | 86.67 |
96-
| Rocks | 668.16 ns | 1,300.93 ns | 71.309 ns | 24.37 | 7.08 | 0.0400 | - | 760 B | 31.67 |
88+
| Method | Mean | Error | StdDev | Ratio | RatioSD | Gen0 | Gen1 | Allocated | Alloc Ratio |
89+
|------------- |--------------:|-------------:|------------:|---------:|---------:|-------:|-------:|----------:|------------:|
90+
| Stub | 36.04 ns | 211.35 ns | 11.58 ns | 1.00 | 0.00 | - | - | 24 B | 1.00 |
91+
| FakeItEasy | 9,730.53 ns | 41,163.62 ns | 2,256.32 ns | 284.24 | 81.51 | 0.2700 | 0.0100 | 5167 B | 215.29 |
92+
| JustMockLite | 314,033.86 ns | 71,902.79 ns | 3,941.23 ns | 9,480.39 | 3,595.73 | 1.8300 | 1.8200 | 34287 B | 1,428.62 |
93+
| Moq | 7,799.30 ns | 25,730.45 ns | 1,410.37 ns | 229.49 | 68.42 | 0.2200 | - | 4112 B | 171.33 |
94+
| NSubstitute | 9,171.73 ns | 31,745.34 ns | 1,740.07 ns | 269.03 | 76.95 | 0.3600 | - | 6896 B | 287.33 |
95+
| PCLMock | 3,867.43 ns | 18,210.21 ns | 998.16 ns | 112.21 | 30.07 | 0.1100 | - | 2080 B | 86.67 |
96+
| Rocks | 777.72 ns | 1,593.74 ns | 87.36 ns | 22.97 | 6.67 | 0.0400 | - | 760 B | 31.67 |
9797

9898
### Verify
9999

100100
Verifies that the method was called
101101

102-
| Method | Mean | Error | StdDev | Ratio | RatioSD | Gen0 | Gen1 | Allocated | Alloc Ratio |
102+
| Method | Mean | Error | StdDev | Ratio | RatioSD | Gen0 | Gen1 | Allocated | Alloc Ratio |
103103
|------------- |--------------:|-------------:|------------:|---------:|---------:|-------:|-------:|----------:|------------:|
104-
| Stub | 33.18 ns | 205.56 ns | 11.27 ns | 1.00 | 0.00 | - | - | 24 B | 1.00 |
105-
| FakeItEasy | 7,960.76 ns | 32,460.79 ns | 1,779.29 ns | 252.69 | 69.50 | 0.2900 | 0.0100 | 5489 B | 228.71 |
106-
| JustMockLite | 112,853.39 ns | 36,112.41 ns | 1,979.44 ns | 3,713.43 | 1,403.99 | 2.1400 | 0.5300 | 40100 B | 1,670.83 |
107-
| Moq | 7,369.41 ns | 25,822.57 ns | 1,415.42 ns | 235.07 | 66.52 | 0.2100 | - | 4072 B | 169.67 |
108-
| NSubstitute | 9,264.27 ns | 36,165.94 ns | 1,982.38 ns | 294.23 | 80.02 | 0.3900 | - | 7464 B | 311.00 |
109-
| PCLMock | 3,666.99 ns | 18,823.35 ns | 1,031.77 ns | 114.95 | 28.37 | 0.1400 | - | 2736 B | 114.00 |
110-
| Rocks | 886.32 ns | 1,496.10 ns | 82.01 ns | 28.69 | 9.07 | 0.0400 | - | 800 B | 33.33 |
104+
| Stub | 33.64 ns | 201.12 ns | 11.02 ns | 1.00 | 0.00 | - | - | 24 B | 1.00 |
105+
| FakeItEasy | 9,186.06 ns | 40,565.27 ns | 2,223.52 ns | 287.89 | 84.23 | 0.2900 | 0.0100 | 5489 B | 228.71 |
106+
| JustMockLite | 134,882.54 ns | 39,368.39 ns | 2,157.91 ns | 4,375.22 | 1,685.94 | 2.1500 | 0.5300 | 40260 B | 1,677.50 |
107+
| Moq | 8,506.66 ns | 25,250.99 ns | 1,384.09 ns | 269.44 | 82.73 | 0.2100 | - | 4072 B | 169.67 |
108+
| NSubstitute | 10,575.58 ns | 38,199.33 ns | 2,093.83 ns | 333.07 | 97.84 | 0.3900 | - | 7464 B | 311.00 |
109+
| PCLMock | 4,121.90 ns | 16,560.56 ns | 907.74 ns | 129.45 | 37.61 | 0.1400 | - | 2736 B | 114.00 |
110+
| Rocks | 1,027.11 ns | 1,942.11 ns | 106.45 ns | 32.66 | 10.03 | 0.0400 | - | 800 B | 33.33 |
111111

0 commit comments

Comments
 (0)