Skip to content

Commit 5c87f3b

Browse files
jensjohaCommit Queue
authored and
Commit Queue
committed
[analyzer] Add benchmark simulating 'flutter.setSubscriptions' and opening new files
This benchmark simulates the current bug at flutter/flutter-intellij#7980 Comparing this benchmark across dart versions also reveals something interesting, here run on the "ImportChain" type with 100 files: Comparing 3.5.4 with 3.6.2 ``` Initial analysis: -2.5724% +/- 1.5022% (-0.16 +/- 0.10) (6.35 -> 6.19) Completion after open of new file: -1.4714% +/- 1.0334% (-0.04 +/- 0.03) (2.92 -> 2.87) peak virtual memory size: 8.6445% +/- 2.4585% (220.40 +/- 62.68) (2549.60 -> 2770.00) total program size (virtual): 9.0756% +/- 2.4049% (226.20 +/- 59.94) (2492.40 -> 2718.60) peak resident set size ("high water mark"): -9.7940% +/- 1.6649% (-58.00 +/- 9.86) (592.20 -> 534.20) size of memory portions (rss): -8.7961% +/- 3.4971% (-47.20 +/- 18.77) (536.60 -> 489.40) ``` Comparing 3.6.2 with 3.7.2 ``` Initial analysis: -8.7696% +/- 2.2759% (-0.54 +/- 0.14) (6.19 -> 5.64) Completion without opening files: 16.5289% +/- 8.3591% (0.07 +/- 0.03) (0.41 -> 0.48) Completion after open of new file: 45.0913% +/- 2.8023% (1.30 +/- 0.08) (2.87 -> 4.17) getAssists call: 21.4736% +/- 2.3049% (0.61 +/- 0.07) (2.86 -> 3.48) peak virtual memory size: -5.9134% +/- 3.8164% (-163.80 +/- 105.72) (2770.00 -> 2606.20) total program size (virtual): -6.5548% +/- 4.0754% (-178.20 +/- 110.79) (2718.60 -> 2540.40) peak resident set size ("high water mark"): -16.3984% +/- 0.5460% (-87.60 +/- 2.92) (534.20 -> 446.60) size of memory portions (rss): -18.0629% +/- 3.2699% (-88.40 +/- 16.00) (489.40 -> 401.00) ``` Where, between 3.6.2 and 3.7.2 these stand out: ``` Completion without opening files: 16.5289% +/- 8.3591% (0.07 +/- 0.03) (0.41 -> 0.48) Completion after open of new file: 45.0913% +/- 2.8023% (1.30 +/- 0.08) (2.87 -> 4.17) getAssists call: 21.4736% +/- 2.3049% (0.61 +/- 0.07) (2.86 -> 3.48) ``` these surely weren't great before, but are much worse now. Change-Id: I6f94f941cda86b1aa9ee7e7a9b1912df85c7acb2 Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/417820 Commit-Queue: Jens Johansen <[email protected]> Reviewed-by: Brian Wilkerson <[email protected]>
1 parent daf1084 commit 5c87f3b

File tree

2 files changed

+175
-0
lines changed

2 files changed

+175
-0
lines changed
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,159 @@
1+
// Copyright (c) 2025, the Dart project authors. Please see the AUTHORS file
2+
// for details. All rights reserved. Use of this source code is governed by a
3+
// BSD-style license that can be found in the LICENSE file.
4+
5+
import '../language_server_benchmark.dart';
6+
import '../legacy_messages.dart';
7+
import 'utils.dart';
8+
9+
/// At least until "https://github.com/flutter/flutter-intellij/issues/7980" is
10+
/// fixed, when a user in IntelliJ in a Flutter project opens a file it's added
11+
/// to a list and never gets off that list. Whenever a file is added to the list
12+
/// (i.e. every time the user opens a file it hasn't opened before) a
13+
/// "flutter.setSubscriptions" request is sent, with every file on the list.
14+
/// At the same time a "edit.getAssists" request for the newly opened file is
15+
/// issued (as well as a few others).
16+
/// While processing the "edit.getAssists" request it (currently) has to wait
17+
/// for the processing of all the requested files from the
18+
/// "flutter.setSubscriptions" request, and any subsequent request is not
19+
/// handled before after the "edit.getAssists" is done processing.
20+
/// Said anther way: No user requests are handled before all the files have
21+
/// been processed.
22+
/// While the list will start out empty, opening many files in a session is a
23+
/// natural way of working so it will over type become many files and this
24+
/// benchmark is thus not completely unrealistic.
25+
Future<void> main(List<String> args) async {
26+
await runHelper(
27+
args,
28+
LegacyManyFilesInFlutterSetSubscriptionsBenchmark.new,
29+
runAsLsp: false,
30+
);
31+
}
32+
33+
class LegacyManyFilesInFlutterSetSubscriptionsBenchmark
34+
extends DartLanguageServerBenchmark {
35+
@override
36+
final Uri rootUri;
37+
@override
38+
final Uri cacheFolder;
39+
40+
final RunDetails runDetails;
41+
42+
LegacyManyFilesInFlutterSetSubscriptionsBenchmark(
43+
super.args,
44+
this.rootUri,
45+
this.cacheFolder,
46+
this.runDetails,
47+
) : super(useLspProtocol: false);
48+
49+
@override
50+
LaunchFrom get launchFrom => LaunchFrom.Dart;
51+
52+
@override
53+
Future<void> afterInitialization() async {
54+
// The main file is an open tab.
55+
await send(
56+
LegacyMessages.setPriorityFiles(largestIdSeen + 1, [
57+
runDetails.mainFile.uri,
58+
]),
59+
);
60+
61+
// Does this change anything?
62+
await Future.delayed(const Duration(milliseconds: 100));
63+
64+
// The user starts typing - types 'ge' in the empty line in main...
65+
await send(
66+
LegacyMessages.updateContent(
67+
largestIdSeen + 1,
68+
runDetails.mainFile.uri,
69+
runDetails.mainFileTypingContent,
70+
),
71+
);
72+
73+
// ...and ask for completion.
74+
var firstCompletionStopwatch = Stopwatch()..start();
75+
var completion1Result =
76+
await (await send(
77+
LegacyMessages.getSuggestions2(
78+
largestIdSeen + 1,
79+
runDetails.mainFile.uri,
80+
runDetails.typingAtOffset,
81+
),
82+
))!.completer.future;
83+
firstCompletionStopwatch.stop();
84+
List<dynamic> completion1Items =
85+
completion1Result['result']['suggestions'] as List;
86+
durationInfo.add(
87+
DurationInfo(
88+
'Completion without opening files',
89+
firstCompletionStopwatch.elapsed,
90+
),
91+
);
92+
if (verbosity >= 0) {
93+
print(
94+
'Got ${completion1Items.length} completion items '
95+
'in ${firstCompletionStopwatch.elapsed}',
96+
);
97+
}
98+
99+
// The user wants to check something. Pretending we've already opened all
100+
// (other) files previously, we're now opening the last one.
101+
await send(
102+
LegacyMessages.getFlutterSetSubscriptions(
103+
largestIdSeen + 1,
104+
runDetails.orderedFileCopies.map((copy) => copy.uri).toList(),
105+
),
106+
);
107+
108+
// So now we have both the main file and the last file open in tabs.
109+
await send(
110+
LegacyMessages.setPriorityFiles(largestIdSeen + 1, [
111+
runDetails.mainFile.uri,
112+
runDetails.orderedFileCopies.last.uri,
113+
]),
114+
);
115+
116+
// The IDE sends a "edit.getAssists" request for the newly opened file at
117+
// offset 0 length 0.
118+
var getAssistsStopwatch = Stopwatch()..start();
119+
var getAssistsFuture = (await send(
120+
LegacyMessages.getAssists(
121+
largestIdSeen + 1,
122+
runDetails.orderedFileCopies.last.uri,
123+
0,
124+
),
125+
))!.completer.future.then((_) => getAssistsStopwatch.stop());
126+
127+
// The user tabs back into the main file and ask for completion again.
128+
Future<Map<String, dynamic>> completionFuture =
129+
(await send(
130+
LegacyMessages.getSuggestions2(
131+
largestIdSeen + 1,
132+
runDetails.mainFile.uri,
133+
runDetails.typingAtOffset,
134+
),
135+
))!.completer.future;
136+
137+
Stopwatch stopwatch = Stopwatch()..start();
138+
var completionResponse = await completionFuture;
139+
List<dynamic> completionItems =
140+
completionResponse['result']['suggestions'] as List;
141+
var completionAfterChange = stopwatch.elapsed;
142+
durationInfo.add(
143+
DurationInfo('Completion after open of new file', completionAfterChange),
144+
);
145+
if (verbosity >= 0) {
146+
print(
147+
'Got ${completionItems.length} completion items '
148+
'in $completionAfterChange',
149+
);
150+
}
151+
152+
// This should be complete already.
153+
await getAssistsFuture;
154+
155+
durationInfo.add(
156+
DurationInfo('getAssists call', getAssistsStopwatch.elapsed),
157+
);
158+
}
159+
}

pkg/analysis_server/tool/benchmark_tools/legacy_messages.dart

+16
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,22 @@ class LegacyMessages {
3232
};
3333
}
3434

35+
static Map<String, dynamic> getFlutterSetSubscriptions(
36+
int id,
37+
List<Uri> files,
38+
) {
39+
return {
40+
'id': '$id',
41+
'method': 'flutter.setSubscriptions',
42+
'params': {
43+
'subscriptions': {
44+
'OUTLINE': [...files.map(_uriToStringWithoutEndingSlash)],
45+
},
46+
// no 'clientRequestTime' on this one it seems.
47+
},
48+
};
49+
}
50+
3551
static Map<String, dynamic> getHover(int id, Uri file, int offset) {
3652
return {
3753
'id': '$id',

0 commit comments

Comments
 (0)