Summary
pasls appears to interpret LSP Position.character as a byte offset into the
line, while the LSP specification defines it as a count of UTF-16 code units
(unless the server negotiates otherwise via positionEncoding).
pasls does not advertise positionEncoding in its initialize result, so per spec
the client is entitled to assume utf-16. The result is that textDocument/definition,
hover and textDocument/references silently return empty whenever any non-ASCII
character appears earlier on the same line as the identifier under the cursor.
There is no error — just no result — which makes it hard to notice.
Environment
pasls build 2026/03/14 (the prebuilt pasls-x86_64-win64 shipped in the
coolchyni.fpctoolkit 1.2.7 VS Code extension)
- Windows 11, FPC 3.2.2 + Lazarus 4.8 (
PP / FPCDIR / LAZARUSDIR all set)
- Driven directly over stdio by a minimal LSP client, so no editor is involved
Reproduction
Source line (a Delphi unit, CP950 on disk; mFileName is a unit-level var
declared on line 210). Six CJK characters precede the identifier:
WriteLog(FormatDateTime('yyyy/mm/dd hh:nn:ss', Now) + ' <6 CJK chars> ' + mFileName);
Two textDocument/definition requests at the same logical position:
character sent |
meaning |
result |
70 |
UTF-16 code units (what the spec requires, and what VS Code sends) |
null |
76 |
byte offset in the file's encoding |
correctly resolves to line 210 |
The same identifier resolves correctly on every pure-ASCII line in the same file
(5 of 5 occurrences), so parsing and symbol resolution themselves are fine.
Not an encoding issue
This is independent of the file's on-disk encoding. The same experiment on a
UTF-8 (with BOM) unit reproduces it: identifier xEXEC_RES, character 49
(UTF-16 units) returns null, while character 63 (UTF-8 byte offset) resolves
correctly. Reading non-UTF-8 sources itself works fine — documentSymbol returns
191 symbols with no mangled names on the CP950 file, presumably thanks to #116.
Impact
Any codebase with non-ASCII string literals or comments loses go-to-definition,
hover and find-references on a large share of its lines. In the file above, 1,276 of
8,560 lines contain non-ASCII characters. Clients cannot work around this, since
VS Code always sends UTF-16 offsets and no positionEncoding is offered.
Possible fixes
- Convert
Position.character from UTF-16 code units to a byte offset when mapping
an LSP position onto the source buffer (and convert back when emitting ranges), or
- advertise
positionEncoding: 'utf-8' in the initialize result and honour the
client's general.positionEncodings capability, so that clients supporting LSP
3.17 negotiation send byte offsets.
Option 1 is what the spec expects by default; option 2 is cheaper but only helps
clients that implement the negotiation.
Summary
paslsappears to interpret LSPPosition.characteras a byte offset into theline, while the LSP specification defines it as a count of UTF-16 code units
(unless the server negotiates otherwise via
positionEncoding).paslsdoes not advertisepositionEncodingin itsinitializeresult, so per specthe client is entitled to assume
utf-16. The result is thattextDocument/definition,hoverandtextDocument/referencessilently return empty whenever any non-ASCIIcharacter appears earlier on the same line as the identifier under the cursor.
There is no error — just no result — which makes it hard to notice.
Environment
paslsbuild2026/03/14(the prebuiltpasls-x86_64-win64shipped in thecoolchyni.fpctoolkit1.2.7 VS Code extension)PP/FPCDIR/LAZARUSDIRall set)Reproduction
Source line (a Delphi unit, CP950 on disk;
mFileNameis a unit-levelvardeclared on line 210). Six CJK characters precede the identifier:
Two
textDocument/definitionrequests at the same logical position:charactersent70null76The same identifier resolves correctly on every pure-ASCII line in the same file
(5 of 5 occurrences), so parsing and symbol resolution themselves are fine.
Not an encoding issue
This is independent of the file's on-disk encoding. The same experiment on a
UTF-8 (with BOM) unit reproduces it: identifier
xEXEC_RES,character49(UTF-16 units) returns
null, whilecharacter63 (UTF-8 byte offset) resolvescorrectly. Reading non-UTF-8 sources itself works fine —
documentSymbolreturns191 symbols with no mangled names on the CP950 file, presumably thanks to #116.
Impact
Any codebase with non-ASCII string literals or comments loses go-to-definition,
hover and find-references on a large share of its lines. In the file above, 1,276 of
8,560 lines contain non-ASCII characters. Clients cannot work around this, since
VS Code always sends UTF-16 offsets and no
positionEncodingis offered.Possible fixes
Position.characterfrom UTF-16 code units to a byte offset when mappingan LSP position onto the source buffer (and convert back when emitting ranges), or
positionEncoding: 'utf-8'in theinitializeresult and honour theclient's
general.positionEncodingscapability, so that clients supporting LSP3.17 negotiation send byte offsets.
Option 1 is what the spec expects by default; option 2 is cheaper but only helps
clients that implement the negotiation.