Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expanded Python syntax file #3609

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Expanded Python syntax file #3609

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
b3f7a25
Expand highlight cases
EllKyGr Jan 14, 2025
899d9db
Color schemes matching expanded python3.yaml
EllKyGr Jan 14, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 54 additions & 0 deletions runtime/colorschemes/catppuccin-mocha_py.micro
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
# Custom color scheme inspired by Catppuccin - Mocha from "https://catppuccin.com/"
# Language related syntax
color-link default "#cdd6f4,#1e1e2e"
color-link comment "italic #777777"
color-link constant "bold #fab387,#1e1e2e" # None, self, Ellipse, cls, default for True and False*
color-link constant.bool "bold #f9e2af,#1e1e2e" # True && False
color-link constant.number "bold #fab387" # New: Float and int
color-link constant.specialChar "italic #f5c2e7" # New: Escape characters and {} inside string
color-link constant.string "#a6e3a1,#1e1e2e" # Single or double quotes, and docstring
color-link constant.string.char "#6A8759,#1e1e2e" # unknown
color-link constant.string.url "italic #777777" # New: does not work*
color-link identifier "italic #f38ba8,#1e1e2e" # built in functions, def functions and function calls with or without dot
color-link identifier.var "bold #cdd6f4" # New: Variables, attributes, placeholders*
color-link identifier.class "bold #f9e2af" # New: As stated and few more*
color-link message "bold #f38ba8,#181825" # As intended both for errors and some others
color-link preproc "italic #89b4fa,#1e1e2e" # Statement preceded by an @
color-link statement "#cba6f7,#1e1e2e" # Several keywords*
color-link symbol "bold #CCCCCC,#1e1e2e" # Default color
color-link type "#f9e2af,#1e1e2e" # Type hint in both inside and outisde of functions
color-link type.keyword "#f9e2af,#1e1e2e" # Built-in types*
color-link symbol "bold #f38ba8" # New*
color-link symbol.brackets "bold #9399b2" # New*
color-link symbol.operator "bold #89dceb" # New*
color-link special "#74c7ec,#1e1e2e" # unknown
color-link underlined "#D33682,#1e1e2e" # unknown
color-link error "bold #CB4B16,#1e1e2e" # unknown
color-link todo "bold #8aadf4,#1e1e2e" # NOTE TODO*

# Micro appearance
color-link hlsearch "#11111b,#89dceb"
color-link statusline "#89dceb,#313244"
color-link tabbar "#242424,#CCCCCC"
color-link indent-char "#4F4F4F,#1e1e2e"
color-link line-number "#7f849c,#313244"
color-link current-line-number "#b4befe,#1e1e2e"
# 37, 38, 39 affected if "diffgutter" is set to true in "settings.json"
color-link diff-added "#a6e3a1"
color-link diff-modified "#89b4fa"
color-link diff-deleted "#f38ba8"
color-link gutter-error "#f38ba8,#1e1e2e" # >> language error
color-link gutter-warning "#f9e2af,#1e1e2e" # >> i.e. spell checking
color-link cursor-line "#45475a"
color-link color-column "#45475a"
color-link divider "#777777" # New
color-link selection "#2C2C2C,#777777" # New

# No extended types; Plain brackets.
color-link type.extended "default"
color-link symbol.tag "#AE381FF,#1e1e2e"
color-link match-brace "#1e1e2e,#7A9EC2"
color-link tab-error "#D75F5F"
color-link trailingws "#bac2de"

# *NOTE* review the python3.yaml for more detail
52 changes: 52 additions & 0 deletions runtime/colorschemes/darcula_py.micro
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# Language related syntax
color-link default "#A5B09E,#242424"
color-link comment "italic #777777"
color-link constant "bold #007364,#242424" # None, self, Ellipse, cls, default for True and False*
color-link constant.bool "bold #CB4B16,#242424" # True && False
color-link constant.number "bold #005A89" # New: Float and int
color-link constant.specialChar "italic #BF723F" # New: Escape characters and {} inside string
color-link constant.string "#6A8759,#242424" # Single or double quotes, and docstring
color-link constant.string.char "#6A8759,#242424" # unknown
color-link constant.string.url "italic #777777" # New: does not work*
color-link identifier "italic #FFC66D,#242424" # built in functions, def functions and function calls with or without dot
color-link identifier.var "bold #A5B09E" # New: Variables, attributes, placeholders*
color-link identifier.class "bold #007364" # New: As stated and few more*
color-link message "bold #D33682,#C242424" # As intended both for errors and some others
color-link preproc "italic #3F8CBF,#242424" # Statement preceded by an @
color-link statement "#915010,#242424" # Several keywords*
color-link symbol "bold #CCCCCC,#242424" # Default color
color-link type "#817CCF,#242424" # Type hint in both inside and outisde of functions
color-link type.keyword "#817CCF,#242424" # Built-in types*
color-link symbol.brackets "bold #AF7B27" # New*
color-link symbol.operator "bold #BF723F" # New*
color-link special "#CC8242,#242424" # unknown
color-link underlined "#D33682,#242424" # unknown
color-link error "bold #CB4B16,#242424" # unknown
color-link todo "bold #D33682,#242424" # NOTE TODO*

# Micro appearance
color-link hlsearch "#CCCCCC,#32593D"
color-link statusline "#FFC66D,#2C2C2C"
color-link tabbar "#242424,#CCCCCC"
color-link indent-char "#4F4F4F,#242424"
color-link line-number "#977000,#2C2C2C"
color-link current-line-number "#FFC66D,#242424"
# 36, 37, 38 affected if "diffgutter" is set to true in "settings.json"
color-link diff-added "#00AF00"
color-link diff-modified "#FFAF00"
color-link diff-deleted "#D70000"
color-link gutter-error "#D33682,#242424" # >> language error
color-link gutter-warning "#36D387,#242424" # >> i.e. spell checking
color-link cursor-line "#2C2C2C"
color-link color-column "#2C2C2C"
color-link divider "#777777" # New
color-link selection "#2C2C2C,#777777" # New

# No extended types; Plain brackets.
color-link type.extended "default"
color-link symbol.tag "#AE81FF,#242424"
color-link match-brace "#242424,#7A9EC2"
color-link tab-error "#D75F5F"
color-link trailingws "#777777"

# *NOTE* review the python3.yaml for more detail
52 changes: 52 additions & 0 deletions runtime/colorschemes/monokai_py.micro
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# Language related syntax
color-link default "#FF9366,#333333"
color-link comment "#756F52,#333333"
color-link constant "bold #7D7075,#333333"
color-link constant.bool "bold #005A89,#333333" # True && False
color-link constant.number "bold #817CCF"
color-link constant.specialChar "italic #747FE6" # New: Escape characters and {} inside string
color-link constant.string "#FFCA7A,#333333"
color-link constant.string.char "#BDE6AD,#333333"
color-link constant.string.url "italic #777777" # New: does not work*
color-link identifier "#FF9366,#333333"
color-link identifier.var "bold #66D9EF" # New: Variables, attributes, placeholders*
color-link identifier.class "bold #AE81FF" # New: As stated and few more*
color-link message "bold #00C799,#C242424" # As intended both for errors and some others
color-link preproc "italic #FFB5FA,#333333" # Statement preceded by an @
color-link statement "#F92672,#333333"
color-link symbol "bold #CCCCCC,#333333" # Default color
color-link type "#007364,#333333" # Type hint in both inside and outisde of functions
color-link type.keyword "#007364,#333333" # Built-in types*
color-link symbol.brackets "bold #B8009A" # New*
color-link symbol.operator "bold #F92672,#333333"
color-link special "#A6E22E,#333333"
color-link underlined "#D33682,#333333"
color-link error "bold #CB4B16,#333333"
color-link todo "bold #D33682,#333333"

# Micro appearance
color-link hlsearch "#777777,#FFCA7A"
color-link statusline "#FFC66D,#3E3E3E"
color-link tabbar "#242424,#CCCCCC"
color-link indent-char "#4F4F4F,#333333"
color-link line-number "#977000,#2C2C2C"
color-link current-line-number "#00C799,#333333"
# 36, 37, 38 affected if "diffgutter" is set to true in "settings.json"
color-link diff-added "#00AF00"
color-link diff-modified "#FFAF00"
color-link diff-deleted "#D70000"
color-link gutter-error "#66D2FF,#333333" # >> language error
color-link gutter-warning "#A5B09E,#333333" # >> i.e. spell checking
color-link cursor-line "#3E3E3E"
color-link color-column "#3E3E3E"
color-link divider "#777777" # New
color-link selection "#2C2C2C,#777777" # New

# No extended types; Plain brackets.
color-link type.extended "default"
color-link symbol.tag "#AE81FF,#333333"
color-link match-brace "#333333,#AE81FF"
color-link tab-error "#D75F5F"
color-link trailingws "#777777"

# *NOTE* review the python3.yaml for more detail
43 changes: 39 additions & 4 deletions runtime/syntax/python3.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,22 +5,50 @@ detect:
header: "^#!.*/(env +)?python(3)?$"

rules:
# built-in objects
- constant: "\\b(Ellipsis|None|self|cls|True|False)\\b"
# built-in attributes
- constant: "\\b(__bases__|__builtin__|__class__|__debug__|__dict__|__doc__|__file__|__members__|__methods__|__name__|__self__)\\b"
# built-in functions
- identifier: "\\b(abs|all|any|ascii|bin|bool|breakpoint|bytearray|bytes|callable|chr|classmethod|compile|complex|delattr|dir|divmod|eval|exec|format|getattr|globals|hasattr|hash|help|hex|id|input|isinstance|issubclass|iter|len|locals|max|min|next|nonlocal|oct|open|ord|pow|print|repr|round|setattr|sorted|sum|vars|__import__)\\b"
- identifier: "\\b(classmethod|enumerate|filter|map|object|property|reversed|slice|staticmethod|super|type|zip)\\b" # New one, previously "type"
# special method names
- identifier: "\\b__(abs|add|and|call|cmp|coerce|complex|concat|contains|delattr|delitem|delslice|del|dict|divmod|div|float|getattr|getitem|getslice|hash|hex|iadd|iand|iconcat|ifloordiv|ilshift|imatmul|imod|imul|init|int|invert|inv|ior|ipow|irshift|isub|iter|itruediv|ixor|len|long|lshift|mod|mul|neg|next|nonzero|oct|or|pos|pow|radd|rand|rcmp|rdivmod|rdiv|repeat|repr|rlshift|rmod|rmul|ror|rpow|rrshift|rshift|rsub|rxor|setattr|setitem|setslice|str|sub|xor)__\\b"
# types
- type: "\\b(bool|bytearray|bytes|classmethod|complex|dict|enumerate|filter|float|frozenset|int|list|map|memoryview|object|property|range|reversed|set|slice|staticmethod|str|super|tuple|type|zip)\\b"
# definitions
- identifier: "def [a-zA-Z_0-9]+"
# Basic variable assign
- identifier.var: "((^|\\s+|\\()[\\w]+(\\s|,)?)+=" # New one
# Variable assignment to another variable
- identifier.var: "=\\s?[\\w]+" # New one
# Variables followed by ':' (for typing), '.' operand (call a function i.e. dict.items()) and as placeholders inside 'for' loop
- identifier.var: "[\\w]+((\\.|\\s)\\b|: )" # New one
# Variables stated after specific keywords
- identifier.var: "(return|while|for|if|and|or|in)\\s[a-zA-Z_0-9]+\\s?" # New one
# Variables followed or preceded by comparative operands.
- identifier.var: "[a-zA-Z0-9_]+\\s(<|>|<=|>=|!=|==)\\s([a-zA-Z0-9_]+)?:?$" # New one
# Variables followed/preceded by normal or compound assignment
- identifier.var: "(\\+|\\-|\\/|\\*)\\s\\w+" # New one
- identifier.var: "(^|\\s+|\\()\\w+\\s(\\+|\\-|\\/|\\*)?=?" # New one
# Variables inside any type of brackets. NOTE: variables with anything but integers or float, won't work.
# Dictionaries highlight variables as 'types' (because of the ':') instead of a regular variable
- identifier.var: "(\\(|\\[|{)([\\w]+|[\\d_]+(\\.[\\d_])*)((,|:)\\s([\\w]+|[\\d_]+(\\.[\\d_])*)?){0,},?(\\)|\\]|})\\s?(:|=)?" # New one
# Iterator slicing and key assignment
- identifier.var: "\\b[\\w]+(\\[[\\w:.\\(\\)\\s/*-+]+\\])" # New one
# Custom function call
- identifier: "[\\w]+\\(" # New one
# Functions called with dot operand
- identifier: "\\.[a-z]+\\(?" # New one
# Import, class and error statements
- identifier.class: "(import|from|as|class|except)\\s[\\w]+((,\\s[\\w]+)+|:)?" # New one
Comment on lines +17 to +40
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't really understand why you want to combine highlighting identifiers with other things like keywords and operators. What purpose does it serve?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because there are not enough groups and subgroups to distinguish one from another. Even if I only addressed the basic x = 5, if you want to use x in a while loop for instance, the variable itself won't be highlighted as you would expect as in pretty much any text editor with this capacities.
Also those keywords and operators won't be highlighted only what follows them, specifically variables (neither numbers nor strings are affected) but I had to add them inside the regex to actually work.
It doesn't look particularly pretty repeating several times identifier.var but it is clear to me that the editor could actually handle more specific subgroups to avoid these inconveniences.

# keywords
- statement: "\\b(and|as|assert|async|await|break|case|class|continue|def|del|elif|else|except|finally|for|from|global|if|import|in|is|lambda|match|nonlocal|not|or|pass|raise|return|try|while|with|yield)\\b"
# decorators
- preproc: "^\\s*@[^(]*"
# From `typing` module and other modules
- type: "(:|->)\\s[a-zA-Z]+(\\[[a-zA-Z]+(,\\s[a-zA-Z]+)?\\])?\\s?:?" # New one
Comment on lines +45 to +46
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Attempting to syntax highlight type annotations specifically is IMO ill-advised, because Python does not have a clear separation of types and runtime variables. The types may be arbitrarily complicated expressions like dict[tuple[str, int], Callable[["MyType", int], float] | None].
The : also conflicts with a bunch of unrelated syntax like if x: y.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Attempting to syntax highlight type annotations specifically is IMO ill-advised, because Python does not have a clear separation of types and runtime variables.

Can you elaborate how highlighting types affects Python's runtime variables?

The types may be arbitrarily complicated expressions like dict[tuple[str, int], Callable[["MyType", int], float] | None].

The attached screenshot shows how types are handled with both the extended syntax file and the darcula_py.micro. As far as I can see it works as intended but I suppose there could be even more complicated
typings.

Types

The : also conflicts with a bunch of unrelated syntax like if x: y.

Fair enough although that falls under the umbrella of good practices, project rules and the use of compliant PEP formatters, case in point the autofmt plugin uses yapf for Python and if you write down that statement it will immediately redacted to the "correct" format.

if x:
    y

Which won't cause the highlight conflict

# Types in basic library (no modules imported)
- type.keyword: "\\b(str|int|float|complex|list|tuple|range|dict|set|frozenset|bool|bytes|bytearray|memoryview|NoneType)\\b" # New one
# built-in objects. NOTE: Changed to this location to take precedence
- constant: "\\b(Ellipsis|None|self|cls|True|False)\\b"
- constant.bool: "\\b(True|False)\\b" # New one
# operators
- symbol.operator: "([~^.:;,+*|=!\\%@]|<|>|/|-|&)"
# parentheses
Expand All @@ -45,13 +73,20 @@ rules:
skip: "\\\\."
rules:
- constant.specialChar: "\\\\."
# f-string format
- constant.specialChar: "\\{[\\w\\d/*-+.\\s\\(\\)]*?\\}" # New one
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This only works for tiny subset of f-string syntax (you may want to read the comments in #3605). Also I think it doesn't make sense to highlight the format groups in regular strings.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll take a look at it


- constant.string:
start: "'"
end: "('|$)"
skip: "\\\\."
rules:
- constant.specialChar: "\\\\."
# f-string format
- constant.specialChar: "\\{[\\w\\d/*-+.\\s\\(\\)]*?\\}" # New one

# NOTE: Requires unique symbols %&#... Currently not working presumably missing specific rules
- constant.string.url: "(http(s)?:\\/\\/w{3}\\.)?[\\w]+(\\.[a-z]{,3})+((\\/[\\w]+){1,})?" # New one
Comment on lines +88 to +89
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I personally would not want to have URLs syntax highlighted as they are not part of Python syntax.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe, though I never implied that was such the case. Rather since there's, apparently, a color group for the urls I assumed some people might actually want to see them highlighted. Otherwise what's the point of the subgroup then?


- comment:
start: "#"
Expand Down