Release 2.0.0 ⚠️ Help Needed! ⚠️

[christopherpickering]

Feature Request

From #167

Using similar logic to linter rule H025 we can indent the templates using blocks instead of going linearly through the code line by line.

This will also match the settings using the compressor.

This will also allow us to use {% blocktrans trimmed %} tags, and format them.

It will also fix poor formatting like this:

{% if %}
  <div>
    {% endif %}
    {% if %}
   </div>
{% endif %}

[christopherpickering]

This is started on branch https://github.com/Riverside-Healthcare/djLint/tree/beast_mode. Basically it will used modified version of pythons html module that is expanded to handle template tags.

If you are familiar with python your help would be welcome! Please reach out if you'd like to build on this branch!

[sveetch]

Hi, what is the status of this ? I thought about some feature requests but i think your big refactoring is probably the highest priority so maybe i could find some time to assist about Python things or to tests. I tried to join the discord to talk about it but it does not work so i'm coming here.

Lastly i created a tool to lint/fix class attribute value that i run just after djLint. I've used the Django template lexer to parse templates without template tag (i'm isolating them out and restore them after the lint, it's a little bit tricky) since they can contain quotes that could break my regex. Although it works pretty well, i assume this could not be efficient as your parser when parsing the whole template and not just attributes.

[christopherpickering]

Thanks, just sent you an email.

[christopherpickering]

Update

I'm making pretty decent progress on the next major version of djLint. Some tests are stating to pass well. I will push updates in a few days. Thanks to @sveetch and others who have helped along the way so far. The formatting method of djlint is completly changing. Below there is a mini diagram showing a basic overview of how formatting is done.

Inspiration for the new method comes from a few places.

• parser (python package html-template-parser and parser.py) > python 3 htmllib
• tree (builder.py) > older version of python bs4 html tree builder
• formatting instructions (tag.py) > prettier's html printer
• file writer (writer.py) > bs4's printer and prettier's writer

Thanks to them for thinking through most of this for us :)

There are a few main differences between djLint's new formatting and traditional formatters.

• djLint will not "autofix" missing html tags.
• nested quotes in attribute values will not be escaped.

Planned changes

• Support C# Razor Pages
• Support more handlebars syntax
• Fast (closes [BUG] djlint --check/--reformat takes 21 seconds on long file. Increase speed? #104). currently takes almost 50% less time to format a file.
• Support formatting of nunjuck SET w/ loose dict style values (closes [BUG] [Formatter] object/array variable whitespace is removed #370, closes [BUG] [Formatter] Whitespaces in Jinja Code #287)
• Support nested if (or other template stuff) statements in html attributes (closes [BUG] [Formatter] Nested if statements in attribute values #443)
• Changes attribute indentation to use tabbed indent similiar to prettiers syntax. Sad to switch but easier in the long run. (closes [FEATURE] [Formatter] Add option for attributes to be indented with std indent vs hanging indent #335)
• Preserve intended and unintended spaces around html tags and template tags. HTML tags will now auto wrap to keep "no space" as well. (closes [BUG] [Formatter] Meaningful whitespace added around filter tag #165, closes [BUG] [Formatter] Refactor inline conditions #182)
• Fixes extra space inside of template syntax... multi spaces consolidated. (closes [FEATURE] Small formatting/linting Jinja issues #262, closes [BUG] [Formatter] Whitespaces in Jinja Code #287)
• Fix quotes inside attributes. Nested quotes will never be escaped like other formaters, but here we assume the dev knows what they are doing and we will alternate quotes between double and single as nesting gets deeper. This is for supporting alpinejs and other js tools. (closes [FEATURE] Small formatting/linting Jinja issues #262)
• Adding an option to disable auto closing void tags. default will close them. (Closes [FEATURE]Is it possible to add a function to fix H017 automatically? #252)

Old Formatting Method:

Used the regex package for all formatting.

graph TD;
   id1(Whitespace is removed where appropriate)-->id2(Whitespace is added where appropriate)
   id2(Whitespace is added where appropriate)-->id3(Short lines consolidated back to a single line)
   id3(Short lines consolidated back to a single line)-->id4(Lines are indented where appropriate);
   id4(Lines are indented where appropriate)-->id5(Attributes are formatted);

Loading

New Formatting Method:

Uses tokens.

graph TD;
   id1(html parsed into elements `parser.py`)-->id2(elements built into a tree `builder.py`)
   id2-->id3(elements in tree are built into a nested list with formatting instructions `tag.py`)
   id3-->id4(new list is written back to file `writer.py`)

Loading

[christopherpickering]

How Can You Help

The top priority is to pass the html formatting tests. Once those are in place the template syntax can be fairly easily added.

1. checkout the beast_mode branch git checkout beast_mode
2. install poetry install
3. run the html tests and find one that fails poetry run pytest tests/test_html/
4. try to fix it! Make sure the other tests don't break ;)

To make development easier, you can create a file called badtest.html in the /test directory. This file will be excluded from git. Add the failing html into that file and then you can cd into the src directory and run djlint with poetry run python -m djlint ../tests/badtest.html --check

[dannypernik]

• we will alternate quotes between double and single as nesting gets deeper.

Will it be possible for the current code to determine whether the top level is single or double quotes? In HTML, I use double-quotes on the top level and single quotes inside. Currently djLint turns the single quotes to double, which VSCode doesn't like:

Then with JS, I use single quotes. If I had to pick one over the other for all files, I'd choose single quotes for the top level. But it would be amazing if it could be made flexible given the file in question!

I wish I could help! I'll check out the docs but doubt I'll be much use.

[christopherpickering]

@dannypernik thanks, can you open a new issue for this? We can probably swap them to single quotes. We use the json5 lib for that.

[dannypernik]

@dannypernik thanks, can you open a new issue for this? We can probably swap them to single quotes. We use the json5 lib for that.

Done. Thank you!

[oliverhaas]

@christopherpickering Can you give a quick update on what the status is and what needs to be done? I would be both open to contributing myself, or sponsoring somebody to get some of the changes mentioned here going (and some other changes/features I need for me work specifically).

I've just tried to get the beast_mode branch running, but there seem to be some issues with the dependencies I couldn't figure out just now. Since the branch is rather old by now, I'm not sure if I should rather migrate the changes to the current master, rather than getting the outdated branch working again.

[krystofbe]

he is not working as a developer anymore, so i don't think we'll see any progress here

https://github.com/christopherpickering

Thanks for visiting. I'm no longer a full time developer.

Reach out if you are interested in becoming owner/maintainer of any of my projects.

[oliverhaas]

Yeah, I actually knew that. I hope that he still finds the time to answer in a few sentences. Otherwise I guess I'll fork and see where this goes.

[krystofbe]

Yeah, I actually knew that. I hope that he still finds the time to answer in a few sentences. Otherwise I guess I'll fork and see where this goes.

I tried to do the same today and got the branch running. Unfortunately 90% of the tests are failing so I guess there is still a lot of work to be done.

[christopherpickering]

Hi @oliverhaas & @krystofbe, sorry I missed this :) Thanks for your contributions here, and to @monosans for keeping the project going.

Basically the goal of the rebuild was to make a template parser (basically an expanded version of pythons html parser), which parses the templates much faster than regex, and then rebuild the parsed stuff in a very similar way to what prettier does for javascript/etc.

These packages were being used there, but are probably in need of an update now:

https://github.com/djlint/html-void-elements
https://github.com/djlint/html-element-attributes
https://github.com/djlint/html-tag-names
https://github.com/djlint/html-styles

These could potentially be merged into one package, but I think keeping them separate lets other people take advantage of the features of the individual packages w/out installing the full djlint.

Doing this should also simplify the djlint code since it is split up into several sub packages.

Yeah, you're right, like 90% of the tests were failing, I was basically trying to work through one test group at a time, the smallest tests to the biggest. The finally formatting of the new method will be slightly different than djlint currently does, so some test output expectations should also be updated.
Unfortunately, I don't have time to chip in other than checking back to see whats going on from time to time :)