[J4] Drop ES5 in favor of delivering ES6 only

[krileon]

With the move to Bootstrap 5 IE support is gone (BS5 does not support IE) and the official stance on IE is that it isn't supported either. I don't think Joomla should be shipping with ES5 files and should be using ES6.

ES5 from ES6 generates larger files, can't use native module support, can't safely scope your code using let/const, can't use arrow functions, can't use native Promises, can't use fetch, and more. Shipping with ES6 would speed up Joomla even more just by dropping support for browsers that are already not supported. Is there still a reason Joomla 4 needs to still ship with ES5?

[Fedik]

can't use native module support

Who said that? js module can contain ES5 code.

But I agree, it will simplify core development (js part) in general.

[Fedik]

Use of a modules is not depend from which code "style" you serve, so it cannot be a point here. That what I wanted to say.

[krileon]

Sorry, don't understand what you mean. The below is what I am referring to.

1. https://caniuse.com/?search=modules
2. https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/import

Native support for modules only exists as part of ES6. Modules in ES5 are done by other means and are not the same.

[Fedik]

It a bit off topic, but this ES5 code "module":

<script type="module">
function foobBar() {alert('It works!');}
foobBar();
</script>

Delivering ES5 does not block you from using "module".

[krileon]

You're mixing an ES6 module script with ES5 code. That makes no sense and will not work in IE11. You'd need a clone of that with nomodule. Nobody would do that so I don't understand your argument here. Anyway this is getting off topic.

[Fedik]

You're mixing an ES6 module script with ES5 code.

Well, that actually is your topic, about ES5/6 code and module, that not related 😄
So now you see my point? 😉

Okay, nevermind

[dgrammatiko]

It's not that easy. Attaching an attribute type=module without knowing you also defer the execution of that script. Most DEVs still use ES5 in their product (and honestly I don't see them moving to ES20XX before the EOL of J3). My point is mixing type=module with script tags without defer (you can't defer inline scripts) is gonna be catastrophic.

FWIW #31485 was attempt shipping modules and not breaking anything

[krileon]

@wilsonge Depends on your target browser and if you need to transpile edge ES6 features depending on browser availability. It's fine to transpile, but we don't need to transpile it to ES5 as we're not targeting any browsers that don't support ES6.

[wilsonge]

My main aim was to allow frontend developers to build applications that were IE compatible for frontend template - even if core ships the es6 code by default (i don't think we can realistically expect template people to do it themselves as overrides). However it was easier to transpile everything rather than just the frontend assets.

[dgrammatiko]

If this is still the aim I need to patch the PR to allow the polyfills loaded (only for IE). This is where the type=module/nomodule shines, it's the progressive enhancement that provides that is invaluable...

[krileon]

I completely understand the decision at the time it was made, but it just doesn't make sense for the current situation. Frontend and backend don't support IE11 with BS5. It doesn't make sense to step backwards either for template developers that don't want to move forward. We need to think of the future and not the past.

IE11 is officially dead in August of this year. Microsoft themselves is also not even going to support IE11 in their own products in August. Doesn't make any sense to target it.

[beat]

Joomla 3 will still be around during its maintenance time (and well after) that for people stuck in the past with IE11. People stuck with IE11 can postpone their Joomla 4 upgrade. Even new projects that need IE11 support can be deployed with Joomla 3, then upgrade to Joomla 4 when they can drop IE11. Thus Joomla 4 really shouldn't bother with officially unmaintained (as of August this year) IE11 imho.

[dgrammatiko]

All it takes is editing either the programmatic way or the JSON files for all the resources. For most of Joomla's file this should be enough

I'm really concerned Joomla 4 is going to be releasing with out of date code and concepts right out of the gate.

Are you willing to do the job in hand or just asking someone else to do it for you? If so, let's talk, if not I'm done here

[krileon]

All it takes is editing either the programmatic way or the JSON files for all the resources. For most of Joomla's file this should be enough

This was already suggested below.

#31485 (comment)

Which you directly said is too late.

#31485 (comment)

Is this still something too late to be doing? The nomodule fallback isn't necessary unless we're using latest ES6 features not available on modern browsers in which case it'd be better to just polyfill. For example Chrome has supported ES6 modules natively since 2017. Only reason to be doing nomodule trick these days is if you need to support IE11, which Joomla 4 does not.

Are you willing to do the job in hand or just asking someone else to do it for you? If so, let's talk, if not I'm done here

The initial point of this discussion was to see if there was even any interest in this. I'm one of the lead developers of Community Builder so my time is extremely limited and wasn't about to dump it into a PR that'd just be rejected.

[beat]

I think this above is a serious and valid concern from @krileon that deserves some thought:

I'm really concerned Joomla 4 is going to be releasing with out of date code and concepts right out of the gate. We want modern developers to pick Joomla and provide them with a modern framework. ES5 and jQuery is not modern. All modern frameworks have dropped such practices with Babel transpiling no longer the default behavior. I want a strong and bright future for Joomla. Not one stuck in the past, again, for reasons unknown to me. Template/Extension developers not wanting to move their code forward isn't a valid reason either.

We all want Joomla 4 to succeed and invert the Joomla 3's curve. A key point is to keep the developers' and users' excitement up.

Joomla 4's PHP framework is modern and a huge step from Joomla 3, kudos dev-team! Let's also bring Javascript's framework into modernity too, specially that it looks like mainly being to remove the transpiling from ES6 to ES5 and adding a deprecated comment to jQuery.

The drop of BS4 in favor of BS5 has generated some excitement and some big modernity hopes on the frontend too.

It's imho worth dropping ES5 in favor of ES6, and while still including and using jQuery at release time (that's fine), just deprecating jQuery. So not a huge release-delayer, but a huge checkmark for Joomla 4's use.

[brianteeman]

It's not hostility although my comment clearly hit a nerve with you. My point is that for 4 years people have been working on joomla 4. For all that time they have been asking for help and assistance and now as we are so close to getting it over the finish line you come along and say that it is essential there are changes. Well I'm sorry but the time to make those changes has long gone. Both your comments and Beat's comments are written as if you have just had this amazing idea. Well others have had that idea for a long time and they've worked hard to try and achieve it with extremely limited resources. When people come along after the fact and devalue the effort and commitments of those that have given so freely of their time it is not surprising that they all eventually give up and move on.

[ghost]

brianteeman what you done to make j4 ES6 only?

[krileon]

It's not hostility although my comment clearly hit a nerve with you.

I'm confused how you don't think "Sorry if I couldnt care less what you think at this time." isn't hostile. You're talking to potential future contributors and that is what you decide to go with. I don't work in hostile environments and neither should anyone else if they have the choice. There's enough negativity in the world right now to deal with and I won't take part of it by choice.

This is a discussion. It's not an issue ticket. It's not a pull request. This doesn't have to be done now or even at all. That's the point of discussing this. If you feel this shouldn't happen now you could've come out of the gate with "Maybe for Joomla 5, but there's concern with making such a change this late into Joomla 4 development". You had hundreds, no thousands, of ways to approach this conversation and you went with the scorched earth option of devaluing someone's opinion with "Sorry if I couldnt care less what you think at this time.".

For all that time they have been asking for help and assistance

I've no idea where they have been asking for assistance. Until recently discussion were not enabled. I've no idea where contributors discussed before this.

I'm sorry for being late to the game to contribute. I manage effectively a code base of over 30 extensions. I'm stretched for time as is. My goal this year was to better manage my time and try to contribute to Joomla to give something back, but it's a bit discouraging if this is the reactions from just making a simple discussion to use code that's already there.

now as we are so close to getting it over the finish line you come along and say that it is essential there are changes

Nobody is making demands here. This again is just a discussion and it's again about the ES6 that's already here.

Well I'm sorry but the time to make those changes has long gone.

Why? You act as if Joomla releasing can't make anymore changes. As if we've hit a wall and can do no more.

Both your comments and Beat's comments are written as if you have just had this amazing idea.

I opened this discussion because it was decided to move forward with Bootstrap 5 which completely removes support for IE11. This means frontend and backend framework do not support IE11. The idea was to move forward with using ES6 exclusively since there is no support for IE11 at all. Previous ideas regard this were attempting to keep frontend IE11 compatibility.

Well others have had that idea for a long time and they've worked hard to try and achieve it with extremely limited resources.

I am extremely grateful for all those who have already contributed ES6 and the move away from jQuery. My discussion is about using that ES6 they've worked so hard to achieve.

When people come along after the fact and devalue the effort and commitments of those that have given so freely of their time it is not surprising that they all eventually give up and move on.

Nobody is devaluing anything here. The ES6 is already done of which is absolutely fantastic. I would've never started this discussion if Joomla 4 was already released as it's too large of a change without doing a major release, but Joomla is in a position to make the switch to the existing ES6 before releasing stable which is why this discussion was made to see if that's something everybody wanted or not.

[brianteeman]

The decision to remove support for IE was made a long time ago and is nothing to do with bootstrap 5

[barsan-md]

Looks like more and more is being inserted into this release of Joomla. Since brianteeman is already infuriated.... May I suggest dropping PHP in favour of Python?

😆
Just kidding, looking forward for the release, and I'm glad to all the contributors that worked on it.

[Fedik]

I think we can delivery ES6 pretty safe. It does not affect extension developers, and no one will notice difference.
It just turn off ES5 transpiler, that all.

But final decision need to be made not by me, well ... you know 😄

[dgrammatiko]

There a bug in example ;)

Of course, there is, I wrote it so that was expected 😉
I know why this was introduced. At that time was a good idea (it was providing a polyfill for CustomEvents, for once, etc) BUT all these are now given so now it feels more like an overhead and will be a point of friction for newcomers. I mean it's easier to search MDN Events than Joomla Events and actually do what you want...

If its possible do it step by step, then better do step by step

Agreed, it's just that you can't unship things without breaking B/C, thus all my points above. It's more like what luggage do the project willing to carry for the next x years and most importantly why. Not my call, just saying loud my thoughts here

[krileon]

I 100% agree that Joomla.Event.dispatch should go as well. It was basically a middlewear for the CustomEvents, which doesn't make sense for ES6.

I also agree with going step by step, but do think some things should be addressed now instead of later. Mainly I'm of the opinion the below is needed to deploy ES6 files.

1. Remove Joomla.Event.dispatch
2. Remove CustomEvents Polyfill
3. Remove WebComponents Polyfill
4. Adjust browserlist to exclude IE11
5. Adjust build process to no longer transpile to ES5

Would you feel that'd be a safe bare minimum?

[dgrammatiko]

There you go: #32171

This treats Web components/Custom Elements as ES Modules

[krileon]

There you go: #32171

This treats Web components/Custom Elements as ES Modules

Wow, awesome @dgrammatiko ! Is there anything I should specifically be testing for? Already switched to branch, ran npm ci, verified modules in source, and checked a few JS behaviors (e.g. admin menu toggling and navigation). Seams ok thus far.

[dgrammatiko]

This affects the alerts, tabs and almost all fields but should be pretty safe

[joeforjoomla]

@dgrammatiko
I've found 2 important side effects of delivering ES6 files with type="module" that is not B/C and forces a lot of changes in extensions for developers.

First of all it's no longer possible to use Bootstrap components through jQuery.

Moreover even if the regular Bootstrap bundle is loaded, the events system of jQuery is totally skipped, requiring to update the code to use for example:
document.querySelector('#selector').addEventListener('shown.bs.modal'....

instead of:
$('#selector').on('shown.bs.modal',...

So what do we do?

[dgrammatiko]

You can check the .zip here https://ci.joomla.org/artifacts/joomla/joomla-cms/4.0-dev/32198/downloads/39697/

[joeforjoomla]

@dgrammatiko i've tried that zip but it's still the same, not working.
I've found that it works if i use single components apart the 'tab' one still not present in jQuery:
HTMLHelper::('bootstrap.popover');
HTMLHelper::('bootstrap.alert');
HTMLHelper::('bootstrap.collapse');
HTMLHelper::('bootstrap.modal');
HTMLHelper::_('bootstrap.tab');

but it does not work using the global method:
HTMLHelper::_ ( 'bootstrap.framework' );

[dgrammatiko]

but it does not work using the global method:

That's a bug, but a weird one, all the framework method does is calling each component individually

[joeforjoomla]

@dgrammatiko it seems to me that jQuery works with Bootstrap only for single modules AND only for single modules loaded BEFORE 'bootstrap.es5.min.js'. For example, loading as below:

<script src="/media/vendor/bootstrap/js/popover.es6.min.js?7a19a487669695576ab08be3e3201892" type="module"></script> <script src="/media/vendor/bootstrap/js/bootstrap.es5.min.js?7a19a487669695576ab08be3e3201892" nomodule defer></script> <script src="/media/vendor/bootstrap/js/alert.es6.min.js?7a19a487669695576ab08be3e3201892" type="module"></script> <script src="/media/vendor/bootstrap/js/collapse.es6.min.js?7a19a487669695576ab08be3e3201892" type="module"></script <script src="/media/vendor/bootstrap/js/modal.es6.min.js?7a19a487669695576ab08be3e3201892" type="module"></script> <script src="/media/vendor/bootstrap/js/tab.es6.min.js?7a19a487669695576ab08be3e3201892" type="module"></script>

i'm able to use 'popovers' without errors:

but not 'alerts':

[dgrammatiko]

Then I guess this is a Bootstrap bug, we are not altering any of their code

[joeforjoomla]

I don't know... i've also tried to remove the loading of bootstrap.es5.min.js but nothing works.

Now i have :

<script src="/media/vendor/bootstrap/js/popover.es6.min.js?7a19a487669695576ab08be3e3201892" type="module"></script> <script src="/media/vendor/bootstrap/js/alert.es6.min.js?7a19a487669695576ab08be3e3201892" type="module"></script> <script src="/media/vendor/bootstrap/js/collapse.es6.min.js?7a19a487669695576ab08be3e3201892" type="module"></script> <script src="/media/vendor/bootstrap/js/modal.es6.min.js?7a19a487669695576ab08be3e3201892" type="module"></script> <script src="/media/vendor/bootstrap/js/tab.es6.min.js?7a19a487669695576ab08be3e3201892" type="module"></script>

and only the first one jQuery('.selector').popover(); works without errors.

[joeforjoomla]

@dgrammatiko I've tried with a clean installation of the build that you sent me,

Modified /Joomla40staging/administrator/components/com_content/src/View/Articles/HtmlView.php

Result, jQuery loaded before Bootstrap and:

Popovers work, modals don't work:

[dgrammatiko]

It's possible to avoid this and still migrate to BS5

What's the magic solution here?

[joeforjoomla]

@dgrammatiko use the jQuery version like the @wilsonge one

[dgrammatiko]

After spending countless hours the last several years converting most of the core javascript to vanilla I think I can safely say, thank you, but I won't do that. You're welcome to do a PR yourself though...

[joeforjoomla]

Well i don't care about how many hours you have spent... i don't agree and i'm tempted to load my own bootstrap in all my extensions instead of invest a lot of work to redo all my extensions. If you prefer you can pay me to hire developers to do that quickly. It's easy to have control of the code of a CMS without take care of people that invest and work with it all around the world. Congratulations.

[krileon]

Well i don't care about how many hours you have spent...

That's extremely rude. @dgrammatiko has put in an absolutely amazing amount of work moving Joomla into the future away from dead jQuery. Going back would be one of the worst mistakes in Joomla history.

i don't agree and i'm tempted to load my own bootstrap

Then do that. We do it with Community Builder since we use BS4 on J3. We do the same with jQuery loaded in noconflict and has worked fine for over 10 years and still works fine now on J4.

instead of invest a lot of work to redo all my extensions.

It's a search and replace in any IDE. Takes like 5 minutes. It only applies to Bootstrap components as well. Your own jQuery plugins and usages will not be impacted by this. Bootstrap 5 documentation directly shows what you need to use to call the components and examples have been provided above. This doesn't require some massive rewrite of your code.

If you think there's an issue with Bootstrap component loading then open an issue ticket and provide your information there. This isn't the place for this.

[dgrammatiko]

i don't agree and i'm tempted to load my own bootstrap in all my extensions instead of invest a lot of work to redo all my extensions

If you have a solution why are you complaining, use your solution. Also for the record, I'm not part of the decision-makers, it just happens that I did a PR that was merged but you think is wrong or whatever. Cool. Let's keep it there.

[dgrammatiko]

So, I'm refactoring the build tools mainly because they're awfully slow. The questions here are:

• what's the Browser minimum version for the ES6+ code? I mean the project should have a safety buffer as JS gets a new version every year but some browsers are sluggish...
• is there a strong signal that the type=module/nomodule thing is something you should ship?

@Fedik @krileon @beat @wilsonge

[krileon]

If the IE 11 nomodule workaround is added I'd like to see it as some sort of flag the Joomla template sets if that template needs to work on IE11 otherwise it just never outputs to the DOM. It just doesn't make any sense to me to add the ES5 nomodule fallback when we can't even test it.

[dgrammatiko]

I'd like to see it as some sort of flag the Joomla template sets if that template needs to work on IE11

I don't have an argument against it but since the code that spits out the script tags is outside of the template there are some other issues (what's the default behaviour, etc). The code that handles the tags is here:

joomla-cms/libraries/src/Document/Renderer/Html/ScriptsRenderer.php

Lines 44 to 132 in a0c9338

	public function render($head, $params = array(), $content = null)
	{
	// Get line endings
	$lnEnd = $this->_doc->_getLineEnd();
	$tab = $this->_doc->_getTab();
	$buffer = '';
	$wam = $this->_doc->getWebAssetManager();
	$assets = $wam->getAssets('script', true);
	// Get a list of inline assets and their relation with regular assets
	$inlineAssets = $wam->filterOutInlineAssets($assets);
	$inlineRelation = $wam->getInlineRelation($inlineAssets);
	// Merge with existing scripts, for rendering
	$assets = array_merge(array_values($assets), $this->_doc->_scripts);
	// Generate script file links
	foreach ($assets as $key => $item)
	{
	// Check whether we have an Asset instance, or old array with attributes
	$asset = $item instanceof WebAssetItemInterface ? $item : null;
	// Add src attribute for non Asset item
	if (!$asset)
	{
	$item['src'] = $key;
	}
	// Check for inline content "before"
	if ($asset && !empty($inlineRelation[$asset->getName()]['before']))
	{
	foreach ($inlineRelation[$asset->getName()]['before'] as $itemBefore)
	{
	$buffer .= $this->renderInlineElement($itemBefore);
	// Remove this item from inline queue
	unset($inlineAssets[$itemBefore->getName()]);
	}
	}
	$buffer .= $this->renderElement($item);
	// Check for inline content "after"
	if ($asset && !empty($inlineRelation[$asset->getName()]['after']))
	{
	foreach ($inlineRelation[$asset->getName()]['after'] as $itemBefore)
	{
	$buffer .= $this->renderInlineElement($itemBefore);
	// Remove this item from inline queue
	unset($inlineAssets[$itemBefore->getName()]);
	}
	}
	}
	// Generate script declarations for assets
	foreach ($inlineAssets as $item)
	{
	$buffer .= $this->renderInlineElement($item);
	}
	// Generate script declarations for old scripts
	foreach ($this->_doc->_script as $type => $contents)
	{
	// Test for B.C. in case someone still store script declarations as single string
	if (\is_string($contents))
	{
	$contents = [$contents];
	}
	foreach ($contents as $content)
	{
	$buffer .= $this->renderInlineElement(
	[
	'type' => $type,
	'content' => $content,
	]
	);
	}
	}
	// Output the custom tags - array_unique makes sure that we don't output the same tags twice
	foreach (array_unique($this->_doc->_custom) as $custom)
	{
	$buffer .= $tab . $custom . $lnEnd;
	}
	return ltrim($buffer, $tab);
	}

[krileon]

I guess for example in theory it'd be a flag set on the web asset manager object. At the top of the default Joomla 4 template for example getWebAssetManager is called. After that a template could set some sort of IE11 flag like $wa->supportIE( true ) or $wa->isES5( true ) then in the renderer file you've linked above when looping the scripts it'd need to decide to output the nomodule if it exists and there's a type=module of same script.

As for default behavior I'd say default off since default template doesn't support IE11 and should be dictated by the what the actual template supports.

[dgrammatiko]

Not exactly relevant but Edge legacy ends in one month: https://blogs.windows.com/msedgedev/2021/02/05/what-you-need-to-know-about-kiosk-mode-when-support-for-microsoft-edge-legacy-ends/

[krileon]

I don't think we need to support Edge Legacy. The not Edge < 79 browserlist value covers excluding non-Chrome versions of Edge.

[dgrammatiko]

The Pr #32315 is introducing type=module/nomodule dual mode. This means that modern (ES2017+) browsers will load the ES2017+ code and browsers with support up to ES2016 will load the ES5 files

Please test

[krileon]

In Joomla article edit I'm seeing "Failed to initialize plugin: jdragndrop" with a JS error for TinyMCE.

IN: /media/vendor/tinymce/tinymce.js?8cddf7e44456cc0accb7ae7521e9ad9d
ON: Line 20021

Failed to initialize plugin: jdragndrop TypeError: Plugin is not a constructor
    at initPlugin (tinymce.js?8cddf7e44456cc0accb7ae7521e9ad9d:26444)
    at Array.<anonymous> (tinymce.js?8cddf7e44456cc0accb7ae7521e9ad9d:26461)
    at Object.each$2 [as each] (tinymce.js?8cddf7e44456cc0accb7ae7521e9ad9d:582)
    at initPlugins (tinymce.js?8cddf7e44456cc0accb7ae7521e9ad9d:26460)
    at init$2 (tinymce.js?8cddf7e44456cc0accb7ae7521e9ad9d:26548)
    at Editor.<anonymous> (tinymce.js?8cddf7e44456cc0accb7ae7521e9ad9d:26675)
    at Array.<anonymous> (tinymce.js?8cddf7e44456cc0accb7ae7521e9ad9d:7964)
    at each$2 (tinymce.js?8cddf7e44456cc0accb7ae7521e9ad9d:582)
    at loadScripts (tinymce.js?8cddf7e44456cc0accb7ae7521e9ad9d:7961)
    at tinymce.js?8cddf7e44456cc0accb7ae7521e9ad9d:7948

[dgrammatiko]

Good catch, fixed with 78e5dfd

[krileon]

Confirmed that's fixed. Found another.

With CodeMirror selected as the editor clicking the Read More button under the editor in article create/edit causes the following.

IN: /media/com_content/js/admin-article-readmore.js?df78d5746e2e5d511e47f5b9f7313983
ON: Line 15

Uncaught TypeError: window.Joomla.editors.instances[editor].getValue is not a function
    at window.insertReadmore (admin-article-readmore.js?df78d5746e2e5d511e47f5b9f7313983:15)
    at HTMLButtonElement.onclick (index.php?option=com_content&view=article&layout=edit:401)

btw, do you want me to put the feedback here or on the ticket?

[dgrammatiko]

Fixed with 23354b8

I think we can move the error reporing to the actual PR

[dgrammatiko]

I think that this one was already implemented in a way that Joomla 4 delivers modern JS scripts for browsers that support it and keep the ES5 scripts for older browsers. The complete removal of the ES5 scripts from the package is a decision that the Production Team will have to make at some point that will serve best the project.

fwiw the PRs that implemented this were:

• bootstrap 5: [4.0][WIP] BS5 JS #31990
• Web Components: [4.0] Treat Web Components and Custom Elements as ES Modules #32171
• Tools redo: [4.0] Deliver ES2015+ javascript #32315