MFE builds failing as a result of unresolved dependency

[stelio]

Hi

I'm attempting to rebuild the mfe tutor images build mfe --no-cache --no-registry-cache, but i'm receiving the following issue

...
 => [communications-i18n 4/5] RUN stat /openedx/app/src/i18n/messages 2> /dev/null || (echo "missing messages folder" && mkdir -p /openedx/app/src/i18n/messages)                                                                                                                                                    0.1s
 => [account-i18n 4/5] RUN stat /openedx/app/src/i18n/messages 2> /dev/null || (echo "missing messages folder" && mkdir -p /openedx/app/src/i18n/messages)                                                                                                                                                           0.1s
 => [course-authoring-i18n 4/5] RUN stat /openedx/app/src/i18n/messages 2> /dev/null || (echo "missing messages folder" && mkdir -p /openedx/app/src/i18n/messages)                                                                                                                                                  0.1s
 => [discussions-i18n 4/5] RUN stat /openedx/app/src/i18n/messages 2> /dev/null || (echo "missing messages folder" && mkdir -p /openedx/app/src/i18n/messages)                                                                                                                                                       0.1s
 => [profile-i18n 4/5] RUN stat /openedx/app/src/i18n/messages 2> /dev/null || (echo "missing messages folder" && mkdir -p /openedx/app/src/i18n/messages)                                                                                                                                                           0.1s
 => [authn-common 4/6] RUN npm install '@edx/brand@npm:@edly-io/indigo-brand-openedx@^2.1.1'                                                                                                                                                                                                                         9.8s
 => [authn-i18n 4/5] RUN stat /openedx/app/src/i18n/messages 2> /dev/null || (echo "missing messages folder" && mkdir -p /openedx/app/src/i18n/messages)                                                                                                                                                             0.1s
 => ERROR [profile-common 5/9] RUN npm install '@edx/frontend-component-header@npm:@edly-io/indigo-frontend-component-header@^3.1.3'                                                                                                                                                                                 4.3s
 => CANCELED [learning-common 5/9] RUN npm install '@edx/frontend-component-header@npm:@edly-io/indigo-frontend-component-header@^3.1.3'                                                                                                                                                                             0.0s
------
 > [profile-common 5/9] RUN npm install '@edx/frontend-component-header@npm:@edly-io/indigo-frontend-component-header@^3.1.3':
4.277 npm ERR! code ERESOLVE
4.281 npm ERR! ERESOLVE unable to resolve dependency tree
4.283 npm ERR!
4.283 npm ERR! While resolving: @edx/[email protected]
4.283 npm ERR! Found: @edx/[email protected]
4.283 npm ERR! node_modules/@edx/frontend-platform
4.283 npm ERR!   @edx/frontend-platform@"5.5.4" from the root project
4.283 npm ERR!
4.283 npm ERR! Could not resolve dependency:
4.283 npm ERR! peer @edx/frontend-platform@"^7.0.0 || ^8.0.0" from @edx/[email protected]
4.283 npm ERR! node_modules/@edx/frontend-component-header
4.283 npm ERR!   @edx/frontend-component-header@"npm:@edly-io/indigo-frontend-component-header@^3.1.3" from the root project
4.283 npm ERR!
4.283 npm ERR! Fix the upstream dependency conflict, or retry
4.284 npm ERR! this command with --force or --legacy-peer-deps
4.284 npm ERR! to accept an incorrect (and potentially broken) dependency resolution.
4.284 npm ERR!
4.284 npm ERR!
4.284 npm ERR! For a full report see:
4.284 npm ERR! /root/.npm/_logs/2024-11-06T07_35_48_092Z-eresolve-report.txt
4.286
4.286 npm ERR! A complete log of this run can be found in: /root/.npm/_logs/2024-11-06T07_35_48_092Z-debug-0.log
------
Dockerfile:589
--------------------
 587 |
 588 |     RUN npm install '@edx/brand@npm:@edly-io/indigo-brand-openedx@^2.1.1'
 589 | >>> RUN npm install '@edx/frontend-component-header@npm:@edly-io/indigo-frontend-component-header@^3.1.3'
 590 |     RUN npm install @edly-io/indigo-frontend-component-footer@^2.0.0
 591 |
--------------------
ERROR: failed to solve: process "/bin/sh -c npm install '@edx/frontend-component-header@npm:@edly-io/indigo-frontend-component-header@^3.1.3'" did not complete successfully: exit code: 1
Error: Command failed with status 1: docker buildx build --tag=docker.io/overhangio/openedx-mfe:17.0.1-indigo --no-cache --output=type=docker /home/ubuntu/.local/share/tutor/env/plugins/mfe/build/mfe

*Note repeated runs yield the same results, so it's not a once off.

I've tried adjusting the associated mfe-dockerfile-post-npm-install-* patches with the following:

1. Installing @edx/frontend-platform RUN npm install '@edx/frontend-platform@^7.0.0' before the @edx/[email protected]
2. Force Installation RUN npm install --force '@edx/frontend-component-header@npm:@edly-io/indigo-frontend-component-header@^3.1.3'
3. Installed with --legacy-peer-deps

But non of these approaches yielded a positive outcome, all attempted builds failed later down the line because of some or other dependancies.

My questions are:

1. What is the ideal way to develop the MFE, where I can test and swap out components etc? I'm running the whole tutor images build mfe --no-cache --no-registry-cache pipeline over and over on a reduced max-parallelism setup, which takes forever to build and often results in a failed build 10-20mins later.

2. What is the most efficient way to build the MFE, should we still be using the max-parallelism = 2 hack?https://docs.tutor.edly.io/troubleshooting.html#high-resource-consumption-by-docker-on-tutor-images-build

3. Do you know what could be the actual root cause of this issue i've described above?

[DawoudSheraz]

Hello. Can you list down which tutor, tutor-mfe, and tutor-indigo versions are you using? There are no active dependencies issues as far as I know. You can look at https://github.com/overhangio/tutor-mfe/?tab=readme-ov-file#troubleshooting to understand how to debug dependencies conflicts.

What is the ideal way to develop the MFE, where I can test and swap out components etc? I'm running the whole tutor images build mfe --no-cache --no-registry-cache pipeline over and over on a reduced max-parallelism setup, which takes forever to build and often results in a failed build 10-20mins later.

What do you mean by swapping out components? If you are asking about developing or customizing MFEs, you can look at https://github.com/overhangio/tutor-mfe/?tab=readme-ov-file#customising-mfes and https://github.com/overhangio/tutor-mfe/?tab=readme-ov-file#mfe-development.

What is the most efficient way to build the MFE, should we still be using the max-parallelism = 2 hack?https://docs.tutor.edly.io/troubleshooting.html#high-resource-consumption-by-docker-on-tutor-images-build

It would be good to use it because multiple npm installs usually result in connection issues. The less the number of parallel builds, the better the chances of image getting built without any connection issues. You might want to check your docker buildx cache size. If the cache size is small, the previous build artifacts might not be cached entirely and it would result in next build re-building the things it should not have to.

[stelio]

Hi Dawoud,

Thanks for the follow-up

Hello. Can you list down which tutor, tutor-mfe, and tutor-indigo versions are you using? There are no active dependencies issues as far as I know. You can look at https://github.com/overhangio/tutor-mfe/?tab=readme-ov-file#troubleshooting to understand how to debug dependencies conflicts.

Admittedly I am using older versions of tutor-mfe with a newer version of tutor-indigo, but this has worked successfully for me in the past:

NAME STATUS VERSION
android installed 17.0.0
cairn installed 17.1.0
credentials installed 17.0.1
discovery installed 17.0.1
ecommerce installed 17.0.2
forum ✅ enabled 17.0.1
indigo ✅ enabled 18.2.2
jupyter installed 17.0.0
mfe ✅ enabled 17.0.1
minio installed 17.0.0
notes installed 17.0.0
webui installed 17.0.1
xqueue installed 17.0.0

What is the ideal way to develop the MFE, where I can test and swap out components etc? I'm running the whole tutor images build mfe --no-cache --no-registry-cache pipeline over and over on a reduced max-parallelism setup, which takes forever to build and often results in a failed build 10-20mins later.

What do you mean by swapping out components? If you are asking about developing or customizing MFEs, you can look at https://github.com/overhangio/tutor-mfe/?tab=readme-ov-file#customising-mfes and https://github.com/overhangio/tutor-mfe/?tab=readme-ov-file#mfe-development.

This answers my question thanks

What is the most efficient way to build the MFE, should we still be using the max-parallelism = 2 hack?https://docs.tutor.edly.io/troubleshooting.html#high-resource-consumption-by-docker-on-tutor-images-build

It would be good to use it because multiple npm installs usually result in connection issues. The less the number of parallel builds, the better the chances of image getting built without any connection issues. You might want to check your docker buildx cache size. If the cache size is small, the previous build artifacts might not be cached entirely and it would result in next build re-building the things it should not have to.

Your caching suggestion has also helped thanks

[DawoudSheraz]

Yes, using different release versions(17 & 18 for instance) is not recommended as it causes compatibility and other issues. Provided the context you have added above, this is not an issue. I am closing this issue, let me know if you have any further questions. Thanks