How to rebuild assets on top of superset image?

[coneybeare]

I am looking to simply replace some assets (images) in the release build, but I am looking to extend the base docker image, rather than fork and customize so that I don't have to continually update the build to get the latest improvements. Starting with a custom Dockerfile, I would have expected this to work, but it does not:

ARG SUPERSET_VERSION=latest
FROM apache/superset:$SUPERSET_VERSION
USER root

RUN apt-get update && apt-get install -y git
COPY overrides /app # contains files like /superset/static/assets/images/loading.gif
WORKDIR /app/superset-frontend
RUN set -eux; \
  curl -sL https://deb.nodesource.com/setup_18.x | bash -; \
  apt-get install -y nodejs;
RUN npm cache clean --force
RUN npm install --force
RUN npm run build

RUN pip install --no-cache sqlalchemy-drill psycopg2-binary redis snowflake-sqlalchemy

WORKDIR /app
USER superset

The build step fails because of course, the raw assets are no longer in the /app/superset-frontend folder.

My approach before this was to copy files into /superset/static/assets/images and then a versioned one into /superset/static/assets at runtime, but that is fragile as the compiled version filenames will change.

Before attempting to solve this and figure out if I can move the raw assets back to rebuild, I thought I would ask the community here, as it feels I am swimming upstream for a relatively simple use case. How would you solve this?

[dosubot[bot]]

Hey @coneybeare! I'm here to help you with any bugs, questions, or contributions you have in mind. Let's tackle this together!

To rebuild assets on top of a Superset Docker image without forking and customizing the base image, you can use a multi-stage build approach. This allows you to modify the assets while ensuring the raw assets are available for the build process. Here's a general approach:

1. Use the existing superset-node stage: This stage is responsible for building the frontend assets. You can modify this stage to include your custom images or assets.

2. Bind mount your custom assets: Use Docker's --mount option to bind mount your custom assets into the build context. This way, you can replace or add new assets without modifying the base image.

3. Rebuild the assets: Use the existing build commands to rebuild the assets with your modifications.

Here's a simplified example of how you might modify the Dockerfile to achieve this:

# Use the existing superset-node stage to build custom assets
FROM --platform=${BUILDPLATFORM} node:20-bullseye-slim AS custom-assets

# Copy the necessary scripts and configurations
COPY docker/ /app/docker/

# Install system dependencies required for node-gyp
RUN /app/docker/apt-install.sh build-essential python3 zstd

# Set working directory
WORKDIR /app/superset-frontend

# Bind mount your custom assets
RUN --mount=type=bind,source=./path/to/your/custom/assets,target=/app/superset-frontend/assets \
    --mount=type=cache,target=/root/.cache \
    --mount=type=cache,target=/root/.npm \
    npm ci

# Rebuild the assets
RUN npm run build

# Continue with the rest of the Dockerfile stages
FROM python-common AS final

# Copy the newly built assets from the custom-assets stage
COPY --from=custom-assets /app/superset/static/assets superset/static/assets

# Continue with the rest of the setup

This approach allows you to inject custom assets into the build process without altering the base image directly. You can replace ./path/to/your/custom/assets with the actual path to your custom assets ^[1].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}

[coneybeare]

@dosu this approach requires a fork of the superset codebase, and editing the included Dockerfile. I know this will work, but I am looking for a solution which I don't have to continually update my branch with upstream changes. I was hoping to start with a prebuilt superset image, then extend it from there. Is that possible?

[dosubot[bot]]

To extend a prebuilt Superset Docker image without forking the codebase, you can use a custom Dockerfile that starts from the official Superset image and then adds your custom assets. Here's a general approach:

1. Start from the Superset Image: Use the official Superset image as the base.

2. Copy Custom Assets: Copy your custom assets into the appropriate directory in the container.

3. Rebuild or Replace Assets: If you need to rebuild assets, ensure that the necessary tools (like Node.js and npm) are available in the image. Alternatively, if you only need to replace static files, you can directly copy them to the target directory.

Here's an example Dockerfile:

ARG SUPERSET_VERSION=latest
FROM apache/superset:$SUPERSET_VERSION

USER root

# Install Node.js and npm if needed for rebuilding assets
RUN apt-get update && apt-get install -y nodejs npm

# Copy your custom assets
COPY overrides/superset/static/assets /app/superset/static/assets

# If rebuilding is necessary, navigate to the frontend directory and rebuild
WORKDIR /app/superset-frontend
RUN npm install
RUN npm run build

USER superset

In this setup, replace overrides/superset/static/assets with the path to your custom assets. This approach allows you to extend the image without modifying the original Dockerfile, and you can update your base image version by changing the SUPERSET_VERSION argument.

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}

[coneybeare]

@dosu how is that any different than my original approach? Which fails because the original assets are not found?

[dosubot[bot]]

To address the issue of missing original assets when extending a prebuilt Superset Docker image, you can ensure that the build process has access to all necessary files by using multi-stage builds and mounting required files. Here's a refined approach:

1. Use Multi-Stage Builds: This separates the build environment from the final runtime environment, ensuring dependencies and assets are managed effectively.

2. Mount Required Files: Use Docker's --mount option to bind mount necessary files like package.json and package-lock.json for installing dependencies, and use --mount=type=cache for caching npm and other build-related files.

3. Copy Assets Correctly: Ensure that the assets are copied from the build stage to the final image correctly. For example:

   COPY --from=superset-node /app/superset/static/assets superset/static/assets

4. Set Environment Variables: Make sure environment variables like DEV_MODE are set appropriately to ensure the frontend build is not skipped.

5. Run the Build Commands: Ensure that npm commands to build the frontend assets are executed successfully.

By following these steps, you can extend the Superset Docker image and rebuild assets without encountering issues related to missing original assets ^[1].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}

[coneybeare]

Dosu didn't really help here, maybe a human has done this before?