Error saving index-cache.yaml with Microsoft Azure as storage backend

[gevalo1]

Helm Chart: chartmuseum/[email protected]
Docker image: ghcr.io/helm/chartmuseum:v0.16.1

---

STORAGE: microsoft
DISABLE_STATEFILES: false

We have a workflow using changesets where we occasionally release a bunch of our applications at once. Through a series of GitHub Actions each application gets built, packaged & uploaded to ChartMuseum, which uploads the Charts to an Azure Storage Account (blob).

We noticed a bunch of these GitHub Actions fail with the following error:

Error saving index-cache.yaml

storage: service returned error: StatusCode=409, ErrorCode=InvalidBlobType, ErrorMessage=The blob type is invalid for this operation.\nRequestId:...\nTime:..., RequestInitiated=..., RequestId=..., API Version=2018-03-28, QueryParameterName=, QueryParameterValue=

---

I assume the issue might be caused by the fact that ChartMuseum is trying to update index-cache.yaml multiple times at the same time, causing Azure to throw a 409 error.

What do you suggest we do to resolve this?
Should we just disable the index-cache.yaml statefile or is there an alternative solution? What is the exact effect if we do this?

[scbizu]

hi @gevalo1 , what about your action actually looks like ? The error message shows that your blob type is not correct , I assume that your chart package is not complete or is broken .

[gevalo1]

Hi @scbizu,

The actual chart packages do get uploaded correctly, and I've configured the cache-interval argument to make sure index-cache.yaml gets updated eventually.

The problem seems to be purely with updating index-cache.yaml while uploading the chart packages.

---

This is the GitHub Action

name: Release

env:
  ...

on:
  release:
    types: [published]

jobs:
  release:
    name: Release
    runs-on: ubuntu-latest
    steps:
      - name: Checkout Repo
        uses: actions/checkout@v4

      - uses: pnpm/action-setup@v3
        with:
          version: ^9.0.6

      - name: Setup Node.js 20.12.2
        uses: actions/setup-node@v4
        with:
          node-version: 20.12.2
          cache: "pnpm"

      - name: Install dependencies
        run: pnpm install

      - name: Set up kubelogin for non-interactive login
        uses: azure/use-kubelogin@v1
        with:
           kubelogin-version: 'v0.1.1'

      - name: Login to Azure ACR
        uses: docker/login-action@v3
        with:
          ...

      - name: Setup helm
        uses: azure/setup-helm@v4
        with:
          ...

      - name: Login to Azure
        uses: azure/login@v2
        with:
          ...

      - name: Setup Kubernetes connection
        uses: azure/aks-set-context@v4
        with:
          ...

      - name: Release
        env:
          ...
        run: |
          pnpm run build
          pnpm i
          pnpm run release

And it's pnpm run release which triggers the actual upload to ChartMuseum. The code looks like this

execSync(`helm dependency update`, { encoding: 'utf-8', stdio: 'inherit' });
execSync(`helm package --app-version ${version} --version ${version}`, { encoding: 'utf-8', stdio: 'inherit' });

const data = new FormData();
data.append('chart', fs.createReadStream(`${chart}-${version}.tgz`));

const config = {
  auth: {
    ...
  },
  method: 'post',
  url: 'https://chartmuseum.[...].com/api/charts',
  headers: {
    ...data.getHeaders(),
  },
  data,
};

axios(config)
  .then((response) => {
    console.log(`ChartMuseum: ${JSON.stringify(response.data)}`);
  })
  .catch((error) => {
    if (axios.isAxiosError(error)) {
      console.error(error.toJSON());
      throw error;
    }

    console.error(error);
    throw error;
  });

[cristianomanzoni]

Hi All.
We're facing the same issue, looking at logs I found an interesting information, our index-cache.yaml can't be uploaded due to a restriction in Azure Storage.
As stated in the KB: https://learn.microsoft.com/en-us/troubleshoot/azure/azure-storage/blobs/connectivity/request-body-large

There's a 4-MB limit for each call to the Azure Storage service. If your file is larger than 4 MB, you must break it in chunks.

Is there any workaround or stable solution? Our index is really much more than 4MB :)

TIA,
CM

[scbizu]

@cristianomanzoni Hi , we already fixed this in new CM release .

see chartmuseum/storage#667 and #701