Issue: Recursive Supabase Edge Function for Paginated Shopify Products

[harsh-makroagency]

Hi everyone,

I've built a Supabase Edge Function that fetches products from a Shopify store. Since Shopify has pagination, I need to retrieve multiple pages of products.

To handle this, I implemented self-calling recursion—invoking the Edge Function within itself to fetch the next page when required. However, I've noticed that Supabase Edge Functions have a limited execution time, which is causing issues when fetching large datasets.

Current Approach

1. The function retrieves products from Shopify via GraphQL.
2. If there are more pages (hasNextPage is true), the function calls itself recursively to fetch the next set of products.
3. Due to execution time limits, the function stops before completing all pages.

Here’s the code snippet that invokes the function for the next page:

const response = await fetch(https://${shopifyStoreDomain}.myshopify.com/admin/api/2024-04/graphql.json, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'X-Shopify-Access-Token': shopifyAccessToken,
},
body: JSON.stringify({ query, variables: { first: 250, after: cursor?.endCursor || null } }),
});

Challenges

1. Supabase Edge Functions have a time limit—recursive calls may not complete fetching all products.
2. Long-running tasks are not ideal in Edge Functions.
3. Shopify rate limits need to be considered when making multiple API calls.