Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Install nodes that require significant memory #626

Open
1 of 2 tasks
knolleary opened this issue May 24, 2022 · 4 comments
Open
1 of 2 tasks

Install nodes that require significant memory #626

knolleary opened this issue May 24, 2022 · 4 comments
Labels
epic A significant feature or piece of work that doesn't easily fit into a single release scope:node-red Features for FlowFuse to boost the vanilla Node-RED experience

Comments

@knolleary
Copy link
Member

knolleary commented May 24, 2022

Some nodes in the Node-RED ecosystem have a larger install-time memory requirement than others. This can lead to installs failing when running in a Stack with a limited memory footprint.

For example, the 256 memory limit we have in FFCloud is fine for most cases, but will fail for a module like node-red-contrib-graphql-server.

This issue is to track how we can improve the user experience around this scenario

  • Node-RED

    • When running npm, node-red doesn't capture the full log output. That can make it hard to know why an install failed. Capturing the full log could produce a lot of noise. It would be interesting to see if the exit code of a killed process is distinct enough for Node-RED to detect and log appropriately
  • FlowForge

    • Memory is defined by the Project Stack. In a k8s environment, that doesn't give an admin any discretion to increase an individual project's memory allocation on an ad hoc basis.
    • One solution would be to define a second stack with more memory allocated.
      • If that was made generally available, we would want to consider attaching an additional cost to the stack to reflect the larger resource usage. This would require Introduce ProjectType #380.
      • An alternative would be to create the stack but mark it 'inactive' - this means it doesn't show up for users. We could then allow an admin to modify the project to use that stack (overriding the checks we have about applying an inactive stack). The user wouldn't be able to modify the stack (because they cannot select inactive stacks) - which would make upgrading their project something admins would have to do for them... the maintenance cost of that is not practical.

Tasks

Preview Give feedback
  1. 3 of 3
    size:M - 3 story
    hardillb
  2. size:XL - 8 story
@knolleary knolleary added the needs-triage Needs looking at to decide what to do label May 24, 2022
@ZJvandeWeg ZJvandeWeg added the scope:node-red Features for FlowFuse to boost the vanilla Node-RED experience label Jun 3, 2022
@sammachin
Copy link
Contributor

Purely from a user perspective it feels like whats needed here is the ability to 'burst' memory usage above the baseline, I'm not sure if there's anything in k8s that would allow for that?

@sammachin sammachin moved this to 📐 Design in ☁️ Product Planning Jun 14, 2022
@sammachin sammachin removed the needs-triage Needs looking at to decide what to do label Jun 14, 2022
@sammachin sammachin changed the title Node installs can fail when memory limits are too low Install nodes that require significant memory Jun 14, 2022
@sammachin sammachin added the story A user-oriented description of a feature label Jun 14, 2022
@sammachin
Copy link
Contributor

I've converted this to a story as I'm not sure its really a bug as much as a limitation of the current platform

@robmarcer
Copy link
Contributor

This came up today for a FF Cloud customer. They can't get node-red-contrib-chatbot to install on a small instance but it does work on medium. I suspect the only resource which is causing that problem is RAM.

@joepavitt
Copy link
Contributor

Currently, when npm is killed as part of startup, we don't communicate well tot he user why this failed. We're seing this issue a lot, we have two methods for resolving:

  1. Better communication of of why the Instance failed to spin up
  2. Stop the instances from falling over

(1) is the quick MVP here, and (2) should be a longer term objective.

@MarianRaphael MarianRaphael added epic A significant feature or piece of work that doesn't easily fit into a single release and removed story A user-oriented description of a feature labels Mar 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
epic A significant feature or piece of work that doesn't easily fit into a single release scope:node-red Features for FlowFuse to boost the vanilla Node-RED experience
Projects
Status: Medium
Development

No branches or pull requests

6 participants