-
Notifications
You must be signed in to change notification settings - Fork 165
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Broken nodejs.org webhook since December 17th #2123
Comments
I don't know much about the webhook, although I can learn I suppose? But hopefully someone else on @nodejs/build knows what to do? |
May I also add that it might be better to rotate the log more frequently since it gets quite huge very fast? |
Here's a couple of pointers I've learned after working with the @nodejs-github-bot which has a similar GitHub webhook setup:
Hopefully the above gives some hints about what might be wrong or where to look 🤷♂ |
weird error messages in the system logs for github-webhook:
but the log file does exist; perhaps something to do with a file rotation, although we don't have an automatic one yet. I've restarted the service but don't have access to nodejs/nodejs.org to trigger a new build to see if it works. Can someone with perms push a new commit or go to the Settings in GitHub and re-trigger a master push event? |
I just merged nodejs/nodejs.org#2872 so it should trigger now |
looks like it's working again for the record, this just required a I'll manually rotate that log not but it does need to be automated by logrotate at some point. |
FYI https://nodejs.org/github-webhook.log seems to be empty for me |
I emptied it, push to master and it should fill 🤞 |
Merged nodejs/nodejs.org#2875 so it should trigger now |
Looks like it logged the deployment now |
Posted in IRC, but this seems to have happened again for the v12.18.2 release post - nodejs/nodejs.org#3289 |
/cc @codebytere Here's the output of the bottom of https://nodejs.org/github-webhook.log:
The bit above that is a successful build, which took 125670 ms. The next two I believe are yesterday's 12.x and 14.x but says build/ansible/www-standalone/resources/scripts/build-site.sh Lines 12 to 14 in 41d831d
So, the webhook isn't the only thing that can trigger a website build, it also gets triggered by changes to the downloads/release/index.json when there's a new release! We have a crontab entry in that which checks every 5 minutes, so it can be fairly quick after you've promoted your releases. So here's my theory: you both did your releases, the auto-build ran after detecting the changes, then you did your blog posts and pushed to nodejs.org but when the webhook triggered it found that build-site.sh was already running so it bailed. We don't have a queueing mechanism built into any of this, that would be the logical thing to add, except it adds a significant amount of complication. This is probably going to require some kind of queueing mechanism to fix it, but that's going to add a significant amount of complexity. I did build one for unofficial-builds which works quite nicely so maybe I could borrow from that. An alternative might be to just remove that crontab job entirely and rely on the blog posts to update the website. Each release gets a blog post so maybe that's reasonable? If you ever need to force an update then just force push something to nodejs/nodejs.org. For now I've just run it manually. In future just remember that pushing any new commit to nodejs/nodejs.org should send a trigger again. Alternatively if you have access to the webhook settings in that repo you can replay the last one to the same effect. |
This seems to be happening again https://nodejs.org/github-webhook.log is showing a commit from 3 weeks ago and the site is not updating |
I'm seeing this is https://nodejs.org/github-webhook.log for nodejs/nodejs.org@ef6cf1f:
in particular it looks like coroutine has failed, which I presume is a dependency for the website? |
src/coroutine.cc is from fibers and the assertion looks like laverdet/node-fibers#451. I'm not sure what version of Node.js the webhook is running but the mention of npm 7.10 would suggest Node.js 16, which fibers is not compatible with. |
which looks like Node.js 16 (from https://hub.docker.com/layers/node/library/node/latest/images/sha256-2b67c5888e26a4427faec4741c9db1445df567c3bd76bc35c59a39f83ba75ba1?context=explore) |
@richardlau seems like we should maybe switch to |
Indeed I did suggest that and have done. PR to sync the copy of the script to what is deployed on the website: #2639 |
The last deploy in https://nodejs.org/github-webhook.log was for nodejs/nodejs.org@a1196dd
The next commit in this repo was 9df1fc4 but it doesn't look like it should affect the hook so maybe something manual happened.
/cc @Trott
The text was updated successfully, but these errors were encountered: