Add a Julia deployment strategy

[asinghvi17]

#16

the script here is mainly copied code from Documenter.jl. The reason to do this instead of using Documenter directly is that Documenter enforces the concept of "stable" and "dev" directories - which we don't want, since all commits to master get immediately deployed to jl.geocompx.org/$chapter.

Note that this does not lock us in to a Julia deployment pipeline - if we want to simply remove this and go back to quarto publish, the only thing we lose is preview docs.

[asinghvi17]

https://jl.geocompx.org/previews/pr18/ 🎉

[asinghvi17]

@Robinlovelace you might be interested in this! It basically deploys all PR builds to a CNAME/previews/PR## path in gh-pages. I'm sure there's a way to simplify the code :D - a lot of it is just managing git.

[Nowosad]

@asinghvi17 would it not be easier to have second .yaml that only runs for PRs?

[Robinlovelace]

@Robinlovelace you might be interested in this! It basically deploys all PR builds to a CNAME/previews/PR## path in gh-pages. I'm sure there's a way to simplify the code :D - a lot of it is just managing git.

For sure, seeing previews of PRs in a publicly available URL would be handy. I've seen it before and netlify provides quick and simple ways of doing it. Is it all happening on GitHub pages? One thing to be aware of is repo sizes if everything gets to committed to gh pages it will grow quick. Already 25 MB in size, check this out also:

geocompjl git:(main) git checkout gh-pages 
branch 'gh-pages' set up to track 'origin/gh-pages'.
Switched to a new branch 'gh-pages'
➜  geocompjl git:(gh-pages) ls
chapters  index.html  previews  references.html  robots.txt  search.json  site_libs  sitemap.xml
➜  geocompjl git:(gh-pages) dust
4.2M       ┌── figure-html                                         │                                                                                        ░█████████ │   8%
4.2M     ┌─┴ 05-raster-vector_files                                │                                                                                        ░█████████ │   8%
5.1M   ┌─┴ chapters                                                │                                                                                        ██████████ │  10%
4.2M   │       ┌── figure-html                                     │                                                             ░░░░░░░░░░░░░░░░░░░░░░░░▒▒▒▓█████████ │   8%
4.2M   │     ┌─┴ 05-raster-vector_files                            │                                                             ░░░░░░░░░░░░░░░░░░░░░░░░▒▒▒▓█████████ │   8%
5.1M   │   ┌─┴ chapters                                            │                                                             ░░░░░░░░░░░░░░░░░░░░░░░░▒▒▒██████████ │  10%
6.4M   │ ┌─┴ PR18                                                  │                                                             ░░░░░░░░░░░░░░░░░░░░░░░░█████████████ │  12%
4.2M   │ │     ┌── figure-html                                     │                                                             ░░░░░░░░░░░░░░░░░░░░░░░░▒▒▒▓█████████ │   8%
4.2M   │ │   ┌─┴ 05-raster-vector_files                            │                                                             ░░░░░░░░░░░░░░░░░░░░░░░░▒▒▒▓█████████ │   8%
5.1M   │ │ ┌─┴ chapters                                            │                                                             ░░░░░░░░░░░░░░░░░░░░░░░░▒▒▒██████████ │  10%
6.4M   │ ├─┴ PR20                                                  │                                                             ░░░░░░░░░░░░░░░░░░░░░░░░█████████████ │  12%
4.2M   │ │     ┌── figure-html                                     │                                                             ░░░░░░░░░░░░░░░░░░░░░░░░▒▒▒▓█████████ │   8%
4.2M   │ │   ┌─┴ 05-raster-vector_files                            │                                                             ░░░░░░░░░░░░░░░░░░░░░░░░▒▒▒▓█████████ │   8%
5.1M   │ │ ┌─┴ chapters                                            │                                                             ░░░░░░░░░░░░░░░░░░░░░░░░▒▒▒██████████ │  10%
6.6M   │ ├─┴ PR23                                                  │                                                             ░░░░░░░░░░░░░░░░░░░░░░░░█████████████ │  13%
 19M   ├─┴ previews                                                │                                                             █████████████████████████████████████ │  38%
 25M   │     ┌── pack-f5acb7d1a8335bd50049c4ac5778cd42fce703e9.pack│                                                ░█████████████████████████████████████████████████ │  50%
 25M   │   ┌─┴ pack                                                │                                                ░█████████████████████████████████████████████████ │  50%
 25M   │ ┌─┴ objects                                               │                                                ░█████████████████████████████████████████████████ │  50%
 25M   ├─┴ .git                                                    │                                                ██████████████████████████████████████████████████ │  50%
 51M ┌─┴ .                  

Contributor experience is important. If you can avoid output files going into the commit history, in whichever branch, that is good (not that geocompr or geocompy did a good job on this, you can maybe learn from our experiences).

[Robinlovelace]

Also: in terms of the code. It's a LOT. Netlify support for this is great btw and could reduce amount of data pushed to Git. As far as I recall we don't push outputs to gh-pages anymore. https://docs.netlify.com/site-deploys/deploy-previews/

[Robinlovelace]

A LOT here, so maybe not the easiest thing to maintain/contribute-to: https://github.com/geocompx/geocompjl/pull/18/files

[asinghvi17]

For sure! I wasn't aware that Netlify does this. Do you think it makes sense to switch the provider for geocompjl to netlify?

Yes, this is all happening on GH pages. We usually force push to gh-pages to avoid this, but that can run into race conditions if multiple PRs build at the same time. Usually it's not too bad, though.

[Robinlovelace]

For sure! I wasn't aware that Netlify does this. Do you think it makes sense to switch the provider for geocompjl to netlify?

Probably.

[Robinlovelace]

We have a free 'supporting open source' Netlify account serving many GB of data each month for all our book readers which is ace.

[Robinlovelace]

May need to do some things in said account to set things up... Maybe worth a call, @Nowosad is good at that kind of thing and I'd like to re-check our settings and learn so up for it at some point too.

[asinghvi17]

That would be great! And it would be nice to switch away from gh-pages :)

[Nowosad]

Hi @asinghvi17,

(side note: I am definitely not an expert in this topics, thus better solutions may exist)

The current workflow is as follows: any push to main takes all of the quarto files and tries to render them to html. When this part is successful, the HTML outputs are pushed to the gh-pages branch. Then the change to gh-pages is detected by netlify, which published the new version of the book at jl.geocompx.com.

From what I understand, the netlify previews only work when the PR is done to the publishing branch of the repo (i.e. gh-pages in our case). However, all of the PR will be made to the main brach, and thus I think this option is not going to work.

One possible alternative is to write another GH action workflow that pushes to another branch after rendering, and then we can connect another url to that branch. The drawback of this approach is that the last PR will overwrite the previous one.

Do you have any other ideas?

[Robinlovelace]

Oh yes regarding gh-pages on our other books. In light of Netlify services and the benefits of keeping repo sizes down, making gh repo clone geocompx/geocomp* quicker for users and potential contributors, I think it could be useful to see the gh-pages-generating actions as redundant to requirements, netlify API keys can be used to ship all contents of the output (_site?) folder without touching git. That feels like a good workflow to me.

As a motivation, try running gh repo clone geocompx/geocompy and see how long it takes.