-
Notifications
You must be signed in to change notification settings - Fork 29
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make travis jobs faster #410
Comments
After setting up the solution described above, I am not convinced this will help to have faster feedback. Here's a set of jobs as an example. When I worked on the patch, I noticed results similar to that one. Moreover, I realized we can't use docker instances on the subcrates jobs like I proposed. The dockers instances are running Ubuntu Precise, and the openzwave-adapter requires a version of gcc that allows Hence, I wonder if we shouldn't use a different CI platform. @samgiles proposed to set up a buildbot like rust's bors. Setting up a production-ready buildbot will likely take some time. I wonder if using taskcluster-github wouldn't be easier to set and scale up. Moreover, combined with Treeherder, it would allow us to display 3-dimensional-matrices of jobs (platform/suites/jobs), whereas Travis stucks us to a sorted linear list. What do you think @fabricedesre ? |
We don't have time now to set up a whole new buildbot infra. My biggest gripe was the intermittent failures which are now fixed. |
I agree. As it would take too much time, let's leave this issue aside, for now. |
Another approach suggested by @fabricedesre : Don't run coverage when we know it's not going to change on PRs. Cases we could be smarter onI think we can also extend that reasoning to : Run only tests that are necessary. For instance:
Exclusion vs inclusionAt first it sounds safer to define exclusion rules (like described in the table above). If we went the other way (aka "when this change happens, run only these steps"), we'd have to declare what steps to run, each time we add/change a piece of code. This can be easily missed. I believe that a test is better if it's run too often rather than never. PRs vs masterThe exclusion rules might be buggy. Based on that, I would recommend to keep running every test on master, and just narrow down the PRs. How we could proceed?A way to define these exclusions rules could be within files like Then a script will:
Going further?For rust code, we can probably avoid to manually define what should not be tested. We could parse |
With the dependencies being in the same repo, Travis results might take an hour overall. Time is split into 2 parts:
I see 2 ways of tackling these issues.
#1. Prevent multiple builds in the same job
In the Linux job particularly, we build foxbox or one of its dependencies many times:
cargo build
is runcargo test
is executed in foxboxcargo test
runs in a subcrate => We build every dependency one more time. This a missing feature in cargo. See RFC: Add workspaces to Cargo rust-lang/rfcs#1525cargo test
with dead code => every single dependency gets recompiled with no dead code optimization.In the cross-compile job, we also build once in release mode the other time in debug.
I would recommend to put each of these different builds in self-contained jobs. Moreover, if subcrates are compiled an tested in their own job, it would allow us to deal with publishing on crates.io in that same job.
#2. Run on the container based infrastructure whenever we can
That infrastructure is faster to spawn and has more available machine than the VM infra. Using them would be ideal.
However, most of our cases requires us to run in a VM:
Hence, the only cases I see we could use the container would be the
cargo test
in subcrates.The text was updated successfully, but these errors were encountered: