add Rook to cross-cloud CI testing #137
Comments
|
We would love to get going on this. Where can we find more info? @denverwilliams @lixuna any pointers? thanks! |
|
Hi, @travisn. We'd welcome help in getting Rook on the CNCF CI dashboard as well as collaborations on CI/CD in the community as well. It sounds like Rook is already doing some CI which maybe someone would like to talk about at the next CNCF CI WG meeting (Jan 22nd). Regarding the cncf.ci dashboard we have short and longer term options. We don't have a quick answer to help you get started unfortunately. While we are working on new documentation and platform updates to allow projects to help directly to add themselves, that work is still in-progress and not yet ready. In the short term adding a project to the cncf.ci dashboard requires re-implementing part of a projects build/deploy/test process in Gitlab and that is not a self-service process unfortunately. I'll go more into that below. Longer term, we are planning a new iteration of the dashboard which includes integration+collaboration with projects after Kubecon Seattle 2018. Maybe we can meet in-person in Seattle at Kubecon to discuss both the short and longer term options. An overview of what how the system currently works should be helpful. Currently the dashboard is populated based on data pulled from the underlying CI platform (Gitlab) or pulled from external CI systems (if we have created an integration). At the moment the only integrations to external CI systems are for build artifacts. K8s is currently built from source (release + HEAD on master). Then k8s is deployed (using the (cross-cloud provisioner) to all of the supported cloud providers. After the builds artifacts are collected for the projects deployments to all of the k8s clusters happens. At which point integration tests can be run if any exists. Some options for integration of Rook deploy+tests on the dashboard:
Option 1 has been the default choice. It would be similar to what you have on Jenkins. Gitlab has a configuration file that has the commands to create a build and run whatever other commands are needed. We would need to see how Rook would be deployed and tested with k8s. Implementation in the various gitlab stages including running the integration tests would be similar to Jenkins. Option 2 is similar, expect the build artifacts are pulled from Rook's Jenkins server (or some other artifact repository). Either option will require taking build and test process used in the Rook Jenkins server and re-implementing in the Gitlab system. Fyi, A future option we hope to provide is pulling all status information from a projects CI system and not running anything in a different system. It would be something like, Rook Jenkins + CNCF.ci Dashboad. Help Jenkins to use cross-cloud for provisioning k8s clusters and deploy Rook to those clusters. Integrate Rook Jenkins results with Cross-Cloud CI status repository which provides data for the Dashboard. The different Cross-Cloud CI components can be used outside of the implementation for the CNCF dashboard. So implementing one of the current options... The first thing would be determining where/how Rook fits in with the current system and dashboard. Since Rook provides a storage orchestrator for K8s I'm not sure where it would fit in and what integration tests we want to highlight. The other projects run on the clusters themselves in containers. We need to determine that first. Then implementation of the steps in Gitlab for build + integration tests. Otherwise that would be something someone on the Cross-Cloud CI team would work on. |
|
@taylor Thanks for all the background on what it might look like to integrate with the CNCF CI. We would really prefer to have the builds and integration tests all running as you described in option 1. Unfortunately I didn't think to think to discuss this at kubecon. We can plan on joining the community meeting next month to discuss. |
|
Thank you for the discussion today in the CI working group meeting. I think the CNCF CI v2.0 slides that were shown are going in a good direction that will result in more CNCF projects integrating with the CNCF CI efforts. The Rook project is definitely interested in integrating as well. From the roadmap, it looks like the March timeframe will add documentation for how to add a project to this effort, so we can engage further then. We initially had hopes for the CNCF hosting the full build/integration environment for us and getting rid of the burden of our own Jenkins server that we are hosting and managing on our own. But it sounds like that is not the direction that will be supported going forward, so we will need to look into how best our existing CI efforts with Jenkins can be integrated into the v2 efforts. |
|
Jared, thanks for participating in the call. The cross-cloud group is
definitely not aiming to take over CI responsibilities for any of the 20
graduated or incubating CNCF projects.
However, that does not mean the Rook project is on it's own for CI. Please
open a request at https://github.com/cncf/servicedesk and we would be happy
to work with you, perhaps by helping transition Rook to a hosted
Jenkins environment, or perhaps an alternative CI platform.
As long as you are producing container images as artifacts in your CI
process, cross-cloud should be able to integrate you into their testing
matrix fairly quickly.
--
Dan Kohn <[email protected]>
Executive Director, Cloud Native Computing Foundation https://www.cncf.io
+1-415-233-1000 https://www.dankohn.com
…On Tue, Jan 22, 2019 at 3:43 PM Jared Watts ***@***.***> wrote:
Thank you for the discussion today in the CI working group meeting
<https://docs.google.com/document/d/1NA4N6PvNEkHX1yzaDFr19Xlru-amRxNi2pliqudmYNA/edit#heading=h.f4yxlzcq0ckl>.
I think the CNCF CI v2.0 slides
<https://docs.google.com/presentation/d/1NbRstXKJU7y7rOV60gV4Hg2TY2sM6l3rche73XpiRRk/edit#slide=id.g4c4fbf5639_0_143>
that were shown are going in a good direction that will result in more CNCF
projects integrating with the CNCF CI efforts. The Rook project is
definitely interested in integrating as well. From the roadmap
<https://docs.google.com/presentation/d/1NbRstXKJU7y7rOV60gV4Hg2TY2sM6l3rche73XpiRRk/edit#slide=id.g4cae776d61_3_44>,
it looks like the March timeframe will add documentation for how to add a
project to this effort, so we can engage further then.
We initially had hopes for the CNCF hosting the full build/integration
environment for us and getting rid of the burden of our own Jenkins server
<https://jenkins.rook.io/> that we are hosting and managing on our own.
But it sounds like that is not the direction that will be supported going
forward, so we will need to look into how best our existing CI efforts
with Jenkins <https://jenkins.rook.io/> can be integrated into the v2
efforts.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#137 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AC8MBuLDh9dZBzDalEWtvfHNegQIucTTks5vF3fvgaJpZM4TCJtF>
.
|
|
Hi, @jbw976. It sounds like we have two efforts that can be worked on independently:
As we already have a Jenkins integration (for ONAP) I think integration with Rook existing Jenkins will lower the effort for adding the project to the dashboard. As we are closer to adding Rook we'll reach out to discuss items like published artifacts for different stages, smoke tests deployments, and e2e tests. Regarding CI for the Rook project I suggest looking at hosted options rather than maintaining your own CI system (as that is not something desired within the project). There are many choices which provide free hosting for open source projects, including:
Hosted CircleCI and TravisCI are being used by many projects in the CNCF and K8s community. If it's decided to continue self-hosting, I suggest looking at alternatives to Jenkins which should be easier to maintain such as Buildbot or CircleCI. |
|
We do not recommend LF Infra management as we are trying to reduce the
workloads that run in that environment and move more stuff to public
services.
--
Dan Kohn <[email protected]>
Executive Director, Cloud Native Computing Foundation https://www.cncf.io
+1-415-233-1000 https://www.dankohn.com
…On Thu, Jan 24, 2019 at 7:32 PM Taylor Carpenter ***@***.***> wrote:
Hi, @jbw976 <https://github.com/jbw976>.
It sounds like we have two efforts that can be worked on independently:
1. adding Rook to the cncf.ci dashboard
2. general CI hosting/help for the rook project
As we already have a Jenkins integration (for ONAP) I think integration
with Rook existing Jenkins will lower the effort for adding the project to
the dashboard. As we are closer to adding Rook we'll reach out to discuss
items like published artifacts for different stages, smoke tests
deployments, and e2e tests.
Regarding CI for the Rook project I suggest looking at hosted options
rather than maintaining your own CI system (as that is not something
desired within the project). There are many choices which provide free
hosting for open source projects, including:
- TravisCI
- OpenLab <https://openlabtesting.org/> (new request
<https://github.com/theopenlab/openlab/issues/new?template=test-request.md>
)
- LF Infra mangement
<https://www.linuxfoundation.org/projects/service/infrastructure-management/>
- Gitlab
- CircleCI - free, but may not work for Rook if you need more than 3
containers.
Hosted CircleCI and TravisCI are being used by many projects in the CNCF
and K8s community.
If it's decided to continue self-hosting, I suggest looking at
alternatives to Jenkins which should be easier to maintain such as
Buildbot <https://buildbot.net/> or CircleCI.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#137 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AC8MBhBMbRTFiFMh_lKlR_cwqAY1PYbpks5vGlC0gaJpZM4TCJtF>
.
|
|
Thanks for the pointers and discussion. It sounds like the short term and easier goal of adding rook to the cncf.ci v2 dashboard will be a great first step. Please let us know when the documentation or process for adding projects (march on the roadmap) is ready and we'll be happy to engage. |
As per discussion on the CI mailing list, we would love for Rook to be integrated with the cross-cloud CI testing provided by the CNCF.
Our current Jenkins setup is hosted on an instance in Amazon EC2 and can be accessed at https://jenkins.rook.io.
That master instance dynamically brings up slaves in EC2 and GCE to deploy various versions of Kubernetes to those public cloud environments and run end to end integration testing there. Here's a recent build from master that shows the small matrix we currently deploy and test on: https://jenkins.rook.io/blue/organizations/jenkins/rook%2Frook/detail/master/559/pipeline
We would love to expand this matrix and integrate with the same CI services/environment that other CNCF projects are using. Thanks!
/cc @bassam @travisn
The text was updated successfully, but these errors were encountered: