ceph: Enable separate builds of ceph & rgw FSALs #16
+295
−28
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,46 @@ | ||
| # | ||
| # from: https://raw.githubusercontent.com/kbsingh/centos-ci-scripts/master/build_python_script.py | ||
| # | ||
| # This script uses the Duffy node management api to get fresh machines to run | ||
| # your CI tests on. Once allocated you will be able to ssh into that machine | ||
| # as the root user and setup the environ | ||
| # | ||
| # XXX: You need to add your own api key below, and also set the right cmd= line | ||
| # needed to run the tests | ||
| # | ||
| # Please note, this is a basic script, there is no error handling and there are | ||
| # no real tests for any exceptions. Patches welcome! | ||
|
|
||
| import json, urllib, subprocess, sys, os | ||
|
|
||
| url_base="http://admin.ci.centos.org:8080" | ||
| # we just build on CentOS-7/x86_64, CentOS-6 does not have 'mock'? | ||
| ver="7" | ||
| arch="x86_64" | ||
| count=1 | ||
| script_url=os.getenv("TEST_SCRIPT") | ||
|
|
||
| # read the API key for Duffy from the ~/duffy.key file | ||
| fo=open("/home/nfs-ganesha/duffy.key") | ||
| api=fo.read().strip() | ||
| fo.close() | ||
|
|
||
| # build the URL to request the system(s) | ||
| get_nodes_url="%s/Node/get?key=%s&ver=%s&arch=%s&count=%s" % (url_base,api,ver,arch,count) | ||
|
|
||
| # request the system | ||
| dat=urllib.urlopen(get_nodes_url).read() | ||
| b=json.loads(dat) | ||
|
|
||
| cmd="""ssh -t -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no root@%s ' | ||
| yum -y install curl && | ||
| curl -o build.sh %s && | ||
| CENTOS_VERSION="%s" CENTOS_ARCH="%s" GERRIT_HOST="%s" GERRIT_PROJECT="%s" GERRIT_REFSPEC="%s" bash build.sh' | ||
| """ % (b['hosts'][0], script_url, os.getenv("CENTOS_VERSION"), os.getenv("CENTOS_ARCH"), os.getenv("GERRIT_HOST"), os.getenv("GERRIT_PROJECT"), os.getenv("GERRIT_REFSPEC")) | ||
| rtn_code=subprocess.call(cmd, shell=True) | ||
|
|
||
| # return the system(s) to duffy | ||
| done_nodes_url="%s/Node/done?key=%s&ssid=%s" % (url_base, api, b['ssid']) | ||
| das=urllib.urlopen(done_nodes_url).read() | ||
|
|
||
| sys.exit(rtn_code) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,95 @@ | ||
| #!/bin/sh | ||
|
|
||
| set -e | ||
|
|
||
| # these variables need to be set | ||
| [ -n "${GERRIT_HOST}" ] | ||
| [ -n "${GERRIT_PROJECT}" ] | ||
| [ -n "${GERRIT_REFSPEC}" ] | ||
|
|
||
| # only use https for now | ||
| GIT_REPO="https://${GERRIT_HOST}/${GERRIT_PROJECT}" | ||
|
|
||
| SHAMAN_REPO_URL="https://shaman.ceph.com/api/repos/ceph/master/latest/centos/7/flavors/default/repo" | ||
| TIME_LIMIT=1200 | ||
| INTERVAL=30 | ||
| REPO_FOUND=0 | ||
|
|
||
| # poll shaman for up to 10 minutes | ||
| while [ "$SECONDS" -le "$TIME_LIMIT" ] | ||
| do | ||
| if `curl --fail -L $SHAMAN_REPO_URL > /etc/yum.repos.d/shaman.repo`; then | ||
| echo "Ceph repo file has been added from shaman" | ||
| REPO_FOUND=1 | ||
| break | ||
| else | ||
| sleep $INTERVAL | ||
| fi | ||
| done | ||
|
|
||
| if [[ "$REPO_FOUND" -eq 0 ]]; then | ||
| echo "Ceph lib repo does NOT exist in shaman" | ||
| exit 1 | ||
| fi | ||
|
|
||
| # basic packages to install | ||
| xargs yum -y install <<< " | ||
| git | ||
| bison | ||
| cmake | ||
| dbus-devel | ||
| flex | ||
| gcc-c++ | ||
| git | ||
| krb5-devel | ||
| libacl-devel | ||
| libblkid-devel | ||
| libcap-devel | ||
| libnfsidmap-devel | ||
| libwbclient-devel | ||
| redhat-rpm-config | ||
| rpm-build | ||
| glusterfs-api-devel | ||
| libcephfs-devel | ||
| librgw-devel | ||
| " | ||
|
|
||
| git clone ${GIT_REPO} | ||
| cd $(basename "${GERRIT_PROJECT}") | ||
| git fetch origin ${GERRIT_REFSPEC} && git checkout FETCH_HEAD | ||
|
|
||
| # update libntirpc | ||
| git submodule update --init || git submodule sync | ||
|
|
||
| # cleanup old build dir | ||
| [ -d build ] && rm -rf build | ||
|
|
||
| mkdir build | ||
| cd build | ||
|
|
||
| ( cmake -DCMAKE_BUILD_TYPE=Maintainer ../src && make rpm ) || touch FAILED | ||
|
|
||
| # dont vote if the subject of the last change includes the word "WIP" | ||
| if ( git log --oneline -1 | grep -q -i -w 'WIP' ) | ||
| then | ||
| echo "Change marked as WIP, not posting result to GerritHub." | ||
| touch WIP | ||
| fi | ||
|
|
||
| # we accept different return values | ||
| # 0 - SUCCESS + VOTE | ||
| # 1 - FAILED + VOTE | ||
| # 10 - SUCCESS + REPORT ONLY (NO VOTE) | ||
| # 11 - FAILED + REPORT ONLY (NO VOTE) | ||
|
|
||
| RET=0 | ||
| if [ -e FAILED ] | ||
| then | ||
| RET=$[RET + 1] | ||
| fi | ||
| if [ -e WIP ] | ||
| then | ||
| RET=$[RET + 10] | ||
| fi | ||
|
|
||
| exit ${RET} | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,154 @@ | ||
| <?xml version='1.0' encoding='UTF-8'?> | ||
| <project> | ||
| <actions/> | ||
| <description></description> | ||
| <keepDependencies>false</keepDependencies> | ||
| <properties> | ||
| <com.coravy.hudson.plugins.github.GithubProjectProperty plugin="[email protected]"> | ||
| <projectUrl>https://github.com/nfs-ganesha/nfs-ganesha/</projectUrl> | ||
| <displayName></displayName> | ||
| </com.coravy.hudson.plugins.github.GithubProjectProperty> | ||
| <com.chikli.hudson.plugin.naginator.NaginatorOptOutProperty plugin="[email protected]"> | ||
| <optOut>false</optOut> | ||
| </com.chikli.hudson.plugin.naginator.NaginatorOptOutProperty> | ||
| <hudson.model.ParametersDefinitionProperty> | ||
| <parameterDefinitions> | ||
| <hudson.model.StringParameterDefinition> | ||
| <name>BOOTSTRAP_SCRIPT</name> | ||
| <description>Python script that reserves a machine from Duffy and starts the ${TEST_SCRIPT}.</description> | ||
| <defaultValue>https://raw.githubusercontent.com/nfs-ganesha/ci-tests/centos-ci/build-fsal_ceph_and_rgw/bootstrap.py</defaultValue> | ||
| </hudson.model.StringParameterDefinition> | ||
| <hudson.model.StringParameterDefinition> | ||
| <name>TEST_SCRIPT</name> | ||
| <description>Test script to execute on the reserved machine, should move to the some NFS-Ganesha upstream testing repo at one point.</description> | ||
| <defaultValue>https://raw.githubusercontent.com/nfs-ganesha/ci-tests/centos-ci/build-fsal_ceph_and_rgw/build-fsal_ceph_and_rgw.sh</defaultValue> | ||
| </hudson.model.StringParameterDefinition> | ||
| <hudson.model.ChoiceParameterDefinition> | ||
| <name>GERRIT_HOST</name> | ||
| <description>Gerrit instance hosting the git repository.</description> | ||
| <choices class="java.util.Arrays$ArrayList"> | ||
| <a class="string-array"> | ||
| <string>review.gerrithub.io</string> | ||
| </a> | ||
| </choices> | ||
| </hudson.model.ChoiceParameterDefinition> | ||
| <hudson.model.ChoiceParameterDefinition> | ||
| <name>GERRIT_PROJECT</name> | ||
| <description>Gerrit project of the git repository.</description> | ||
| <choices class="java.util.Arrays$ArrayList"> | ||
| <a class="string-array"> | ||
| <string>ffilz/nfs-ganesha</string> | ||
| </a> | ||
| </choices> | ||
| </hudson.model.ChoiceParameterDefinition> | ||
| <hudson.model.ChoiceParameterDefinition> | ||
| <name>GERRIT_REFSPEC</name> | ||
| <description>Patch set from the upstream git repository to build from.</description> | ||
| <choices class="java.util.Arrays$ArrayList"> | ||
| <a class="string-array"> | ||
| <string>refs/changes/81/276581/1</string> | ||
| </a> | ||
| </choices> | ||
| </hudson.model.ChoiceParameterDefinition> | ||
| <hudson.model.ChoiceParameterDefinition> | ||
| <name>CENTOS_VERSION</name> | ||
| <description>CentOS version to build the RPMs for.</description> | ||
| <choices class="java.util.Arrays$ArrayList"> | ||
| <a class="string-array"> | ||
| <string>7</string> | ||
| <string>6</string> | ||
|
There was a problem hiding this comment. Are you sure you want to provide testing for CentOS-6? |
||
| </a> | ||
| </choices> | ||
| </hudson.model.ChoiceParameterDefinition> | ||
| <hudson.model.ChoiceParameterDefinition> | ||
| <name>CENTOS_ARCH</name> | ||
| <description>CentOS architecture to build the RPMs for.</description> | ||
| <choices class="java.util.Arrays$ArrayList"> | ||
| <a class="string-array"> | ||
| <string>x86_64</string> | ||
| <string>i386</string> | ||
|
There was a problem hiding this comment. Are there i386 compatible RPMs in the shaman repository? |
||
| </a> | ||
| </choices> | ||
| </hudson.model.ChoiceParameterDefinition> | ||
| </parameterDefinitions> | ||
| </hudson.model.ParametersDefinitionProperty> | ||
| </properties> | ||
| <scm class="hudson.scm.NullSCM"/> | ||
| <assignedNode>nfs-ganesha</assignedNode> | ||
| <canRoam>false</canRoam> | ||
| <disabled>false</disabled> | ||
| <blockBuildWhenDownstreamBuilding>false</blockBuildWhenDownstreamBuilding> | ||
| <blockBuildWhenUpstreamBuilding>false</blockBuildWhenUpstreamBuilding> | ||
| <triggers> | ||
| <com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.GerritTrigger plugin="[email protected]"> | ||
| <spec></spec> | ||
| <gerritProjects> | ||
| <com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.data.GerritProject> | ||
| <compareType>PLAIN</compareType> | ||
| <pattern>ffilz/nfs-ganesha</pattern> | ||
| <branches> | ||
| <com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.data.Branch> | ||
| <compareType>ANT</compareType> | ||
| <pattern>**</pattern> | ||
| </com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.data.Branch> | ||
| </branches> | ||
| <disableStrictForbiddenFileVerification>false</disableStrictForbiddenFileVerification> | ||
| </com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.data.GerritProject> | ||
| </gerritProjects> | ||
| <skipVote> | ||
| <onSuccessful>false</onSuccessful> | ||
| <onFailed>false</onFailed> | ||
| <onUnstable>false</onUnstable> | ||
| <onNotBuilt>false</onNotBuilt> | ||
| </skipVote> | ||
| <silentMode>true</silentMode> | ||
| <notificationLevel></notificationLevel> | ||
| <silentStartMode>false</silentStartMode> | ||
| <escapeQuotes>true</escapeQuotes> | ||
| <noNameAndEmailParameters>false</noNameAndEmailParameters> | ||
| <dependencyJobsNames></dependencyJobsNames> | ||
| <readableMessage>false</readableMessage> | ||
| <buildStartMessage></buildStartMessage> | ||
| <buildFailureMessage></buildFailureMessage> | ||
| <buildSuccessfulMessage></buildSuccessfulMessage> | ||
| <buildUnstableMessage></buildUnstableMessage> | ||
| <buildNotBuiltMessage></buildNotBuiltMessage> | ||
| <buildUnsuccessfulFilepath></buildUnsuccessfulFilepath> | ||
| <customUrl></customUrl> | ||
| <serverName>nfs-ganesha-gerrithub</serverName> | ||
| <triggerOnEvents> | ||
| <com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.events.PluginPatchsetCreatedEvent> | ||
| <excludeDrafts>true</excludeDrafts> | ||
| <excludeTrivialRebase>false</excludeTrivialRebase> | ||
| <excludeNoCodeChange>true</excludeNoCodeChange> | ||
| </com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.events.PluginPatchsetCreatedEvent> | ||
| <com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.events.PluginCommentAddedContainsEvent> | ||
| <commentAddedCommentContains>recheck ceph and rgw</commentAddedCommentContains> | ||
| </com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.events.PluginCommentAddedContainsEvent> | ||
| </triggerOnEvents> | ||
| <dynamicTriggerConfiguration>false</dynamicTriggerConfiguration> | ||
| <triggerConfigURL></triggerConfigURL> | ||
| <triggerInformationAction/> | ||
| </com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.GerritTrigger> | ||
| </triggers> | ||
| <concurrentBuild>false</concurrentBuild> | ||
| <builders> | ||
| <hudson.tasks.Shell> | ||
| <command># no need for verbose output | ||
| set +x | ||
|
|
||
| # do not immediately fail on an error | ||
| set +e | ||
|
|
||
| # download and run the bootstrap script | ||
| curl -o bootstrap.py ${BOOTSTRAP_SCRIPT} | ||
| python bootstrap.py | ||
| RET=$? | ||
|
|
||
| # exit with SUCCESS or FAIL only | ||
| exit ${EXIT}</command> | ||
|
|
||
| </hudson.tasks.Shell> | ||
| </builders> | ||
| <publishers/> | ||
| <buildWrappers/> | ||
| </project> | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You should use
${CENTOS_VERSION}instead of 7. CentOS-6 can be provided through the jobs too (and that should probably fail while installing dependencies).