Skip to content

Init testcontainer integration tests using Ansible agent #169

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Dec 22, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Jenkinsfile
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
*/
buildPlugin(
forkCount: '1C', // run this number of tests in parallel for faster feedback. If the number terminates with a 'C', the value will be multiplied by the number of available CPU cores
useContainerAgent: true, // Set to `false` if you need to use Docker for containerized tests
useContainerAgent: false, // Set to `false` if you need to use Docker for containerized tests
configurations: [
[platform: 'linux', jdk: 21],
[platform: 'windows', jdk: 17],
Expand Down
129 changes: 15 additions & 114 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,8 @@ Jenkins jobs.

3. Repeat for any additional desired installations

There is no automatic ansible installation possible using Global Tools.

### OS User PATH

Ansible can also be added to the PATH user used by the Jenkins executor
Expand All @@ -40,6 +42,18 @@ guide.

------------------------------------------------------------------------

## Supported versions

The plugin is tested against supported ansible-core versions (https://endoflife.date/ansible-core). It might work with older versions, but this is not guaranteed.

See `PipelineTest.java`

```java
private static Stream<String> ansibleVersions() {
return Stream.of("2.14.13", "2.15.8", "2.16.2");
}
```

## Adhoc

[Adhoc commands](http://docs.ansible.com/ansible/latest/intro_adhoc.html) allow
Expand Down Expand Up @@ -289,122 +303,9 @@ a "Secret text" or a "Secret file".

------------------------------------------------------------------------

## Open Issues

[View issues in Jira](https://issues.jenkins.io/secure/IssueNavigator.jspa?reset=true&jqlQuery=component%20=%20ansible-plugin%20AND%20status%20in%20%28Open,%20%22In%20Progress%22,%20Reopened%29&tempMax=1000&src=confmacro)

------------------------------------------------------------------------

## Changelog

#### Version 1.0 (26 March 2018)

* [Fix security issue](https://jenkins.io/security/advisory/2018-03-26/#SECURITY-630):
Do not disable host key verification by default. **This may
break existing configurations as host key verification will
be enabled everywhere by default.**

#### Version 0.8.0 (16 Jan 2018)

* Add support for Ansible Vault
[JENKINS-48499](https://issues.jenkins.io/browse/JENKINS-48499)
* Add hostKeyChecking option to pipeline [JENKINS-42445](https://issues.jenkins.io/browse/JENKINS-42445)

#### Version 0.6.2 (3 Jan 2017)

* Fix blocker bug when launched from a pipeline
[JENKINS-40780](https://issues.jenkins.io/browse/JENKINS-40780)

#### Version 0.6.1 (1 Jan 2017)

* Use latest parent project definition in order to deploy
plugin (thanks
to [alecharp](https://github.com/alecharp) for the help and
the PR)

#### Version 0.6 (31 Dec 2016)

**WARN: 0.6.x version will be the last one to support Jenkins 1.xxx and
Ansible 1.x - The 0.7.x and next releases will require Jenkins 2.32.1
(or higher) and Ansible 2.2 (or higher)**

* Add a "do not specify" option for
inventory [JENKINS-34627](https://issues.jenkins.io/browse/JENKINS-34627)
* Support inventoryContent in pipeline (thanks
to [leewin12](https://github.com/leewin12) for the PR)
* Add support of extra variables in jobdsl (thanks
to [pawbur](https://github.com/pawbur) for the PR)
* Support empty forks (number of parallel processes)
parameter [JENKINS-39438](https://issues.jenkins.io/browse/JENKINS-39438)
* Escape '%' character in private key path (thanks
to [ewollesen](https://github.com/ewollesen) for the PR)
* Omit ansible option when expanded environment variable is
empty (thanks to [vjestin](https://github.com/vjestin) for
the PR)
* Add the --forks parameter configurable in pipeline step
(thanks to
[anguswilliams](https://github.com/anguswilliams) for the
PR)
* Fix usage of environment variable in ansiblePlaybook
pipeline step (thanks to
[thomasKalmar](https://github.com/thomasKalmar)
and [barthorre](https://github.com/barthorre) for the
PR) [JENKINS-38289](https://issues.jenkins.io/browse/JENKINS-38289)

#### Version 0.5 (5 May 2016)

* Add support for ansible extra variables
[JENKINS-29863](https://issues.jenkins.io/browse/JENKINS-29863)
* Improve Pipeline plugin
integration [JENKINS-32911](https://issues.jenkins.io/browse/JENKINS-32911)
* Add the possibility to use the default inventory file
(thanks to Johann Schmitz for the PR)
* Add colorized output in pipeline jobs (thanks to
Kirill Merkushev for the PR)
* Make Jenkins build variables available as environment
variables for ansible (thanks to Kevin Mooney for the
PR) [JENKINS-29284](https://issues.jenkins.io/browse/JENKINS-29284)

#### Version 0.4 (25 December 2015)

* Support for password protected SSH
keys [JENKINS-30656](https://issues.jenkins.io/browse/JENKINS-30656)
* Initial support for the workflow
plugin [JENKINS-30398](https://issues.jenkins.io/browse/JENKINS-30398)
* Add support for Job DSL plugin (thanks to Kirill Merkushev
for the
PR) [JENKINS-31790](https://issues.jenkins.io/browse/JENKINS-31790)

#### Version 0.3.1 (15 July 2015)

* Fix execution on slave
nodes [JENKINS-29294](https://issues.jenkins.io/browse/JENKINS-29294)

#### Version 0.3 (20 June 2015)

* Add support for password based SSH authentication (with
sshpass)
* Environment variables can be used in Module and Module
arguments text field in Ad-hoc command builder
* Environment variables can be used in inline inventory text
box
[JENKINS-28547](https://issues.jenkins.io/browse/JENKINS-28547)

#### Version 0.2 (11 May 2015)

* Fix NullPointerException when no credentials are selected
* Fix --skippedTags parameter configuration which was ignored
* Fix NullPointerException and print an error message in the
build console when the inventory is not set in the job
configuration

#### Version 0.1 (01 May 2015)

* Initial version

This plugin gives the possibility to run [Ansible](http://www.ansible.com/) ad-hoc command or playbooks as a build step.

[![Build Status](https://ci.jenkins.io/buildStatus/icon?job=Plugins/ansible-plugin/master)](https://ci.jenkins.io/job/Plugins/job/ansible-plugin/job/master/)
Changelog is now published on GitHub release.

## Using Jenkins Build and Environment Variables

Expand Down
37 changes: 33 additions & 4 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,10 @@
<gitHubRepo>jenkinsci/ansible-plugin</gitHubRepo>
<jenkins.version>2.426.2</jenkins.version>
<spotless.check.skip>false</spotless.check.skip>

<!-- Test dependencies version -->
<testcontainer.version>1.19.3</testcontainer.version>

</properties>
<dependencyManagement>
<dependencies>
Expand Down Expand Up @@ -66,12 +70,12 @@
<groupId>org.jenkins-ci.plugins</groupId>
<artifactId>ssh-credentials</artifactId>
</dependency>
<!-- Test dependencies -->
<dependency>
<groupId>org.jenkins-ci.plugins.workflow</groupId>
<artifactId>workflow-step-api</artifactId>
<optional>true</optional>
</dependency>
<!-- Test -->
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
Expand All @@ -83,16 +87,15 @@
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.jenkins-ci.plugins.workflow</groupId>
<artifactId>workflow-basic-steps</artifactId>
<groupId>org.jenkins-ci.plugins</groupId>
<artifactId>ssh-slaves</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.jenkins-ci.plugins.workflow</groupId>
<artifactId>workflow-cps</artifactId>
<scope>test</scope>
</dependency>
<!-- Test plugins -->
<dependency>
<groupId>org.jenkins-ci.plugins.workflow</groupId>
<artifactId>workflow-job</artifactId>
Expand All @@ -108,6 +111,32 @@
<artifactId>mockito-core</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>junit-jupiter</artifactId>
<version>${testcontainer.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>testcontainers</artifactId>
<version>${testcontainer.version}</version>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>org.apache.commons</groupId>
<artifactId>commons-compress</artifactId>
</exclusion>
<exclusion>
<groupId>org.jetbrains</groupId>
<artifactId>annotations</artifactId>
</exclusion>
<exclusion>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
</exclusion>
</exclusions>
</dependency>
</dependencies>
<repositories>
<repository>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -239,7 +239,9 @@ public void perform(
? CredentialsProvider.findCredentialById(vaultCredentialsId, StandardCredentials.class, run)
: null);
invocation.setVaultTmpPath(
StringUtils.isNotBlank(vaultTmpPath) ? new FilePath(new File(vaultTmpPath)) : null);
StringUtils.isNotBlank(vaultTmpPath)
? new FilePath(computer.getChannel(), new File(vaultTmpPath).getAbsolutePath())
: null);
invocation.setExtraVars(extraVars);
invocation.setAdditionalParameters(additionalParameters);
invocation.setDisableHostKeyCheck(disableHostKeyChecking);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -266,6 +266,7 @@ public void perform(
throws InterruptedException, IOException {
try {
CLIRunner runner = new CLIRunner(run, ws, launcher, listener);
Computer computer = node.toComputer();
String exe = AnsibleInstallation.getExecutable(
ansibleName, AnsibleCommand.ANSIBLE_PLAYBOOK, node, listener, envVars);
AnsiblePlaybookInvocation invocation = new AnsiblePlaybookInvocation(exe, run, ws, listener, envVars);
Expand Down Expand Up @@ -295,7 +296,9 @@ public void perform(
run)
: null);
invocation.setVaultTmpPath(
StringUtils.isNotBlank(vaultTmpPath) ? new FilePath(new File(vaultTmpPath)) : null);
StringUtils.isNotBlank(vaultTmpPath)
? new FilePath(computer.getChannel(), new File(vaultTmpPath).getAbsolutePath())
: null);
invocation.setExtraVars(extraVars);
invocation.setAdditionalParameters(additionalParameters);
invocation.setDisableHostKeyCheck(disableHostKeyChecking);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -125,6 +125,7 @@ public void perform(
throws InterruptedException, IOException {
try {
CLIRunner runner = new CLIRunner(run, ws, launcher, listener);
Computer computer = node.toComputer();
String exe = AnsibleInstallation.getExecutable(
ansibleName, AnsibleCommand.ANSIBLE_VAULT, node, listener, envVars);
AnsibleVaultInvocation invocation = new AnsibleVaultInvocation(exe, run, ws, listener, envVars);
Expand All @@ -144,7 +145,9 @@ public void perform(
run)
: null);
invocation.setVaultTmpPath(
StringUtils.isNotBlank(vaultTmpPath) ? new FilePath(new File(vaultTmpPath)) : null);
StringUtils.isNotBlank(vaultTmpPath)
? new FilePath(computer.getChannel(), new File(vaultTmpPath).getAbsolutePath())
: null);
invocation.setContent(content);
invocation.setInput(input);
invocation.setOutput(output);
Expand Down
3 changes: 3 additions & 0 deletions src/main/java/org/jenkinsci/plugins/ansible/Utils.java
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ static FilePath createSshKeyFile(FilePath key, FilePath tmpPath, SSHUserPrivateK
static FilePath createSshAskPassFile(
FilePath script, FilePath tmpPath, SSHUserPrivateKey credentials, boolean inThisDir)
throws IOException, InterruptedException {
tmpPath.mkdirs();
StringBuilder sb = new StringBuilder();
sb.append("#! /bin/sh\n").append("/bin/echo \"" + Secret.toString(credentials.getPassphrase()) + "\"");
script = tmpPath.createTextTempFile("ssh", ".sh", sb.toString(), inThisDir);
Expand All @@ -69,6 +70,7 @@ static FilePath createSshAskPassFile(
static FilePath createVaultPasswordFile(FilePath key, FilePath tmpPath, FileCredentials credentials)
throws IOException, InterruptedException {
try (InputStream content = credentials.getContent()) {
tmpPath.mkdirs();
key = tmpPath.createTempFile("vault", ".password");
key.copyFrom(content);
key.chmod(0400);
Expand All @@ -87,6 +89,7 @@ static FilePath createVaultPasswordFile(FilePath key, FilePath tmpPath, FileCred
*/
static FilePath createVaultPasswordFile(FilePath key, FilePath tmpPath, StringCredentials credentials)
throws IOException, InterruptedException {
tmpPath.mkdirs();
key = tmpPath.createTextTempFile(
"vault", ".password", credentials.getSecret().getPlainText(), true);
key.chmod(0400);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,16 +6,12 @@
import static org.hamcrest.Matchers.hasEntry;
import static org.mockito.Mockito.*;

import com.cloudbees.jenkins.plugins.sshcredentials.SSHUserPrivateKey;
import com.cloudbees.plugins.credentials.common.StandardUsernamePasswordCredentials;
import hudson.EnvVars;
import hudson.model.AbstractBuild;
import hudson.model.BuildListener;
import hudson.model.TaskListener;
import hudson.util.ArgumentListBuilder;
import hudson.util.Secret;
import java.util.Map;
import org.junit.Ignore;
import org.junit.Test;
import org.mockito.ArgumentCaptor;

Expand Down Expand Up @@ -128,66 +124,6 @@ public void secure_by_default_SEC_630() throws Exception {
assertThat((Map<String, String>) argument.getValue(), hasEntry("ANSIBLE_FORCE_COLOR", "true"));
}

@Test
@Ignore("build.getWorkspace() cannot be mocked")
public void should_handle_private_key_credentials() throws Exception {
// Given
Inventory inventory = new InventoryPath("/tmp/hosts");
SSHUserPrivateKey pkey = mock(SSHUserPrivateKey.class);
when(pkey.getUsername()).thenReturn("mylogin");
BuildListener listener = mock(BuildListener.class);
CLIRunner runner = mock(CLIRunner.class);
AbstractBuild<?, ?> build = mock(AbstractBuild.class);
when(build.getEnvironment(any(TaskListener.class))).thenReturn(new EnvVars());
AnsibleAdHocCommandInvocation invocation =
new AnsibleAdHocCommandInvocation("/usr/local/bin/ansible", build, listener);
invocation.setHostPattern("localhost");
invocation.setInventory(inventory);
invocation.setModule("ping");
invocation.setCredentials(pkey);
invocation.setForks(5);
// When
invocation.execute(runner);
// Then
ArgumentCaptor<ArgumentListBuilder> argument = ArgumentCaptor.forClass(ArgumentListBuilder.class);
verify(runner).execute(argument.capture(), anyMap());

assertThat(
argument.getValue().toString(),
is("/usr/local/bin/ansible localhost -i /tmp/hosts -m ping -f 5 --private-key .+ -u mylogin"));
}

@Test
@Ignore("Secret can neither be instanced nor mocked")
public void should_handle_password_credentials() throws Exception {
// Given
Inventory inventory = new InventoryPath("/tmp/hosts");
StandardUsernamePasswordCredentials password = mock(StandardUsernamePasswordCredentials.class);
when(password.getUsername()).thenReturn("mylogin");
when(password.getPassword()).thenReturn(Secret.fromString("aStrongSecretPassword"));
BuildListener listener = mock(BuildListener.class);
CLIRunner runner = mock(CLIRunner.class);
AbstractBuild<?, ?> build = mock(AbstractBuild.class);
when(build.getEnvironment(any(TaskListener.class))).thenReturn(new EnvVars());
AnsibleAdHocCommandInvocation invocation =
new AnsibleAdHocCommandInvocation("/usr/local/bin/ansible", build, listener);
invocation.setHostPattern("localhost");
invocation.setInventory(inventory);
invocation.setModule("ping");
invocation.setCredentials(password);
invocation.setForks(5);
// When
invocation.execute(runner);
// Then
ArgumentCaptor<ArgumentListBuilder> argument = ArgumentCaptor.forClass(ArgumentListBuilder.class);
verify(runner).execute(argument.capture(), anyMap());

assertThat(
argument.getValue().toString(),
is("sshpass ****** /usr/local/bin/ansible localhost -i /tmp/hosts -m ping -f 5 " + "-u"
+ " mylogin -k"));
}

@Test
public void should_handle_variables() throws Exception {
// Given
Expand Down
Loading