Cookbook cache collisions in chef server location

[karmix]

Version:

cookbook-omnifetch 0.11.1

Environment:

Linux, but not operating system specific.

Scenario:

When working with cookbooks from multiple chef server locations, chef may corrupt its cache, resulting in unexpected errors.

Cache keys for cookbooks from chef server locations are made up of only the cookbook name and version. Different sources can provide different cookbooks with the same name and version, resulting in cache collisions. To uniquely identify a cookbook, we should also include the server name, server port, and organization when constructing the cache key.

Steps to Reproduce:

The following commands create two orgs in chef-zero, upload a different cookbook to each org, create policies using each cookbook, install/cache the cookbooks, then try to push a policy to chef-zero resulting in an error message about not supporting modifications to cached cookbooks.

# Create temporary workspace
TMPDIR=$(mktemp -d)
export CHEF_WORKSTATION_HOME=$TMPDIR/chef-workstation
mkdir -p $CHEF_WORKSTATION_HOME
cd $TMPDIR

# Generate dummy key for signing API calls.
openssl genrsa -out client.pem

# Create config.rb
mkdir -p org-{a,b}/.chef
cat >org-a/.chef/config.rb <<END_OF_CONFIG
node_name       'node'
client_name     'client'
client_key      '$TMPDIR/client.pem'
END_OF_CONFIG
cp org-a/.chef/config.rb org-b/.chef/
echo "chef_server_url 'http://localhost:8889/organizations/org-a'" >>org-a/.chef/config.rb
echo "chef_server_url 'http://localhost:8889/organizations/org-b'" >>org-b/.chef/config.rb

# Start chef-zero
mkdir -p $TMPDIR/chef-zero
chef-zero --multi-org --file-store $TMPDIR/chef-zero &

# Create organizations
KNIFE_HOME=$TMPDIR/org-a/.chef knife org create org-a "Organization A"
KNIFE_HOME=$TMPDIR/org-b/.chef knife org create org-b "Organization B"

# Upload two cookbooks named baseline-1.0.0 with
#   identical metadata:
mkdir -p org-{a,b}/baseline/recipes
cat >org-a/baseline/metadata.rb <<END_OF_METADATA
name 'baseline'
version '1.0.0'
END_OF_METADATA
cp org-a/baseline/metadata.rb org-b/baseline/metadata.rb
#   different default recipes:
echo "file '/etc/org-a-baseline'" >>org-a/baseline/recipes/default.rb
echo "file '/etc/org-b-baseline'" >>org-b/baseline/recipes/default.rb

KNIFE_HOME=$TMPDIR/org-a/.chef knife cookbook upload baseline -o $TMPDIR/org-a
KNIFE_HOME=$TMPDIR/org-b/.chef knife cookbook upload baseline -o $TMPDIR/org-b

# Create Policyfiles
cat >org-a/Policyfile.rb <<END_OF_POLICYFILE
name 'org-a-policy'
default_source :chef_server, 'http://localhost:8889/organizations/org-a'
run_list 'baseline'
END_OF_POLICYFILE
cat >org-b/Policyfile.rb <<END_OF_POLICYFILE
name 'org-b-policy'
default_source :chef_server, 'http://localhost:8889/organizations/org-b'
run_list 'baseline'
END_OF_POLICYFILE

# Install baseline from org-a (caches baseline-1.0.0 from org A)
KNIFE_HOME=$TMPDIR/org-a/.chef chef install org-a/Policyfile.rb
# Install baseline from org-b (replaces cache with baseline-1.0.0 from orb B)
KNIFE_HOME=$TMPDIR/org-b/.chef chef install org-b/Policyfile.rb

# Upload policy for org-a (signature no longer matches and aborts)
KNIFE_HOME=$TMPDIR/org-a/.chef chef push org-a/Policyfile.lock.json

Expected Result:

In the last step, chef should successfully push the cookbook artifact for org-a's baseline cookbook to the chef server.

Actual Result:

When chef is told to push the cookbooks to org-a, it aborts with the following error:

Error: Invalid lockfile data
Reason: (ChefCLI::CachedCookbookModified) Cached cookbook `baseline' (1.0.0) has been modified since the lockfile was generated. Cached cookbooks cannot be modified. (full path: `/tmp/tmp.7E9h0QtanB/chef-workstation/cache/cookbooks/baseline-1.0.0')

The user never modified cookbooks in the cache; chef overwrote its own cached cookbook with another due to a cache key collision.