Add chef-vault implementation #minor #101

lanej · 2018-08-08T04:21:45Z

Currently only supports version 3 of the encryption/decryption scheme: See https://github.com/chef/chef/blob/7c9178e5dd4ec7b5a75993c148b087bac6e5b53d/lib/chef/encrypted_data_bag_item/encryptor.rb#L162-L222

Encrypt data and update a data bag
Decrypt data with the provided client key
Add client
Add admin

Fairly minimal and the testing is weak but I thought I'd share before progressing further.

This change is

lanej · 2018-08-08T04:27:36Z

It would be great to replace these shell commands : https://github.com/hashicorp/terraform/blob/c0793c84fdbea8c4f3f7597cad57bcca38b21ebf/builtin/provisioners/chef/resource_provisioner.go#L569-L617

MarkGibbons · 2019-07-21T17:52:01Z

Looks nice. It's taking me while to get up to speed on the code base. I'll try to get this evaluated over the next month.

lanej · 2019-07-22T21:27:44Z

@MarkGibbons great! LMK what you think and I'll fix the conflicts.

MarkGibbons · 2020-01-13T14:28:57Z

I don't see an already written way to list the available vault items.
ok - fixed
May want to create a databag if it doesn't already exist when creating data bag items. Nothing hard about creating a data bag.
ok -fixed.

MarkGibbons · 2020-01-13T14:32:53Z

Nice code, I'm writing examples and testing against a chef server. See https://github.com/go-chef/chef/tree/vault if you want to see. I merged your branch into my vault branch and will merge it once testing and maybe a couple other feature get added.

MarkGibbons · 2019-12-27T14:20:20Z

search_test.go

@@ -31,7 +31,6 @@ func TestSearch_Get(t *testing.T) {
 		"users":  "http://localhost:4000/search/users",
 	}
 	if !reflect.DeepEqual(indexes, wantedIdx) {


Note to self. Figure out if the deleted error message is needed.

MarkGibbons · 2020-01-11T15:30:20Z

vault.go

+const algorithm string = "aes-256-gcm"
+const supportedVersion int = 3
+
+var errUnsupportedVersion = errors.New("Only Version 3 encrypted values are supported")


Version 3 of what? Comment needs to be clearer.

V3 of the encrypted data bag support from chef. PR comments have a nice link.

Update comments in my version.

MarkGibbons · 2020-01-12T15:09:55Z

vault.go

+	Keys         *VaultItemKeys
+	Name         string
+	Vault        string
+	VaultService *VaultService


Trying to understand why vault service is part of the vault item structure.

Pass the client connection to Item processing. Need to be clear that get is required before decrypt or update.

MarkGibbons · 2020-01-12T15:11:39Z

vault.go

+}
+
+// CreateItem creates a vault item and keys databag
+func (vs *VaultService) CreateItem(vaultName, itemName string) (*VaultItem, error) {


Looks like Create Item creates an empty vault and then Update adds the content. Note to self, verify this and add doc to make it clear.

Also need to create the data bag before creating the vault items.
OK

MarkGibbons · 2020-01-12T15:16:54Z

vault.go

+	return item, nil
+}
+
+// UpdateItem sets the item data, encrypts with a shared key, and then encrypts the shared key with each authorized client key in the <item>_keys data bag


Is this function a total replacement of the contents of the vault item? Figure it out and doc. Any local data concept (knife -m solo concept) or does it always hit the server (knife -m client)

Not sure what mode does. Looks like the code is always aimed at the server.

MarkGibbons · 2020-01-12T18:24:51Z

vault.go

+		return nil, err
+	}
+
+	keysItem := map[string]interface{}{


How are the admins, client, search_query set and used? Figure that out, create code examples.

I think adding user to an item needs to be clearer.
Also may need to handle bad users (old keys).

MarkGibbons · 2020-01-16T13:54:10Z

vault.go

+	keys := map[string]bool{}
+
+	for name := range *databags {
+		if vaultName := strings.TrimSuffix(name, "_keys"); vaultName != name {


I don't believe this code finds chef vault data bags. The vaults don't have the _keys name, the items within the vaults have those names.

MarkGibbons · 2020-01-18T12:17:42Z

vault.go

+	databagData := DataBagItem(itemData)
+
+	item.DataBagItem = &databagData
+


We need to reencrypt the shared secret for the users in admins and clients as is we can't specify admins or clients which makes the feature hard to use. The search feature is also not implemented. I'm looking at adding admins and clients.

Corrected a few spelling mistakes along the way.

Add a compatible client implementation to the chef-vault ruby version.

The testing could be much better. I will throw the crypto tests I have in next. This testing is much more useful with a live chef server.

lanej · 2020-01-26T23:32:07Z

I rebased master but it seems that your other branch is pretty far along. This code base has come long way since I opened this PR. Let me know if I can help with anything.

MarkGibbons · 2020-01-28T15:14:45Z

I'm noodling at adding users, clients and getting the search function to work. In this line

userEncodedSecret, err := EncodeSharedSecret(vs.client.Auth.PrivateKey, sharedSecret)

in the CreateItem method. I think that is supposed to be a PublicKey. The encode and decode methods in crypt both use the PrivateKey. Both of those things and adding code to deal with multiple users and nodes need to be there IMO. If you have time, cool. Otherwise I'll likely get it. Might take me a while. You can see my branch, local vault branch, to see what I've done already. I've been adding tests against a chef server that gets spun up.

thanks,
Mark

bhoriuchi · 2022-06-19T06:41:49Z

not sure if this is still an active effort but noticed only v3 supported. i did some similar work on https://github.com/bhoriuchi/go-chef-crypto and have v1 and v2 if interested in adding to this. also one issue i ran into to note is that the ruby base64 package adds new lines every 60 chars when you use knife so https://github.com/bhoriuchi/go-chef-crypto/blob/master/crypto.go#L211-L216 is used to replicate that.

MarkGibbons · 2022-07-07T00:00:12Z

I'll look at this one again over the next month. The last time I looked at this I went down some really nasty rabbit holes.

dhubler · 2022-09-08T15:56:58Z

I'm just looking to support encryption/decryption (not vault nec.)
https://docs.chef.io/data_bags/#encrypt
I see @bhoriuchi library and from my perspective, if that was integrated into "go-chef" then I'd have what I need. If that is not possible, would there be a way to provide a "hook" into the current go-chef library for users to combine two libs on their own?

MarkGibbons · 2022-09-09T01:27:22Z

Adding support for chef vault to this api may be too much effort for too little return.
@dhubler & @bhoriuchi adding better encryption support is something that makes sense. Could the code to do that be turned in to a pull request?

dhubler · 2022-09-09T12:35:35Z

I don't see why not. Haven't started, but initial thought would be to add go-chef-encrypt as dependency and submit PR with it integrated into go-chef API. If this sounds ok to you, I'll open a separate issue and post PR there.

…

On Thu, Sep 8, 2022 at 9:27 PM Mark Gibbons ***@***.***> wrote: Adding support for chef vault to this api may be too much effort for too little return. @dhubler <https://github.com/dhubler> & @bhoriuchi <https://github.com/bhoriuchi> adding better encryption support is something that makes sense. Could the code to do that be turned in to a pull request? — Reply to this email directly, view it on GitHub <#101 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAACA7T4ODCX6ECNPRCSXULV5KHALANCNFSM4FONAKCA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

lanej force-pushed the vault branch from 8f0b1de to e29282d Compare August 8, 2018 04:24

MarkGibbons self-assigned this May 17, 2019

MarkGibbons changed the title ~~Add chef-vault implementation~~ Add chef-vault implementation #minor Jan 8, 2020

MarkGibbons reviewed Jan 13, 2020

View reviewed changes

MarkGibbons reviewed Jan 16, 2020

View reviewed changes

MarkGibbons reviewed Jan 18, 2020

View reviewed changes

lanej added 7 commits January 26, 2020 15:24

fix: build warnings relating to missing formatting directives

12ee3e0

Corrected a few spelling mistakes along the way.

feature(vault): initial commit

1049521

Add a compatible client implementation to the chef-vault ruby version.

refactor(vault): remove redundant methods. add create and read tests

cd77489

test(vault): verify deletes

a773f27

refactor(vault): update and decrypt with a test

16cab19

The testing could be much better. I will throw the crypto tests I have in next. This testing is much more useful with a live chef server.

test(crypto): verify encrypt/decrypt vault data

a45c966

test: remove extraneous print statement

99bf053

lanej force-pushed the vault branch from 9c42d9a to 99bf053 Compare January 26, 2020 23:26

dhubler mentioned this pull request Sep 8, 2022

Any plans to integrate with https://github.com/go-chef/chef bhoriuchi/go-chef-crypto#3

Open

dhubler mentioned this pull request Sep 12, 2022

Support encryption and decryption #242

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add chef-vault implementation #minor #101

Add chef-vault implementation #minor #101

lanej commented Aug 8, 2018 •

edited

Loading

lanej commented Aug 8, 2018 •

edited

Loading

MarkGibbons commented Jul 21, 2019

lanej commented Jul 22, 2019

MarkGibbons commented Jan 13, 2020 •

edited

Loading

MarkGibbons commented Jan 13, 2020

MarkGibbons Dec 27, 2019

MarkGibbons Jan 13, 2020

MarkGibbons Jan 11, 2020

MarkGibbons Jan 13, 2020

MarkGibbons Jan 14, 2020

MarkGibbons Jan 12, 2020

MarkGibbons Jan 13, 2020

MarkGibbons Jan 13, 2020

MarkGibbons Jan 12, 2020

MarkGibbons Jan 13, 2020

MarkGibbons Jan 12, 2020

MarkGibbons Jan 13, 2020

MarkGibbons Jan 12, 2020

MarkGibbons Jan 13, 2020

MarkGibbons Jan 16, 2020

MarkGibbons Jan 18, 2020

lanej commented Jan 26, 2020

MarkGibbons commented Jan 28, 2020

bhoriuchi commented Jun 19, 2022 •

edited

Loading

MarkGibbons commented Jul 7, 2022

dhubler commented Sep 8, 2022

MarkGibbons commented Sep 9, 2022

dhubler commented Sep 9, 2022 via email

		databagData := DataBagItem(itemData)

		item.DataBagItem = &databagData

Add chef-vault implementation #minor #101

Are you sure you want to change the base?

Add chef-vault implementation #minor #101

Conversation

lanej commented Aug 8, 2018 • edited Loading

lanej commented Aug 8, 2018 • edited Loading

MarkGibbons commented Jul 21, 2019

lanej commented Jul 22, 2019

MarkGibbons commented Jan 13, 2020 • edited Loading

MarkGibbons commented Jan 13, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lanej commented Jan 26, 2020

MarkGibbons commented Jan 28, 2020

bhoriuchi commented Jun 19, 2022 • edited Loading

MarkGibbons commented Jul 7, 2022

dhubler commented Sep 8, 2022

MarkGibbons commented Sep 9, 2022

dhubler commented Sep 9, 2022 via email

lanej commented Aug 8, 2018 •

edited

Loading

lanej commented Aug 8, 2018 •

edited

Loading

MarkGibbons commented Jan 13, 2020 •

edited

Loading

bhoriuchi commented Jun 19, 2022 •

edited

Loading