Secrets management, encryption and git; beyond cf-secret

Hi all, I've recently become involved in a long-established CFEngine 
deployment. My previous CFEngine ended about 2018, and we didn't use Git at 
the time, so I'm looking for some advice with regard to security posture.

I'm currently working in an environment predominantly 3.15.<varies>, with a 
few outliers in 3.9.x and 3.21.x.

The issue I'm trying to address is that currently there are a lot of 
unencrypted secrets in the Git repository for our CFEngine policies, which 
while highly objectionable, is understandable given that cf-secret landed 
in 3.16.0 (and maybe 3.15.3).

Reading into cf-secret I find it lacks some capabilities I would have 
expected. I'm looking for community experience for keeping unencrypted 
secrets out of Git. I will accept having encrypted secrets in Git as a 
suitable middle-ground.

I see no capability within CFEngine to have variables that are stored in 
the likes of a secrets-manager such as AWS Parameter Store, or Azure Key 
Vault, Hashicorp Vault.

Ideally I'd want the only unencrypted secrets to be in their rendered 
configuration files, which might get inserted by an edit_line, 
edit_template or file copy promise (similar to Ansible Vault)

I'm _not_ looking to introduce an application dependency on configured 
applications to use the likes of HashiCorp Vault, or even to require them 
to obtain credentials in any way different to how they currently do; 
writing unencrypted secrets into an application's configuration file is 
completely desirable for these requirements (and yes, this could be 
improved further). So long as the variables are not present unencrypted 
elsewhere, particularly in Git. Bonus points if they aren't present 
unencrypted on the policy hub.


Idea #1: cf-secret
* Upgrade the fleet to be 3.16.0 or later (preferably targeting latest LTS 
release)
* Ensure all secrets are embedded in a data variables that comes from a 
file that can be sourced (eg. data.json), and that each server has its own 
version, encrypted according to its key.
* Now we have potentially many copies for each original file, so we need to 
solve for securing that original file and updating the various encrypted 
copies.

eg. data-original.json, data-host1.json, data-host2.json

Doesn't feel like it would be particularly scalable.


Idea #1b: Use Mozilla SOPS instead of cf-secret

I've encountered this problem in other similar situations before, and had 
discovered (but not used much) SOPS: https://github.com/mozilla/sops

It is more general than cf-secret (particularly in the ways it can encrypt 
files).... but is not so different to cf-secret. Likely it just has a 
different set of trade-offs (particularly if you use GPG), although I'm 
likely to trust it more than cf-secret, considering the credentials of the 
person who created it.


Idea #2: Provision the secrets outside of CFEngine using CI/CD.

This environment already has a practice of using CI/CD deployment, where 
secrets are injected by the CD tooling. In this case, we could do the 
following:

* introduce some data file (eg. a data.json) or files (eg. one per module?) 
that are stored in a particular location on each server. These files could 
either be unencrypted, or encrypted via cf-secret.
* it would remain preferable if the variable definitions (eg. naming, 
structure) would remain in Git, but the values be injected elsewhere.
* CFEngine should be configured to abort (partially?) if the file has not 
been delivered.
* the secret data could be pushed by CD technology (eg. Jenkins, Team City, 
Bamboo).
* it would likely entail enrolling many more servers in CD
* an alternative to using CD would be to use another push-based mechanism 
such as Ansible.

I'm thinking this potential solution is best suited for an on-premises 
environment that has no pre-existing secrets-management API.


Idea #3: CFEngine pulls data from a Secrets Management API

For environments that do have a Secrets Management API (eg. Azure Key 
Vault, AWS Secrets Manager, Hashicorp Vault), then we could:

* deliver a script (using CFEngine) that will query the API to obtain the 
secrets and write the result into a JSON or YAML output that can be parsed 
into a data variable. If this in a cloud environment, use Instance Roles 
etc. to govern and audit access; otherwise you need to solve for seeding 
access key for API.
* CFEngine should be configured to abort (partially?) if the data could not 
be obtained.

This solution is most-attractive, particularly in an environment where 
technology such as AWS Secrets Manager is in play, where it can rotate 
database credentials etc.



For my client's current need, I suspect that Idea #2 (CI/CD) will be the 
most appropriate, given what they do (not) have available.

If anyone has any resources (worked examples) or deployment experience 
(gotchas, success-stories), I would be really happy to receive them.


One other question.... if you create a data variable (containing 
unencrypted secrets) from a command that emits yaml/json... does that 
variable get persisted anywhere? I'm thinking its likely that it will get 
remembered for a period of time, presumably somewhere under /var/cfengine/

Thank you very much for your time and expertise,
Cameron


-- 


--





-- 
You received this message because you are subscribed to the Google Groups "help-cfengine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To view this discussion on the web visit https://groups.google.com/d/msgid/help-cfengine/4f325327-fd6d-4b29-930a-d26b13d717b5n%40googlegroups.com.