PRESENTATION.org

State of the CFEngine

{{{color(orange,Happy to be back)}}}

Releases since 2018-02-05

15 releases!

Version	Date
3.10.4	2018-05-03
3.7.8	2018-05-03
3.12.0	2018-06-28
3.10.5	2018-11-16
3.13.0	2018-11-23
3.12.1	2018-11-30
3.10.6	2019-05-10
3.12.2	2019-05-20
3.12.2-2	2019-05-29
3.14.0	2019-07-01
3.12.2-3	2019-07-25
3.14.0-2	2019-07-29
3.12.3	2019-12-11
3.10.7	2019-12-11
3.15.0	2019-12-19

2 LTS series reached End of Life

Version	Date
3.7.8	2018-05-03
3.10.7	2019-12-11

2 LTS series born

Version	Date
3.12.0	2018-06-28
3.15.0	2019-12-19

Contributions

2009

	Core	MPF	Docs
Commits	528	0	0
Contributors	2	1	1
New Contributors	1	0	0

2010

	Core	MPF	Docs
Commits	932	0	0
Contributors	3	1	1
New Contributors	1	0	0

2011

	Core	MPF	Docs
Commits	1695	44	0
Contributors	9	5	1
New Contributors	6	5	0

2012

	Core	MPF	Docs
Commits	1808	40	0
Contributors	32	12	1
New Contributors	23	8	0

2013

	Core	MPF	Docs
Commits	2865	219	929
Contributors	51	38	28
New Contributors	40	29	29

2014

	Core	MPF	Docs
Commits	2411	518	1200
Contributors	48	34	34
New Contributors	16	21	17

2015

	Core	MPF	Docs
Commits	699	245	334
Contributors	29	22	21
New Contributors	9	8	7

2016

	Core	MPF	Docs
Commits	587	228	383
Contributors	23	20	15
New Contributors	6	8	4

2017

	Core	MPF	Docs
Commits	244	154	186
Contributors	23	13	17
New Contributors	9	3	7

2018

	Core	MPF	Docs
Commits	492	163	231
Contributors	18	22	11
New Contributors	7	9	7

2019

	Core	MPF	Docs
Commits	612	255	172
Contributors	22	20	15
New Contributors	9	8	3

2020

	Core	MPF	Docs
Commits	17	12	7
Contributors	4	5	4
New Contributors	0	0	1

Core

basename()

• New in 3.15.0
• Convenient and familiar

bundle agent example_basename
{
  vars:
      "basename" -> { "CFE-3196" }
        string => basename( $(this.promise_filename) );

      "basename_wo_extension" -> { "CFE-3196" }
        string => basename( $(this.promise_filename), ".cf" );
  reports:

      "basename = '$(basename)'";
      "basename without '.cf' extension = '$(basename_wo_extension)'";
}

# cf-agent --bundlesequence example_basename --no-lock --log-level info --file /home/nickanderson/src/presentations/state-of-the-cfengine/cfengine3-NiK6sU
    info: Using command line specified bundlesequence
R: basename = 'cfengine3-NiK6sU'
R: basename without '.cf' extension = 'cfengine3-NiK6sU'

classfiltercsv()

• New in 3.15.0
• Useful to reduce logic and speed policy

classfiltercsv() Input

ClassExpr,Sort,Token,Value
# This is a comment
any,A,net.ipv4.ip_forward,ANYVALUE
example_class1,z,net.ipv4.ip_forward,ANYVALUE
example_class2,a,net.ipv4.ip_forward,127.0.0.3
not_defined,Z,net.ipv4.ip_forward,NOT_DEFINED
example_class3.example_class2.example_class1,1,net.ipv4.ip_forward,127.0.0.4
also_undefined,0,net.ipv4.ip_forward,NOT_DEFINED

• NOTE: CRLF (Windows/DOS) line endings required per RFC

classfiltercsv() Example Policy

bundle agent example_classfiltercsv
{
  classes:
      "example_class1";
      "example_class2";
      "example_class3";

  vars:
      "data_file" string => "$(this.promise_dirname)/data/classfiltercsv.csv";
      "d" data => classfiltercsv($(data_file), "true", 0, 1);

  reports:
      "Filtered data: $(with)" with => string_mustache("{{%-top-}}", d);
}

classfiltercsv() Policy Output

R: Filtered data: [
  {
    "Sort": "A",
    "Token": "net.ipv4.ip_forward",
    "Value": "ANYVALUE"
  },
  {
    "Sort": "a",
    "Token": "net.ipv4.ip_forward",
    "Value": "127.0.0.3"
  },
  {
    "Sort": "z",
    "Token": "net.ipv4.ip_forward",
    "Value": "127.0.0.4"
  },
  {
    "Sort": "z",
    "Token": "net.ipv4.ip_forward",
    "Value": "ANYVALUE"
  }
]

classfiltercsv() For more…

Checkout Jay Goldberg’s talk “Data-Driven Configuration with CFEngine’s classfiltercsv function” tomorrow, 16:50–17:15, B.2.009

hash_to_int()

• New in 3.12.0
• Useful for distributed grouping and orchestration (re-implement select_class)

bundle agent example_hash_to_int
{
  vars:
      "hello"  int => hash_to_int(0, 1000, "hello");
      "world"  int => hash_to_int(0, 1000, "world");

      # Hash can vary on hostkey or policy hub:
      "hour"   int => hash_to_int(0, 24, "$(sys.key_digest)");
      "minute" int => hash_to_int(0, 60, "$(sys.policy_hub)");

  reports:
      "'hello' hashed to: $(hello)";
      "'world' hashed to: $(world)";

}

    info: Using command line specified bundlesequence
R: 'hello' hashed to: 172
R: 'world' hashed to: 760

string_replace()

• New in 3.12.1
• Convenience, more simple version of regex_replace()

bundle agent example_string_replace
{
  vars:
      "my_string" string => "This is not the string you are looking for";

  reports:
      "$(with)" with => string_replace( $(my_string), "not ", "" );
}

R: This is the string you are looking for

read_module_protocol()

• New in 3.15.0
• Useful for leveraging cached output from module protocol
• More light weight and less noisy than using cat

read_module_protocol() Example usage

bundle agent cache_maintenance
# Creates a module protocol cache, refreshes it if minute is 30-35
{
  vars:
      "file"
        string => "$(this.promise_dirname)/cached_module";

  classes:
      "cache_refresh"
        if => not(fileexists("$(file)"));
    Min30_35::
      "cache_refresh";

  files:
    cache_refresh::
      "$(file)"
        create => "true",
        template_method => "inline_mustache",
        edit_template_string => "=my_variable=$(sys.date)";
}

read_module_protocol() Example usage continued …

bundle agent demo
# Demonstrates read_module_protocol function, prints a variable from it
{
  classes:
      "cache_was_read"
        if => read_module_protocol("$(cache_maintenance.file)");

  reports:
    cache_was_read::
      "Module cache was read!";
      "cached_module.my_variable = $(cached_module.my_variable)";
}
bundle agent __main__
{
  methods:
      "cache_maintenance"
        handle => "cache_maintenance_done";
      "demo"
        depends_on => { "cache_maintenance_done" };
}

R: Module cache was read!
R: cached_module.my_variable = Mon Feb  3 05:32:05 2020

read_module_protocol() Example output

R: Module cache was read!
R: cached_module.my_variable = Mon Feb  3 05:32:05 2020

body copy_from missing_ok

• New in 3.12.0
• Promise KEPT if source file is missing (except for remote
• Useful for opportunistic copies (host specific data)

body copy_from missing_ok Example

bundle agent main
{
  files:
      "/tmp/my.file"
        copy_from => local_if_available( "/tmp/$(sys.fqhost).json" ),
        classes => results( "bundle", "copy_from_local" );

      "/tmp/my.file"
        copy_from => server_if_available( "/tmp/$(sys.fqhost).json" ),
        classes => results( "bundle", "copy_from_server" );

  reports:
      "$(with)"
        with => string_mustache( "Classes:$(const.n){{#-top-}}{{.}}$(const.n){{/-top-}}",
                                 sort( classesmatching( "copy_from_.*" ), lex));
}
body copy_from local_if_available(file)
{
        source => "$(file)";
        missing_ok => "true";
}
body copy_from server_if_available(file)
{
        source => "$(file)";
        missing_ok => "true";
        servers => { "127.0.0.1" };
}

body copy_from missing_ok Example output

   error: No suitable server found
R: Classes:
copy_from_local_kept
copy_from_local_reached
copy_from_server_error
copy_from_server_failed
copy_from_server_not_kept
copy_from_server_reached

R: Yet/tmp/nickanderson-thinkpad-w550s.json doesn't exist

body control hub query_timeout

• New in 3.12.0
• Enterprise specific

body storage mount_type panfs & cifs

• New in 3.15.0
• Thanks SURF Sara!

body packge_module interpreter

• New in 3.13.0, 3.12.2

body package_module module_path

• New in 3.13.0, 3.12.2

commands promise inform

• New in 3.15.0
• Useful for suppressing noise from commands that do not produce output

bundle agent main
{
  commands:
      "$(paths.false)";
      "$(paths.true)" inform => "false";
}

    info: Executing 'no timeout' ... '/bin/false'
   error: Finished command related to promiser '/bin/false' -- an error occurred, returned 1
    info: Completed execution of '/bin/false'

files promise edit_template_string

• New in 3.12.0
• Useful for self contained policy

bundle agent main
{
  files:
    "/tmp/now.txt"
      create => "true",
      edit_template_string => "{{{sys.date}}}";
  reports:
    "/tmp/now.txt"
      printfile => cat( $(this.promiser) );
}

    info: Created file '/tmp/now.txt', mode 0600
R: /tmp/now.txt

Multiple augments

• New in 3.12.0

Multiple augments: def.json

{
    "vars": {
      "my_var": "This is my_var, set in def.json",
      "my_platform_specific_var": "This is my_platform_specific_var, set in def.json"
  },
    "augments": [ "$(sys.flavor).json" ]
}

Multiple augments: $(sys.flavor).json

{
    "vars": {
      "my_platform_specific_var": "This is my_platform_specific_var, set in ubuntu_19.json"
  }
}

Multiple augments: policy

bundle agent main
{
    reports:
      "$(sys.flavor)";
      "$(def.my_var)";
      "$(def.my_platform_specific_var)";
} 

Multiple augments: Policy run and output

cf-agent -KIf ./example.cf 

R: ubuntu_19
R: This is my_var, set in def.json
R: This is my_platform_specific_var, set in ubuntu_19.json

Augments classes by class expression

• New in 3.14.0, 3.12.2

Improved CI

• Valgrind
• ASAM
• LGTM

Enterprise

• Federated Reporting
• Policy Analyzer
• Alert conditions
  • File changes
  • Custom SQL
• Dashboard widgets
  • Newly bootstrapped hosts trend
  • Decommissioned hosts trend
• Improved report builder
  • Extend with custom schema
• File integrity monitoring UI
• Fine grained permissions for RBAC
• Export/Import settings
• APIs
  • Inventory
  • File changes
• New OOTB Inventory
  • AWS EC2 region, instance type, instance id, availability zone
  • System UUID
  • Setuid files
  • Root owned setuid files

Tooling

cf-remote

Install CFEngine on remote hosts, provision vms on cloud providers

cf-check

Validate and repair lmdbs

cf-bottom

Tom, the new cfengine team member, triggers builds in jenkins

???

Thank you

• Made with spacemacs, using org-mode, ob-cfengine3, and org-reveal, with love