Diego 0.1431.0

Changes from 0.1430.0 to 0.1431.0

• Depends on garden-linux-release v0.305.0.

Breaking changes

Auction request payloads

Associated to Diego story "The auction should only send resources and identifiers back and forth". This may cause instance downtime during an upgrade from 0.1430.0 and earlier.

Mutual SSL Auth to BBS

Associated to Diego story "All communication with the BBS should be secured via mutually-authenticated SSL". By default, the BBS will now require mutual SSL authentication for access to its API endpoints. If this is enabled, components on an older release will be unable to communicate with the BBS when deploying an update, so cells may be unable to evacuate.

To configure the BBS with SSL correctly, it is easiest to use the scripts/generate-bbs-certs script to generate a CA certificate and key and certificates and keys for the BBS server and its clients. The contents of these certificates and client and server keys must then be included in the deployment manifest. If using the spiff-based manifest-generation tooling, these values can be included in the property-overrides.yml stub once and will flow to the BBS server and its clients.

Significant changes

• DesiredLRP data should be split across separate records
• As a BBS client, I can efficiently get frequently accessed data for all DesiredLRPs in a domain
• NSYNC's bulker should fetch the minimal set of DesiredLRP data
• Route-Emitter's bulk loop should fetch the minimal set of DesiredLRP data
• If a migration fails, BOSH aborts the deploy and I should be able to BOSH deploy the previously deployed release and recover.
• If no /version key is present in etcd, the BBS should not run any migrations
• As a Diego developer, I would like to run vizzini as an errand
• As a Diego operator, I can specify a set of decryption keys to use to decrypt data at rest, with the BBS migrating data to the designated active key in the set
• Diego etcd on bosh-lite should default to requiring ssl
• As a Diego operator, I can opt out of the SSH DATs that do not use the plugin
• vizzini test errand runs against BBS with mutual SSL auth enabled
• Provide vizzini job with BBS URL and local consul agent URL

BOSH job changes

• Added vizzini job to run the vizzini test suite as an errand.

BOSH property changes

• Added acceptance_tests.skip_ssh_without_plugin_tests: When true, skip SSH DATs that do not use the SSH plugin.
• Added properties for vizzini job:
  • vizzini.bbs.api_location: Address for vizzini to reach the BBS.
  • vizzini.routable_domain_suffix: Domain to use for vizzini to register routes during the test.
  • vizzini.nodes: Number of tests to run in parallel.
  • vizzini.verbose: Whether to log verbosely during the test run.
• Added BBS encryption properties:
  • diego.bbs.encryption_keys: List of label/passphrase pairs available to the BBS for encryption.
  • diego.bbs.active_key_label: Label of the encryption key to be used to encrypt the database.
• Added BBS mutual SSL auth properties:
  • Properties for BBS server job:
    • diego.bbs.require_ssl: whether the BBS requires SSL for communication.
    • diego.bbs.ca_cert: CA certificate used to sign BBS client and server SSL certificates.
    • diego.bbs.server_cert: SSL certificate that the BBS presents.
    • diego.bbs.server_key: Private key paired with the BBS's SSL certificate.
  • New BBS properties for client jobs:
    • Properties:
      • diego.*.bbs.ca_cert
      • diego.*.bbs.client_cert
      • diego.*.bbs.client_key
      • diego.*.bbs.require_ssl
    • Jobs:
      • auctioneer
      • converger
      • nsync
      • receptor
      • rep
      • route_emitter
      • ssh_proxy
      • stager
      • tps
      • vizzini
• Changed diego.*.bbs.api_url to diego.*.bbs.api_location for all jobs using the old property.
• Removed etcd communication properties from Diego core jobs:
  • Properties:
    • diego.*.etcd.machines
    • diego.*.etcd.ca_cert
    • diego.*.etcd.client_cert
    • diego.*.etcd.client_key
    • diego.*.etcd.require_ssl
  • Jobs:
    • auctioneer
    • converger
    • receptor
    • rep