jira.xml

<?xml version="1.0" encoding="utf-8" ?>
<!--
#    \\ SPIKE: Secure your secrets with SPIFFE.
#  \\\\\ Copyright 2024-present SPIKE contributors.1
# \\\\\\\ SPDX-License-Identifier: Apache-2.0
-->
<stuff>
  <purpose>
    <target>Our goal is to have a minimally delightful product.</target>
    <target>Strive not to add features just for the sake of adding
      features.
    </target>
    <target>Half-assed features shall be completed before adding more
      features.
    </target>
    <meta>
      Minimally Delightful Product Requirements:
      - A Kubernetes SPIKE deployment
      ✅ Minimal policy enforcement
      - Minimal integration tests
      ✅ A demo workload that uses SPIKE to test things out as a consumer.
      ✅ A golang SDK
    </meta>
  </purpose>
  <watch>
    <issue>
      There is ongoing work on kubernetification.
    </issue>
    <issue>
      There is ongoing work on CLI documentation.
    </issue>
  </watch>
  <immediate-backlog>
    <aside>
      <issue>
        turtle book markdownification
        * Try it with pandoc.
        * automate some minimal script.
      </issue>
      <issue>
        * cut a new vsecm version too.
        * there are also open PRs there.
      </issue>
      <issue>
        aes256-cbc
      </issue>
      <issue>
        jay:U2FsdGVkX1+VhdGia1yk+JAUSraXj60ZA2ydT9TuHmQBE1sWLcMLb5z0B76sCqpJVGi1GQCl8BnnoV5kznYneQ==
      </issue>
    </aside>
    <isuse>
      godoc updates required on this repo, and also SDK.
      check and ensure ALL files are up-to-date.
    </isuse>
    <issue>
      address todo items inside source code after a release cut.

      Also, things to test before the release cut:
      1. basic secret creation
      2. basic secret listing
      3. nexus crash and recover
      4. total system crash and recover
      5. demo application and policy creation
    </issue>
    <issue>
      this should be configurable:
      ctxC, cancel := context.WithTimeout(context.Background(), 30*time.Second)
      defer cancel()
    </issue>
    <issue>
      if crashed and trying to restore (instead of recover)
      you get this:

      spike (main)$ spike operator recover
      2025/02/24 21:30:35 recover: Problem parsing response body
      unexpected end of JSON input

      error could be more explanatory
    </issue>
    <issue>
      spike (main)$ spike operator restore
      (your input will be hidden as you paste/type it)
      Enter recovery shard: 2025/02/24 22:09:34 recover: Problem parsing
      response body
      unexpected end of JSON input

      ^ this happens when the input is invalid; a more explanatory error message
      would be great.
    </issue>
    <issue>
      sanitize perms

      err := api.CreatePolicy(name, spiffeIddPattern, pathPattern, perms)
      if err != nil {
      fmt.Printf("Error: %v\n", err)
      return
      }
    </issue>
    <issue>
      return cobra.Command{
      Use: "delete policy-id",
      Short: "Delete a policy",
      Args: cobra.ExactArgs(1),
      Run: func(cmd *cobra.Command, args []string) {
      api := spike.NewWithSource(source)

      // TOD: sanitize policy id.
      // also validate other command line arguments too if it makes sense.
      // better to stop bad data at the client (but still not trust the
      // client fully)

      Go through all cobra commands and validate/sanitize what needs to
      be.
    </issue>
    <issue>
      use dig too, since dig works better than parsing hosts file in mac os

      # Check if getent is available
      if command -v getent >/dev/null 2>&1; then
      HOSTS_ANSWER=$(getent hosts "$SPIRE_SERVER_DOMAIN")
      else
      # Fallback for systems without getent (like macOS)
      HOSTS_ANSWER=$(grep "$SPIRE_SERVER_DOMAIN" /etc/hosts | grep -v "^#")
      fi
    </issue>
    <issue>
      To docs:

      additional comments.

      wrt `mlock`;
      * it requires root privileges or `CAP_IPC_LOCK` in Linux.
      * there is no windows equivalent.

      however without `mlock` if swap is on, zeroing out memory won't be good
      enough as we won't have any control ove whatever is written on swap.

      --

      @kfox1111 has suggested to add "turning swap off" to the
      [production guide](https://spike.ist/operations/production/) as a
      recommendation.

      Turning swap off is recommended for bare-metal k8s control plane and
      worker nodes anyway; so chances are, it will be turned off if a modern
      containerized (doker, k8s, etc) environment.

      If turning swap is not an option, encrypting swap could be another option
      too.

      --

      may want to put a note in docs or something to run with swap off.
      not sure if the sanitation bits would work with swap on a system.
      ^
      production guides.
      ^
      also describe how enabling swap might impact the security posture.
    </issue>
    <issue>
      sanitize keeper id and shard

      request := net.HandleRequest[
      reqres.ShardContributionRequest, reqres.ShardContributionResponse](
      requestBody, w,
      reqres.ShardContributionResponse{Err: data.ErrBadInput},
      )
      if request == nil {
      return errors.ErrParseFailure
      }

      shard := request.Shard
      id := request.KeeperId

    </issue>
    <issue>
      validate spiffe id and other parameters
      for this and also other keeper endpoints

      func RouteShard(
      w http.ResponseWriter, r *http.Request, audit *log.AuditEntry,
      ) error {
      const fName = "routeContribute"
      log.AuditRequest(fName, r, audit, log.AuditCreate)

      requestBody := net.ReadRequestBody(w, r)
      if requestBody == nil {
      return errors.ErrReadFailure
      }

      here is an example that does that:
      func RoutePutPolicy(
      w http.ResponseWriter, r *http.Request, audit *log.AuditEntry,
      ) error {
      const fName = "routePutPolicy"
      log.AuditRequest(fName, r, audit, log.AuditCreate)

      requestBody := net.ReadRequestBody(w, r)
      if requestBody == nil {
      return errors.ErrParseFailure
      }

      request := net.HandleRequest[
      reqres.PolicyCreateRequest, reqres.PolicyCreateResponse](
      requestBody, w,
      reqres.PolicyCreateResponse{Err: data.ErrBadInput},
      )
      if request == nil {
      return errors.ErrReadFailure
      }

      err := guardPutPolicyRequest(*request, w, r)
      if err != nil {
      return err
      }

    </issue>
    <issue>
      Ensure the system works w/o keepers in "in memory" mode.
      also document it and also create a video out of it.
    </issue>
    <issue>
      this should be configurable:
      ticker := time.NewTicker(5 * time.Minute)
    </issue>
    <issue>
      nil check wherever Backend() is called.
      var be backend.Backend
    </issue>
  </immediate-backlog>
  <later>
    <issue>
      newSecreteUndeleteCommand

      Run: func(cmd *cobra.Command, args []string) {
      // O: we can pass this as a predicate to newSecretUndeleteCommand,
      as a HOF.
      trust.Authenticate(spiffeId)
    </issue>
    <issue>
      add retries to everything under:
      app/nexus/internal/state/persist
      ^ they all talk to db; and sqlite can temporarily lock for
      a variety of reasons.
    </issue>
    <issue priority="important" severity="medium">
      if a keeper crashes it has to wait for the next nexus cycle which is
      suboptimal. Instead, nexus can send a ping that returns an overall
      status
      of keeper (i.e. if it's populated or not)
      this can be more frequent than hydration; and once nexus realizes
      keeper
      is down, it can rehydrate it.

      in addition; nexus can first check the sha hash of the keeper's shard.
      before resending; if the hashes match, it won't restransmit the shard.
    </issue>
    <issue>
      path sanitization:
      ^(?!/)([a-zA-Z0-9._-]+)(/[a-zA-Z0-9._-]+)*$

      Hashi Vault is not as script as the above regex; but I think this
      gives a nice balance.

      ^(?!/) → Ensures the path does not start with / (Vault does not require a leading /).
      ([a-zA-Z0-9._-]+) → The first segment must consist of alphanumeric characters, dots, underscores, or dashes.
      (/[a-zA-Z0-9._-]+)* → Subsequent segments must follow the same pattern and be separated by /.
      No double slashes (//) allowed.
      No spaces, backslashes, or reserved URL characters allowed.
      Example Valid Paths:
      ✅ secret/myapp/config
      ✅ secrets/db-creds/admin-user
      ✅ tenantA/projectX/env1/key

      Example Invalid Paths:
      ❌ /secret/myapp/config (leading /)
      ❌ secret//double-slash (double /)
      ❌ secret\path (backslash used)
      ❌ secret path/with space (contains spaces)
      ❌ secret#invalid?path (reserved URL characters)
    </issue>
    <issue>
      this is for policy creation:

      allowed := state.CheckAccess(
      spiffeid.String(), "*",
      []data.PolicyPermission{data.PermissionSuper},
      )

      instead of a wildcard, maybe have a predefined path
      for access check like "/spike/system/acl"

      also disallow people creating secrets etc under
      /spike/system
    </issue>

    <issue>
      path is still stored plain in DB; use HMAC instead.

      CREATE TABLE "secrets" (
      "hashed_path"	BLOB NOT NULL,
      "version"	INTEGER NOT NULL,
      "nonce"	BLOB NOT NULL,
      "encrypted_data"	BLOB NOT NULL,
      "created_time"	DATETIME NOT NULL,
      "deleted_time"	DATETIME,
      PRIMARY KEY("hashed_path","version")
      );

      same is true for policies; we need to keep them encrypted too.
    </issue>
    <issue>
      Also, create a script in ./hack that does that.
      (i.e. something that forces Nexus to reset its root key
      upon next restart; and see how it impacts the system
      see SQLite db with a db viewer
      or maybe let that script delete the database too.)

      // TODO: if you stop nexus, delete the tombstone file, and
      restart nexus,
      // (and no keeper returns a shard and returns 404)
      // it will reset its root key and update the keepers to store
      the new
      // root key. This is not an attack vector, because an adversary
      who can
      // delete the tombstone file, can also delete the backing store.
      /// Plus no sensitive data is exposed; it's just all data is
      inaccessible
      // now because the root key is lost for good. In either
      // case, for production systems, the backing store needs to be
      backed up
      // and the root key needs to be backed up in a secure place too.
      // ^ add these to the documentation.
    </issue>
    <issue>
      spike operator reset:
      deletes and recreates the ~/.spike folder
      restarts the initialization flow to rekey keepers.

      volkan@spike:~/Desktop/WORKSPACE/spike$ spike secret get /db
      Error reading secret: post: Problem connecting to peer

      ^ I get an error instead of a "secret not found" message.
    </issue>
    <issue>
      verify if the keeper has shard before resending it:
      send hash of the shard first
      if keeper says “I have it”, don’t send the actual shard.
      this will make things extra secure.
    </issue>
    <issue>
      Fleet management:
      - There is a management plane cluster
      - There is a control plane cluster
      - There are workload clusters connected to the control plane
      - All of those are their own trust domains.
      - There is MP-CP connectivity
      - There is CP-WL connectivity
      - MP has a central secrets store
      - WL and CP need secrets
      - Securely dispatch them without "ever" using Kubernetes secrets.
      - Have an alternative that uses ESO and a restricted secrets
      namespace
      that no one other than SPIKE components can see into.
    </issue>
    <issue>
      Wrt DR:

      A 404 response from a keeper meens that it does not have the shard
      if threshold number of keepers do not have the shard too, then there is
      no way that nexus can recover; so does it mean it should stop polling?

      Not quite. Because later, we can have ways to seed keepers, maybe through
      other nexuses, maybe via cloning from a backup keeper, maybe using a
      secure keeper API (i.e. imitating nexus)

      It's better, and simpler to keep the polling always running.
      It makes less assumptions that way.

      make this an ADR.
    </issue>
    <issue>
      configure SPIKE to rekey itself as per NIST guidelines.
      Also maybe `spike operator rekey` to manually initiate that.
      `spike operator rekey` will also change the shamir shares, wheras the
      internal rekey will just change the encryption key, leaving the shamir
      shares intact.
    </issue>
    <issue>
      This is OpenBao's production deployment checlist; check of if any of
      those also applies for SPIKE, of if we need different/additional items
      for SPIKE.

      From the context, OpenBao can be swapped with SPIKE Nexus, I think:

      End-to-End TLS. OpenBao should always be used with TLS in production. If
      intermediate load balancers or reverse proxies are used to front OpenBao,
      they should not terminate TLS. This way traffic is always encrypted in
      transit to OpenBao and minimizes risks introduced by intermediate layers.

      Single Tenancy. OpenBao should be the only main process running on a
      machine. This reduces the risk that another process running on the same
      machine is compromised and can interact with OpenBao. This can be
      accomplished by using OpenBao Helm's affinity configurable.

      Enable Auditing. OpenBao supports several auditing backends. Enabling
      auditing provides a history of all operations performed by OpenBao and
      provides a forensics trail in the case of misuse or compromise. Audit logs
      securely hash any sensitive data, but access should still be restricted to
      prevent any unintended disclosures. OpenBao Helm includes a configurable
      auditStorage option that provisions a persistent volume to store audit
      logs.

      Immutable Upgrades. OpenBao relies on an external storage backend for
      persistence, and this decoupling allows the servers running OpenBao to be
      managed immutably. When upgrading to new versions, new servers with the
      upgraded version of OpenBao are brought online. They are attached to the
      same shared storage backend and unsealed. Then the old servers are
      destroyed. This reduces the need for remote access and upgrade
      orchestration which may introduce security gaps. See the upgrade section
      for instructions on upgrading OpenBao on Kubernetes.

      Upgrade Frequently. OpenBao is actively developed, and updating frequently
      is important to incorporate security fixes and any changes in default
      settings such as key lengths or cipher suites. Subscribe to the OpenBao
      mailing list and GitHub CHANGELOG for updates.

      Restrict Storage Access. OpenBao encrypts all data at rest, regardless of
      which storage backend is used. Although the data is encrypted, an attacker
      with arbitrary control can cause data corruption or loss by modifying or
      deleting keys. Access to the storage backend should be restricted to only
      OpenBao to avoid unauthorized access or operations.
    </issue>
    <issue>
      a mode that enables the admin to load shares to keepers.
      this will be essentially using the keeper REST API and acting as
      SPIKE Pilot with a recover svid.
      The benefit would be; we can have a dedicated recover and restore binaries
      without having to expose the pilot binary.
      Or we can update the keepers, even if we don't have access to SPIKE Nexus
      or SPIKE Nexus is down.
      maybe by using a "seeder" SPIFFE ID.
      but I'm also not sure if it's worth it.
      it would mean more APIs to secure; it would also mean keeping spike
      keepers more intelligent (instead of keeping them dumb)
    </issue>
    <issue>
      A /stats endpoint.

      A dedicated /stats endpoint will be implemented to provide real-time
      metrics about:
      Total number of secrets managed.
      Status of the key-value store.
      Resource utilization metrics (e.g., CPU, memory).
      This endpoint will support integration with monitoring tools for enhanced
      observability.
      These measures will ensure comprehensive monitoring and troubleshooting.
    </issue>
    <issue>
      By design, we regard memory as the source of truth.
      This means that backing store might miss some secrets.
      Find ways to reduce the likelihood of this happening.
      1. Implement exponential retries.
      2. Implement a health check to ensure backing store is up.
      3. Create background jobs to sync the backing store.
    </issue>
    <issue>
      all components shall have
      liveness and readiness endpoints
      (or maybe we can design it once we k8s...ify things.
    </issue>
    <issue>
      in development mode, nexus shall act as a single binary:
      - you can create secrets and policies via `nexus create policy` etc

      that can be done by sharing
      "github.com/spiffe/spike/app/spike/internal/cmd"
      between nexus and pilot

      this can even be an optional flag on nexus
      (i.e. SPIKE_NEXUS_ENABLE_PILOT_CLI)
      running ./nexus will start a server
      but run
      ning nexus with args will register secrets and policies.
    </issue>
    <issue>
      Consider using OSS Security Scorecard:
      https://github.com/vmware-tanzu/secrets-manager/security/code-scanning/tools/Scorecard/status
    </issue>
    <issue>
      SPIKE automatic rotation of encryption key.
      the shards will create a root key and the root key will encrypt the
      encryption key.
      so SPIKE can rotate the encryption key in the background and encrypt
      it with the new root key.
      this way, we won't have to rotate the shards to rotate the
      encryption key.
    </issue>
    <issue>
      SPIKE CSI Driver

      the CSI Secrets Store driver enables users to create
      `SecretProviderClass` objects. These objects define which secret
      provider
      to use and what secrets to retrieve. When pods requesting CSI
      volumes are
      made, the CSI Secrets Store driver sends the request to the OpenBao
      CSI
      provider if the provider is `vault`. The CSI provider then uses the
      specified `SecretProviderClass` and the pod’s service account to
      retrieve
      the secrets from OpenBao and mount them into the pod’s CSI volume.
      Note
      that the secret is retrieved from SPIKE Nexus and populated to the
      CSI
      secrets store volume during the `ContainerCreation` phase.
      Therefore, pods
      are blocked from starting until the secrets are read from SPIKE and
      written to the volume.
    </issue>
    <issue>
      shall we implement rate limiting; or should that be out of scope
      (i.e. to be implemented by the user.
    </issue>
    <issue>
      more fine grained policy management

      1. an explicit deny will override allows
      2. have allowed/disallowed/required parameters
      3. etc.

      # This section grants all access on "secret/*". further restrictions
      can be
      # applied to this broad policy, as shown below.
      path "secret/*" {
      capabilities = ["create", "read", "update", "patch", "delete",
      "list", "scan"]
      }

      # Even though we allowed secret/*, this line explicitly denies
      # secret/super-secret. this takes precedence.
      path "secret/super-secret" {
      capabilities = ["deny"]
      }

      # Policies can also specify allowed, disallowed, and required
      parameters. here
      # the key "secret/restricted" can only contain "foo" (any value) and
      "bar" (one
      # of "zip" or "zap").
      path "secret/restricted" {
      capabilities = ["create"]
      allowed_parameters = {
      "foo" = []
      "bar" = ["zip", "zap"]
      }

      but also, instead of going deep down into the policy rabbit hole,
      maybe
      it's better to rely on well-established policy engines like OPA.

      A rego-based evaluation will give allow/deny decisions, which SPIKE
      Nexus
      can then honor.

      Think about pros/cons of each approach. -- SPIKE can have a
      good-enough
      default policy engine, and for more sophisticated functionality we
      can
      leverage OPA.
    </issue>
    <issue>
      key rotation

      NIST rotation guidance

      Periodic rotation of the encryption keys is recommended, even in the
      absence of compromise. Due to the nature of the AES-256-GCM
      encryption
      used, keys should be rotated before approximately 232
      encryptions have been performed, following the guidelines of NIST
      publication 800-38D.

      SPIKE will automatically rotate the backend encryption key prior to
      reaching
      232 encryption operations by default.

      also support manual key rotation
    </issue>
    <issue>
      Do an internal security analysis / threat model for spike.
    </issue>
    <issue>
      TODO in-memory "dev mode" for SPIKE #spike (i.e. in memory mode will
      not be default)
      nexus --dev or something similar (maybe an env var)
    </issue>
    <issue>
      Use SPIKE in lieu of encryption as a service (similar to transit
      secrets)
    </issue>
    <issue>
      dynamic secrets
    </issue>
    <issue>
      use case:
      one time access to an extremely limited subset of secrets
      (maybe using a one time, or time-bound token)
      but also consider if SPIKE needs tokens at all; I think we can
      piggyback
      most of the authentication to SPIFFE and/or JWT -- having to convert
      various kinds of tokens into internal secrets store tokens is not
      that much needed.
    </issue>
    <issue>
      - TODO Telemetry
      - core system metrics
      - audit log metrics
      - authentication metrics
      - database metrics
      - policy metrics
      - secrets metrics
    </issue>
    <issue>
      "token" secret type
      - will be secure random
      - will have expiration
    </issue>
    <issue>
      double-encryption of nexus-keeper comms (in case mTLS gets
      compromised, or
      SPIRE is configured to use an upstream authority that is
      compromised, this
      will provide end-to-end encryption and an additional layer of
      security
      over
      the existing PKI)
    </issue>

    <issue>
      * Implement strict API access controls:
      * Use mTLS for all API connections
      * Enforce SPIFFE-based authentication
      * Implement rate limiting to prevent brute force attacks
      * Configure request validation:
      * Validate all input parameters
      * Implement request size limits
      * Set appropriate timeout values
      * Audit API usage:
      * Log all API requests
      * Monitor for suspicious patterns
      * Regular review of API access logs
      ----
      * Enable comprehensive auditing:
      * Log all secret access attempts
      * Track configuration changes
      * Monitor system events
      * Implement compliance controls:
      * Regular compliance checks
      * Documentation of security controls
      * Periodic security assessments
      ---
      * Tune for security and performance:
      * Optimize TLS session handling
      * Configure appropriate connection pools
      * Set proper cache sizes
      * Monitor performance metrics:
      * Track response times
      * Monitor error rates
      * Alert on performance degradation
    </issue>

    <issue>
      ability to clone a keeper cluster to another standby keeper cluster
      (for redundancy).
      this way, if the set of keepers become not operational, we can
      hot-switch to the other keeper cluster.
      the assumption here is the redundant keeper cluster either remains
      healthy, or is somehow snapshotted -- since the shards are in
      memory, snapshotting will be hard. -- but stil it's worth thinking
      about.
      an alternative option would be to simplyh increase the number of
      keepers.
    </issue>
    <issue>
      ErrPolicyExists = errors.New("policy already exists")
      ^ this error is never used; check why.
    </issue>
    <issue>
      work on the "named admin" feature (using Keycloak as an OIDC
      provider)
      This is required for "named admin" feature.
    </issue>
    <issue>
      BootstrapOrDie()
      if we are certain that SPIKE nexus cannot bootstrap, maybe it can
      just kill itself.
    </issue>

    <issue>
      Consider using google kms, azure keyvault, and other providers
      (including an external SPIKE deployment) for root key recovery.
      question to consider is whether it's really needed
      second question to consider is what to link kms to (keepers or
      nexus?)
      keepers would be better because we'll back up the shards only then.
      or google kms can be used as an alternative to keepers
      (i.e., store encrypted dek, with the encrypted root key on nexus;
      only kms can decrypt it -- but, to me, it does not provide any
      additional advantage since if you are on the machine, you can talk
      to
      google kms anyway)
    </issue>
    <issue>
      dev mode with "zero" keepers.
    </issue>
    <issue>
      remove symbols when packaging binaries for release.
    </issue>
    <issue severity="important" priority="above-normal">
      consider db backend as untrusted
      i.e. encrypt everything you store there; including policies.
      (that might already be the case actually) -- if so, document it
      in the website.
    </issue>
    <issue>
      exponentially back off here

      log.Log().Info("tick", "msg", "Waiting for keepers to initialize")
      time.Sleep(5 * time.Second)

      or maybe not; I'm not sure if it's worth the effort.
      or maybe this algorithm has changed already; needs to be
      double-checked.
    </issue>

    <issue kind="good-first-issue"
           ref="https://github.com/spiffe/spike/issues/80">
      validations:

      along with the error code, also return some explanatory message

      instead of this for example

      err = validation.ValidateSpiffeIdPattern(spiffeIdPattern)
      if err != nil {
      responseBody := net.MarshalBody(reqres.PolicyCreateResponse{
      Err: data.ErrBadInput,
      }, w)
      net.Respond(http.StatusBadRequest, responseBody, w)
      return err
      }

      do this

      err = validation.ValidateSpiffeIdPattern(spiffeIdPattern)
      if err != nil {
      responseBody := net.MarshalBody(reqres.PolicyCreateResponse{
      Err: data.ErrBadInput,
      Reason: "Invalid spiffe id pattern. Matcher should be a regex that
      can match a spiffe id"
      }, w)
      net.Respond(http.StatusBadRequest, responseBody, w)
      return err
      }
    </issue>
    <issue>
      control these with flags.
      i.e. the starter script can optionally NOT automatically
      start nexus or keepers.

      #echo ""
      #echo "Waiting before SPIKE Keeper 1..."
      #sleep 5
      #run_background "./hack/start-keeper-1.sh"
      #echo ""
      #echo "Waiting before SPIKE Keeper 2..."
      #sleep 5
      #run_background "./hack/start-keeper-2.sh"
      #echo ""
      #echo "Waiting before SPIKE Keeper 3..."
      #sleep 5
      #run_background "./hack/start-keeper-3.sh"

      #echo ""
      #echo "Waiting before SPIKE Nexus..."
      #sleep 5
      #run_background "./hack/start-nexus.sh"
    </issue>
    <issue kind="research">
      spike, but as an http proxy?
      like, point http(s)_proxy at this thing, connect to it using your spiffe id, the proxy can use policies and the client spiffeid to allow/disallow an http proxy request
      No; but not a bad idea indeed. -- will look around; I'll share if I find something related.
    </issue>

  </later>
  <runner-up>
    <issue>
      dev mode, single binary.
    </issue>
    <issue>
      use Mlock optionally

      Also we can either "try and shrug" using `mlock` as in

      ```go
      err := unix.Mlock(data)
      if err != nil {
      // do nothing
      }

      // Zero out memory
      for i := range data {
      data[i] = 0
      }
      ```

      Or we can put the mlock option behind a feature flag.

      If the user has granted `CAP_IPC_LOCK` to SPIKE Nexus, then they can also
      set something like `SPIKE_NEXUS_USE_MLOCK="true"` as an env var, and try
      locking memory if the variable is set (will be "false" by default).

      Also research more about mlock before jumping at this.

      https://github.com/spiffe/spike/issues/68
    </issue>
    <issue>
      // TDO: Yes memory is the source of truth; but at least
      // attempt some exponential retries before giving up.
      if err := be.StoreSecret(ctx, path, *secret); err != nil {
      // Log error but continue - memory is the source of truth
      log.Log().Warn(fName,
      "msg", "Failed to cache secret",
      "path", path,
      "err", err.Error(),
      )
      }

      SQLLite can error out if there is a blocked transaction or
      a integrity issue, which a retry can fix it.
    </issue>
    <issue>
      seed, err := crypto.Aes256Seed()

      create an override function that return a byte slice instead of a string.
      while at that, we can create a pointer to a byte array too
      (since the size will be fixed for 256bits anyways)

      We can reuse that instead of the [32]byte array creation in the source code.
    </issue>

    <issue>
      adding a timeout or circuit breaker to the infinite loop in BootstrapBackingStoreWithNewRootKey, I was suggesting a way to prevent the function from running indefinitely if something goes wrong with keeper initialization.
      The current implementation uses:
      for {
      // Ensure to get a success response from ALL keepers eventually.
      exit := iterateKeepersToBootstrap(
      keepers, rootShares, successfulKeepers, source,
      )
      if exit {
      return
      }

      log.Log().Info(fName, "msg", "Waiting for keepers to initialize")
      // TODO: make the time configurable.
      time.Sleep(5 * time.Second)
      }
      This loop will continue forever until all keepers are successfully initialized. While this makes sense for normal operation, there are scenarios where this could become problematic:

      If one or more keepers are permanently unavailable or unreachable
      If there's a persistent network issue preventing communication
      If there's a configuration issue that makes successful initialization impossible

      A potential improvement would be to add:
      maxAttempts := env.GetMaxBootstrapAttempts() // Could be configurable
      attempts := 0

      ctx, cancel := context.WithTimeout(context.Background(), env.GetBootstrapTimeout())
      defer cancel()

      for {
      select {
      case lt-ctx.Done():
      log.Log().Warn(fName, "msg", "Bootstrap timed out after waiting threshold")
      // Implement fallback strategy or escalate the issue
      return
      default:
      attempts++
      if maxAttempts > 0 ++ attempts > maxAttempts {
      log.Log().Warn(fName, "msg", "Exceeded maximum bootstrap attempts")
      // Implement fallback strategy or escalate the issue
      return
      }

      exit := iterateKeepersToBootstrap(
      keepers, rootShares, successfulKeepers, source,
      )
      if exit {
      return
      }

      log.Log().Info(fName, "msg", "Waiting for keepers to initialize",
      "attempt", attempts, "maxAttempts", maxAttempts)
      time.Sleep(5 * time.Second)
      }
      }
      This approach provides:

      A maximum number of attempts (configurable)
      A total timeout for the entire operation (configurable)
      Better observability of progress through attempt counting

      In highly reliable systems, you might want the bootstrap to keep trying forever, but even then, it's valuable to have observability into how long it's been trying and an option to break the loop if needed.
    </issue>
    <issue>
      JSON Security:

      Marshaling/unmarshaling sensitive data with standard JSON functions can leave traces in memory
      Consider if a more secure serialization approach is needed for sensitive data

      Regarding more secure JSON serialization approaches for sensitive data, here are several strategies you could consider:

      Custom Secure Marshaler/Unmarshaler:

      Implement custom JSON marshal/unmarshal methods that use secure memory practices
      Include zero-out operations immediately after the data is no longer needed
      Use fixed-size arrays rather than slices to avoid heap allocations


      Buffer Pooling with Zeroing:

      Create a pool of fixed-size buffers for serialization operations
      Zero out buffers before returning them to the pool
      This minimizes allocations and ensures cleanup


      Memory-Safe Serialization Libraries:

      Look into security-focused serialization libraries that handle memory better than standard encoding/json
      Some cryptographic libraries include serialization functions designed for sensitive data


      Encrypted Serialization:

      Serialize to an encrypted format directly
      The data stays encrypted in memory except during actual use


      Use of Go's io.Writer Interface:

      Marshal directly to a pre-allocated buffer using json.Encoder
      This gives more control over the underlying memory


      Manual Field-by-Field Handling:

      For extremely sensitive structures, avoid automatic JSON marshaling entirely
      json marshaling and unmarshaling can leave traces in memory.

      Instead, construct the JSON manually field by field, zeroing intermediate
      buffers



      A practical implementation might look like:
      func secureJSONMarshal(v interface{}) ([]byte, error) {
      // Pre-allocate buffer of reasonable size to avoid resizing
      buffer := make([]byte, 0, 1024)

      // Use buffer as writer to avoid intermediate string allocations
      b := bytes.NewBuffer(buffer)
      encoder := json.NewEncoder(b)

      // Disable HTML escaping to reduce copying
      encoder.SetEscapeHTML(false)

      err := encoder.Encode(v)
      if err != nil {
      return nil, err
      }

      // Get result as byte slice
      result := b.Bytes()

      // Create a copy that we'll return
      finalResult := make([]byte, len(result))
      copy(finalResult, result)

      // Zero out the original buffer
      for i := range result {
      result[i] = 0
      }

      return finalResult, nil
      }
      For unmarshaling, we could use a similar approach with a pre-allocated
      structure and careful buffer management.
    </issue>
    <issue>
    FIPS compliance in Go is gaining traction;
    if we don't want to have a FIPS-non-compliant mode (for performance,
    etc, reasons), than it looks relatively simpler to implement such a
    thing with recent Go.
    Take a look at these and devise a plan:
    * https://github.com/golang/go/issues/69536
    * https://go.dev/doc/security/fips140

    Go 1.24 + Go Native Cryptography Module is preferred for non-Linux platforms.

    Prerequisites:
    Go FIPS (aka BoringCrypto) on Linux requires CGO therefore glibc is required.
    any modern glibc based Linux distro to build it

    C Complier and Linker
    GCC or clang version is usually independent to FIPS
    any modern gcc or clang for CGO as long as the build succeeds.

    ensure C compile and linker flags passed to CGO
    (CGO_CFLAGS  and CGO_LDFLAGS)
    see https://pkg.go.dev/cmd/cgo

    // fips_only.go can be imported for linux-only targes.
    package main
    import _ "crypto/tls/fipsonly"
    ^ research a bit more about this.

    $ strings "$PROGRAM" | grep -ci boringcrypto_FIPS_mode
    _cgo_12e963b132a6_Cfunc__goboringcrypto_FIPS_mode
    _goboringcrypto_FIPS_mode
    crypto/internal/boring._Cfunc__goboringcrypto_FIPS_mode
    _goboringcrypto_FIPS_mode
    _cgo_12e963b132a6_Cfunc__goboringcrypto_FIPS_mode
    crypto/internal/boring._Cfunc__goboringcrypto_FIPS_mode.abi0
    crypto/internal/boring._cgo_12e963b132a6_Cfunc__goboringcrypto_FIPS_mode
    You may use integrate this verify-fips.sh to your build script.

    #!/bin/bash

    set -x

    PROGRAM="$1"
    GOVERSION="$2"
    [ -z "$GOVERSION" ] and GOVERSION="go1.23." # the target go version

    declare -A SYMBOLS_MAP=(
    ["Cfunc__goboringcrypto"]=1
    ["goboringcrypto"]=1
    ["crypto/internal/boring/sig.BoringCrypto"]=1
    ["crypto/internal/boring/sig.StandardCrypto"]=0
    ["$GOVERSION"]=1
    ["X:boringcrypto"]=1
    )

    for symbol in "${!SYMBOLS_MAP[@]}"; do
    count=$(strings "$PROGRAM" | grep -ci "$symbol")
    expected="${SYMBOLS_MAP[$symbol]}"

    if [ "$expected" -eq 1 ]; then
    if [ "$count" -gt 0 ]; then
    echo "Symbol: $symbol Count: $count. PASS"
    else
    echo "Symbol: $symbol Count: $count. FAIL, expects > 0"
    exit 1
    fi
    else
    if [ "$count" -eq 0 ]; then
    echo "Symbol: $symbol Count: $count. PASS"
    else
    echo "Symbol: $symbol Count: $count. FAIL, expects == 0"
    exit 1
    fi
    fi
    done
    goversion
    ^
    this would work
      on stripped binaries too
    </issue>
    <issue>
      Go 1.24 introduces native support to FIPS 140-3. The Go Cryptographic
      Module in 1.24 will be FIPS 140-3 certified.

      It will not require linking to BoringCrypto/BoringSSL nor enabling CGO.

      This approach if it works, is more preferrable; research about it.,
    </issue>
    <issue>
      some of the memory resets can be done by the generic library at the SDK;
      check them out.
    </issue>

    <issue>
      spike policy write my-policy ./the-policy-file.yaml
      also check out: https://developer.hashicorp.com/vault/docs/concepts/policies
      to see if we can amend any updates to the policy rules
      (one such update, for example, is limiting what kind of attributes are
      allowed, but we should discuss whether that much granularity is worth the
      hassle)
    </issue>
    <issue>
      strip binaries of smybols before build
    </issue>


    <issue>
      read policies from a yaml or a json file and create them.
    </issue>
    <issue>
      test that the timeout results in an error.

      ctx, cancel := context.WithTimeout(
      context.Background(), env.DatabaseOperationTimeout(),
      )
      defer cancel()

      cachedPolicy, err := retry.Do(ctx, func() (*data.Policy, error) {
      return be.LoadPolicy(ctx, id)
      })

    </issue>
    <issue>
      ability to lock nexus programmatically.
      when locked, nexus will deny almost all operations
      locking is done by executing nexus binary with a certain command
      line flag.
      (i.e. there is no API access, you'll need to physically exec the
      ./nexus
      binary -- regular svid verifications are still required)
      only a superadmin can lock or unlock nexus.

      ^ instead of that, you can run a script that removes all SVID
      registrations. That will effectively result in the same thing.
    </issue>
    <issue>
      consider using NATS for cross trust boundary (or nor) secret
      federation
    </issue>
    <issue>
      over the break, I dusted off
      https://github.com/spiffe/helm-charts-hardened/pull/166 and started
      playing with the new k8s built in cel based mutation functionality.
      the k8s cel support is a little rough, but I was able to do a whole
      lot in it, and think I can probably get it to work for everything.
      once 1.33 hits, I think it will be even easier.
      I mention this, as I think spike may want similar functionality?
      csi driver, specify secrets to fetch to volume automatically, keep
      it up to date, and maybe poke the process once refreshed
    </issue>
    <issue>
      wrt: secure erasing shards and the root key >>
      It would be interesting to try and chat with some of the folks under
      the cncf
      (That's a good idea indeed; I'm noting it down.)
    </issue>
    <issue severity="important" urgency="moderate">
      // TDO: check all database operations (secrets, policies, metadata)
      and
      // ensure that they are retried with exponential backooff.
    </issue>

    <issue>
      we need a "reset" command for the restore operation in case
      we pushed an incorrect set of shards.
    </issue>
    <issue>
      better play with OIDC and keycloak sometime.
    </issue>
    <issue>
      attribute-based policy control

      path "secret/restricted" {
      capabilities = ["create"]
      allowed_parameters = {
      "foo" = []
      "bar" = ["zip", "zap"]
      }
      }
    </issue>
    <issue>
      Note: this is non-trivial, but doable.

      Periodic rotation of the encryption keys is recommended, even in the
      absence of compromise. Due to the nature of the AES-256-GCM
      encryption
      used, keys should be rotated before approximately 232 encryptions
      have
      been performed, following the guidelines of NIST publication
      800-38D.

      This can be achieved by having a separate encryption key protected
      by
      the root key and rotating the encryption key, and maybe maintaining
      a
      keyring. This way, we won't have to rotate shards to rotate the
      encryption
      key and won't need to change the shards -- this will also allow the
      encryption key to be rotated behind-the-scenes automatically as per
      NIST guidance.
    </issue>
    <issue severity="important" priority="elevated">
      write an adr about why those asnyc are snyc from now on:

      // TOD we don't have any retry for policies or for recovery info.
      // they are equally important.

      // TO: these xsync operations can cause race conditions
      //
      // 1. process a writes secret
      // 2. process b marks secret as deleted
      // 3. in memory we write then delete
      // 4. but to the backing store it goes as delete then write.
      // 5. memory: secret deleted; backing store: secret exists.
      //
      // to solve it; have a queue of operations (as a go channel)
      // and do not consume the next operation until the current
      // one is complete.
      //
      // have one channel for each resource:
      // - secrets
      // - policies
      // - key recovery info.
      //
      // Or as an alternative; make these xsync operations sync
      // and wait for them to complete before reporting success.
      // this will make the architecture way simpler without needing
      // to rely on channels.
    </issue>
    <issue>
      the backing store is considered untrusted and it stores
      encrypted information
      todo: if it's "really" untrusted then maybe it's better to encrypt
      everything
      (including metadata) -- check how other secrets managers does this.
    </issue>
    <issue kind="good-first-issue">
      create a `spike status` command that shows the current status an
      stats
      of SPIKE Nexus (i.e. whether root key initialized; how many secrets
      are
      in the system etc.)
      It does not have to be too detailed; we can always amend it later.
    </issue>
    <issue>
      Kubernetification
    </issue>
    <issue>
      v.1.0.0 Requirements:
      - Having S3 as a backing store
    </issue>
    <issue>
      Consider a health check / heartbeat between Nexus and Keeper.
      This can be more frequent than the root key sync interval.
    </issue>
    <issue>
      Unit tests and coverage reports.
      Create a solid integration test before.
    </issue>
    <issue>
      Test automation.
    </issue>


    <issue>
      spike dev mode
    </issue>
    <issue>
      document limits and maximums of SPIKE (such as key length, path
      lenght, policy size etc)
    </issue>
    <issue>
      double encryption when passing secrets around
      (can be optional for client-nexus interaction; and can be mandatory
      for
      tools that transcend trust boundaries (as in a relay / message queue
      that
      may be used for secrets federation)
    </issue>
    <issue>
      active/standby HA mode
    </issue>
    <issue>
      pattern-based random secret generation
    </issue>
    <issue>
      admin ui
    </issue>
    <issue>
      guidelines about how to backup and restore
    </issue>
    <issue>
      - AWS KMS support for keepers
      - Azure keyvault support for keepers
      - GCP kms support for keepers
      - HSM support for keepers
      - OCI kms support for keepers
      - keepers storing their shards in a separate SPIKE deployment
      (i.e. SPIKE using another SPIKE to restore root token)
    </issue>
    <issue>
      postgresql backend
    </issue>
    <issue>
      audit targets:
      - file
      - syslog
      - socket
      (if audit targets are enabled then command will not execute unless
      an
      audit trail is started)
    </issue>
    <issue>
      have a mode where nexus does not create tables upon initialization
      (i.e. the admin is expected to create them beforehand)
    </issue>
    <issue>
      maybe ha mode

      HA Mode in OpenBao: In HA mode, OpenBao operates with one active server
      and multiple standby servers. The active server processes all requests,
      while standby servers redirect requests to the active instance. If the
      active server fails, one of the standby servers takes over as the new
      active instance. This mechanism relies on PostgreSQL's ability to manage
      locks and ensure consistency across nodes35.
      Limitations:
      The PostgreSQL backend for OpenBao is community-supported and considered
      in an early preview stage, meaning it may have breaking changes or limited
      testing in production environments2.
      While PostgreSQL supports replication and failover mechanisms for its own
      HA, these features operate independently of OpenBao's HA mode. Proper
      configuration and monitoring of the PostgreSQL cluster are essential to
      ensure database-level resilience
    </issue>
    <issue>
      OIDC authentication for named admins.
    </issue>
    <issue>
      SPIKE Dynamic secret sidecar injector
    </issue>
  </runner-up>
  <future reason="no immediate product value">
    <issue>
      get an OpenSSF badge sometime.
    </issue>
  </future>
  <backlog>

    <issue kind="v1.0-requirement">
      - Run SPIKE in Kubernetes too.
    </issue>
    <issue kind="v1.0-requirement">
      - Postgres support as a backing store.
    </issue>
    <issue kind="v1.0-requirement">
      - Ability to channel audit logs to a log aggregator.
    </issue>
    <issue kind="v1.0-requirement">
      - OIDC integration: Ability to connect to an identity provider.
    </issue>
    <issue kind="v1.0-requirement">
      - ESO (External Secrets Operator) integration
    </issue>
    <issue kind="v1.0-requirement">
      - An ADMIN UI (linked to OIDC probably)
    </issue>
    <issue kind="v1.0-requirement">
      - Ability to use the RESTful API without needing an SDK.
      That could be hard though since we rely on SPIFFE authentication and
      SPIFFE workload API to gather certs: We can use a tool to automate
      that
      part. But it's not that hard either if I know where my certs are:
      `curl --cert /path/to/svid_cert.pem --key /path/to/svid_key.pem
      https://mtls.example.com/resource`
    </issue>
    <issue kind="v1.0-requirement">
      > 80% unit test coverage
    </issue>
    <issue kind="v1.0-requirement">
      Fuzzing for the user-facing API
    </issue>
    <isssue kind="v1.0-requirement">
      100% Integration test (all features will have automated integration
      tests
      in all possible environments)
    </isssue>
    <issue kind="user-request">
      Ability to add custom metadata to secrets.
    </issue>
    <issue>
      We need use cases in the website
      - Policy-based access control for workloads
      - Secret CRUD operations
      - etc
    </issue>
    <issue kind="v1.0-requirement">
      Postgresql support for backing store.
    </issue>
    <issue>
      maybe a default auditor SPIFFEID that can only read stuff (for
      Pilot;
      not for named admins; named admins will use the policy system
      instead)
    </issue>
    <issue>
      Optionally not create tables and other ddl during backing store
      creation
    </issue>
    <issue>
      SPIKE Pilot to ingest a policy YAML file(s) to create policies.
      (similar to kubectl)
    </issue>
    <issue>
      - SPIKE Keep Sanity Tests
      - Ensure that the root key is stored in SPIKE Keep's memory.
      - Ensure that SPIKE Keep can return the root key back to SPIKE
      Nexus.
    </issue>
    <issue>
      Try SPIKE on a Mac.
    </issue>
    <issue>
      Try SPIKE on an x-86 Linux.
    </issue>
    <issue>
      not an adr, but a feature flag to have it start w/o a prepopulated db.
      that can also be useful for brownfield deployments.

      --

      based on the following, maybe move SQLite "create table" ddls to a
      separate file.
      Still a "tool" or a "job" can do that out-of-band.

      update: for SQLite it does not matter as SQLite does not have a
      concept
      of RBAC; creating a db is equivalent to creating a file.
      For other databases, it can be considered, so maybe write an ADR for
      that.

      ADR:

      It's generally considered better security practice to create the
      schema
      out-of-band (separate from the application) for several reasons:

      Principle of Least Privilege:

      The application should only have the permissions it needs for
      runtime
      (INSERT, UPDATE, SELECT, etc.)
      Schema modification rights (CREATE TABLE, ALTER TABLE, etc.) are not
      needed during normal operation
      This limits potential damage if the application is compromised


      Change Management:

      Database schema changes can be managed through proper migration
      tools
      Changes can be reviewed, versioned, and rolled back if needed
      Prevents accidental schema modifications during application restarts


      Environment Consistency:

      Ensures all environments (dev, staging, prod) have identical schemas
      Reduces risk of schema drift between environments
      Makes it easier to track schema changes in version control
    </issue>


    <qa>
      <issue>
        - SPIKE Nexus Sanity Tests
        - Ensure SPIKE Nexus caches the root key in memory.
        - Ensure SPIKE Nexus reads from SPIKE keep if it does not have
        the root
        key.
        - Ensure SPIKE Nexus saves the encrypted root key to the
        database.
        - Ensure SPIKE Nexus caches the user's session key.
        - Ensure SPIKE Nexus removes outdated session keys.
        - Ensure SPIKE Nexus does not re-init (without manual
        intervention)
        after
        being initialized.
        - Ensure SPIKE Nexus adheres to the bootstrapping sequence
        diagram.
        - Ensure SPIKE Nexus backs up the admin token by encrypting it
        with the
        root
        key and storing in the database.
        - Ensure SPIKE Nexus stores the initialization tombstone in the
        database.
      </issue>
      <issue>
        - SPIKE Pilot Sanity Tests
        - Ensure SPIKE Pilot denies any operation if SPIKE Nexus is not
        initialized.
        - Ensure SPIKE Pilot can warn if SPIKE Nexus is unreachable
        - Ensure SPIKE Pilot does not indefinitely hang up if SPIRE is
        not
        there.
        - Ensure SPIKE Pilot can get and set a secret.
        - Ensure SPIKE Pilot can do a force reset.
        - Ensure SPIKE Pilot can recover the root password.
        - Ensure that after `spike init` you have a password-encrypted
        root key
        in the db.
        - Ensure that you can recover the password-encrypted root key.
      </issue>
    </qa>


  </backlog>
  <future>
    <issue kind="idea">
      ideation:

      state is expensive to maintain.
      thats one of the reasons cloud services try and decouple/minimize
      state
      as such, the fewest number of state stores I can get away with
      reasonably the better
      and the state stores that are light weight are much better then the
      state stores that are heavy weight.
      there is no more heavyyweight state store than a network attached
      sql sever.


      It makes sense to argue that "we already have paid the expensive
      cost of a postgresql,
      so we just want to use that rather then add anything else". That,
      can make sense.

      but for those of us not carrying a postgresql already, its better
      not to have to have one added.

      so... it makes sense to make the backing store "plugin-based"

      3 backends that people might want for different reasons:
      * s3 - be stateless for your own admin needs, state managed by
      someone else
      * k8s crds - you are already maintaining an etcd cluster. Might as
      well reuse it
      * postgresql - you maintain a postgresql and want to reuse that

      the first two initially feel different then postgresql code wise...
      they are document stores.
      But posgres is pretty much json-compatible; besides SPIKE does not
      have a complicated ata model.
      So, we can find a common ground and treat all databases that are
      plugged-in as forms of document stores.

      It could keep the code to talk to the db to a real minimum.

      The files should be encrypted by the spike key, so should be fine
      just putting in a k8s crd or something without a second layer of
      encryption
      that can be a big selling point. already have a small k8s cluster?
      just add this component and now you have a secret store too. no
      hassle.
    </issue>
    <issue kind="idea">
      use custom resources as backing store;
      since everything is encrypted and not many people want a fast
      secrets creation throughtput it woudl be useful.
      because then you can do `helm install spiffe/spire` and use it
      without any state tracking.
    </issue>
    <issue kind="idea">
      for k8s instructions (docs)
      Might recommend deploying with the chart rather then from scratch,
      which has a lot of those settings. then we can call out the settings
      in the chart on how to do it
    </issue>
    <issue>
      update the guides: PSP is not a thing anymore; better update it
      to Pod Security Standards.
    </issue>
    <issue>
      An external secrets store (such as Hashi Vault) can use SPIKE Nexus
      to
      auto-unseal itself.
    </issue>

    <issue>
      multiple keeper clusters:

      keepers:
      - nodes: [n1, n2, n3, n4, n5]
      - nodes: [dr1, dr2]

      if it cant assemble back from the first pool, it could try the next
      pool, which could be stood up only during disaster recovery.
    </issue>
    <issue>
      a tool to read from one cluster of keepers to hydrate a different
      cluster of keepers.
    </issue>

    <issue>
      since OPA knows REST, can we expose a policy evaluation endpoint to
      help OPA augment/extend SPIKE policy decisions?
    </issue>
    <issue>
      maybe create an interface for kv, so we can have thread-safe
      variants too.
    </issue>

    <issue>
      maybe create a password manager tool as an example use case
    </issue>

    <issue>
      A `stats` endpoint to show the overall
      system utilization
      (how many secrets; how much memory, etc)
    </issue>
    <issue>
      maybe inspire admin UI from keybase
      https://keybase.io/v0lk4n/devices
      for that, we need an admin ui first :)
      for that we need keycloak to experiment with first.
    </issue>

    <issue>
      the current docs are good and all but they are not good for seo; we
      might
      want to convert to something like zola later down the line
    </issue>

    <issues>
      wrt ADR-0014:
      Maybe we should use something S3-compatible as primary storage
      instead of sqlite.
      But that can wait until we implement other features.

      Besides, Postgres support will be something that some of the
      stakeholders
      want to see too.
    </issues>


    <issue>
      SPIKE Dev Mode:

      * Single binary
      * `keeper` functionality runs in memory
      * `nexus` uses an in-memory store, and its functionality is in the
      single
      binary too.
      * only networking is between the binary and SPIRE Agent.
      * For development only.

      The design should be maintainable with code reuse and should not
      turn into
      maintaining two separate projects.
    </issue>
    <issue>
      rate limiting to api endpoints.
    </issue>
    <issue>
      * super admin can create regular admins and other super admins.
      * super admin can assign backup admins.
      (see drafts.txt for more details)
    </issue>
    <issue>
      Each keeper is backed by a TPM.
    </issue>
    <issue>
      Do some static analysis.
    </issue>
    <to-plan>
      <issue>
        S3 (or compatible) backing store
      </issue>
      <issue>
        File-based backing store
      </issue>
      <issue>
        In memory backing store
      </issue>
      <issue>
        Kubernetes Deployment
      </issue>
    </to-plan>
    <issue>
      Initial super admin can create other admins.
      So that, if an admin leaves, the super admin can delete them.
      or if the password of an admin is compromised, the super admin can
      reset it.
    </issue>
    <issue>
      - Security Measures (SPIKE Nexus)
      - Encrypting the root key with admin password is good
      Consider adding salt to the password encryption
      - Maybe add a key rotation mechanism for the future
    </issue>
    <issue>
      - Error Handling
      - Good use of exponential retries
      - Consider adding specific error types/codes for different failure
      scenarios
      - Might want to add cleanup steps for partial initialization
      failures
    </issue>
    <issue>
      Ability to stream logs and audit trails outside of std out.
    </issue>
    <issue>
      Audit logs should write to a separate location.
    </issue>
    <issue>
      Create a dedicated OIDC resource server (that acts like Pilot but
      exposes
      a
      restful API for things like CI/CD integration.
    </issue>
    <issue>
      HSM integration (i.e. root key is managed/provided by an HSM, and
      the key
      ever leaves the trust boundary of the HSM.
    </issue>
    <issue>
      Ability to rotate the root key (automatic via Nexus).
    </issue>
    <issue>
      Ability to rotate the admin token (manual).
    </issue>
    <issue>
      Encourage to create users instead of relying on the system user.
    </issue>
  </future>
</stuff>