Merge remote-tracking branch 'origin/main' into feature/oidc

README.md

@@ -48,35 +48,36 @@
 
 ## Introduction
 
-**[Infisical](https://infisical.com)** is the open source secret management platform that teams use to centralize their secrets like API keys, database credentials, and configurations.
+**[Infisical](https://infisical.com)** is the open source secret management platform that teams use to centralize their application configuration and secrets like API keys and database credentials as well as manage their internal PKI.
 
-We're on a mission to make secret management more accessible to everyone, not just security teams, and that means redesigning the entire developer experience from ground up.
+We're on a mission to make security tooling more accessible to everyone, not just security teams, and that means redesigning the entire developer experience from ground up.
 
 ## Features
 
-- **[User-friendly dashboard](https://infisical.com/docs/documentation/platform/project)** to manage secrets across projects and environments (e.g. development, production, etc.). 
-- **[Client SDKs](https://infisical.com/docs/sdks/overview)** to fetch secrets for your apps and infrastructure on demand. 
-- **[Infisical CLI](https://infisical.com/docs/cli/overview)** to fetch and inject secrets into any framework in local development and CI/CD. 
-- **[Infisical API](https://infisical.com/docs/api-reference/overview/introduction)** to perform CRUD operation on secrets, users, projects, and any other resource in Infisical. 
-- **[Native integrations](https://infisical.com/docs/integrations/overview)** with platforms like [GitHub](https://infisical.com/docs/integrations/cicd/githubactions), [Vercel](https://infisical.com/docs/integrations/cloud/vercel), [AWS](https://infisical.com/docs/integrations/cloud/aws-secret-manager), and tools like [Terraform](https://infisical.com/docs/integrations/frameworks/terraform), [Ansible](https://infisical.com/docs/integrations/platforms/ansible), and more. 
+- **[User-friendly dashboard](https://infisical.com/docs/documentation/platform/project)** to manage secrets across projects and environments (e.g. development, production, etc.).
+- **[Client SDKs](https://infisical.com/docs/sdks/overview)** to fetch secrets for your apps and infrastructure on demand.
+- **[Infisical CLI](https://infisical.com/docs/cli/overview)** to fetch and inject secrets into any framework in local development and CI/CD.
+- **[Infisical API](https://infisical.com/docs/api-reference/overview/introduction)** to perform CRUD operation on secrets, users, projects, and any other resource in Infisical.
+- **[Native integrations](https://infisical.com/docs/integrations/overview)** with platforms like [GitHub](https://infisical.com/docs/integrations/cicd/githubactions), [Vercel](https://infisical.com/docs/integrations/cloud/vercel), [AWS](https://infisical.com/docs/integrations/cloud/aws-secret-manager), and tools like [Terraform](https://infisical.com/docs/integrations/frameworks/terraform), [Ansible](https://infisical.com/docs/integrations/platforms/ansible), and more.
 - **[Infisical Kubernetes operator](https://infisical.com/docs/documentation/getting-started/kubernetes)** to managed secrets in k8s, automatically reload deployments, and more.
-- **[Infisical Agent](https://infisical.com/docs/infisical-agent/overview)** to inject secrets into your applications without modifying any code logic. 
+- **[Infisical Agent](https://infisical.com/docs/infisical-agent/overview)** to inject secrets into your applications without modifying any code logic.
 - **[Self-hosting and on-prem](https://infisical.com/docs/self-hosting/overview)** to get complete control over your data.
-- **[Secret versioning](https://infisical.com/docs/documentation/platform/secret-versioning)** and **[Point-in-Time Recovery](https://infisical.com/docs/documentation/platform/pit-recovery)** to version every secret and project state. 
-- **[Audit logs](https://infisical.com/docs/documentation/platform/audit-logs)** to record every action taken in a project. 
-- **[Role-based Access Controls](https://infisical.com/docs/documentation/platform/role-based-access-controls)** to create permission sets on any resource in Infisica and assign those to user or machine identities. 
+- **[Secret versioning](https://infisical.com/docs/documentation/platform/secret-versioning)** and **[Point-in-Time Recovery](https://infisical.com/docs/documentation/platform/pit-recovery)** to version every secret and project state.
+- **[Audit logs](https://infisical.com/docs/documentation/platform/audit-logs)** to record every action taken in a project.
+- **[Role-based Access Controls](https://infisical.com/docs/documentation/platform/role-based-access-controls)** to create permission sets on any resource in Infisica and assign those to user or machine identities.
 - **[Simple on-premise deployments](https://infisical.com/docs/self-hosting/overview)** to AWS, Digital Ocean, and more.
-- **[Secret Scanning and Leak Prevention](https://infisical.com/docs/cli/scanning-overview)** to prevent secrets from leaking to git. 
+- **[Internal PKI](https://infisical.com/docs/documentation/platform/pki/private-ca)** to create Private CA hierarchies and start issuing and managing X.509 digital certificates.
+- **[Secret Scanning and Leak Prevention](https://infisical.com/docs/cli/scanning-overview)** to prevent secrets from leaking to git.
 
 And much more.
 
 ## Getting started
 
 Check out the [Quickstart Guides](https://infisical.com/docs/getting-started/introduction)
 
-| Use Infisical Cloud                                                                                                                                     | Deploy Infisical on premise                                                                                                                                                                                                                                                                                                                                                                                                                                           |
-| ------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| The fastest and most reliable way to <br> get started with Infisical is signing up <br> for free to [Infisical Cloud](https://app.infisical.com/login). |  <br> View all [deployment options](https://infisical.com/docs/self-hosting/overview) |
+| Use Infisical Cloud                                                                                                                                     | Deploy Infisical on premise                                                          |
+| ------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------ |
+| The fastest and most reliable way to <br> get started with Infisical is signing up <br> for free to [Infisical Cloud](https://app.infisical.com/login). | <br> View all [deployment options](https://infisical.com/docs/self-hosting/overview) |
 
 ### Run Infisical locally
 

backend/src/db/migrations/20240609133400_private-key-handoff.ts

@@ -0,0 +1,61 @@
+import { Knex } from "knex";
+
+import { TableName } from "../schemas";
+
+export async function up(knex: Knex): Promise<void> {
+  const doesPasswordFieldExist = await knex.schema.hasColumn(TableName.UserEncryptionKey, "hashedPassword");
+  const doesPrivateKeyFieldExist = await knex.schema.hasColumn(
+    TableName.UserEncryptionKey,
+    "serverEncryptedPrivateKey"
+  );
+  const doesPrivateKeyIVFieldExist = await knex.schema.hasColumn(
+    TableName.UserEncryptionKey,
+    "serverEncryptedPrivateKeyIV"
+  );
+  const doesPrivateKeyTagFieldExist = await knex.schema.hasColumn(
+    TableName.UserEncryptionKey,
+    "serverEncryptedPrivateKeyTag"
+  );
+  const doesPrivateKeyEncodingFieldExist = await knex.schema.hasColumn(
+    TableName.UserEncryptionKey,
+    "serverEncryptedPrivateKeyEncoding"
+  );
+  if (await knex.schema.hasTable(TableName.UserEncryptionKey)) {
+    await knex.schema.alterTable(TableName.UserEncryptionKey, (t) => {
+      if (!doesPasswordFieldExist) t.string("hashedPassword");
+      if (!doesPrivateKeyFieldExist) t.text("serverEncryptedPrivateKey");
+      if (!doesPrivateKeyIVFieldExist) t.text("serverEncryptedPrivateKeyIV");
+      if (!doesPrivateKeyTagFieldExist) t.text("serverEncryptedPrivateKeyTag");
+      if (!doesPrivateKeyEncodingFieldExist) t.text("serverEncryptedPrivateKeyEncoding");
+    });
+  }
+}
+
+export async function down(knex: Knex): Promise<void> {
+  const doesPasswordFieldExist = await knex.schema.hasColumn(TableName.UserEncryptionKey, "hashedPassword");
+  const doesPrivateKeyFieldExist = await knex.schema.hasColumn(
+    TableName.UserEncryptionKey,
+    "serverEncryptedPrivateKey"
+  );
+  const doesPrivateKeyIVFieldExist = await knex.schema.hasColumn(
+    TableName.UserEncryptionKey,
+    "serverEncryptedPrivateKeyIV"
+  );
+  const doesPrivateKeyTagFieldExist = await knex.schema.hasColumn(
+    TableName.UserEncryptionKey,
+    "serverEncryptedPrivateKeyTag"
+  );
+  const doesPrivateKeyEncodingFieldExist = await knex.schema.hasColumn(
+    TableName.UserEncryptionKey,
+    "serverEncryptedPrivateKeyEncoding"
+  );
+  if (await knex.schema.hasTable(TableName.UserEncryptionKey)) {
+    await knex.schema.alterTable(TableName.UserEncryptionKey, (t) => {
+      if (doesPasswordFieldExist) t.dropColumn("hashedPassword");
+      if (doesPrivateKeyFieldExist) t.dropColumn("serverEncryptedPrivateKey");
+      if (doesPrivateKeyIVFieldExist) t.dropColumn("serverEncryptedPrivateKeyIV");
+      if (doesPrivateKeyTagFieldExist) t.dropColumn("serverEncryptedPrivateKeyTag");
+      if (doesPrivateKeyEncodingFieldExist) t.dropColumn("serverEncryptedPrivateKeyEncoding");
+    });
+  }
+}

backend/src/db/schemas/user-encryption-keys.ts

@@ -21,7 +21,12 @@ export const UserEncryptionKeysSchema = z.object({
   tag: z.string(),
   salt: z.string(),
   verifier: z.string(),
-  userId: z.string().uuid()
+  userId: z.string().uuid(),
+  hashedPassword: z.string().nullable().optional(),
+  serverEncryptedPrivateKey: z.string().nullable().optional(),
+  serverEncryptedPrivateKeyIV: z.string().nullable().optional(),
+  serverEncryptedPrivateKeyTag: z.string().nullable().optional(),
+  serverEncryptedPrivateKeyEncoding: z.string().nullable().optional()
 });
 
 export type TUserEncryptionKeys = z.infer<typeof UserEncryptionKeysSchema>;

backend/src/ee/services/ldap-config/ldap-config-service.ts

@@ -73,7 +73,13 @@ type TLdapConfigServiceFactoryDep = {
   >;
   userDAL: Pick<
     TUserDALFactory,
-    "create" | "findOne" | "transaction" | "updateById" | "findUserEncKeyByUserIdsBatch" | "find"
+    | "create"
+    | "findOne"
+    | "transaction"
+    | "updateById"
+    | "findUserEncKeyByUserIdsBatch"
+    | "find"
+    | "findUserEncKeyByUserId"
   >;
   userAliasDAL: Pick<TUserAliasDALFactory, "create" | "findOne">;
   permissionService: Pick<TPermissionServiceFactory, "getOrgPermission">;
@@ -592,12 +598,14 @@ export const ldapConfigServiceFactory = ({
     });
 
     const isUserCompleted = Boolean(user.isAccepted);
+    const userEnc = await userDAL.findUserEncKeyByUserId(user.id);
 
     const providerAuthToken = jwt.sign(
       {
         authTokenType: AuthTokenType.PROVIDER_TOKEN,
         userId: user.id,
         username: user.username,
+        hasExchangedPrivateKey: Boolean(userEnc?.serverEncryptedPrivateKey),
         ...(user.email && { email: user.email, isEmailVerified: user.isEmailVerified }),
         firstName,
         lastName,

backend/src/ee/services/saml-config/saml-config-service.ts

@@ -41,7 +41,10 @@ import { TCreateSamlCfgDTO, TGetSamlCfgDTO, TSamlLoginDTO, TUpdateSamlCfgDTO } f
 
 type TSamlConfigServiceFactoryDep = {
   samlConfigDAL: Pick<TSamlConfigDALFactory, "create" | "findOne" | "update" | "findById">;
-  userDAL: Pick<TUserDALFactory, "create" | "findOne" | "transaction" | "updateById" | "findById">;
+  userDAL: Pick<
+    TUserDALFactory,
+    "create" | "findOne" | "transaction" | "updateById" | "findById" | "findUserEncKeyByUserId"
+  >;
   userAliasDAL: Pick<TUserAliasDALFactory, "create" | "findOne">;
   orgDAL: Pick<
     TOrgDALFactory,
@@ -452,6 +455,7 @@ export const samlConfigServiceFactory = ({
     await licenseService.updateSubscriptionOrgMemberCount(organization.id);
 
     const isUserCompleted = Boolean(user.isAccepted);
+    const userEnc = await userDAL.findUserEncKeyByUserId(user.id);
     const providerAuthToken = jwt.sign(
       {
         authTokenType: AuthTokenType.PROVIDER_TOKEN,
@@ -464,6 +468,7 @@ export const samlConfigServiceFactory = ({
         organizationId: organization.id,
         organizationSlug: organization.slug,
         authMethod: authProvider,
+        hasExchangedPrivateKey: Boolean(userEnc?.serverEncryptedPrivateKey),
         authType: UserAliasType.SAML,
         isUserCompleted,
         ...(relayState

backend/src/ee/services/secret-snapshot/snapshot-dal.ts

@@ -331,8 +331,8 @@ export const snapshotDALFactory = (db: TDbClient) => {
    * Prunes excess snapshots from the database to ensure only a specified number of recent snapshots are retained for each folder.
    *
    * This function operates in three main steps:
-   * 1. Pruning snapshots from root/non-versioned folders.
-   * 2. Pruning snapshots from versioned folders.
+   * 1. Pruning snapshots from current folders.
+   * 2. Pruning snapshots from non-current folders (versioned ones).
    * 3. Removing orphaned snapshots that do not belong to any existing folder or folder version.
    *
    * The function processes snapshots in batches, determined by the `PRUNE_FOLDER_BATCH_SIZE` constant,
@@ -350,7 +350,7 @@ export const snapshotDALFactory = (db: TDbClient) => {
 
     try {
       let uuidOffset = "00000000-0000-0000-0000-000000000000";
-      // cleanup snapshots from root/non-versioned folders
+      // cleanup snapshots from current folders
       // eslint-disable-next-line no-constant-condition, no-unreachable-loop
       while (true) {
         const folderBatch = await db(TableName.SecretFolder)
@@ -382,12 +382,11 @@ export const snapshotDALFactory = (db: TDbClient) => {
               .join(TableName.Environment, `${TableName.Environment}.id`, `${TableName.SecretFolder}.envId`)
               .join(TableName.Project, `${TableName.Project}.id`, `${TableName.Environment}.projectId`)
               .join("snapshot_cte", "snapshot_cte.id", `${TableName.Snapshot}.id`)
-              .whereNull(`${TableName.SecretFolder}.parentId`)
               .whereRaw(`snapshot_cte.row_num > ${TableName.Project}."pitVersionLimit"`)
               .delete();
           } catch (err) {
             logger.error(
-              `Failed to prune snapshots from root/non-versioned folders in range ${batchEntries[0]}:${
+              `Failed to prune snapshots from current folders in range ${batchEntries[0]}:${
                 batchEntries[batchEntries.length - 1]
               }`
             );
@@ -399,7 +398,7 @@ export const snapshotDALFactory = (db: TDbClient) => {
         }
       }
 
-      // cleanup snapshots from versioned folders
+      // cleanup snapshots from non-current folders
       uuidOffset = "00000000-0000-0000-0000-000000000000";
       // eslint-disable-next-line no-constant-condition
       while (true) {
@@ -440,7 +439,7 @@ export const snapshotDALFactory = (db: TDbClient) => {
               .delete();
           } catch (err) {
             logger.error(
-              `Failed to prune snapshots from versioned folders in range ${batchEntries[0]}:${
+              `Failed to prune snapshots from non-current folders in range ${batchEntries[0]}:${
                 batchEntries[batchEntries.length - 1]
               }`
             );

backend/src/lib/config/env.ts

@@ -29,7 +29,7 @@ const envSchema = z
     DB_USER: zpStr(z.string().describe("Postgres database username").optional()),
     DB_PASSWORD: zpStr(z.string().describe("Postgres database password").optional()),
     DB_NAME: zpStr(z.string().describe("Postgres database name").optional()),
-
+    BCRYPT_SALT_ROUND: z.number().default(12),
     NODE_ENV: z.enum(["development", "test", "production"]).default("production"),
     SALT_ROUNDS: z.coerce.number().default(10),
     INITIAL_ORGANIZATION_NAME: zpStr(z.string().optional()),

backend/src/lib/crypto/srp.ts

@@ -6,7 +6,7 @@ import tweetnacl from "tweetnacl-util";
 
 import { TUserEncryptionKeys } from "@app/db/schemas";
 
-import { decryptSymmetric, encryptAsymmetric, encryptSymmetric } from "./encryption";
+import { decryptSymmetric128BitHexKeyUTF8, encryptAsymmetric, encryptSymmetric } from "./encryption";
 
 export const generateSrpServerKey = async (salt: string, verifier: string) => {
   // eslint-disable-next-line new-cap
@@ -97,7 +97,13 @@ export const generateUserSrpKeys = async (email: string, password: string) => {
   };
 };
 
-export const getUserPrivateKey = async (password: string, user: TUserEncryptionKeys) => {
+export const getUserPrivateKey = async (
+  password: string,
+  user: Pick<
+    TUserEncryptionKeys,
+    "protectedKeyTag" | "protectedKey" | "protectedKeyIV" | "encryptedPrivateKey" | "iv" | "salt" | "tag"
+  >
+) => {
   const derivedKey = await argon2.hash(password, {
     salt: Buffer.from(user.salt),
     memoryCost: 65536,
@@ -108,17 +114,18 @@ export const getUserPrivateKey = async (password: string, user: TUserEncryptionK
     raw: true
   });
   if (!derivedKey) throw new Error("Failed to derive key from password");
-  const key = decryptSymmetric({
-    ciphertext: user.protectedKey!,
-    iv: user.protectedKeyIV!,
-    tag: user.protectedKeyTag!,
-    key: derivedKey.toString("base64")
+  const key = decryptSymmetric128BitHexKeyUTF8({
+    ciphertext: user.protectedKey as string,
+    iv: user.protectedKeyIV as string,
+    tag: user.protectedKeyTag as string,
+    key: derivedKey
   });
-  const privateKey = decryptSymmetric({
+
+  const privateKey = decryptSymmetric128BitHexKeyUTF8({
     ciphertext: user.encryptedPrivateKey,
     iv: user.iv,
     tag: user.tag,
-    key
+    key: Buffer.from(key, "hex")
   });
   return privateKey;
 };

backend/src/server/routes/v1/admin-router.ts

@@ -80,6 +80,7 @@ export const registerAdminRouter = async (server: FastifyZodProvider) => {
     schema: {
       body: z.object({
         email: z.string().email().trim(),
+        password: z.string().trim(),
         firstName: z.string().trim(),
         lastName: z.string().trim().optional(),
         protectedKey: z.string().trim(),

backend/src/server/routes/v1/identity-kubernetes-auth-router.ts

@@ -198,7 +198,7 @@ export const registerIdentityKubernetesRouter = async (server: FastifyZodProvide
       }),
       response: {
         200: z.object({
-          identityKubernetesAuth: IdentityKubernetesAuthsSchema
+          identityKubernetesAuth: IdentityKubernetesAuthResponseSchema
         })
       }
     },

backend/src/server/routes/v1/password-router.ts

@@ -51,7 +51,8 @@ export const registerPasswordRouter = async (server: FastifyZodProvider) => {
         encryptedPrivateKeyIV: z.string().trim(),
         encryptedPrivateKeyTag: z.string().trim(),
         salt: z.string().trim(),
-        verifier: z.string().trim()
+        verifier: z.string().trim(),
+        password: z.string().trim()
       }),
       response: {
         200: z.object({