BCDA-9392: Test Container Database (#1207)

## 🎫 Ticket

https://jira.cms.gov/browse/BCDA-9392

## 🛠 Changes

**TestDatabaseContainer type that wraps golang
[testcontainers](https://golang.testcontainers.org/modules/postgres/)
package**. Why?
- BCDA api is using pgx, pgx pool, and sql/db for database connections;
wrapper will create a connection for any of these, which eliminates the
need to create a connection in each test suite/file.
- Calling NewTestDatabaseContainer ensures that the same exact container
setup is called for each test, each test suite, or each file and removes
the burden of configuring for the developer.
- The wrapper has methods such as ExecuteFile and ExecuteDir, which will
help enforce the location and naming of testdata across the repository.
- README has been created for example and intended usage.

**api_test.go used for initial tests to apply containers**
- Example file/suite that was used to try the test container.
- TestJobStatusNotComplete was the primary test that was updated/tested
for the containers

**db/testdata directory creation**
- Standard directory naming for test data.
- README created for example and intended usage

## ℹ️ Context

**The goal of this PR is to**:
- set a standard that can be used across all tests, which will improve
readability and speed up development time
- implement a solution that can be incrementally rolled out with other
dev work with no disruption to current tests or CI/CD.
- reduce development burden when creating tests
- improve test reliability, since the database will not have potential
state changes in between tests

Currently, a single container database is used across all tests, with
some or no cleanup, depending on the tests. There is a risk that tests
are not running in a "clean" state; the shared database could be altered
from a previous test which can affect the accuracy and reliability of
all tests.

Additionally, there is no standard of how we do database integration
tests or unit tests. For database integration tests, we should have the
same setup for the database every time, across all tests that will
utilize the database. This will ensure that the tests are in a good
state before running and their outcome will not affect other tests.

For unit tests, we should be mocking the database each time, but this PR
only address integration test strategies. Because this container
strategy would be implemented at whatever granularity is desired (test
file, suite, sub test, etc) and does not change the current test
database that is used across all tests, it will not affect our current
test suite and our tests can be gradually updated over time, as
developers are working in various package.

With the new changes, a test database container will be created and each
time a new test runs, the database will be restored back to a good known
state with a snapshot, which will ensure a clean state for each run.
Testing data and additional snapshots can be created as needed,
depending on the testing scenario.

<!-- If any of the following security implications apply, this PR must
not be merged without Stephen Walter's approval. Explain in this section
and add @SJWalter11 as a reviewer.
  - Adds a new software dependency or dependencies.
  - Modifies or invalidates one or more of our security controls.
  - Stores or transmits data that was not stored or transmitted before.
- Requires additional review of security implications for other reasons.
-->

## 🧪 Validation

Tests passin, v2/test_api.go file updated to use new test containers
(arbitrarily chosen), and tests added for container.go.

---------

Co-authored-by: Parwinder Bhagat <[email protected]>

Author: laurenkrugen-navapbc

bcda/api/v2/api_test.go

@@ -0,0 +1,47 @@
+# Test Database Container
+
+## Purpose
+To have an idempotent database for each test run.
+
+## Implementation Strategy
+
+`TestDatabaseContainer` is a lightweight wrapper of https://golang.testcontainers.org/modules/postgres/. Because BCDA uses pgx, pgxpool, and sql/db for database connections, this wrapper type will implement methods to utilize any of the aforementioned connection types.
+
+This type also implements methods to apply migrations and seed the database with an initial set of necessary data for the BCDA application to execute database operations.
+
+
+## How To Use
+
+1. Create the container in the setup of the test suite; this is the longest running step.
+2. Create the database connection in the setup of the test or the setup of the subtest.
+3. (optional) Seed any additional test data with TestDatabaseContainer.ExecuteFile() or TestDatabaseContainer.ExecuteDir(). For details on seed data, please consult the README.md in ./testdata
+4. Restore a snapshot in the test teardown.
+
+*Note*: Database snapshots cannot be created or restored if a database connection still exists.
+
+```
+type FooTestSuite struct {
+	suite.Suite
+	db          *sql.DB    // example; pgx or pool can also be used
+	dbContainer db.TestDatabaseContainer.  // example; this is optional to be part of the test suite
+}
+
+func (s *FooTestSuite) SetupSuite() {
+	ctr, err := db.NewTestDatabaseContainer()
+	require.NoError(s.T(), err)
+	s.dbContainer = ctr
+}
+
+
+func (s *FooTestSuite) SetupTest() {
+    db, err := s.dbContainer.NewSqlDbConnection()
+	require.NoError(s.T(), err)
+	s.db = db
+}
+
+func (s *FooTestSuite) TearDownTest() {
+	s.db.Close()
+	err := s.dbContainer.RestoreSnapshot().   // example, you can restore from another desired snapshot
+	require.NoError(s.T(), err)
+}
+```

db/README.md

@@ -0,0 +1,276 @@
+package db
+
+import (
+	"context"
+	"database/sql"
+	"errors"
+	"fmt"
+	"os"
+	"path/filepath"
+	"strings"
+
+	"github.com/golang-migrate/migrate/v4"
+	_ "github.com/golang-migrate/migrate/v4/database/postgres"
+	_ "github.com/golang-migrate/migrate/v4/source/file"
+	"github.com/jackc/pgx/v5"
+	"github.com/jackc/pgx/v5/pgxpool"
+	"github.com/testcontainers/testcontainers-go/modules/postgres"
+)
+
+const postgresImage = "postgres:16-alpine"
+
+type TestDatabaseContainer struct {
+	Container        *postgres.PostgresContainer
+	ConnectionString string
+	migrations       string
+	testdata         string
+}
+
+// ExecuteFile will execute a *.sql file for a database container.
+// Sql files for testing purposes should be under a package's testdata/ directory.
+func (td *TestDatabaseContainer) ExecuteFile(path string) (int64, error) {
+	ctx := context.Background()
+	var rows int64
+
+	file, err := os.Stat(filepath.Clean(path))
+	if err != nil {
+		return rows, fmt.Errorf("failed to stat file: %w", err)
+	}
+
+	if filepath.Ext(file.Name()) != ".sql" {
+		return rows, fmt.Errorf("failed execute file: not a .sql file")
+	}
+
+	content, err := os.ReadFile(filepath.Clean(path))
+	if err != nil {
+		return rows, fmt.Errorf("failed to open file: %w", err)
+	}
+
+	sql := string(content)
+
+	pgx, err := td.NewPgxConnection()
+	if err != nil {
+		return rows, fmt.Errorf("failed to connect to container database: %w", err)
+	}
+	defer pgx.Close(ctx)
+	result, err := pgx.Exec(ctx, sql)
+
+	if err != nil {
+		return rows, fmt.Errorf("failed to execute sql: %w", err)
+	}
+	rows = result.RowsAffected()
+	if rows == 0 {
+		return rows, fmt.Errorf("zero rows affected")
+	}
+
+	return rows, err
+}
+
+// ExecuteFile will execute all *.sql files for the provided dirpath.
+// Is it recommended to use the package's testdata/ directory to add test files.
+// A package's testdata/ dir can be retrieved with GetTestDataDir().
+func (td *TestDatabaseContainer) ExecuteDir(dirpath string) error {
+	var err error
+	testDir, err := os.Stat(dirpath)
+	if err != nil {
+		return fmt.Errorf("failed to get testdata directory: %w", err)
+	}
+
+	if !testDir.IsDir() {
+		return errors.New("failed to get directory; path is not a directory")
+	}
+
+	err = filepath.Walk(dirpath, func(path string, info os.FileInfo, err error) error {
+		if err != nil {
+			return fmt.Errorf("error accessing path %s: %w", path, err)
+		}
+		if !info.IsDir() && (filepath.Ext(info.Name()) == ".sql") {
+			_, err = td.ExecuteFile(path)
+			if err != nil {
+				return fmt.Errorf("failed to execute sql file %s with error: %w", path, err)
+			}
+		}
+		return err
+	})
+	return err
+}
+
+// CreateSnapshot will create a snapshot for a given name. Close any active connections to the database
+// before taking a snapshot.
+func (td *TestDatabaseContainer) CreateSnapshot(name string) error {
+	err := td.Container.Snapshot(context.Background(), postgres.WithSnapshotName(name))
+	if err != nil {
+		return fmt.Errorf("failed to restore container database snapshot: %w", err)
+	}
+	return nil
+}
+
+// RestoreSnapshot will restore the snapshot that is taken after the database container
+// has had the initial migrations and data seed applied. If no name is provided, it will restore
+// the default snapshot. "Base" will restore the database to it's init state.
+func (td *TestDatabaseContainer) RestoreSnapshot(name string) error {
+	err := td.Container.Restore(context.Background(), postgres.WithSnapshotName(name))
+	if err != nil {
+		return fmt.Errorf("failed to restore container database snapshot: %w", err)
+	}
+	return nil
+}
+
+// Return a pgx connection for a given database container.
+func (td *TestDatabaseContainer) NewPgxConnection() (*pgx.Conn, error) {
+	pgx, err := pgx.Connect(context.Background(), td.ConnectionString)
+	if err != nil {
+		return nil, fmt.Errorf("failed to open connection to container database: %w", err)
+	}
+	return pgx, nil
+}
+
+// Return a sql/db connection for a given database container.
+func (td *TestDatabaseContainer) NewSqlDbConnection() (*sql.DB, error) {
+	db, err := sql.Open("postgres", td.ConnectionString+"sslmode=disable")
+	if err != nil {
+		return nil, fmt.Errorf("failed to open connection to container database: %w", err)
+	}
+	return db, nil
+}
+
+// Return a pgx pool for a given database container.
+func (td *TestDatabaseContainer) NewPgxPoolConnection() (*pgxpool.Pool, error) {
+	pool, err := pgxpool.New(context.Background(), td.ConnectionString)
+	if err != nil {
+		return nil, fmt.Errorf("failed to create pool for container database: %w", err)
+	}
+	return pool, nil
+}
+
+// runMigrations runs the production migrations to the local database so there is no drift between prod and local development.
+func (td *TestDatabaseContainer) runMigrations() error {
+	m, err := migrate.New("file:"+td.migrations, td.ConnectionString+"sslmode=disable")
+	if err != nil {
+		return fmt.Errorf("failed to get migrations: %w", err)
+	}
+	err = m.Up()
+	if err != nil {
+		return fmt.Errorf("failed to apply migrations: %w", err)
+	}
+	err, _ = m.Close()
+	if err != nil {
+		return fmt.Errorf("failed to close database: %w", err)
+	}
+	return nil
+}
+
+// initSeed will apply the baseline data to the the database with newly run migrations.
+// For applying test or scenario specific data, utilize ExecuteFile or ExecuteDir.
+func (td *TestDatabaseContainer) initSeed() error {
+	err := td.ExecuteDir(td.testdata)
+	if err != nil {
+		return fmt.Errorf("failed to seed database container: %w", err)
+	}
+	return nil
+}
+
+// Returns a new postgres container with migrations from db/migrations/bcda applied and seed
+// data from db/testdata applied.
+func NewTestDatabaseContainer() (TestDatabaseContainer, error) {
+	ctx := context.Background()
+	c, err := postgres.Run(ctx,
+		postgresImage,
+		postgres.WithDatabase("bcda"),
+		postgres.WithUsername("toor"),
+		postgres.WithPassword("foobar"),
+		postgres.BasicWaitStrategies())
+
+	if err != nil {
+		return TestDatabaseContainer{}, fmt.Errorf("failed to create database container: %w", err)
+	}
+
+	conn, err := c.ConnectionString(ctx)
+	if err != nil {
+		return TestDatabaseContainer{}, fmt.Errorf("failed to get connection string for container database: %w", err)
+	}
+
+	tdc := TestDatabaseContainer{
+		Container:        c,
+		ConnectionString: conn,
+	}
+
+	err = tdc.getSetupDirs()
+	if err != nil {
+		return TestDatabaseContainer{}, fmt.Errorf("failed to get testdata or migrations dirs: %w", err)
+	}
+
+	err = tdc.runMigrations()
+	if err != nil {
+		return TestDatabaseContainer{}, fmt.Errorf("failed to apply migrations to container database: %w", err)
+	}
+
+	err = tdc.initSeed()
+	if err != nil {
+		return TestDatabaseContainer{}, fmt.Errorf("failed to add test data to container database: %w", err)
+	}
+
+	err = tdc.CreateSnapshot("Base")
+	if err != nil {
+		return TestDatabaseContainer{}, err
+	}
+
+	return tdc, nil
+
+}
+
+// GetTestDataDir is a helper function that will return the testdata directory for package in which
+// it is invoked. If the testdata directory has been created in another package or the files exist
+// outside the package, they will not be found.
+func GetTestDataDir() (string, error) {
+	currentDir, err := os.Getwd()
+	if err != nil {
+		return "", fmt.Errorf("failed to get current working directory: %w", err)
+	}
+
+	testDir := filepath.Join(filepath.Clean(currentDir), "testdata")
+	_, err = os.Stat(testDir)
+	if err != nil {
+		return "", fmt.Errorf("failed to get testdata directory: %w", err)
+	}
+
+	return testDir, err
+}
+
+// getSetupDirs ensures that we get the db/testdata and migrations directories no matter where NewTestDatabaseContainer is called.
+func (td *TestDatabaseContainer) getSetupDirs() error {
+	currentDir, err := os.Getwd()
+	if err != nil {
+		return fmt.Errorf("failed to get current working directory: %w", err)
+	}
+	testDir := filepath.Join("db", "testdata")
+	migrationsDir := filepath.Join("db", "migrations", "bcda")
+	dirPaths := []string{testDir, migrationsDir}
+
+	for _, v := range dirPaths {
+		for {
+			targetPath := filepath.Join(filepath.Clean(currentDir), filepath.Clean(v))
+			_, err := os.Stat(targetPath)
+			if err == nil {
+				if strings.Contains(v, "testdata") {
+					td.testdata = targetPath
+				}
+				if strings.Contains(v, "migrations") {
+					td.migrations = targetPath
+				}
+				break
+			}
+			if !os.IsNotExist(err) {
+				return fmt.Errorf("error checking path %s: %w", targetPath, err)
+			}
+
+			parentDir := filepath.Dir(currentDir)
+			if parentDir == currentDir {
+				return fmt.Errorf("file or directory '%s' not found in parent directories", "db/testdata")
+			}
+			currentDir = parentDir
+		}
+	}
+	return nil
+
+}