Skip to content

Refactor msar.alter_columns function #4427

@mathemancer

Description

@mathemancer

Problem

The msar.alter_columns function take in a JSON blob defining alterations for a table's columns, then performs all (or most) of those alterations as a single SQL statement. E.g.,

ALTER TABLE customers
  ALTER COLUMN contact TYPE mathesar_types.email,
  ALTER COLUMN contact SET DEFAULT '[email protected]',
  ALTER COLUMN contact SET NOT NULL,
  ALTER COLUMN name SET DEFAULT 'John Doe',
  ALTER COLUMN name SET NOT NULL,
  DROP COLUMN old_id_no,
  ALTER COLUMN website TYPE mathesar_types.uri;

While our actual code that generates this statement is reasonably clean and readable, the resulting statement can be arbitrarily large, complex, and unreadable.

  • Error responses are terrible, since they involve the whole statement.
  • Debugging is therefore difficult for users and developers.
  • Testing is difficult since input and output are similarly complicated.
  • Modifying the code is difficult, since while the functions generating the statement are separate, they have to interact in a dependent way to make the overall statement execute properly.

Proposed solution

We should refactor this function so that it instead runs each alteration as a separate subroutine. The result should involve looping through function calls that run separate queries:

ALTER TABLE customers ALTER COLUMN contact TYPE mathesar_types.email;
ALTER TABLE customers ALTER COLUMN contact SET DEFAULT '[email protected]';
ALTER TABLE customers ALTER COLUMN contact SET NOT NULL;
ALTER TABLE customers ALTER COLUMN name SET DEFAULT 'John Doe';
ALTER TABLE customers ALTER COLUMN name SET NOT NULL;
ALTER TABLE customers DROP COLUMN old_id_no;
ALTER TABLE customers ALTER COLUMN website TYPE mathesar_types.uri;

Note that each individual statement in the example would be run one at a time by subroutines called by the msar.alter_columns function. Moreover, each function called should be callable from Python if desired.

Additional context

This will cause a (very minor) performance drain when altering tables, but

  • Empirical testing shows the drain will be less than 10%.
  • Alteration operations like these don't happen often.
  • The simplicity and maintainability is well worth this performance cost for now.

Sub-issues

Metadata

Metadata

Assignees

Labels

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions