Rule Extensibility

Extensible Automation

Logic Bank automates transaction logic for databases - multi-table constraints and derivations, and actions such as sending messages or emails. It is based on Python and SQLAlchemy, an Object Relational Mapper (ORM).

Such backend logic is a significant element of database systems, often nearly half.

    Logic Bank can reduce 95% of your backend code by 40X,
    using a combination of spreadsheet-like Rules and (Python) code.

While 95% automation is impressive, it's not enough.
Virtually all systems include complex database logic that cannot be automated, as well as non-database logic such as sending emails or messages.

It's therefore imperative to provide extensibility, using standard languages and paradigms. In this article, we'll briefly review the spreadsheet like rules, but the main focus is on 2 levels of extensibility:

• Event Handlers - "ad hoc" and generic Python functions

• Extensible Rules - learn how to define new rule types, providing

  • Reuse - such rules can be used in multiple applications

  • Discovery - such rules need to be discoverable by colleagues

Spreadsheet-like Rules Automate the "Cocktail Napkin Spec"

We’ve all seen how a clear specification - just a few lines - balloons into hundreds of lines of code. This leads to the key design objective for Logic Bank:

Introduce Spreadsheet-like Rules to
Automate the "Cocktail Napkin Spec"

The subsections below show how to declare, activate and execute rules.

Declare

Below is the implementation of our cocktail napkin spec to check credit:

In the diagram above, the rules are declared on lines 40-49. These 5 rules shown in the screen shot replace several hundred lines of code (40X), as shown here.

Activate

Activate the rules while opening your database:

session_maker = sqlalchemy.orm.sessionmaker()
session_maker.configure(bind=engine)
session = session_maker()

LogicBank.activate(session=session, activator=declare_logic)

The activate function:

• invokes declare_logic() (shown in the screen shot above) to load and verify the rules
• installs Logic Bank event handlers to listen to SQLAlchemy before_flush events

Execute

Rules are declarative: you do not need to call them, or order them. Internally, the Logic Bank event handlers call the Logic Bank rule engine to perform these services. For more on declarative, click here.

Event Handlers - Ad Hoc Python Functions

While rules are powerful, they cannot automate everything. That leads to the second key design objective:

Rules must be complemented by code for extensibility,
and manageability (debugging, source control, etc). 

Python code is straightforward: your event handler is passed a LogicRow, which includes

• row - an instance of a SQLAlchemy mapped class (here, Order)

  • Use code completion to access attributes (column values)
  • Observe SQLAlchemy rows provide access to related rows (e.g., sales_rep.Manager.FirstName)

• old_row - prior contents

• logic_row - wraps row and old_row, and includes useful state such as nest level, ins_upd_dlt etc. It is also the rule engine executor for the wrapped row instance, via methods for insert / update and delete.

Python extensibility is shown on line 51, invoking the Python event-handler code on line 32:

def congratulate_sales_rep(row: Order, old_row: Order, logic_row: LogicRow):
    if logic_row.ins_upd_dlt == "ins":  # logic engine fills parents for insert
        sales_rep = row.SalesRep  # type : Employee
        if sales_rep is None:
            logic_row.log("no salesrep for this order")
        else:
            logic_row.log(f'Hi, {sales_rep.Manager.FirstName}, congratulate {sales_rep.FirstName} on their new order')

This is just a simple example that logs a message... a real event might generate a RESTful call, or send an email message.

The red dot in the left margin is a breakpoint, illustrating that you can standard Python debugger services - breakpoints, variable inspection, single step, etc.

Event Types: Early, (normal) Event, Commit Event

SQLAlchemy before_flush events expose a list of rows altered by the client to the Logic Bank rule engine. For each row, it creates a logic_row, and initiates the insert / update / delete method as appropriate. In the course of rule execution, it invokes Early and normal events:

• Early Events run prior to rule execution
• Events run after rule execution

After all the rows have been processed, the rule engine cycles through them again, executing and CommitEvents. Since this is subsequent to row logic, all derivations have been performed, including multi-table rules. In particular, parent rows reflect child sums and counts.

Generic Event Handlers

Events described above are typically "tied" to a particular table (mapped class). You can also create reusable extensions: Generic Event Handlers that apply to all tables, typically driven by meta data.

In nw/logic/logic.py, you will find:

def handle_all(logic_row: LogicRow):
    row = logic_row.row
    if logic_row.ins_upd_dlt == "ins" and hasattr(row, "CreatedOn"):
        row.CreatedOn = datetime.datetime.now()
        logic_row.log("early_row_event_all_classes - handle_all sets 'Created_on"'')

Rule.early_row_event_all_classes(early_row_event_all_classes=handle_all)

This illustrates you can provide reusable services (here, time and date stamping) based on aspects such as your naming conventions.

Extended Rules

You can create Rule extensions that extend the 3 Event classes, using standard Python techniques shown below. In this example, we want to define a reusable audit rule, that operates as shown in the upper left code window:

NWRuleExtension.nw_copy_row(copy_from=Employee,
                            copy_to=EmployeeAudit,
                            copy_when=lambda logic_row:
                                logic_row.are_attributes_changed([Employee.Salary, Employee.Title]))

This rule monitors Employee changes: if the Salary or Title are changed, a row is written to EmployeeAudit, where it is initialized with like-named attributes copied from Employee.

Our rule extension operates are summarized in the diagram:

1. logic.py (upper left code window) calls...

2. nw.rule_extensions.py (lower left code window), which invokes the constructor of our extended rule...

3. nw_copy.py (upper right panel, above) is the implementation of our extended rule

Rule Discovery

Creating an extended rule is of little value if colleagues don't find it. Emails get lost. The new rule needs to be integrated into their IDE.

nw.rule_extensions.py "publishes" the rule, and provides documentation, in conjunction with the Python / IDE (here, PyCharm). This enables you to build multiple rules, and make them all IDE discoverable, just by adding these helper functions.

Rule Definition

The constructor creates a rule instance of our “audit” rule, and saves the rule parameters in the rule instance (here, copy_to etc).

Note this class extends Event.

It then calls the superclass constructor. This logs the rule into the “rule bank”, an internal data structure that stores the rules for subsequent rule execution on session.commit().

Rule Execution

Since our rule extends Event, it's execute() method is called when events are executed, with the logic_row as a parameter. Our code thus executes at runtime, with access to:

• with rule parameters available as instance variables
• the row (wrapped in logic_row)