-
Notifications
You must be signed in to change notification settings - Fork 7
Rule Extensibility
Logic Bank automates transaction logic for databases - multi-table constraints and derivations, and actions such as sending messages or emails. It is based on Python and SQLAlchemy, an Object Relational Mapper (ORM).
Such backend logic is a significant element of database systems, often nearly half.
Logic Bank can reduce 95% of your backend code by 40X,
using a combination of spreadsheet-like Rules and (Python) code.
While 95% automation is impressive, it's not enough. Virtually all systems include complex database logic that cannot be automated, as well as non-database logic such as sending emails or messages.
It's therefore imperative to provide extensibility, using standard languages and paradigms. In this article, we'll discuss:
-
Spreadsheet-like rules - we'll briefly review these, but the main focus is on 2 levels of extensibility...
-
Event Handlers - Python functions:
-
"ad hoc" functins that apply to a specific table
-
"generic" functions that apply to all tables
-
-
Extensible Rules - learn how to define new rule types, providing
-
Reuse - such rules can be used in multiple applications
-
Discovery - such rules need to be discoverable by colleagues
-
We’ve all seen how a clear specification - just a few lines - balloons into hundreds of lines of code. This leads to the key design objective for Logic Bank:
Introduce Spreadsheet-like Rules to
Automate the "Cocktail Napkin Spec"
The subsections below show how to declare, activate and execute rules.
Below is the implementation of our cocktail napkin spec to check credit:
In the diagram above, the rules are declared on lines 40-49. These 5 rules shown in the screen shot replace several hundred lines of code (40X), as shown here.
Activate the rules while opening your database:
session_maker = sqlalchemy.orm.sessionmaker()
session_maker.configure(bind=engine)
session = session_maker()
LogicBank.activate(session=session, activator=declare_logic)
The activate function:
- invokes
declare_logic()
(shown in the screen shot above) to load and verify the rules - installs Logic Bank event handlers to listen
to SQLAlchemy
before_flush
events
Rules are declarative: you do not need to call them, or order them. Internally, the Logic Bank event handlers call the Logic Bank rule engine to perform these services. For more on declarative, click here.
While rules are powerful, they cannot automate everything. That leads to the second key design objective:
Rules must be complemented by code for extensibility,
and manageability (debugging, source control, etc).
Python code is straightforward: your event handler is
passed a LogicRow
, which includes
-
row
- an instance of a SQLAlchemy mapped class (here,Order
)- Use code completion to access attributes (column values)
- Observe SQLAlchemy rows provide access to related rows (e.g.,
sales_rep.Manager.FirstName
)
-
old_row
- prior contents -
logic_row
- wraps row and old_row, and includes useful state such as nest level, ins_upd_dlt etc. It is also the rule engine executor for the wrapped row instance, via methods for insert / update and delete.
Python extensibility is shown on line 51, invoking the Python event-handler code on line 32:
def congratulate_sales_rep(row: Order, old_row: Order, logic_row: LogicRow):
if logic_row.ins_upd_dlt == "ins": # logic engine fills parents for insert
sales_rep = row.SalesRep # type : Employee
if sales_rep is None:
logic_row.log("no salesrep for this order")
else:
logic_row.log(f'Hi, {sales_rep.Manager.FirstName}, congratulate {sales_rep.FirstName} on their new order')
This is just a simple example that logs a message... a real event might generate a RESTful call, or send an email message.
The red dot in the left margin is a breakpoint, illustrating that you can standard Python debugger services - breakpoints, variable inspection, single step, etc.
SQLAlchemy before_flush
events expose a list of rows altered by the client
to the Logic Bank rule engine. For each row, it creates a logic_row
,
and initiates the insert / update / delete method as appropriate. In the
course of rule execution, it invokes Early and normal events:
-
Early Events
run prior to rule execution -
Events
run after rule execution
After all the rows have been processed, the rule engine cycles
through them again, executing and CommitEvents
. Since this
is subsequent to row logic, all derivations have been performed,
including multi-table rules. In particular, parent rows reflect
child sums and counts.
Events described above are typically "tied" to a particular table (mapped class). You can also create reusable extensions: Generic Event Handlers that apply to all tables, typically driven by meta data.
In nw/logic/logic.py
,
you will find:
def handle_all(logic_row: LogicRow):
row = logic_row.row
if logic_row.ins_upd_dlt == "ins" and hasattr(row, "CreatedOn"):
row.CreatedOn = datetime.datetime.now()
logic_row.log("early_row_event_all_classes - handle_all sets 'Created_on"'')
Rule.early_row_event_all_classes(early_row_event_all_classes=handle_all)
This illustrates you can provide reusable services (here, time and date stamping) based on aspects such as your naming conventions.
You can create Rule extensions that extend the 3 Event classes, using standard Python techniques shown below. In this example, we want to define a reusable audit rule, that operates as shown in the upper left code window:
NWRuleExtension.nw_copy_row(copy_from=Employee,
copy_to=EmployeeAudit,
copy_when=lambda logic_row:
logic_row.are_attributes_changed([Employee.Salary, Employee.Title]))
This rule monitors Employee
changes: if the Salary
or Title
are changed, a row is written to EmployeeAudit
,
where it is initialized with like-named attributes copied from Employee
.
Our rule extension operates are summarized in the diagram:
-
logic.py
(upper left code window) calls... -
nw.rule_extensions.py
(lower left code window), which invokes the constructor of our extended rule... -
nw_copy.py
(upper right panel, above) is the implementation of our extended rule
Creating an extended rule is of little value if colleagues don't find it. Emails get lost. The new rule needs to be integrated into their IDE.
nw.rule_extensions.py
"publishes" the rule, and provides
documentation, in conjunction with the Python / IDE (here, PyCharm).
This enables you to build multiple rules, and make them all
IDE discoverable as shown below, just by adding these helper functions.
The constructor (__init__
) creates a rule instance of our “audit” rule,
and saves the rule parameters in the rule instance (here, copy_to
etc).
Note this class extends Event.
It then calls the superclass constructor. This logs the rule into the
“rule bank”, an internal data structure that stores the rules
for subsequent rule execution on session.commit()
.
Since our rule extends Event, it's execute()
instance method is called
when events are executed, with the logic_row
as a parameter.
Our code thus executes at runtime, with access to:
- with rule parameters available as instance variables
- the row (wrapped in
logic_row
)
For example, this executes the lambda function self.copy_when
saved in the rule instance, passing the logic_row
as an argument:
copy_from = logic_row
do_copy = self.copy_when(copy_from)
Copy was a pretty simple example. The objective was to illustrate the technique, so that you can detect your own patters, and devise reusable solutions, and publish them to your team.
The Logic Bank runtime engine includes copy_row
in the runtime.
The Logic Bank runtime also includes a much more powerful example: Allocation.
User Project Operations
Logic Bank Internals