Design Doc ‐ Blob filtering and on‐the‐go pruning based on filtering predicates

Requirements:

• AIDB should support on-the-go filtering of rows (blobs or derived rows) based on the filtering predicates.
• Maintain the code readability

Proposal:

Main Idea:

One very simple way of achieving this can be:

Changing how we prepare the inputs for the next inference engine here

Steps:

• Convert the filtering predicates in the query in CNF form.
• All the tables/columns that are not the output of any inference service are assumed to be meta tables.
• Determine the set of filtering predicates that can be satisfied using the currently executed inference engines. Initially, filtering predicates related to meta tables are taken.
• Generate a SQL query based on the current set of filtering predicates.
• Execute the SQL query to prepare the inputs for the next inference engine.

E.g.

Let's say the query is:

SELECT blob.id
FROM blobs, objects, obj_colors
WHERE blobs.time > 10 and objects.type = car and obj_colors.color = red

Inference engine order will be [object -> color]

• Execute the query to get the inputs for object inference service

SELECT blob.id
FROM blobs
WHERE blobs.time > 10

• Execute the query to get the inputs for color inference service

SELECT blob.id
FROM blobs, objects
WHERE blobs.time > 10 and objects.type = car