Temporary tables

[hatyo]

This introduces temporary tables planning and execution. A temporary table is an in-memory buffer that holds a list of QueryResult objects.

They offer means for storing temporary computation results, and can be leveraged to serve as SQL temporary tables in the future. The following APIs are introduced:

1. TempTable, this wraps a Queue of QueryResult objects, it is mutable, and in its current version, not thread-safe.
2. Two logical expressions are introduced to model table queue source / sink:
   a. TableValuedCorrelationScanExpression enables scanning from a table-valued correlation, such as a TempTable.
   b. TempTableInsertExpression enables inserting into a TempTable .
3. Two physical operators are introduced that correspond to an implementation of the above expressions; TableValuedCorrelationScanPlan and TempTableInsertPlan.
4. Expression implementation rules.

Few tests are added to illustrate the planning capabilities and execution correctness over temporary tables.

Few extra details:

• Added Ser/De logic for QueryResult.
• The TempTable can currently serialize its state, i.e. the underlying queue elements.

Few examples:

Inserting into a temp table:

insert into t2 values ((1, 'foo'), (2, 'bar'))

the logical plan looks like this:

while the physical plan looks like this:

Selecting from a temp table: the following is a logical plan representing a join between two named temp tables:

select * from t1, t2

and the physical NLJ implementation of the join:

This solves #2961

[MMcM]

I agree that Internal is too short for what is now a fairly wide variety of special bindings. But BindingType suggests to me that all bindings would be of one of these types, when these are still just the special ones and normally named parameters aren't one of these types. Also, the serialized enum is PBindingKind. It seems that perhaps a longer consistent name is needed.

[normen662]

The only bindings that do not have an Internal are named bindings coming from the heuristic planner. I think everything in the cascades planner actually has a corresponding Internal (mostly CORRELATION or CONSTANT so far).

[hatyo]

I removed the newly added enum value TABLE_QUEUE effectively reverting the change, but I renamed Internal to BindingKind (not BindingType) to align with the corresponding PB definition in record_query_plan.proto:

fdb-record-layer/fdb-record-layer-core/src/main/proto/record_query_plan.proto

Line 999 in 4ccb4a5

enum PBindingKind {

[MMcM]

Although one might prefer to build a ConstantObjectValue and/or CONSTANT binding, from some higher level query data structure, it works today to call CascadesPlanner with a RecordQuery containing a ParameterComparison and run that plan with an EvaluationContext with a Bindings with a corresponding named parameter. In other words, an external binding kind. These untyped / neutral bindings are not specific to the heuristic planner.

[hatyo]

I reverted the name back to how it was before. It looks like introducing an External binding could cause wider code changes that are outside the scope of this PR. Let's discuss this as a follow up item.

[MMcM]

Because this exhausts, it does not work to have one part of the plan filling and another part emptying. One must, I think, do all the adds and then getReadCursor to process the queue. Does that mean that the table queue itself ought to have a mode for which direction it is currently handling and callers must set that explicitly?

[hatyo]

It is intended to be used such that there are no interleaving reads and writes, to address that, the consumer must make sure that these accesses are synchronised correctly e.g. by means of read-write lock, otherwise it would might break any expected continuation semantics imposed by the consumer.

[foundationdb-ci]

Result of fdb-record-layer-pr on Linux CentOS 7

• Commit ID: 97337ec
• Duration 0:52:40
• Result: ✅ SUCCEEDED
• Error: N/A
• Build Log terminal output (available for 30 days)
• Build Workspace zip file of the working directory (available for 30 days)