Following recent feedback on this forum post - Post
I’ve restructured my proposal to prioritize the foundational ORM primitives rather than jumping directly to SRF routing.
Revised phased approach:
CompositeField— Enabling expressions to declare tuple return types. The existing ColPairs (composite PK support) andallows_composite_expressionsflag show the ORM is already moving in this direction.- output_field on
alias_mapmembers — Having BaseTable and Join define their own output_field so that lookup resolution in names_to_path() can transition fromself.model→self.base_table.output_field. This delegates SQL generation to expressions instead of hardcoding it in the compiler. - SRF support via LATERAL JOIN — With the above foundations, a
TableValuedFunctionalias_map member naturally integrates: add_annotation() detectsset_returning=True, routes the expression toalias_mapwith aCompositeFieldoutput, and replaces the annotation with a Col reference. The compiler emitsLATERAL generate_series(...)in the FROM clause.
One specific question: Once Phase 2 makes alias_map members self-describing via
output_field, would you expect TableValuedFunction to integrate through Query.join() (with a lightweight adaptation for members that lack join_cols), or is direct alias_map insertion with manual refcount management the cleaner approach? TVFs don’t participate in join deduplication or promotion, so I lean toward direct insertion — but I’d value your perspective on long-term maintainability.
Updated AST ( Abstract Syntax Tree )
User writes:
.annotate(series=GenerateSeries(1, 10)).filter(series__gt=5)
│
▼
┌────────────────────────┐
│ Query.add_annotation() │
│ (set_returning=True) │
└────────────┬───────────┘
│
┌──────────────────┴──────────────────┐
▼ ▼
[Phase 2: FROM clause] [Phase 1: SELECT/WHERE]
alias_map["series"] = annotations["series"] =
TableValuedFunction( Col("series",
expr=GenerateSeries(1, 10), CompositeField(
alias="series", IntegerField("value")
output_field=CompositeField( )
IntegerField("value") )
)
) │
│ │
▼ ▼
get_from_clause() iterates resolve_ref("series")
alias_map, calls as_sql() returns the Col object
│ │
▼ ▼
LATERAL generate_series(1, 10) "series"."value" > 5
AS "series"("value")
─────────────────────────────────────────────────────────────
FINAL SQL:
SELECT "myapp_model"."id", "series"."value"
FROM "myapp_model",
LATERAL generate_series(1, 10) AS "series"("value")
WHERE "series"."value" > 5
─────────────────────────────────────────────────────────────