This issue opens some feedbacks to existing documents. It will keep being appended.
docs/design_docs/concepts/planner-pipeline.md
- The pipeline is not type-safe: The first 3 nodes are all things being planned while the last node is a planner
docs/design_docs/concepts/glossary.md
- The definition of
QueryWorkload is unclear. Does it consider
- Query arrival pattern?
- Query repetition?
- Query predictability?
- Data arrival pattern?
- Data type?
- Have we removed the concept of DAG in the new glossary?
- In the
PhysicalPlan section, it says PhysicalPlan includes "runtime topology of data lifecycle stages, placement of computation, transmission, storage", and that PhysicalPlan is owned by downstream. However, as far as I know, currently ASAPPlanner is also sort of planning for stuffs like transmission and storage, so the boundary is not clear to me.
- I suggest in this glossary document we also specify what terms can be used interchangeably, like
pre-ASAP-IR and pre-ASAP-plan; post-ASAP-IR, post-ASAP-plan and post-ASAP-DAG.
docs/design_docs/concepts/pre-asap-ir.md
- Concepts are clear to me, but I don't understand why
Aggregation is a counterpart of Relational and Time. Since in my understanding Aggregation belongs to Relational, while Relational and Time sort of like counterparts to each other since relational data have no notion of order.
- I think the relationship between
pre-ASAP-IR and Node are lacking! For example, imagine the following questions from a new-comer:
- Is
pre-ASAP-IR composed of Nodes? In what manner? What are the "edges"? Is pre-ASAP-IR a tree, or a DAG?
- Is
pre-ASAP-IR a per-query data structure, or a batch-query-level data structure?
- What information does
pre-ASAP-IR encode? Does it include query arrival pattern? query predictability? so on and so forth... Or it encodes query intention only?
- placeholder
docs/design_docs/concepts/post-asap-ir.md
- Same question as in
docs/design_docs/concepts/pre-asap-ir.md: What is the relationship between post-ASAP-IR and Node? What information does post-ASAP-IR encode?
- What is the relation among "Exact work and composition nodes", "ASAP specific nodes" and "pre-ASAP nodes"?
docs/design_docs/architecture/README.md
- This document need rebaking since we are now formally articulating the input and output, and that the new input-output behavior document is currently not consistent with the README file.
- placeholder
docs/design_docs/architecture/input-output-workflow.md
- Please check the PR comments.
- placeholder
This issue opens some feedbacks to existing documents. It will keep being appended.
docs/design_docs/concepts/planner-pipeline.mddocs/design_docs/concepts/glossary.mdQueryWorkloadis unclear. Does it considerPhysicalPlansection, it saysPhysicalPlanincludes "runtime topology of data lifecycle stages, placement of computation, transmission, storage", and thatPhysicalPlanis owned by downstream. However, as far as I know, currentlyASAPPlanneris also sort of planning for stuffs like transmission and storage, so the boundary is not clear to me.pre-ASAP-IRandpre-ASAP-plan;post-ASAP-IR,post-ASAP-planandpost-ASAP-DAG.docs/design_docs/concepts/pre-asap-ir.mdAggregationis a counterpart ofRelationalandTime. Since in my understandingAggregationbelongs toRelational, whileRelationalandTimesort of like counterparts to each other since relational data have no notion of order.pre-ASAP-IRandNodeare lacking! For example, imagine the following questions from a new-comer:pre-ASAP-IRcomposed ofNodes? In what manner? What are the "edges"? Ispre-ASAP-IRa tree, or a DAG?pre-ASAP-IRa per-query data structure, or a batch-query-level data structure?pre-ASAP-IRencode? Does it include query arrival pattern? query predictability? so on and so forth... Or it encodes query intention only?docs/design_docs/concepts/post-asap-ir.mddocs/design_docs/concepts/pre-asap-ir.md: What is the relationship betweenpost-ASAP-IRandNode? What information doespost-ASAP-IRencode?docs/design_docs/architecture/README.mddocs/design_docs/architecture/input-output-workflow.md