Distributed_query_processing

Postgres-XC is capable of distributing or replicating user tables across the datanodes. The query processing engine takes advantage of this distribution or replication to improve query performance. For replicated tables, the operations like join, sorting, grouping etc. are delegated completely to the datanode. By directing different queries to different datanodes, where a relation is replicated, overall throughput can be improved. For distributed tables, these operations are distributed (delegated) across the datanodes in parallel. The delegation of operation is possible if the operations do not involve any unshippable expressions. An expression is unshippable, if it can not be evaluated on the datanode. Following sections describe, distributed processing of these operations.

Coordinator and datanode communicate using SQL + libpq interface, thus the delegation of operations happens by sending queries with corresponding clauses.

Since the discussion applies to tables as well as the results of the queries, we use the word relation, which comprises both the data from a table or result of a query.

• Fast Query Shipping (FQS)
• Sorting
• Grouping and aggregation
• Joins

Fast Query Shipping (FQS)

Fast query shipping is not an operation per say but a method to detect whether the whole query is shippable to the datanodes or not and to ship it to the datanodes without planning the query at the coordinator. If a query is completely shippable to the datanode/s, coordinator just acts as an intermediary between the client and the datanode. If a query can not be FQSed, it is planned at the coordinator and each operation involved in query is delegated to the datanodes, if possible.

Sorting

For a replicated relation, if the sort keys (the expressions by which the result is sorted) are shippable, the sorted result is obtained from one of the datanodes, where the relation is replicated. For distributed relations, if the sort keys are shippable, sorted results are obtained from the datanodes where the relation is distributed. These sorted runs are then merged at the coordinator to get the completely sorted results. In order to delegate sorting, ORDER BY clause with appropriate keys is added to the query being sent to the datanode.

Grouping and aggregation

For replicated relaiton, if the GROUP BY expressions and aggregates are shippable, the aggregated and grouped results are obtained from one of the datanodes where the relation is replicated. For distributed relations, if the GROUP BY expressions and aggregates are shippable, the rows on each of the datanode are grouped and partially aggregated on corresponding datanode. The partially aggregated results are combined at the coordinator to obtain the final aggregate values. If the GROUP BY expression contains bare distribution column, the final aggregates are obtained from the datanode, since all the rows corresponding to a group are available on same datanode. For the purpose of obtaining partially aggregated results, the aggregation is carried out in three steps, a. transition b. combination c. finalisation. Steps a and c are same as PostgreSQL, whereas step b is carried out on the results of step a obtained from datanodes. Steps b and c are carried out on coordinator.

Joins