== Physical Plan ==
AdaptiveSparkPlan (10)
+- == Final Plan ==
ObjectHashAggregate (6)
+- ShuffleQueryStage (5), Statistics(sizeInBytes=8.6 KiB, rowCount=50)
+- Exchange (4)
+- ObjectHashAggregate (3)
+- * Project (2)
+- * Scan ExistingRDD Delta Table State #29 - hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/prp/supplier-minimum-profile/_delta_log (1)
+- == Initial Plan ==
ObjectHashAggregate (9)
+- Exchange (8)
+- ObjectHashAggregate (7)
+- Project (2)
+- Scan ExistingRDD Delta Table State #29 - hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/prp/supplier-minimum-profile/_delta_log (1)
(1) Scan ExistingRDD Delta Table State #29 - hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/prp/supplier-minimum-profile/_delta_log [codegen id : 1]
Output [10]: [txn#24455, add#24456, remove#24457, metaData#24458, protocol#24459, cdc#24460, checkpointMetadata#24461, sidecar#24462, domainMetadata#24463, commitInfo#24464]
Arguments: [txn#24455, add#24456, remove#24457, metaData#24458, protocol#24459, cdc#24460, checkpointMetadata#24461, sidecar#24462, domainMetadata#24463, commitInfo#24464], Delta Table State #29 - hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/prp/supplier-minimum-profile/_delta_log MapPartitionsRDD[632] at $anonfun$recordDeltaOperationInternal$1 at DatabricksLogging.scala:128, ExistingRDD, UnknownPartitioning(0)
(2) Project [codegen id : 1]
Output [6]: [txn#24455, add#24456, remove#24457, metaData#24458, protocol#24459, domainMetadata#24463]
Input [10]: [txn#24455, add#24456, remove#24457, metaData#24458, protocol#24459, cdc#24460, checkpointMetadata#24461, sidecar#24462, domainMetadata#24463, commitInfo#24464]
(3) ObjectHashAggregate
Input [6]: [txn#24455, add#24456, remove#24457, metaData#24458, protocol#24459, domainMetadata#24463]
Keys: []
Functions [10]: [partial_collect_set(domainMetadata#24463, 0, 0), partial_collect_set(txn#24455, 0, 0), partial_count(protocol#24459), partial_sum(add#24456.size), partial_last(metaData#24458, true), partial_count(metaData#24458), partial_last(protocol#24459, true), partial_count(remove#24457), partial_count(add#24456), partial_count(txn#24455)]
Aggregate Attributes [12]: [buf#24602, buf#24603, count#24604L, sum#24605L, last#24606, valueSet#24607, count#24608L, last#24609, valueSet#24610, count#24611L, count#24612L, count#24613L]
Results [12]: [buf#24614, buf#24615, count#24616L, sum#24617L, last#24618, valueSet#24619, count#24620L, last#24621, valueSet#24622, count#24623L, count#24624L, count#24625L]
(4) Exchange
Input [12]: [buf#24614, buf#24615, count#24616L, sum#24617L, last#24618, valueSet#24619, count#24620L, last#24621, valueSet#24622, count#24623L, count#24624L, count#24625L]
Arguments: SinglePartition, ENSURE_REQUIREMENTS, [plan_id=5684]
(5) ShuffleQueryStage
Output [12]: [buf#24614, buf#24615, count#24616L, sum#24617L, last#24618, valueSet#24619, count#24620L, last#24621, valueSet#24622, count#24623L, count#24624L, count#24625L]
Arguments: 0
(6) ObjectHashAggregate
Input [12]: [buf#24614, buf#24615, count#24616L, sum#24617L, last#24618, valueSet#24619, count#24620L, last#24621, valueSet#24622, count#24623L, count#24624L, count#24625L]
Keys: []
Functions [10]: [collect_set(domainMetadata#24463, 0, 0), collect_set(txn#24455, 0, 0), count(protocol#24459), sum(add#24456.size), last(metaData#24458, true), count(metaData#24458), last(protocol#24459, true), count(remove#24457), count(add#24456), count(txn#24455)]
Aggregate Attributes [10]: [collect_set(domainMetadata#24463, 0, 0)#24526, collect_set(txn#24455, 0, 0)#24525, count(protocol#24459)#24524L, sum(add#24456.size)#24519L, last(metaData#24458) ignore nulls#24527, count(metaData#24458)#24523L, last(protocol#24459) ignore nulls#24528, count(remove#24457)#24522L, count(add#24456)#24521L, count(txn#24455)#24520L]
Results [14]: [collect_set(domainMetadata#24463, 0, 0)#24526 AS domainMetadata#24529, null AS fileSizeHistogram#24530, collect_set(txn#24455, 0, 0)#24525 AS setTransactions#24531, null AS numDeletedRecordsOpt#24532, count(protocol#24459)#24524L AS numOfProtocol#24533L, coalesce(sum(add#24456.size)#24519L, 0) AS sizeInBytes#24534L, last(metaData#24458) ignore nulls#24527 AS metadata#24535, count(metaData#24458)#24523L AS numOfMetadata#24536L, null AS deletedRecordCountsHistogramOpt#24537, null AS numDeletionVectorsOpt#24538, last(protocol#24459) ignore nulls#24528 AS protocol#24539, count(remove#24457)#24522L AS numOfRemoves#24540L, count(add#24456)#24521L AS numOfFiles#24541L, count(txn#24455)#24520L AS numOfSetTransactions#24542L]
(7) ObjectHashAggregate
Input [6]: [txn#24455, add#24456, remove#24457, metaData#24458, protocol#24459, domainMetadata#24463]
Keys: []
Functions [10]: [partial_collect_set(domainMetadata#24463, 0, 0), partial_collect_set(txn#24455, 0, 0), partial_count(protocol#24459), partial_sum(add#24456.size), partial_last(metaData#24458, true), partial_count(metaData#24458), partial_last(protocol#24459, true), partial_count(remove#24457), partial_count(add#24456), partial_count(txn#24455)]
Aggregate Attributes [12]: [buf#24602, buf#24603, count#24604L, sum#24605L, last#24606, valueSet#24607, count#24608L, last#24609, valueSet#24610, count#24611L, count#24612L, count#24613L]
Results [12]: [buf#24614, buf#24615, count#24616L, sum#24617L, last#24618, valueSet#24619, count#24620L, last#24621, valueSet#24622, count#24623L, count#24624L, count#24625L]
(8) Exchange
Input [12]: [buf#24614, buf#24615, count#24616L, sum#24617L, last#24618, valueSet#24619, count#24620L, last#24621, valueSet#24622, count#24623L, count#24624L, count#24625L]
Arguments: SinglePartition, ENSURE_REQUIREMENTS, [plan_id=5674]
(9) ObjectHashAggregate
Input [12]: [buf#24614, buf#24615, count#24616L, sum#24617L, last#24618, valueSet#24619, count#24620L, last#24621, valueSet#24622, count#24623L, count#24624L, count#24625L]
Keys: []
Functions [10]: [collect_set(domainMetadata#24463, 0, 0), collect_set(txn#24455, 0, 0), count(protocol#24459), sum(add#24456.size), last(metaData#24458, true), count(metaData#24458), last(protocol#24459, true), count(remove#24457), count(add#24456), count(txn#24455)]
Aggregate Attributes [10]: [collect_set(domainMetadata#24463, 0, 0)#24526, collect_set(txn#24455, 0, 0)#24525, count(protocol#24459)#24524L, sum(add#24456.size)#24519L, last(metaData#24458) ignore nulls#24527, count(metaData#24458)#24523L, last(protocol#24459) ignore nulls#24528, count(remove#24457)#24522L, count(add#24456)#24521L, count(txn#24455)#24520L]
Results [14]: [collect_set(domainMetadata#24463, 0, 0)#24526 AS domainMetadata#24529, null AS fileSizeHistogram#24530, collect_set(txn#24455, 0, 0)#24525 AS setTransactions#24531, null AS numDeletedRecordsOpt#24532, count(protocol#24459)#24524L AS numOfProtocol#24533L, coalesce(sum(add#24456.size)#24519L, 0) AS sizeInBytes#24534L, last(metaData#24458) ignore nulls#24527 AS metadata#24535, count(metaData#24458)#24523L AS numOfMetadata#24536L, null AS deletedRecordCountsHistogramOpt#24537, null AS numDeletionVectorsOpt#24538, last(protocol#24459) ignore nulls#24528 AS protocol#24539, count(remove#24457)#24522L AS numOfRemoves#24540L, count(add#24456)#24521L AS numOfFiles#24541L, count(txn#24455)#24520L AS numOfSetTransactions#24542L]
(10) AdaptiveSparkPlan
Output [14]: [domainMetadata#24529, fileSizeHistogram#24530, setTransactions#24531, numDeletedRecordsOpt#24532, numOfProtocol#24533L, sizeInBytes#24534L, metadata#24535, numOfMetadata#24536L, deletedRecordCountsHistogramOpt#24537, numDeletionVectorsOpt#24538, protocol#24539, numOfRemoves#24540L, numOfFiles#24541L, numOfSetTransactions#24542L]
Arguments: isFinalPlan=true