crp-order-qty-opt-service - Details for Query 3093

Submitted Time: 2026/08/10 09:35:09
Duration: 5 s
Succeeded Jobs: 4689 4690 4691 4693 4694 4695 4696 4697 4698

Plan Visualization

Show the Stage ID and Task ID that corresponds to the max metric

digraph G { 0 [id="node0" labelType="html" label=" AdaptiveSparkPlan " tooltip="AdaptiveSparkPlan isFinalPlan=false"]; 1 [id="node1" labelType="html" label="ObjectHashAggregate number of output rows: 522 time in aggregation build total (min, med, max (stageId: taskId)) 578 ms (6 ms, 57 ms, 60 ms (stage 61378.0: task 150704)) spill size total (min, med, max (stageId: taskId)) 0.0 B (0.0 B, 0.0 B, 0.0 B (stage 61378.0: task 150699)) number of sort fallback tasks: 0" tooltip="ObjectHashAggregate(keys=[supplierGroup#724150, supplier#724145, plant#724144, deliveryDateTime#724148, subrange#723984, null#724582, productGroup#724005], functions=[collect_list(struct#724156, 0, 0)])"]; 2 [id="node2" labelType="html" label="AQEShuffleRead number of partitions: 11 partition data size total (min, med, max (stageId: taskId)) 677.1 KiB (50.9 KiB, 60.2 KiB, 72.9 KiB (driver)) number of coalesced partitions: 11" tooltip="AQEShuffleRead coalesced"]; 3 [id="node3" labelType="html" label="Exchange shuffle records written: 2,556 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 2,556 local bytes read total (min, med, max (stageId: taskId)) 322.5 KiB (24.0 KiB, 29.8 KiB, 34.1 KiB (stage 61378.0: task 150706)) merged fetch fallback count: 0 local blocks read: 310 remote merged chunks fetched: 0 remote blocks read: 308 data size total (min, med, max (stageId: taskId)) 1162.4 KiB (32.8 KiB, 45.8 KiB, 57.1 KiB (stage 61375.0: task 150678)) local merged bytes read: 0.0 B local merged chunks fetched: 0 shuffle write time total (min, med, max (stageId: taskId)) 40 ms (1 ms, 1 ms, 1 ms (stage 61375.0: task 150672)) remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 fetch wait time total (min, med, max (stageId: taskId)) 0 ms (0 ms, 0 ms, 0 ms (stage 61378.0: task 150699)) remote bytes read total (min, med, max (stageId: taskId)) 323.3 KiB (23.8 KiB, 28.4 KiB, 36.7 KiB (stage 61378.0: task 150704)) number of partitions: 25 remote reqs duration total (min, med, max (stageId: taskId)) 15 ms (1 ms, 1 ms, 2 ms (stage 61378.0: task 150699)) remote bytes read to disk: 0.0 B shuffle bytes written total (min, med, max (stageId: taskId)) 645.8 KiB (20.0 KiB, 25.7 KiB, 30.2 KiB (stage 61375.0: task 150671))" tooltip="Exchange hashpartitioning(supplierGroup#724150, supplier#724145, plant#724144, deliveryDateTime#724148, subrange#723984, null#724582, productGroup#724005, 25), ENSURE_REQUIREMENTS, [plan_id=286668]"]; 4 [id="node4" labelType="html" label="ObjectHashAggregate number of output rows: 2,556 time in aggregation build total (min, med, max (stageId: taskId)) 720 ms (1 ms, 3 ms, 69 ms (stage 61375.0: task 150671)) spill size total (min, med, max (stageId: taskId)) 0.0 B (0.0 B, 0.0 B, 0.0 B (stage 61375.0: task 150674)) number of sort fallback tasks: 0" tooltip="ObjectHashAggregate(keys=[supplierGroup#724150, supplier#724145, plant#724144, deliveryDateTime#724148, subrange#723984, null AS null#724582, productGroup#724005], functions=[partial_collect_list(struct#724156, 0, 0)])"]; subgraph cluster5 { isCluster="true"; id="cluster5"; label="WholeStageCodegen (7)\n \nduration: total (min, med, max (stageId: taskId))\n845 ms (1 ms, 3 ms, 81 ms (stage 61375.0: task 150671))"; tooltip="WholeStageCodegen (7)"; 6 [id="node6" labelType="html" label=" Project " tooltip="Project [supplierGroup#724150, supplier#724145, plant#724144, deliveryDateTime#724148, subrange#723984, productGroup#724005, struct(product, product#724143, orderDateTime, orderDateTime#724147, availabilityDateTime, availabilityDateTime#724149, nextAvailabilityDateTime, nextAvailabilityDateTime#724151, mrpController, mrpController#723954, sourceOfSupplyUuid, sourceOfSupplyUuid#724146, purchasingOrg, purchasingOrg#723986, purchasingGroup, purchasingGroup#723985, isFallbackOrderOnly, isFallbackOrderOnly#724153) AS struct#724156]"]; 7 [id="node7" labelType="html" label="BroadcastHashJoin number of output rows: 2,940" tooltip="BroadcastHashJoin [product#724143], [product#724004], LeftOuter, BuildRight, false"]; 8 [id="node8" labelType="html" label=" Project " tooltip="Project [sourceOfSupplyUuid#724146, plant#724144, product#724143, supplier#724145, orderDateTime#724147, deliveryDateTime#724148, availabilityDateTime#724149, supplierGroup#724150, nextAvailabilityDateTime#724151, isFallBackOrderOnly#724153, mrpController#723954, subrange#723984, purchasingGroup#723985, purchasingOrg#723986]"]; 9 [id="node9" labelType="html" label="BroadcastHashJoin number of output rows: 2,940" tooltip="BroadcastHashJoin [sourceOfSupplyUuid#724146], [sourceOfSupplyUuid#723983], LeftOuter, BuildRight, false"]; 10 [id="node10" labelType="html" label=" Project " tooltip="Project [plant#724144, product#724143, supplier#724145, sourceOfSupplyUuid#724146, orderDateTime#724147, deliveryDateTime#724148, availabilityDateTime#724149, supplierGroup#724150, nextAvailabilityDateTime#724151, isFallBackOrderOnly#724153, mrpController#723954]"]; 11 [id="node11" labelType="html" label="BroadcastHashJoin number of output rows: 2,940" tooltip="BroadcastHashJoin [plant#724144, product#724143], [plant#723953, product#723952], LeftOuter, BuildRight, false"]; } 12 [id="node12" labelType="html" label="Exchange shuffle records written: 2,940 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 2,940 local bytes read total (min, med, max (stageId: taskId)) 106.1 KiB (2.3 KiB, 4.2 KiB, 6.5 KiB (stage 61375.0: task 150682)) merged fetch fallback count: 0 local blocks read: 104 remote merged chunks fetched: 0 remote blocks read: 75 data size total (min, med, max (stageId: taskId)) 643.1 KiB (5.3 KiB, 24.9 KiB, 59.7 KiB (stage 61363.0: task 150519)) local merged bytes read: 0.0 B local merged chunks fetched: 0 shuffle write time total (min, med, max (stageId: taskId)) 19 ms (0 ms, 0 ms, 3 ms (stage 61363.0: task 150640)) remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 fetch wait time total (min, med, max (stageId: taskId)) 0 ms (0 ms, 0 ms, 0 ms (stage 61375.0: task 150674)) remote bytes read total (min, med, max (stageId: taskId)) 76.9 KiB (449.0 B, 3.3 KiB, 4.8 KiB (stage 61375.0: task 150689)) number of partitions: 25 remote reqs duration total (min, med, max (stageId: taskId)) 4 ms (0 ms, 0 ms, 1 ms (stage 61375.0: task 150677)) remote bytes read to disk: 0.0 B shuffle bytes written total (min, med, max (stageId: taskId)) 183.0 KiB (2.3 KiB, 7.1 KiB, 14.4 KiB (stage 61363.0: task 150519))" tooltip="Exchange hashpartitioning(product#724143, plant#724144, 25), REPARTITION_BY_NUM, [plan_id=286110]"]; subgraph cluster13 { isCluster="true"; id="cluster13"; label="WholeStageCodegen (1)\n \nduration: total (min, med, max (stageId: taskId))\n15.1 s (440 ms, 608 ms, 698 ms (stage 61363.0: task 150622))"; tooltip="WholeStageCodegen (1)"; 14 [id="node14" labelType="html" label="ColumnarToRow number of output rows: 2,940 number of input batches: 198" tooltip="ColumnarToRow"]; } 15 [id="node15" labelType="html" label="Scan parquet number of files read: 198 scan time total (min, med, max (stageId: taskId)) 14.7 s (435 ms, 589 ms, 692 ms (stage 61363.0: task 150622)) metadata time: 0 ms size of files read: 894.9 KiB number of output rows: 2,940" tooltip="FileScan parquet [product#724143,plant#724144,supplier#724145,sourceOfSupplyUuid#724146,orderDateTime#724147,deliveryDateTime#724148,availabilityDateTime#724149,supplierGroup#724150,nextAvailabilityDateTime#724151,isFallBackOrderOnly#724153] Batched: true, DataFilters: [], Format: Parquet, Location: InMemoryFileIndex(1 paths)[hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanac..., PartitionFilters: [], PushedFilters: [], ReadSchema: struct<product:string,plant:string,supplier:string,sourceOfSupplyUuid:string,orderDateTime:timest..."]; 16 [id="node16" labelType="html" label="BroadcastExchange time to broadcast: 1 ms time to build: 8 ms time to collect: 38 ms number of output rows: 2,747 data size: 2.1 MiB" tooltip="BroadcastExchange HashedRelationBroadcastMode(List(input[1, string, true], input[0, string, true]),false), [plan_id=286522]"]; 17 [id="node17" labelType="html" label="Exchange shuffle records written: 2,747 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 2,747 local bytes read total (min, med, max (stageId: taskId)) 134.8 KiB (5.2 KiB, 5.4 KiB, 5.8 KiB (stage 61373.0: task 150655)) merged fetch fallback count: 0 local blocks read: 25 remote merged chunks fetched: 0 remote blocks read: 0 data size: 318.9 KiB local merged bytes read: 0.0 B local merged chunks fetched: 0 shuffle write time: 1 ms remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 fetch wait time total (min, med, max (stageId: taskId)) 0 ms (0 ms, 0 ms, 0 ms (stage 61373.0: task 150648)) remote bytes read: 0.0 B number of partitions: 25 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written: 134.8 KiB" tooltip="Exchange RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=286126]"]; subgraph cluster18 { isCluster="true"; id="cluster18"; label="WholeStageCodegen (2)\n \nduration: 96 ms"; tooltip="WholeStageCodegen (2)"; 19 [id="node19" labelType="html" label=" Project " tooltip="Project [product#723947.internalRefUUID AS product#723952, plant#723946.internalRefUUID AS plant#723953, productionAspect#723949.productPlanningPlants.mrpController.internalRefUUID AS mrpController#723954]"]; 20 [id="node20" labelType="html" label="Filter number of output rows: 2,747" tooltip="Filter (isnotnull(plant#723946.internalRefUUID) AND isnotnull(product#723947.internalRefUUID))"]; 21 [id="node21" labelType="html" label="ColumnarToRow number of output rows: 2,747 number of input batches: 1" tooltip="ColumnarToRow"]; } 22 [id="node22" labelType="html" label="Scan parquet number of files read: 1 scan time: 92 ms dynamic partition pruning time: 0 ms metadata time: 763 ms size of files read: 179.9 KiB number of output rows: 2,747 number of partitions read: 1" tooltip="FileScan parquet [plant#723946,product#723947,productionAspect#723949,systemId#723944] Batched: true, DataFilters: [isnotnull(plant#723946.internalRefUUID), isnotnull(product#723947.internalRefUUID)], Format: Parquet, Location: PreparedDeltaFileIndex(1 paths)[hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20...., PartitionFilters: [], PushedFilters: [IsNotNull(plant.internalRefUUID), IsNotNull(product.internalRefUUID)], ReadSchema: struct<plant:struct<internalRefUUID:string>,product:struct<internalRefUUID:string>,productionAspe..."]; 23 [id="node23" labelType="html" label="BroadcastExchange time to broadcast: 1 ms time to build: 27 ms time to collect: 429 ms number of output rows: 4,343 data size: 2.3 MiB" tooltip="BroadcastExchange HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=286281]"]; subgraph cluster24 { isCluster="true"; id="cluster24"; label="WholeStageCodegen (5)\n \nduration: total (min, med, max (stageId: taskId))\n0 ms (0 ms, 0 ms, 0 ms (stage 61368.0: task 150553))"; tooltip="WholeStageCodegen (5)"; 25 [id="node25" labelType="html" label=" Project " tooltip="Project [internalUUID#723960 AS sourceOfSupplyUuid#723983, supplierSubrange#723975 AS subrange#723984, _extract_internalRefUUID#724268 AS purchasingGroup#723985, _extract_internalRefUUID#724269 AS purchasingOrg#723986]"]; } 26 [id="node26" labelType="html" label="Exchange shuffle records written: 4,343 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 4,343 local bytes read total (min, med, max (stageId: taskId)) 177.8 KiB (7.0 KiB, 7.1 KiB, 7.2 KiB (stage 61368.0: task 150557)) merged fetch fallback count: 0 local blocks read: 25 remote merged chunks fetched: 0 remote blocks read: 0 data size: 522.9 KiB local merged bytes read: 0.0 B local merged chunks fetched: 0 shuffle write time: 1 ms remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 fetch wait time total (min, med, max (stageId: taskId)) 0 ms (0 ms, 0 ms, 0 ms (stage 61368.0: task 150553)) remote bytes read: 0.0 B number of partitions: 25 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written: 177.8 KiB" tooltip="Exchange RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=286148]"]; subgraph cluster27 { isCluster="true"; id="cluster27"; label="WholeStageCodegen (3)\n \nduration: 141 ms"; tooltip="WholeStageCodegen (3)"; 28 [id="node28" labelType="html" label=" Project " tooltip="Project [internalUUID#723960, purchasingGroup#723966.internalRefUUID AS _extract_internalRefUUID#724268, purchasingOrganization#723967.internalRefUUID AS _extract_internalRefUUID#724269, supplierSubrange#723975]"]; 29 [id="node29" labelType="html" label="Filter number of output rows: 4,343" tooltip="Filter isnotnull(internalUUID#723960)"]; 30 [id="node30" labelType="html" label="ColumnarToRow number of output rows: 4,343 number of input batches: 2" tooltip="ColumnarToRow"]; } 31 [id="node31" labelType="html" label="Scan parquet number of files read: 1 scan time: 136 ms dynamic partition pruning time: 0 ms metadata time: 0 ms size of files read: 315.0 KiB number of output rows: 4,343 number of partitions read: 1" tooltip="FileScan parquet [internalUUID#723960,purchasingGroup#723966,purchasingOrganization#723967,supplierSubrange#723975,systemId#723959] Batched: true, DataFilters: [isnotnull(internalUUID#723960)], Format: Parquet, Location: PreparedDeltaFileIndex(1 paths)[hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20...., PartitionFilters: [], PushedFilters: [IsNotNull(internalUUID)], ReadSchema: struct<internalUUID:string,purchasingGroup:struct<internalRefUUID:string>,purchasingOrganization:..."]; 32 [id="node32" labelType="html" label="BroadcastExchange time to broadcast: 0 ms time to build: 1 ms time to collect: 376 ms number of output rows: 159 data size: 2.0 MiB" tooltip="BroadcastExchange HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=286364]"]; subgraph cluster33 { isCluster="true"; id="cluster33"; label="WholeStageCodegen (6)\n \nduration: total (min, med, max (stageId: taskId))\n4 ms (0 ms, 0 ms, 1 ms (stage 61370.0: task 150579))"; tooltip="WholeStageCodegen (6)"; 34 [id="node34" labelType="html" label=" Project " tooltip="Project [internalUUID#723991 AS product#724004, null AS productGroup#724005]"]; } 35 [id="node35" labelType="html" label="Exchange shuffle records written: 159 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 159 local bytes read total (min, med, max (stageId: taskId)) 7.5 KiB (289.0 B, 299.0 B, 336.0 B (stage 61370.0: task 150596)) merged fetch fallback count: 0 local blocks read: 25 remote merged chunks fetched: 0 remote blocks read: 0 data size: 7.5 KiB local merged bytes read: 0.0 B local merged chunks fetched: 0 shuffle write time: 1 ms remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 fetch wait time total (min, med, max (stageId: taskId)) 0 ms (0 ms, 0 ms, 0 ms (stage 61370.0: task 150579)) remote bytes read: 0.0 B number of partitions: 25 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written: 7.5 KiB" tooltip="Exchange RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=286171]"]; subgraph cluster36 { isCluster="true"; id="cluster36"; label="WholeStageCodegen (4)\n \nduration: 94 ms"; tooltip="WholeStageCodegen (4)"; 37 [id="node37" labelType="html" label=" Project " tooltip="Project [internalUUID#723991]"]; 38 [id="node38" labelType="html" label="Filter number of output rows: 159" tooltip="Filter isnotnull(internalUUID#723991)"]; 39 [id="node39" labelType="html" label="ColumnarToRow number of output rows: 159 number of input batches: 1" tooltip="ColumnarToRow"]; } 40 [id="node40" labelType="html" label="Scan parquet number of files read: 1 scan time: 93 ms dynamic partition pruning time: 0 ms metadata time: 0 ms size of files read: 36.0 KiB number of output rows: 159 number of partitions read: 1" tooltip="FileScan parquet [internalUUID#723991,systemId#723990] Batched: true, DataFilters: [isnotnull(internalUUID#723991)], Format: Parquet, Location: PreparedDeltaFileIndex(1 paths)[hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20...., PartitionFilters: [], PushedFilters: [IsNotNull(internalUUID)], ReadSchema: struct<internalUUID:string>"]; 1->0; 2->1; 3->2; 4->3; 6->4; 7->6; 8->7; 9->8; 10->9; 11->10; 12->11; 14->12; 15->14; 16->11; 17->16; 19->17; 20->19; 21->20; 22->21; 23->9; 25->23; 26->25; 28->26; 29->28; 30->29; 31->30; 32->7; 34->32; 35->34; 37->35; 38->37; 39->38; 40->39; }

Details

== Physical Plan ==
AdaptiveSparkPlan (67)
+- == Current Plan ==
   ResultQueryStage (42)
   +- ObjectHashAggregate (41)
      +- AQEShuffleRead (40)
         +- ShuffleQueryStage (39), Statistics(sizeInBytes=1162.4 KiB, rowCount=2.56E+3)
            +- Exchange (38)
               +- ObjectHashAggregate (37)
                  +- * Project (36)
                     +- * BroadcastHashJoin LeftOuter BuildRight (35)
                        :- * Project (25)
                        :  +- * BroadcastHashJoin LeftOuter BuildRight (24)
                        :     :- * Project (14)
                        :     :  +- * BroadcastHashJoin LeftOuter BuildRight (13)
                        :     :     :- ShuffleQueryStage (4), Statistics(sizeInBytes=643.1 KiB, rowCount=2.94E+3)
                        :     :     :  +- Exchange (3)
                        :     :     :     +- * ColumnarToRow (2)
                        :     :     :        +- Scan parquet  (1)
                        :     :     +- BroadcastQueryStage (12), Statistics(sizeInBytes=2.1 MiB, rowCount=2.75E+3)
                        :     :        +- BroadcastExchange (11)
                        :     :           +- ShuffleQueryStage (10), Statistics(sizeInBytes=318.9 KiB, rowCount=2.75E+3)
                        :     :              +- Exchange (9)
                        :     :                 +- * Project (8)
                        :     :                    +- * Filter (7)
                        :     :                       +- * ColumnarToRow (6)
                        :     :                          +- Scan parquet  (5)
                        :     +- BroadcastQueryStage (23), Statistics(sizeInBytes=2.3 MiB, rowCount=4.34E+3)
                        :        +- BroadcastExchange (22)
                        :           +- * Project (21)
                        :              +- ShuffleQueryStage (20), Statistics(sizeInBytes=522.9 KiB, rowCount=4.34E+3)
                        :                 +- Exchange (19)
                        :                    +- * Project (18)
                        :                       +- * Filter (17)
                        :                          +- * ColumnarToRow (16)
                        :                             +- Scan parquet  (15)
                        +- BroadcastQueryStage (34), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
                           +- BroadcastExchange (33)
                              +- * Project (32)
                                 +- ShuffleQueryStage (31), Statistics(sizeInBytes=7.5 KiB, rowCount=159)
                                    +- Exchange (30)
                                       +- * Project (29)
                                          +- * Filter (28)
                                             +- * ColumnarToRow (27)
                                                +- Scan parquet  (26)
+- == Initial Plan ==
   ObjectHashAggregate (66)
   +- Exchange (65)
      +- ObjectHashAggregate (64)
         +- Project (63)
            +- BroadcastHashJoin LeftOuter BuildRight (62)
               :- Project (56)
               :  +- BroadcastHashJoin LeftOuter BuildRight (55)
               :     :- Project (49)
               :     :  +- BroadcastHashJoin LeftOuter BuildRight (48)
               :     :     :- Exchange (43)
               :     :     :  +- Scan parquet  (1)
               :     :     +- BroadcastExchange (47)
               :     :        +- Exchange (46)
               :     :           +- Project (45)
               :     :              +- Filter (44)
               :     :                 +- Scan parquet  (5)
               :     +- BroadcastExchange (54)
               :        +- Project (53)
               :           +- Exchange (52)
               :              +- Project (51)
               :                 +- Filter (50)
               :                    +- Scan parquet  (15)
               +- BroadcastExchange (61)
                  +- Project (60)
                     +- Exchange (59)
                        +- Project (58)
                           +- Filter (57)
                              +- Scan parquet  (26)


(1) Scan parquet 
Output [10]: [product#724143, plant#724144, supplier#724145, sourceOfSupplyUuid#724146, orderDateTime#724147, deliveryDateTime#724148, availabilityDateTime#724149, supplierGroup#724150, nextAvailabilityDateTime#724151, isFallBackOrderOnly#724153]
Batched: true
Location: InMemoryFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-workload-determination-service/out/workload-determination-result/10000002108/results]
ReadSchema: struct<product:string,plant:string,supplier:string,sourceOfSupplyUuid:string,orderDateTime:timestamp,deliveryDateTime:timestamp,availabilityDateTime:timestamp,supplierGroup:string,nextAvailabilityDateTime:timestamp,isFallBackOrderOnly:boolean>

(2) ColumnarToRow [codegen id : 1]
Input [10]: [product#724143, plant#724144, supplier#724145, sourceOfSupplyUuid#724146, orderDateTime#724147, deliveryDateTime#724148, availabilityDateTime#724149, supplierGroup#724150, nextAvailabilityDateTime#724151, isFallBackOrderOnly#724153]

(3) Exchange
Input [10]: [product#724143, plant#724144, supplier#724145, sourceOfSupplyUuid#724146, orderDateTime#724147, deliveryDateTime#724148, availabilityDateTime#724149, supplierGroup#724150, nextAvailabilityDateTime#724151, isFallBackOrderOnly#724153]
Arguments: hashpartitioning(product#724143, plant#724144, 25), REPARTITION_BY_NUM, [plan_id=286110]

(4) ShuffleQueryStage
Output [10]: [product#724143, plant#724144, supplier#724145, sourceOfSupplyUuid#724146, orderDateTime#724147, deliveryDateTime#724148, availabilityDateTime#724149, supplierGroup#724150, nextAvailabilityDateTime#724151, isFallBackOrderOnly#724153]
Arguments: 0

(5) Scan parquet 
Output [4]: [plant#723946, product#723947, productionAspect#723949, systemId#723944]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-productplant]
PushedFilters: [IsNotNull(plant.internalRefUUID), IsNotNull(product.internalRefUUID)]
ReadSchema: struct<plant:struct<internalRefUUID:string>,product:struct<internalRefUUID:string>,productionAspect:struct<productPlanningPlants:struct<mrpController:struct<internalRefUUID:string>>>>

(6) ColumnarToRow [codegen id : 2]
Input [4]: [plant#723946, product#723947, productionAspect#723949, systemId#723944]

(7) Filter [codegen id : 2]
Input [4]: [plant#723946, product#723947, productionAspect#723949, systemId#723944]
Condition : (isnotnull(plant#723946.internalRefUUID) AND isnotnull(product#723947.internalRefUUID))

(8) Project [codegen id : 2]
Output [3]: [product#723947.internalRefUUID AS product#723952, plant#723946.internalRefUUID AS plant#723953, productionAspect#723949.productPlanningPlants.mrpController.internalRefUUID AS mrpController#723954]
Input [4]: [plant#723946, product#723947, productionAspect#723949, systemId#723944]

(9) Exchange
Input [3]: [product#723952, plant#723953, mrpController#723954]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=286126]

(10) ShuffleQueryStage
Output [3]: [product#723952, plant#723953, mrpController#723954]
Arguments: 1

(11) BroadcastExchange
Input [3]: [product#723952, plant#723953, mrpController#723954]
Arguments: HashedRelationBroadcastMode(List(input[1, string, true], input[0, string, true]),false), [plan_id=286522]

(12) BroadcastQueryStage
Output [3]: [product#723952, plant#723953, mrpController#723954]
Arguments: 6

(13) BroadcastHashJoin [codegen id : 7]
Left keys [2]: [plant#724144, product#724143]
Right keys [2]: [plant#723953, product#723952]
Join type: LeftOuter
Join condition: None

(14) Project [codegen id : 7]
Output [11]: [plant#724144, product#724143, supplier#724145, sourceOfSupplyUuid#724146, orderDateTime#724147, deliveryDateTime#724148, availabilityDateTime#724149, supplierGroup#724150, nextAvailabilityDateTime#724151, isFallBackOrderOnly#724153, mrpController#723954]
Input [13]: [product#724143, plant#724144, supplier#724145, sourceOfSupplyUuid#724146, orderDateTime#724147, deliveryDateTime#724148, availabilityDateTime#724149, supplierGroup#724150, nextAvailabilityDateTime#724151, isFallBackOrderOnly#724153, product#723952, plant#723953, mrpController#723954]

(15) Scan parquet 
Output [5]: [internalUUID#723960, purchasingGroup#723966, purchasingOrganization#723967, supplierSubrange#723975, systemId#723959]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-sourceofsupply-sourceofsupply]
PushedFilters: [IsNotNull(internalUUID)]
ReadSchema: struct<internalUUID:string,purchasingGroup:struct<internalRefUUID:string>,purchasingOrganization:struct<internalRefUUID:string>,supplierSubrange:string>

(16) ColumnarToRow [codegen id : 3]
Input [5]: [internalUUID#723960, purchasingGroup#723966, purchasingOrganization#723967, supplierSubrange#723975, systemId#723959]

(17) Filter [codegen id : 3]
Input [5]: [internalUUID#723960, purchasingGroup#723966, purchasingOrganization#723967, supplierSubrange#723975, systemId#723959]
Condition : isnotnull(internalUUID#723960)

(18) Project [codegen id : 3]
Output [4]: [internalUUID#723960, purchasingGroup#723966.internalRefUUID AS _extract_internalRefUUID#724268, purchasingOrganization#723967.internalRefUUID AS _extract_internalRefUUID#724269, supplierSubrange#723975]
Input [5]: [internalUUID#723960, purchasingGroup#723966, purchasingOrganization#723967, supplierSubrange#723975, systemId#723959]

(19) Exchange
Input [4]: [internalUUID#723960, _extract_internalRefUUID#724268, _extract_internalRefUUID#724269, supplierSubrange#723975]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=286148]

(20) ShuffleQueryStage
Output [4]: [internalUUID#723960, _extract_internalRefUUID#724268, _extract_internalRefUUID#724269, supplierSubrange#723975]
Arguments: 2

(21) Project [codegen id : 5]
Output [4]: [internalUUID#723960 AS sourceOfSupplyUuid#723983, supplierSubrange#723975 AS subrange#723984, _extract_internalRefUUID#724268 AS purchasingGroup#723985, _extract_internalRefUUID#724269 AS purchasingOrg#723986]
Input [4]: [internalUUID#723960, _extract_internalRefUUID#724268, _extract_internalRefUUID#724269, supplierSubrange#723975]

(22) BroadcastExchange
Input [4]: [sourceOfSupplyUuid#723983, subrange#723984, purchasingGroup#723985, purchasingOrg#723986]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=286281]

(23) BroadcastQueryStage
Output [4]: [sourceOfSupplyUuid#723983, subrange#723984, purchasingGroup#723985, purchasingOrg#723986]
Arguments: 4

(24) BroadcastHashJoin [codegen id : 7]
Left keys [1]: [sourceOfSupplyUuid#724146]
Right keys [1]: [sourceOfSupplyUuid#723983]
Join type: LeftOuter
Join condition: None

(25) Project [codegen id : 7]
Output [14]: [sourceOfSupplyUuid#724146, plant#724144, product#724143, supplier#724145, orderDateTime#724147, deliveryDateTime#724148, availabilityDateTime#724149, supplierGroup#724150, nextAvailabilityDateTime#724151, isFallBackOrderOnly#724153, mrpController#723954, subrange#723984, purchasingGroup#723985, purchasingOrg#723986]
Input [15]: [plant#724144, product#724143, supplier#724145, sourceOfSupplyUuid#724146, orderDateTime#724147, deliveryDateTime#724148, availabilityDateTime#724149, supplierGroup#724150, nextAvailabilityDateTime#724151, isFallBackOrderOnly#724153, mrpController#723954, sourceOfSupplyUuid#723983, subrange#723984, purchasingGroup#723985, purchasingOrg#723986]

(26) Scan parquet 
Output [2]: [internalUUID#723991, systemId#723990]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-product-v2]
PushedFilters: [IsNotNull(internalUUID)]
ReadSchema: struct<internalUUID:string>

(27) ColumnarToRow [codegen id : 4]
Input [2]: [internalUUID#723991, systemId#723990]

(28) Filter [codegen id : 4]
Input [2]: [internalUUID#723991, systemId#723990]
Condition : isnotnull(internalUUID#723991)

(29) Project [codegen id : 4]
Output [1]: [internalUUID#723991]
Input [2]: [internalUUID#723991, systemId#723990]

(30) Exchange
Input [1]: [internalUUID#723991]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=286171]

(31) ShuffleQueryStage
Output [1]: [internalUUID#723991]
Arguments: 3

(32) Project [codegen id : 6]
Output [2]: [internalUUID#723991 AS product#724004, null AS productGroup#724005]
Input [1]: [internalUUID#723991]

(33) BroadcastExchange
Input [2]: [product#724004, productGroup#724005]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=286364]

(34) BroadcastQueryStage
Output [2]: [product#724004, productGroup#724005]
Arguments: 5

(35) BroadcastHashJoin [codegen id : 7]
Left keys [1]: [product#724143]
Right keys [1]: [product#724004]
Join type: LeftOuter
Join condition: None

(36) Project [codegen id : 7]
Output [7]: [supplierGroup#724150, supplier#724145, plant#724144, deliveryDateTime#724148, subrange#723984, productGroup#724005, struct(product, product#724143, orderDateTime, orderDateTime#724147, availabilityDateTime, availabilityDateTime#724149, nextAvailabilityDateTime, nextAvailabilityDateTime#724151, mrpController, mrpController#723954, sourceOfSupplyUuid, sourceOfSupplyUuid#724146, purchasingOrg, purchasingOrg#723986, purchasingGroup, purchasingGroup#723985, isFallbackOrderOnly, isFallbackOrderOnly#724153) AS struct#724156]
Input [16]: [sourceOfSupplyUuid#724146, plant#724144, product#724143, supplier#724145, orderDateTime#724147, deliveryDateTime#724148, availabilityDateTime#724149, supplierGroup#724150, nextAvailabilityDateTime#724151, isFallBackOrderOnly#724153, mrpController#723954, subrange#723984, purchasingGroup#723985, purchasingOrg#723986, product#724004, productGroup#724005]

(37) ObjectHashAggregate
Input [7]: [supplierGroup#724150, supplier#724145, plant#724144, deliveryDateTime#724148, subrange#723984, productGroup#724005, struct#724156]
Keys [7]: [supplierGroup#724150, supplier#724145, plant#724144, deliveryDateTime#724148, subrange#723984, null AS null#724582, productGroup#724005]
Functions [1]: [partial_collect_list(struct#724156, 0, 0)]
Aggregate Attributes [1]: [buf#724501]
Results [8]: [supplierGroup#724150, supplier#724145, plant#724144, deliveryDateTime#724148, subrange#723984, null#724582, productGroup#724005, buf#724502]

(38) Exchange
Input [8]: [supplierGroup#724150, supplier#724145, plant#724144, deliveryDateTime#724148, subrange#723984, null#724582, productGroup#724005, buf#724502]
Arguments: hashpartitioning(supplierGroup#724150, supplier#724145, plant#724144, deliveryDateTime#724148, subrange#723984, null#724582, productGroup#724005, 25), ENSURE_REQUIREMENTS, [plan_id=286668]

(39) ShuffleQueryStage
Output [8]: [supplierGroup#724150, supplier#724145, plant#724144, deliveryDateTime#724148, subrange#723984, null#724582, productGroup#724005, buf#724502]
Arguments: 7

(40) AQEShuffleRead
Input [8]: [supplierGroup#724150, supplier#724145, plant#724144, deliveryDateTime#724148, subrange#723984, null#724582, productGroup#724005, buf#724502]
Arguments: coalesced

(41) ObjectHashAggregate
Input [8]: [supplierGroup#724150, supplier#724145, plant#724144, deliveryDateTime#724148, subrange#723984, null#724582, productGroup#724005, buf#724502]
Keys [7]: [supplierGroup#724150, supplier#724145, plant#724144, deliveryDateTime#724148, subrange#723984, null#724582, productGroup#724005]
Functions [1]: [collect_list(struct#724156, 0, 0)]
Aggregate Attributes [1]: [collect_list(struct#724156, 0, 0)#724166]
Results [8]: [supplierGroup#724150, supplier#724145, plant#724144, deliveryDateTime#724148, subrange#723984, null AS supplyingPlant#724155, productGroup#724005, collect_list(struct#724156, 0, 0)#724166 AS productOrderSchedule#724157]

(42) ResultQueryStage
Output [8]: [supplierGroup#724150, supplier#724145, plant#724144, deliveryDateTime#724148, subrange#723984, supplyingPlant#724155, productGroup#724005, productOrderSchedule#724157]
Arguments: 9

(43) Exchange
Input [10]: [product#724143, plant#724144, supplier#724145, sourceOfSupplyUuid#724146, orderDateTime#724147, deliveryDateTime#724148, availabilityDateTime#724149, supplierGroup#724150, nextAvailabilityDateTime#724151, isFallBackOrderOnly#724153]
Arguments: hashpartitioning(product#724143, plant#724144, 25), REPARTITION_BY_NUM, [plan_id=286021]

(44) Filter
Input [4]: [plant#723946, product#723947, productionAspect#723949, systemId#723944]
Condition : (isnotnull(plant#723946.internalRefUUID) AND isnotnull(product#723947.internalRefUUID))

(45) Project
Output [3]: [product#723947.internalRefUUID AS product#723952, plant#723946.internalRefUUID AS plant#723953, productionAspect#723949.productPlanningPlants.mrpController.internalRefUUID AS mrpController#723954]
Input [4]: [plant#723946, product#723947, productionAspect#723949, systemId#723944]

(46) Exchange
Input [3]: [product#723952, plant#723953, mrpController#723954]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=286025]

(47) BroadcastExchange
Input [3]: [product#723952, plant#723953, mrpController#723954]
Arguments: HashedRelationBroadcastMode(List(input[1, string, true], input[0, string, true]),false), [plan_id=286045]

(48) BroadcastHashJoin
Left keys [2]: [plant#724144, product#724143]
Right keys [2]: [plant#723953, product#723952]
Join type: LeftOuter
Join condition: None

(49) Project
Output [11]: [plant#724144, product#724143, supplier#724145, sourceOfSupplyUuid#724146, orderDateTime#724147, deliveryDateTime#724148, availabilityDateTime#724149, supplierGroup#724150, nextAvailabilityDateTime#724151, isFallBackOrderOnly#724153, mrpController#723954]
Input [13]: [product#724143, plant#724144, supplier#724145, sourceOfSupplyUuid#724146, orderDateTime#724147, deliveryDateTime#724148, availabilityDateTime#724149, supplierGroup#724150, nextAvailabilityDateTime#724151, isFallBackOrderOnly#724153, product#723952, plant#723953, mrpController#723954]

(50) Filter
Input [5]: [internalUUID#723960, purchasingGroup#723966, purchasingOrganization#723967, supplierSubrange#723975, systemId#723959]
Condition : isnotnull(internalUUID#723960)

(51) Project
Output [4]: [internalUUID#723960, purchasingGroup#723966.internalRefUUID AS _extract_internalRefUUID#724268, purchasingOrganization#723967.internalRefUUID AS _extract_internalRefUUID#724269, supplierSubrange#723975]
Input [5]: [internalUUID#723960, purchasingGroup#723966, purchasingOrganization#723967, supplierSubrange#723975, systemId#723959]

(52) Exchange
Input [4]: [internalUUID#723960, _extract_internalRefUUID#724268, _extract_internalRefUUID#724269, supplierSubrange#723975]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=286031]

(53) Project
Output [4]: [internalUUID#723960 AS sourceOfSupplyUuid#723983, supplierSubrange#723975 AS subrange#723984, _extract_internalRefUUID#724268 AS purchasingGroup#723985, _extract_internalRefUUID#724269 AS purchasingOrg#723986]
Input [4]: [internalUUID#723960, _extract_internalRefUUID#724268, _extract_internalRefUUID#724269, supplierSubrange#723975]

(54) BroadcastExchange
Input [4]: [sourceOfSupplyUuid#723983, subrange#723984, purchasingGroup#723985, purchasingOrg#723986]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=286049]

(55) BroadcastHashJoin
Left keys [1]: [sourceOfSupplyUuid#724146]
Right keys [1]: [sourceOfSupplyUuid#723983]
Join type: LeftOuter
Join condition: None

(56) Project
Output [14]: [sourceOfSupplyUuid#724146, plant#724144, product#724143, supplier#724145, orderDateTime#724147, deliveryDateTime#724148, availabilityDateTime#724149, supplierGroup#724150, nextAvailabilityDateTime#724151, isFallBackOrderOnly#724153, mrpController#723954, subrange#723984, purchasingGroup#723985, purchasingOrg#723986]
Input [15]: [plant#724144, product#724143, supplier#724145, sourceOfSupplyUuid#724146, orderDateTime#724147, deliveryDateTime#724148, availabilityDateTime#724149, supplierGroup#724150, nextAvailabilityDateTime#724151, isFallBackOrderOnly#724153, mrpController#723954, sourceOfSupplyUuid#723983, subrange#723984, purchasingGroup#723985, purchasingOrg#723986]

(57) Filter
Input [2]: [internalUUID#723991, systemId#723990]
Condition : isnotnull(internalUUID#723991)

(58) Project
Output [1]: [internalUUID#723991]
Input [2]: [internalUUID#723991, systemId#723990]

(59) Exchange
Input [1]: [internalUUID#723991]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=286038]

(60) Project
Output [2]: [internalUUID#723991 AS product#724004, null AS productGroup#724005]
Input [1]: [internalUUID#723991]

(61) BroadcastExchange
Input [2]: [product#724004, productGroup#724005]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=286053]

(62) BroadcastHashJoin
Left keys [1]: [product#724143]
Right keys [1]: [product#724004]
Join type: LeftOuter
Join condition: None

(63) Project
Output [7]: [supplierGroup#724150, supplier#724145, plant#724144, deliveryDateTime#724148, subrange#723984, productGroup#724005, struct(product, product#724143, orderDateTime, orderDateTime#724147, availabilityDateTime, availabilityDateTime#724149, nextAvailabilityDateTime, nextAvailabilityDateTime#724151, mrpController, mrpController#723954, sourceOfSupplyUuid, sourceOfSupplyUuid#724146, purchasingOrg, purchasingOrg#723986, purchasingGroup, purchasingGroup#723985, isFallbackOrderOnly, isFallbackOrderOnly#724153) AS struct#724156]
Input [16]: [sourceOfSupplyUuid#724146, plant#724144, product#724143, supplier#724145, orderDateTime#724147, deliveryDateTime#724148, availabilityDateTime#724149, supplierGroup#724150, nextAvailabilityDateTime#724151, isFallBackOrderOnly#724153, mrpController#723954, subrange#723984, purchasingGroup#723985, purchasingOrg#723986, product#724004, productGroup#724005]

(64) ObjectHashAggregate
Input [7]: [supplierGroup#724150, supplier#724145, plant#724144, deliveryDateTime#724148, subrange#723984, productGroup#724005, struct#724156]
Keys [7]: [supplierGroup#724150, supplier#724145, plant#724144, deliveryDateTime#724148, subrange#723984, null AS null#724500, productGroup#724005]
Functions [1]: [partial_collect_list(struct#724156, 0, 0)]
Aggregate Attributes [1]: [buf#724501]
Results [8]: [supplierGroup#724150, supplier#724145, plant#724144, deliveryDateTime#724148, subrange#723984, null#724500, productGroup#724005, buf#724502]

(65) Exchange
Input [8]: [supplierGroup#724150, supplier#724145, plant#724144, deliveryDateTime#724148, subrange#723984, null#724500, productGroup#724005, buf#724502]
Arguments: hashpartitioning(supplierGroup#724150, supplier#724145, plant#724144, deliveryDateTime#724148, subrange#723984, null#724500, productGroup#724005, 25), ENSURE_REQUIREMENTS, [plan_id=286058]

(66) ObjectHashAggregate
Input [8]: [supplierGroup#724150, supplier#724145, plant#724144, deliveryDateTime#724148, subrange#723984, null#724500, productGroup#724005, buf#724502]
Keys [7]: [supplierGroup#724150, supplier#724145, plant#724144, deliveryDateTime#724148, subrange#723984, null#724500, productGroup#724005]
Functions [1]: [collect_list(struct#724156, 0, 0)]
Aggregate Attributes [1]: [collect_list(struct#724156, 0, 0)#724166]
Results [8]: [supplierGroup#724150, supplier#724145, plant#724144, deliveryDateTime#724148, subrange#723984, null AS supplyingPlant#724155, productGroup#724005, collect_list(struct#724156, 0, 0)#724166 AS productOrderSchedule#724157]

(67) AdaptiveSparkPlan
Output [8]: [supplierGroup#724150, supplier#724145, plant#724144, deliveryDateTime#724148, subrange#723984, supplyingPlant#724155, productGroup#724005, productOrderSchedule#724157]
Arguments: isFinalPlan=false

SQL / DataFrame Properties

Name	Value
spark.activity.hdlfsEnabled	true
spark.activity.isolateSession	true
spark.sql.parquet.fieldId.read.enabled	true
spark.sql.parquet.fieldId.write.enabled	true
temporal.activity.id	813bf27f-8afc-34b1-9af3-26d0f5c0568f
temporal.activity.type	EnhanceWorklistByAttributes