crp-order-qty-opt-service - Details for Query 4243

Submitted Time: 2026/08/10 13:11:08
Duration: 3 s
Succeeded Jobs: 6420 6421 6422 6424 6425 6426 6427 6428 6429

Plan Visualization

Show the Stage ID and Task ID that corresponds to the max metric

digraph G { 0 [id="node0" labelType="html" label=" AdaptiveSparkPlan " tooltip="AdaptiveSparkPlan isFinalPlan=false"]; 1 [id="node1" labelType="html" label="ObjectHashAggregate number of output rows: 40 time in aggregation build: 58 ms spill size: 0.0 B number of sort fallback tasks: 0" tooltip="ObjectHashAggregate(keys=[supplierGroup#992843, supplier#992838, plant#992837, deliveryDateTime#992841, subrange#992677, null#993275, productGroup#992698], functions=[collect_list(struct#992849, 0, 0)])"]; 2 [id="node2" labelType="html" label="AQEShuffleRead number of partitions: 1 partition data size: 60.3 KiB number of coalesced partitions: 1" tooltip="AQEShuffleRead coalesced"]; 3 [id="node3" labelType="html" label="Exchange shuffle records written: 155 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 155 local bytes read: 30.5 KiB merged fetch fallback count: 0 local blocks read: 68 remote merged chunks fetched: 0 remote blocks read: 62 data size total (min, med, max (stageId: taskId)) 69.0 KiB (2.1 KiB, 4.2 KiB, 6.3 KiB (stage 84179.0: task 201620)) local merged bytes read: 0.0 B local merged chunks fetched: 0 shuffle write time total (min, med, max (stageId: taskId)) 14 ms (0 ms, 0 ms, 0 ms (stage 84179.0: task 201623)) remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 fetch wait time: 0 ms remote bytes read: 27.1 KiB number of partitions: 25 remote reqs duration: 3 ms remote bytes read to disk: 0.0 B shuffle bytes written total (min, med, max (stageId: taskId)) 57.6 KiB (1744.0 B, 3.2 KiB, 4.9 KiB (stage 84179.0: task 201620))" tooltip="Exchange hashpartitioning(supplierGroup#992843, supplier#992838, plant#992837, deliveryDateTime#992841, subrange#992677, null#993275, productGroup#992698, 25), ENSURE_REQUIREMENTS, [plan_id=384950]"]; 4 [id="node4" labelType="html" label="ObjectHashAggregate number of output rows: 155 time in aggregation build total (min, med, max (stageId: taskId)) 1.0 s (0 ms, 1 ms, 102 ms (stage 84179.0: task 201618)) spill size total (min, med, max (stageId: taskId)) 0.0 B (0.0 B, 0.0 B, 0.0 B (stage 84179.0: task 201618)) number of sort fallback tasks: 0" tooltip="ObjectHashAggregate(keys=[supplierGroup#992843, supplier#992838, plant#992837, deliveryDateTime#992841, subrange#992677, null AS null#993275, productGroup#992698], functions=[partial_collect_list(struct#992849, 0, 0)])"]; subgraph cluster5 { isCluster="true"; id="cluster5"; label="WholeStageCodegen (7)\n \nduration: total (min, med, max (stageId: taskId))\n1.1 s (0 ms, 1 ms, 115 ms (stage 84179.0: task 201618))"; tooltip="WholeStageCodegen (7)"; 6 [id="node6" labelType="html" label=" Project " tooltip="Project [supplierGroup#992843, supplier#992838, plant#992837, deliveryDateTime#992841, subrange#992677, productGroup#992698, struct(product, product#992836, orderDateTime, orderDateTime#992840, availabilityDateTime, availabilityDateTime#992842, nextAvailabilityDateTime, nextAvailabilityDateTime#992844, mrpController, mrpController#992647, sourceOfSupplyUuid, sourceOfSupplyUuid#992839, purchasingOrg, purchasingOrg#992679, purchasingGroup, purchasingGroup#992678, isFallbackOrderOnly, isFallbackOrderOnly#992846) AS struct#992849]"]; 7 [id="node7" labelType="html" label="BroadcastHashJoin number of output rows: 170" tooltip="BroadcastHashJoin [product#992836], [product#992697], LeftOuter, BuildRight, false"]; 8 [id="node8" labelType="html" label=" Project " tooltip="Project [sourceOfSupplyUuid#992839, plant#992837, product#992836, supplier#992838, orderDateTime#992840, deliveryDateTime#992841, availabilityDateTime#992842, supplierGroup#992843, nextAvailabilityDateTime#992844, isFallBackOrderOnly#992846, mrpController#992647, subrange#992677, purchasingGroup#992678, purchasingOrg#992679]"]; 9 [id="node9" labelType="html" label="BroadcastHashJoin number of output rows: 170" tooltip="BroadcastHashJoin [sourceOfSupplyUuid#992839], [sourceOfSupplyUuid#992676], LeftOuter, BuildRight, false"]; 10 [id="node10" labelType="html" label=" Project " tooltip="Project [plant#992837, product#992836, supplier#992838, sourceOfSupplyUuid#992839, orderDateTime#992840, deliveryDateTime#992841, availabilityDateTime#992842, supplierGroup#992843, nextAvailabilityDateTime#992844, isFallBackOrderOnly#992846, mrpController#992647]"]; 11 [id="node11" labelType="html" label="BroadcastHashJoin number of output rows: 170" tooltip="BroadcastHashJoin [plant#992837, product#992836], [plant#992646, product#992645], LeftOuter, BuildRight, false"]; } 12 [id="node12" labelType="html" label="Exchange shuffle records written: 170 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 170 local bytes read total (min, med, max (stageId: taskId)) 10.2 KiB (391.0 B, 398.0 B, 920.0 B (stage 84179.0: task 201616)) merged fetch fallback count: 0 local blocks read: 25 remote merged chunks fetched: 0 remote blocks read: 6 data size total (min, med, max (stageId: taskId)) 37.2 KiB (2.2 KiB, 2.2 KiB, 4.4 KiB (stage 84168.0: task 201473)) local merged bytes read: 0.0 B local merged chunks fetched: 0 shuffle write time total (min, med, max (stageId: taskId)) 7 ms (0 ms, 0 ms, 0 ms (stage 84168.0: task 201480)) remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 fetch wait time total (min, med, max (stageId: taskId)) 0 ms (0 ms, 0 ms, 0 ms (stage 84179.0: task 201618)) remote bytes read total (min, med, max (stageId: taskId)) 2.3 KiB (390.0 B, 398.0 B, 796.0 B (stage 84179.0: task 201621)) number of partitions: 25 remote reqs duration total (min, med, max (stageId: taskId)) 0 ms (0 ms, 0 ms, 0 ms (stage 84179.0: task 201624)) remote bytes read to disk: 0.0 B shuffle bytes written total (min, med, max (stageId: taskId)) 12.5 KiB (529.0 B, 782.0 B, 1269.0 B (stage 84168.0: task 201473))" tooltip="Exchange hashpartitioning(product#992836, plant#992837, 25), REPARTITION_BY_NUM, [plan_id=384382]"]; subgraph cluster13 { isCluster="true"; id="cluster13"; label="WholeStageCodegen (1)\n \nduration: total (min, med, max (stageId: taskId))\n2.9 s (77 ms, 185 ms, 325 ms (stage 84168.0: task 201474))"; tooltip="WholeStageCodegen (1)"; 14 [id="node14" labelType="html" label="ColumnarToRow number of output rows: 170 number of input batches: 31" tooltip="ColumnarToRow"]; } 15 [id="node15" labelType="html" label="Scan parquet number of files read: 31 scan time total (min, med, max (stageId: taskId)) 2.6 s (76 ms, 151 ms, 290 ms (stage 84168.0: task 201474)) metadata time: 0 ms size of files read: 129.3 KiB number of output rows: 170" tooltip="FileScan parquet [product#992836,plant#992837,supplier#992838,sourceOfSupplyUuid#992839,orderDateTime#992840,deliveryDateTime#992841,availabilityDateTime#992842,supplierGroup#992843,nextAvailabilityDateTime#992844,isFallBackOrderOnly#992846] Batched: true, DataFilters: [], Format: Parquet, Location: InMemoryFileIndex(1 paths)[hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanac..., PartitionFilters: [], PushedFilters: [], ReadSchema: struct<product:string,plant:string,supplier:string,sourceOfSupplyUuid:string,orderDateTime:timest..."]; 16 [id="node16" labelType="html" label="BroadcastExchange time to broadcast: 2 ms time to build: 12 ms time to collect: 34 ms number of output rows: 2,747 data size: 2.1 MiB" tooltip="BroadcastExchange HashedRelationBroadcastMode(List(input[1, string, true], input[0, string, true]),false), [plan_id=384852]"]; 17 [id="node17" labelType="html" label="Exchange shuffle records written: 2,747 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 2,747 local bytes read total (min, med, max (stageId: taskId)) 134.8 KiB (5.2 KiB, 5.4 KiB, 5.8 KiB (stage 84177.0: task 201599)) merged fetch fallback count: 0 local blocks read: 25 remote merged chunks fetched: 0 remote blocks read: 0 data size: 318.9 KiB local merged bytes read: 0.0 B local merged chunks fetched: 0 shuffle write time: 1 ms remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 fetch wait time total (min, med, max (stageId: taskId)) 0 ms (0 ms, 0 ms, 0 ms (stage 84177.0: task 201591)) remote bytes read: 0.0 B number of partitions: 25 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written: 134.8 KiB" tooltip="Exchange RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=384398]"]; subgraph cluster18 { isCluster="true"; id="cluster18"; label="WholeStageCodegen (2)\n \nduration: 103 ms"; tooltip="WholeStageCodegen (2)"; 19 [id="node19" labelType="html" label=" Project " tooltip="Project [product#992640.internalRefUUID AS product#992645, plant#992639.internalRefUUID AS plant#992646, productionAspect#992642.productPlanningPlants.mrpController.internalRefUUID AS mrpController#992647]"]; 20 [id="node20" labelType="html" label="Filter number of output rows: 2,747" tooltip="Filter (isnotnull(plant#992639.internalRefUUID) AND isnotnull(product#992640.internalRefUUID))"]; 21 [id="node21" labelType="html" label="ColumnarToRow number of output rows: 2,747 number of input batches: 1" tooltip="ColumnarToRow"]; } 22 [id="node22" labelType="html" label="Scan parquet number of files read: 1 scan time: 100 ms dynamic partition pruning time: 0 ms metadata time: 538 ms size of files read: 179.9 KiB number of output rows: 2,747 number of partitions read: 1" tooltip="FileScan parquet [plant#992639,product#992640,productionAspect#992642,systemId#992637] Batched: true, DataFilters: [isnotnull(plant#992639.internalRefUUID), isnotnull(product#992640.internalRefUUID)], Format: Parquet, Location: PreparedDeltaFileIndex(1 paths)[hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20...., PartitionFilters: [], PushedFilters: [IsNotNull(plant.internalRefUUID), IsNotNull(product.internalRefUUID)], ReadSchema: struct<plant:struct<internalRefUUID:string>,product:struct<internalRefUUID:string>,productionAspe..."]; 23 [id="node23" labelType="html" label="BroadcastExchange time to broadcast: 4 ms time to build: 27 ms time to collect: 129 ms number of output rows: 4,343 data size: 2.3 MiB" tooltip="BroadcastExchange HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=384553]"]; subgraph cluster24 { isCluster="true"; id="cluster24"; label="WholeStageCodegen (5)\n \nduration: total (min, med, max (stageId: taskId))\n1 ms (0 ms, 0 ms, 1 ms (stage 84172.0: task 201488))"; tooltip="WholeStageCodegen (5)"; 25 [id="node25" labelType="html" label=" Project " tooltip="Project [internalUUID#992653 AS sourceOfSupplyUuid#992676, supplierSubrange#992668 AS subrange#992677, _extract_internalRefUUID#992961 AS purchasingGroup#992678, _extract_internalRefUUID#992962 AS purchasingOrg#992679]"]; } 26 [id="node26" labelType="html" label="Exchange shuffle records written: 4,343 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 4,343 local bytes read total (min, med, max (stageId: taskId)) 177.8 KiB (7.0 KiB, 7.1 KiB, 7.2 KiB (stage 84172.0: task 201503)) merged fetch fallback count: 0 local blocks read: 25 remote merged chunks fetched: 0 remote blocks read: 0 data size: 522.9 KiB local merged bytes read: 0.0 B local merged chunks fetched: 0 shuffle write time: 1 ms remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 fetch wait time total (min, med, max (stageId: taskId)) 0 ms (0 ms, 0 ms, 0 ms (stage 84172.0: task 201488)) remote bytes read: 0.0 B number of partitions: 25 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written: 177.8 KiB" tooltip="Exchange RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=384420]"]; subgraph cluster27 { isCluster="true"; id="cluster27"; label="WholeStageCodegen (3)\n \nduration: 140 ms"; tooltip="WholeStageCodegen (3)"; 28 [id="node28" labelType="html" label=" Project " tooltip="Project [internalUUID#992653, purchasingGroup#992659.internalRefUUID AS _extract_internalRefUUID#992961, purchasingOrganization#992660.internalRefUUID AS _extract_internalRefUUID#992962, supplierSubrange#992668]"]; 29 [id="node29" labelType="html" label="Filter number of output rows: 4,343" tooltip="Filter isnotnull(internalUUID#992653)"]; 30 [id="node30" labelType="html" label="ColumnarToRow number of output rows: 4,343 number of input batches: 2" tooltip="ColumnarToRow"]; } 31 [id="node31" labelType="html" label="Scan parquet number of files read: 1 scan time: 136 ms dynamic partition pruning time: 0 ms metadata time: 0 ms size of files read: 315.0 KiB number of output rows: 4,343 number of partitions read: 1" tooltip="FileScan parquet [internalUUID#992653,purchasingGroup#992659,purchasingOrganization#992660,supplierSubrange#992668,systemId#992652] Batched: true, DataFilters: [isnotnull(internalUUID#992653)], Format: Parquet, Location: PreparedDeltaFileIndex(1 paths)[hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20...., PartitionFilters: [], PushedFilters: [IsNotNull(internalUUID)], ReadSchema: struct<internalUUID:string,purchasingGroup:struct<internalRefUUID:string>,purchasingOrganization:..."]; 32 [id="node32" labelType="html" label="BroadcastExchange time to broadcast: 1 ms time to build: 11 ms time to collect: 115 ms number of output rows: 159 data size: 2.0 MiB" tooltip="BroadcastExchange HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=384636]"]; subgraph cluster33 { isCluster="true"; id="cluster33"; label="WholeStageCodegen (6)\n \nduration: total (min, med, max (stageId: taskId))\n2 ms (0 ms, 0 ms, 1 ms (stage 84174.0: task 201494))"; tooltip="WholeStageCodegen (6)"; 34 [id="node34" labelType="html" label=" Project " tooltip="Project [internalUUID#992684 AS product#992697, null AS productGroup#992698]"]; } 35 [id="node35" labelType="html" label="Exchange shuffle records written: 159 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 159 local bytes read total (min, med, max (stageId: taskId)) 7.5 KiB (289.0 B, 299.0 B, 336.0 B (stage 84174.0: task 201520)) merged fetch fallback count: 0 local blocks read: 25 remote merged chunks fetched: 0 remote blocks read: 0 data size: 7.5 KiB local merged bytes read: 0.0 B local merged chunks fetched: 0 shuffle write time: 1 ms remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 fetch wait time total (min, med, max (stageId: taskId)) 0 ms (0 ms, 0 ms, 0 ms (stage 84174.0: task 201494)) remote bytes read: 0.0 B number of partitions: 25 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written: 7.5 KiB" tooltip="Exchange RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=384443]"]; subgraph cluster36 { isCluster="true"; id="cluster36"; label="WholeStageCodegen (4)\n \nduration: 155 ms"; tooltip="WholeStageCodegen (4)"; 37 [id="node37" labelType="html" label=" Project " tooltip="Project [internalUUID#992684]"]; 38 [id="node38" labelType="html" label="Filter number of output rows: 159" tooltip="Filter isnotnull(internalUUID#992684)"]; 39 [id="node39" labelType="html" label="ColumnarToRow number of output rows: 159 number of input batches: 1" tooltip="ColumnarToRow"]; } 40 [id="node40" labelType="html" label="Scan parquet number of files read: 1 scan time: 153 ms dynamic partition pruning time: 0 ms metadata time: 0 ms size of files read: 36.0 KiB number of output rows: 159 number of partitions read: 1" tooltip="FileScan parquet [internalUUID#992684,systemId#992683] Batched: true, DataFilters: [isnotnull(internalUUID#992684)], Format: Parquet, Location: PreparedDeltaFileIndex(1 paths)[hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20...., PartitionFilters: [], PushedFilters: [IsNotNull(internalUUID)], ReadSchema: struct<internalUUID:string>"]; 1->0; 2->1; 3->2; 4->3; 6->4; 7->6; 8->7; 9->8; 10->9; 11->10; 12->11; 14->12; 15->14; 16->11; 17->16; 19->17; 20->19; 21->20; 22->21; 23->9; 25->23; 26->25; 28->26; 29->28; 30->29; 31->30; 32->7; 34->32; 35->34; 37->35; 38->37; 39->38; 40->39; }

Details

== Physical Plan ==
AdaptiveSparkPlan (67)
+- == Current Plan ==
   ResultQueryStage (42)
   +- ObjectHashAggregate (41)
      +- AQEShuffleRead (40)
         +- ShuffleQueryStage (39), Statistics(sizeInBytes=69.0 KiB, rowCount=155)
            +- Exchange (38)
               +- ObjectHashAggregate (37)
                  +- * Project (36)
                     +- * BroadcastHashJoin LeftOuter BuildRight (35)
                        :- * Project (25)
                        :  +- * BroadcastHashJoin LeftOuter BuildRight (24)
                        :     :- * Project (14)
                        :     :  +- * BroadcastHashJoin LeftOuter BuildRight (13)
                        :     :     :- ShuffleQueryStage (4), Statistics(sizeInBytes=37.2 KiB, rowCount=170)
                        :     :     :  +- Exchange (3)
                        :     :     :     +- * ColumnarToRow (2)
                        :     :     :        +- Scan parquet  (1)
                        :     :     +- BroadcastQueryStage (12), Statistics(sizeInBytes=2.1 MiB, rowCount=2.75E+3)
                        :     :        +- BroadcastExchange (11)
                        :     :           +- ShuffleQueryStage (10), Statistics(sizeInBytes=318.9 KiB, rowCount=2.75E+3)
                        :     :              +- Exchange (9)
                        :     :                 +- * Project (8)
                        :     :                    +- * Filter (7)
                        :     :                       +- * ColumnarToRow (6)
                        :     :                          +- Scan parquet  (5)
                        :     +- BroadcastQueryStage (23), Statistics(sizeInBytes=2.3 MiB, rowCount=4.34E+3)
                        :        +- BroadcastExchange (22)
                        :           +- * Project (21)
                        :              +- ShuffleQueryStage (20), Statistics(sizeInBytes=522.9 KiB, rowCount=4.34E+3)
                        :                 +- Exchange (19)
                        :                    +- * Project (18)
                        :                       +- * Filter (17)
                        :                          +- * ColumnarToRow (16)
                        :                             +- Scan parquet  (15)
                        +- BroadcastQueryStage (34), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
                           +- BroadcastExchange (33)
                              +- * Project (32)
                                 +- ShuffleQueryStage (31), Statistics(sizeInBytes=7.5 KiB, rowCount=159)
                                    +- Exchange (30)
                                       +- * Project (29)
                                          +- * Filter (28)
                                             +- * ColumnarToRow (27)
                                                +- Scan parquet  (26)
+- == Initial Plan ==
   ObjectHashAggregate (66)
   +- Exchange (65)
      +- ObjectHashAggregate (64)
         +- Project (63)
            +- BroadcastHashJoin LeftOuter BuildRight (62)
               :- Project (56)
               :  +- BroadcastHashJoin LeftOuter BuildRight (55)
               :     :- Project (49)
               :     :  +- BroadcastHashJoin LeftOuter BuildRight (48)
               :     :     :- Exchange (43)
               :     :     :  +- Scan parquet  (1)
               :     :     +- BroadcastExchange (47)
               :     :        +- Exchange (46)
               :     :           +- Project (45)
               :     :              +- Filter (44)
               :     :                 +- Scan parquet  (5)
               :     +- BroadcastExchange (54)
               :        +- Project (53)
               :           +- Exchange (52)
               :              +- Project (51)
               :                 +- Filter (50)
               :                    +- Scan parquet  (15)
               +- BroadcastExchange (61)
                  +- Project (60)
                     +- Exchange (59)
                        +- Project (58)
                           +- Filter (57)
                              +- Scan parquet  (26)


(1) Scan parquet 
Output [10]: [product#992836, plant#992837, supplier#992838, sourceOfSupplyUuid#992839, orderDateTime#992840, deliveryDateTime#992841, availabilityDateTime#992842, supplierGroup#992843, nextAvailabilityDateTime#992844, isFallBackOrderOnly#992846]
Batched: true
Location: InMemoryFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-workload-determination-service/out/workload-determination-result/10000002111/results]
ReadSchema: struct<product:string,plant:string,supplier:string,sourceOfSupplyUuid:string,orderDateTime:timestamp,deliveryDateTime:timestamp,availabilityDateTime:timestamp,supplierGroup:string,nextAvailabilityDateTime:timestamp,isFallBackOrderOnly:boolean>

(2) ColumnarToRow [codegen id : 1]
Input [10]: [product#992836, plant#992837, supplier#992838, sourceOfSupplyUuid#992839, orderDateTime#992840, deliveryDateTime#992841, availabilityDateTime#992842, supplierGroup#992843, nextAvailabilityDateTime#992844, isFallBackOrderOnly#992846]

(3) Exchange
Input [10]: [product#992836, plant#992837, supplier#992838, sourceOfSupplyUuid#992839, orderDateTime#992840, deliveryDateTime#992841, availabilityDateTime#992842, supplierGroup#992843, nextAvailabilityDateTime#992844, isFallBackOrderOnly#992846]
Arguments: hashpartitioning(product#992836, plant#992837, 25), REPARTITION_BY_NUM, [plan_id=384382]

(4) ShuffleQueryStage
Output [10]: [product#992836, plant#992837, supplier#992838, sourceOfSupplyUuid#992839, orderDateTime#992840, deliveryDateTime#992841, availabilityDateTime#992842, supplierGroup#992843, nextAvailabilityDateTime#992844, isFallBackOrderOnly#992846]
Arguments: 0

(5) Scan parquet 
Output [4]: [plant#992639, product#992640, productionAspect#992642, systemId#992637]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-productplant]
PushedFilters: [IsNotNull(plant.internalRefUUID), IsNotNull(product.internalRefUUID)]
ReadSchema: struct<plant:struct<internalRefUUID:string>,product:struct<internalRefUUID:string>,productionAspect:struct<productPlanningPlants:struct<mrpController:struct<internalRefUUID:string>>>>

(6) ColumnarToRow [codegen id : 2]
Input [4]: [plant#992639, product#992640, productionAspect#992642, systemId#992637]

(7) Filter [codegen id : 2]
Input [4]: [plant#992639, product#992640, productionAspect#992642, systemId#992637]
Condition : (isnotnull(plant#992639.internalRefUUID) AND isnotnull(product#992640.internalRefUUID))

(8) Project [codegen id : 2]
Output [3]: [product#992640.internalRefUUID AS product#992645, plant#992639.internalRefUUID AS plant#992646, productionAspect#992642.productPlanningPlants.mrpController.internalRefUUID AS mrpController#992647]
Input [4]: [plant#992639, product#992640, productionAspect#992642, systemId#992637]

(9) Exchange
Input [3]: [product#992645, plant#992646, mrpController#992647]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=384398]

(10) ShuffleQueryStage
Output [3]: [product#992645, plant#992646, mrpController#992647]
Arguments: 1

(11) BroadcastExchange
Input [3]: [product#992645, plant#992646, mrpController#992647]
Arguments: HashedRelationBroadcastMode(List(input[1, string, true], input[0, string, true]),false), [plan_id=384852]

(12) BroadcastQueryStage
Output [3]: [product#992645, plant#992646, mrpController#992647]
Arguments: 6

(13) BroadcastHashJoin [codegen id : 7]
Left keys [2]: [plant#992837, product#992836]
Right keys [2]: [plant#992646, product#992645]
Join type: LeftOuter
Join condition: None

(14) Project [codegen id : 7]
Output [11]: [plant#992837, product#992836, supplier#992838, sourceOfSupplyUuid#992839, orderDateTime#992840, deliveryDateTime#992841, availabilityDateTime#992842, supplierGroup#992843, nextAvailabilityDateTime#992844, isFallBackOrderOnly#992846, mrpController#992647]
Input [13]: [product#992836, plant#992837, supplier#992838, sourceOfSupplyUuid#992839, orderDateTime#992840, deliveryDateTime#992841, availabilityDateTime#992842, supplierGroup#992843, nextAvailabilityDateTime#992844, isFallBackOrderOnly#992846, product#992645, plant#992646, mrpController#992647]

(15) Scan parquet 
Output [5]: [internalUUID#992653, purchasingGroup#992659, purchasingOrganization#992660, supplierSubrange#992668, systemId#992652]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-sourceofsupply-sourceofsupply]
PushedFilters: [IsNotNull(internalUUID)]
ReadSchema: struct<internalUUID:string,purchasingGroup:struct<internalRefUUID:string>,purchasingOrganization:struct<internalRefUUID:string>,supplierSubrange:string>

(16) ColumnarToRow [codegen id : 3]
Input [5]: [internalUUID#992653, purchasingGroup#992659, purchasingOrganization#992660, supplierSubrange#992668, systemId#992652]

(17) Filter [codegen id : 3]
Input [5]: [internalUUID#992653, purchasingGroup#992659, purchasingOrganization#992660, supplierSubrange#992668, systemId#992652]
Condition : isnotnull(internalUUID#992653)

(18) Project [codegen id : 3]
Output [4]: [internalUUID#992653, purchasingGroup#992659.internalRefUUID AS _extract_internalRefUUID#992961, purchasingOrganization#992660.internalRefUUID AS _extract_internalRefUUID#992962, supplierSubrange#992668]
Input [5]: [internalUUID#992653, purchasingGroup#992659, purchasingOrganization#992660, supplierSubrange#992668, systemId#992652]

(19) Exchange
Input [4]: [internalUUID#992653, _extract_internalRefUUID#992961, _extract_internalRefUUID#992962, supplierSubrange#992668]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=384420]

(20) ShuffleQueryStage
Output [4]: [internalUUID#992653, _extract_internalRefUUID#992961, _extract_internalRefUUID#992962, supplierSubrange#992668]
Arguments: 2

(21) Project [codegen id : 5]
Output [4]: [internalUUID#992653 AS sourceOfSupplyUuid#992676, supplierSubrange#992668 AS subrange#992677, _extract_internalRefUUID#992961 AS purchasingGroup#992678, _extract_internalRefUUID#992962 AS purchasingOrg#992679]
Input [4]: [internalUUID#992653, _extract_internalRefUUID#992961, _extract_internalRefUUID#992962, supplierSubrange#992668]

(22) BroadcastExchange
Input [4]: [sourceOfSupplyUuid#992676, subrange#992677, purchasingGroup#992678, purchasingOrg#992679]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=384553]

(23) BroadcastQueryStage
Output [4]: [sourceOfSupplyUuid#992676, subrange#992677, purchasingGroup#992678, purchasingOrg#992679]
Arguments: 4

(24) BroadcastHashJoin [codegen id : 7]
Left keys [1]: [sourceOfSupplyUuid#992839]
Right keys [1]: [sourceOfSupplyUuid#992676]
Join type: LeftOuter
Join condition: None

(25) Project [codegen id : 7]
Output [14]: [sourceOfSupplyUuid#992839, plant#992837, product#992836, supplier#992838, orderDateTime#992840, deliveryDateTime#992841, availabilityDateTime#992842, supplierGroup#992843, nextAvailabilityDateTime#992844, isFallBackOrderOnly#992846, mrpController#992647, subrange#992677, purchasingGroup#992678, purchasingOrg#992679]
Input [15]: [plant#992837, product#992836, supplier#992838, sourceOfSupplyUuid#992839, orderDateTime#992840, deliveryDateTime#992841, availabilityDateTime#992842, supplierGroup#992843, nextAvailabilityDateTime#992844, isFallBackOrderOnly#992846, mrpController#992647, sourceOfSupplyUuid#992676, subrange#992677, purchasingGroup#992678, purchasingOrg#992679]

(26) Scan parquet 
Output [2]: [internalUUID#992684, systemId#992683]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-product-v2]
PushedFilters: [IsNotNull(internalUUID)]
ReadSchema: struct<internalUUID:string>

(27) ColumnarToRow [codegen id : 4]
Input [2]: [internalUUID#992684, systemId#992683]

(28) Filter [codegen id : 4]
Input [2]: [internalUUID#992684, systemId#992683]
Condition : isnotnull(internalUUID#992684)

(29) Project [codegen id : 4]
Output [1]: [internalUUID#992684]
Input [2]: [internalUUID#992684, systemId#992683]

(30) Exchange
Input [1]: [internalUUID#992684]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=384443]

(31) ShuffleQueryStage
Output [1]: [internalUUID#992684]
Arguments: 3

(32) Project [codegen id : 6]
Output [2]: [internalUUID#992684 AS product#992697, null AS productGroup#992698]
Input [1]: [internalUUID#992684]

(33) BroadcastExchange
Input [2]: [product#992697, productGroup#992698]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=384636]

(34) BroadcastQueryStage
Output [2]: [product#992697, productGroup#992698]
Arguments: 5

(35) BroadcastHashJoin [codegen id : 7]
Left keys [1]: [product#992836]
Right keys [1]: [product#992697]
Join type: LeftOuter
Join condition: None

(36) Project [codegen id : 7]
Output [7]: [supplierGroup#992843, supplier#992838, plant#992837, deliveryDateTime#992841, subrange#992677, productGroup#992698, struct(product, product#992836, orderDateTime, orderDateTime#992840, availabilityDateTime, availabilityDateTime#992842, nextAvailabilityDateTime, nextAvailabilityDateTime#992844, mrpController, mrpController#992647, sourceOfSupplyUuid, sourceOfSupplyUuid#992839, purchasingOrg, purchasingOrg#992679, purchasingGroup, purchasingGroup#992678, isFallbackOrderOnly, isFallbackOrderOnly#992846) AS struct#992849]
Input [16]: [sourceOfSupplyUuid#992839, plant#992837, product#992836, supplier#992838, orderDateTime#992840, deliveryDateTime#992841, availabilityDateTime#992842, supplierGroup#992843, nextAvailabilityDateTime#992844, isFallBackOrderOnly#992846, mrpController#992647, subrange#992677, purchasingGroup#992678, purchasingOrg#992679, product#992697, productGroup#992698]

(37) ObjectHashAggregate
Input [7]: [supplierGroup#992843, supplier#992838, plant#992837, deliveryDateTime#992841, subrange#992677, productGroup#992698, struct#992849]
Keys [7]: [supplierGroup#992843, supplier#992838, plant#992837, deliveryDateTime#992841, subrange#992677, null AS null#993275, productGroup#992698]
Functions [1]: [partial_collect_list(struct#992849, 0, 0)]
Aggregate Attributes [1]: [buf#993194]
Results [8]: [supplierGroup#992843, supplier#992838, plant#992837, deliveryDateTime#992841, subrange#992677, null#993275, productGroup#992698, buf#993195]

(38) Exchange
Input [8]: [supplierGroup#992843, supplier#992838, plant#992837, deliveryDateTime#992841, subrange#992677, null#993275, productGroup#992698, buf#993195]
Arguments: hashpartitioning(supplierGroup#992843, supplier#992838, plant#992837, deliveryDateTime#992841, subrange#992677, null#993275, productGroup#992698, 25), ENSURE_REQUIREMENTS, [plan_id=384950]

(39) ShuffleQueryStage
Output [8]: [supplierGroup#992843, supplier#992838, plant#992837, deliveryDateTime#992841, subrange#992677, null#993275, productGroup#992698, buf#993195]
Arguments: 7

(40) AQEShuffleRead
Input [8]: [supplierGroup#992843, supplier#992838, plant#992837, deliveryDateTime#992841, subrange#992677, null#993275, productGroup#992698, buf#993195]
Arguments: coalesced

(41) ObjectHashAggregate
Input [8]: [supplierGroup#992843, supplier#992838, plant#992837, deliveryDateTime#992841, subrange#992677, null#993275, productGroup#992698, buf#993195]
Keys [7]: [supplierGroup#992843, supplier#992838, plant#992837, deliveryDateTime#992841, subrange#992677, null#993275, productGroup#992698]
Functions [1]: [collect_list(struct#992849, 0, 0)]
Aggregate Attributes [1]: [collect_list(struct#992849, 0, 0)#992859]
Results [8]: [supplierGroup#992843, supplier#992838, plant#992837, deliveryDateTime#992841, subrange#992677, null AS supplyingPlant#992848, productGroup#992698, collect_list(struct#992849, 0, 0)#992859 AS productOrderSchedule#992850]

(42) ResultQueryStage
Output [8]: [supplierGroup#992843, supplier#992838, plant#992837, deliveryDateTime#992841, subrange#992677, supplyingPlant#992848, productGroup#992698, productOrderSchedule#992850]
Arguments: 9

(43) Exchange
Input [10]: [product#992836, plant#992837, supplier#992838, sourceOfSupplyUuid#992839, orderDateTime#992840, deliveryDateTime#992841, availabilityDateTime#992842, supplierGroup#992843, nextAvailabilityDateTime#992844, isFallBackOrderOnly#992846]
Arguments: hashpartitioning(product#992836, plant#992837, 25), REPARTITION_BY_NUM, [plan_id=384293]

(44) Filter
Input [4]: [plant#992639, product#992640, productionAspect#992642, systemId#992637]
Condition : (isnotnull(plant#992639.internalRefUUID) AND isnotnull(product#992640.internalRefUUID))

(45) Project
Output [3]: [product#992640.internalRefUUID AS product#992645, plant#992639.internalRefUUID AS plant#992646, productionAspect#992642.productPlanningPlants.mrpController.internalRefUUID AS mrpController#992647]
Input [4]: [plant#992639, product#992640, productionAspect#992642, systemId#992637]

(46) Exchange
Input [3]: [product#992645, plant#992646, mrpController#992647]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=384297]

(47) BroadcastExchange
Input [3]: [product#992645, plant#992646, mrpController#992647]
Arguments: HashedRelationBroadcastMode(List(input[1, string, true], input[0, string, true]),false), [plan_id=384317]

(48) BroadcastHashJoin
Left keys [2]: [plant#992837, product#992836]
Right keys [2]: [plant#992646, product#992645]
Join type: LeftOuter
Join condition: None

(49) Project
Output [11]: [plant#992837, product#992836, supplier#992838, sourceOfSupplyUuid#992839, orderDateTime#992840, deliveryDateTime#992841, availabilityDateTime#992842, supplierGroup#992843, nextAvailabilityDateTime#992844, isFallBackOrderOnly#992846, mrpController#992647]
Input [13]: [product#992836, plant#992837, supplier#992838, sourceOfSupplyUuid#992839, orderDateTime#992840, deliveryDateTime#992841, availabilityDateTime#992842, supplierGroup#992843, nextAvailabilityDateTime#992844, isFallBackOrderOnly#992846, product#992645, plant#992646, mrpController#992647]

(50) Filter
Input [5]: [internalUUID#992653, purchasingGroup#992659, purchasingOrganization#992660, supplierSubrange#992668, systemId#992652]
Condition : isnotnull(internalUUID#992653)

(51) Project
Output [4]: [internalUUID#992653, purchasingGroup#992659.internalRefUUID AS _extract_internalRefUUID#992961, purchasingOrganization#992660.internalRefUUID AS _extract_internalRefUUID#992962, supplierSubrange#992668]
Input [5]: [internalUUID#992653, purchasingGroup#992659, purchasingOrganization#992660, supplierSubrange#992668, systemId#992652]

(52) Exchange
Input [4]: [internalUUID#992653, _extract_internalRefUUID#992961, _extract_internalRefUUID#992962, supplierSubrange#992668]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=384303]

(53) Project
Output [4]: [internalUUID#992653 AS sourceOfSupplyUuid#992676, supplierSubrange#992668 AS subrange#992677, _extract_internalRefUUID#992961 AS purchasingGroup#992678, _extract_internalRefUUID#992962 AS purchasingOrg#992679]
Input [4]: [internalUUID#992653, _extract_internalRefUUID#992961, _extract_internalRefUUID#992962, supplierSubrange#992668]

(54) BroadcastExchange
Input [4]: [sourceOfSupplyUuid#992676, subrange#992677, purchasingGroup#992678, purchasingOrg#992679]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=384321]

(55) BroadcastHashJoin
Left keys [1]: [sourceOfSupplyUuid#992839]
Right keys [1]: [sourceOfSupplyUuid#992676]
Join type: LeftOuter
Join condition: None

(56) Project
Output [14]: [sourceOfSupplyUuid#992839, plant#992837, product#992836, supplier#992838, orderDateTime#992840, deliveryDateTime#992841, availabilityDateTime#992842, supplierGroup#992843, nextAvailabilityDateTime#992844, isFallBackOrderOnly#992846, mrpController#992647, subrange#992677, purchasingGroup#992678, purchasingOrg#992679]
Input [15]: [plant#992837, product#992836, supplier#992838, sourceOfSupplyUuid#992839, orderDateTime#992840, deliveryDateTime#992841, availabilityDateTime#992842, supplierGroup#992843, nextAvailabilityDateTime#992844, isFallBackOrderOnly#992846, mrpController#992647, sourceOfSupplyUuid#992676, subrange#992677, purchasingGroup#992678, purchasingOrg#992679]

(57) Filter
Input [2]: [internalUUID#992684, systemId#992683]
Condition : isnotnull(internalUUID#992684)

(58) Project
Output [1]: [internalUUID#992684]
Input [2]: [internalUUID#992684, systemId#992683]

(59) Exchange
Input [1]: [internalUUID#992684]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=384310]

(60) Project
Output [2]: [internalUUID#992684 AS product#992697, null AS productGroup#992698]
Input [1]: [internalUUID#992684]

(61) BroadcastExchange
Input [2]: [product#992697, productGroup#992698]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=384325]

(62) BroadcastHashJoin
Left keys [1]: [product#992836]
Right keys [1]: [product#992697]
Join type: LeftOuter
Join condition: None

(63) Project
Output [7]: [supplierGroup#992843, supplier#992838, plant#992837, deliveryDateTime#992841, subrange#992677, productGroup#992698, struct(product, product#992836, orderDateTime, orderDateTime#992840, availabilityDateTime, availabilityDateTime#992842, nextAvailabilityDateTime, nextAvailabilityDateTime#992844, mrpController, mrpController#992647, sourceOfSupplyUuid, sourceOfSupplyUuid#992839, purchasingOrg, purchasingOrg#992679, purchasingGroup, purchasingGroup#992678, isFallbackOrderOnly, isFallbackOrderOnly#992846) AS struct#992849]
Input [16]: [sourceOfSupplyUuid#992839, plant#992837, product#992836, supplier#992838, orderDateTime#992840, deliveryDateTime#992841, availabilityDateTime#992842, supplierGroup#992843, nextAvailabilityDateTime#992844, isFallBackOrderOnly#992846, mrpController#992647, subrange#992677, purchasingGroup#992678, purchasingOrg#992679, product#992697, productGroup#992698]

(64) ObjectHashAggregate
Input [7]: [supplierGroup#992843, supplier#992838, plant#992837, deliveryDateTime#992841, subrange#992677, productGroup#992698, struct#992849]
Keys [7]: [supplierGroup#992843, supplier#992838, plant#992837, deliveryDateTime#992841, subrange#992677, null AS null#993193, productGroup#992698]
Functions [1]: [partial_collect_list(struct#992849, 0, 0)]
Aggregate Attributes [1]: [buf#993194]
Results [8]: [supplierGroup#992843, supplier#992838, plant#992837, deliveryDateTime#992841, subrange#992677, null#993193, productGroup#992698, buf#993195]

(65) Exchange
Input [8]: [supplierGroup#992843, supplier#992838, plant#992837, deliveryDateTime#992841, subrange#992677, null#993193, productGroup#992698, buf#993195]
Arguments: hashpartitioning(supplierGroup#992843, supplier#992838, plant#992837, deliveryDateTime#992841, subrange#992677, null#993193, productGroup#992698, 25), ENSURE_REQUIREMENTS, [plan_id=384330]

(66) ObjectHashAggregate
Input [8]: [supplierGroup#992843, supplier#992838, plant#992837, deliveryDateTime#992841, subrange#992677, null#993193, productGroup#992698, buf#993195]
Keys [7]: [supplierGroup#992843, supplier#992838, plant#992837, deliveryDateTime#992841, subrange#992677, null#993193, productGroup#992698]
Functions [1]: [collect_list(struct#992849, 0, 0)]
Aggregate Attributes [1]: [collect_list(struct#992849, 0, 0)#992859]
Results [8]: [supplierGroup#992843, supplier#992838, plant#992837, deliveryDateTime#992841, subrange#992677, null AS supplyingPlant#992848, productGroup#992698, collect_list(struct#992849, 0, 0)#992859 AS productOrderSchedule#992850]

(67) AdaptiveSparkPlan
Output [8]: [supplierGroup#992843, supplier#992838, plant#992837, deliveryDateTime#992841, subrange#992677, supplyingPlant#992848, productGroup#992698, productOrderSchedule#992850]
Arguments: isFinalPlan=false

SQL / DataFrame Properties

Name	Value
spark.activity.hdlfsEnabled	true
spark.activity.isolateSession	true
spark.sql.parquet.fieldId.read.enabled	true
spark.sql.parquet.fieldId.write.enabled	true
temporal.activity.id	1a43b787-92b1-3a6e-ab08-25a6e19b9a16
temporal.activity.type	EnhanceWorklistByAttributes