crp-order-qty-opt-service - Details for Query 84

Submitted Time: 2026/02/06 10:33:26
Duration: 2 s
Succeeded Jobs: 175 177 178 179 180 181 182

Show the Stage ID and Task ID that corresponds to the max metric

digraph G { 0 [labelType="html" label=" AdaptiveSparkPlan "]; 1 [labelType="html" label="ObjectHashAggregate number of output rows: 179 time in aggregation build total (min, med, max (stageId: taskId)) 114 ms (18 ms, 19 ms, 20 ms (stage 335.0: task 2547)) spill size total (min, med, max (stageId: taskId)) 0.0 B (0.0 B, 0.0 B, 0.0 B (stage 335.0: task 2548)) number of sort fallback tasks: 0"]; 2 [labelType="html" label="AQEShuffleRead number of partitions: 6 partition data size total (min, med, max (stageId: taskId)) 346.9 KiB (51.3 KiB, 57.9 KiB, 65.6 KiB (driver)) number of coalesced partitions: 6"]; 3 [labelType="html" label="Exchange shuffle records written: 1,032 local merged chunks fetched: 0 shuffle write time total (min, med, max (stageId: taskId)) 45 ms (1 ms, 1 ms, 2 ms (stage 332.0: task 2543)) remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 1,032 local bytes read total (min, med, max (stageId: taskId)) 163.3 KiB (20.2 KiB, 30.0 KiB, 34.4 KiB (stage 335.0: task 2550)) fetch wait time total (min, med, max (stageId: taskId)) 0 ms (0 ms, 0 ms, 0 ms (stage 335.0: task 2548)) remote bytes read total (min, med, max (stageId: taskId)) 165.4 KiB (17.9 KiB, 31.6 KiB, 38.0 KiB (stage 335.0: task 2547)) merged fetch fallback count: 0 local blocks read: 257 remote merged chunks fetched: 0 remote blocks read: 249 data size total (min, med, max (stageId: taskId)) 445.4 KiB (13.9 KiB, 17.6 KiB, 21.8 KiB (stage 332.0: task 2520)) local merged bytes read: 0.0 B number of partitions: 25 remote reqs duration total (min, med, max (stageId: taskId)) 28 ms (4 ms, 5 ms, 5 ms (stage 335.0: task 2550)) remote bytes read to disk: 0.0 B shuffle bytes written total (min, med, max (stageId: taskId)) 328.7 KiB (10.9 KiB, 12.9 KiB, 16.0 KiB (stage 332.0: task 2527))"]; 4 [labelType="html" label="ObjectHashAggregate number of output rows: 1,032 time in aggregation build total (min, med, max (stageId: taskId)) 123 ms (1 ms, 2 ms, 11 ms (stage 332.0: task 2524)) spill size total (min, med, max (stageId: taskId)) 0.0 B (0.0 B, 0.0 B, 0.0 B (stage 332.0: task 2525)) number of sort fallback tasks: 0"]; subgraph cluster5 { isCluster="true"; label="WholeStageCodegen (6)\n \nduration: total (min, med, max (stageId: taskId))\n292 ms (1 ms, 2 ms, 35 ms (stage 332.0: task 2528))"; 6 [labelType="html" label=" Project "]; 7 [labelType="html" label="BroadcastHashJoin number of output rows: 1,196"]; 8 [labelType="html" label=" Project "]; 9 [labelType="html" label="BroadcastHashJoin number of output rows: 1,196"]; 10 [labelType="html" label=" Project "]; 11 [labelType="html" label="Filter number of output rows: 1,196"]; } 12 [labelType="html" label="Window spill size total (min, med, max (stageId: taskId)) 0.0 B (0.0 B, 0.0 B, 0.0 B (stage 332.0: task 2525))"]; 13 [labelType="html" label="WindowGroupLimit number of output rows: 1,196"]; subgraph cluster14 { isCluster="true"; label="WholeStageCodegen (5)\n \nduration: total (min, med, max (stageId: taskId))\n705 ms (6 ms, 8 ms, 69 ms (stage 332.0: task 2524))"; 15 [labelType="html" label="Sort sort time total (min, med, max (stageId: taskId)) 0 ms (0 ms, 0 ms, 0 ms (stage 332.0: task 2525)) peak memory total (min, med, max (stageId: taskId)) 51.6 MiB (2.1 MiB, 2.1 MiB, 2.1 MiB (stage 332.0: task 2525)) spill size total (min, med, max (stageId: taskId)) 0.0 B (0.0 B, 0.0 B, 0.0 B (stage 332.0: task 2525))"]; } 16 [labelType="html" label="Exchange shuffle records written: 1,196 local merged chunks fetched: 0 shuffle write time total (min, med, max (stageId: taskId)) 21 ms (0 ms, 0 ms, 1 ms (stage 322.0: task 2396)) remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 1,196 local bytes read total (min, med, max (stageId: taskId)) 87.5 KiB (1647.0 B, 3.5 KiB, 6.2 KiB (stage 332.0: task 2527)) fetch wait time total (min, med, max (stageId: taskId)) 20 ms (0 ms, 1 ms, 2 ms (stage 332.0: task 2534)) remote bytes read total (min, med, max (stageId: taskId)) 79.1 KiB (717.0 B, 3.4 KiB, 5.1 KiB (stage 332.0: task 2532)) merged fetch fallback count: 0 local blocks read: 89 remote merged chunks fetched: 0 remote blocks read: 83 data size total (min, med, max (stageId: taskId)) 252.3 KiB (3.2 KiB, 10.3 KiB, 20.7 KiB (stage 322.0: task 2393)) local merged bytes read: 0.0 B number of partitions: 25 remote reqs duration total (min, med, max (stageId: taskId)) 83 ms (3 ms, 3 ms, 5 ms (stage 332.0: task 2523)) remote bytes read to disk: 0.0 B shuffle bytes written total (min, med, max (stageId: taskId)) 166.5 KiB (2.6 KiB, 6.7 KiB, 11.9 KiB (stage 322.0: task 2393))"]; subgraph cluster17 { isCluster="true"; label="WholeStageCodegen (1)\n \nduration: total (min, med, max (stageId: taskId))\n16.2 s (554 ms, 636 ms, 795 ms (stage 322.0: task 2397))"; 18 [labelType="html" label="ColumnarToRow number of output rows: 1,196 number of input batches: 200"]; } 19 [labelType="html" label="Scan parquet number of files read: 200 scan time total (min, med, max (stageId: taskId)) 15.9 s (546 ms, 628 ms, 784 ms (stage 322.0: task 2397)) metadata time: 0 ms size of files read: 757.7 KiB number of output rows: 1,196"]; 20 [labelType="html" label="BroadcastExchange time to broadcast: 1 ms time to build: 2 ms time to collect: 72 ms number of output rows: 1,867 data size: 2.1 MiB"]; 21 [labelType="html" label="Exchange shuffle records written: 1,867 local merged chunks fetched: 0 shuffle write time: 1 ms remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 1,867 local bytes read total (min, med, max (stageId: taskId)) 93.4 KiB (3.5 KiB, 3.7 KiB, 4.0 KiB (stage 328.0: task 2473)) fetch wait time total (min, med, max (stageId: taskId)) 0 ms (0 ms, 0 ms, 0 ms (stage 328.0: task 2470)) remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 25 remote merged chunks fetched: 0 remote blocks read: 0 data size: 221.3 KiB local merged bytes read: 0.0 B number of partitions: 25 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written: 93.4 KiB"]; subgraph cluster22 { isCluster="true"; label="WholeStageCodegen (2)\n \nduration: 119 ms"; 23 [labelType="html" label=" Project "]; 24 [labelType="html" label="Filter number of output rows: 1,867"]; 25 [labelType="html" label="ColumnarToRow number of output rows: 1,867 number of input batches: 1"]; } 26 [labelType="html" label="Scan parquet number of files read: 1 scan time: 116 ms dynamic partition pruning time: 0 ms metadata time: 808 ms size of files read: 108.6 KiB number of output rows: 1,867 number of partitions read: 1"]; 27 [labelType="html" label="BroadcastExchange time to broadcast: 3 ms time to build: 7 ms time to collect: 101 ms number of output rows: 3,798 data size: 2.1 MiB"]; subgraph cluster28 { isCluster="true"; label="WholeStageCodegen (4)\n \nduration: total (min, med, max (stageId: taskId))\n0 ms (0 ms, 0 ms, 0 ms (stage 330.0: task 2493))"; 29 [labelType="html" label=" Project "]; } 30 [labelType="html" label="Exchange shuffle records written: 3,798 local merged chunks fetched: 0 shuffle write time: 1 ms remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 3,798 local bytes read total (min, med, max (stageId: taskId)) 154.2 KiB (6.0 KiB, 6.2 KiB, 6.3 KiB (stage 330.0: task 2516)) fetch wait time total (min, med, max (stageId: taskId)) 0 ms (0 ms, 0 ms, 0 ms (stage 330.0: task 2493)) remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 25 remote merged chunks fetched: 0 remote blocks read: 0 data size: 455.7 KiB local merged bytes read: 0.0 B number of partitions: 25 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written: 154.2 KiB"]; subgraph cluster31 { isCluster="true"; label="WholeStageCodegen (3)\n \nduration: 132 ms"; 32 [labelType="html" label=" Project "]; 33 [labelType="html" label="Filter number of output rows: 3,798"]; 34 [labelType="html" label="ColumnarToRow number of output rows: 3,798 number of input batches: 1"]; } 35 [labelType="html" label="Scan parquet number of files read: 1 scan time: 128 ms dynamic partition pruning time: 0 ms metadata time: 0 ms size of files read: 263.0 KiB number of output rows: 3,798 number of partitions read: 1"]; 1->0; 2->1; 3->2; 4->3; 6->4; 7->6; 8->7; 9->8; 10->9; 11->10; 12->11; 13->12; 15->13; 16->15; 18->16; 19->18; 20->9; 21->20; 23->21; 24->23; 25->24; 26->25; 27->7; 29->27; 30->29; 32->30; 33->32; 34->33; 35->34; }

AdaptiveSparkPlan isFinalPlan=true

ObjectHashAggregate(keys=[supplierGroup#29630, supplier#29625, plant#29624, deliveryDateTime#29628, subrange#29775], functions=[collect_list(struct#29829, 0, 0)])

AQEShuffleRead coalesced

Exchange hashpartitioning(supplierGroup#29630, supplier#29625, plant#29624, deliveryDateTime#29628, subrange#29775, 25), ENSURE_REQUIREMENTS, [plan_id=9930]

ObjectHashAggregate(keys=[supplierGroup#29630, supplier#29625, plant#29624, deliveryDateTime#29628, subrange#29775], functions=[partial_collect_list(struct#29829, 0, 0)])

Project [supplierGroup#29630, supplier#29625, plant#29624, deliveryDateTime#29628, subrange#29775, struct(product, product#29623, orderDateTime, orderDateTime#29627, availabilityDateTime, availabilityDateTime#29629, nextAvailabilityDateTime, nextAvailabilityDateTime#29631, mrpController, mrpController#29615, sourceOfSupplyUuid, sourceOfSupplyUuid#29626, purchasingOrg, purchasingOrg#29777, purchasingGroup, purchasingGroup#29776) AS struct#29829]

BroadcastHashJoin [sourceOfSupplyUuid#29626], [sourceOfSupplyUuid#29774], LeftOuter, BuildRight, false

Project [plant#29624, product#29623, supplier#29625, sourceOfSupplyUuid#29626, orderDateTime#29627, deliveryDateTime#29628, availabilityDateTime#29629, supplierGroup#29630, nextAvailabilityDateTime#29631, mrpController#29615]

BroadcastHashJoin [plant#29624, product#29623], [plant#29614, product#29613], LeftOuter, BuildRight, false

Project [product#29623, plant#29624, supplier#29625, sourceOfSupplyUuid#29626, orderDateTime#29627, deliveryDateTime#29628, availabilityDateTime#29629, supplierGroup#29630, nextAvailabilityDateTime#29631]

Filter (rank#29786 = 1)

WholeStageCodegen (6)

Window [row_number() windowspecdefinition(product#29623, plant#29624, availabilityDateTime#29629 ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) AS rank#29786], [product#29623, plant#29624], [availabilityDateTime#29629 ASC NULLS FIRST]

WindowGroupLimit [product#29623, plant#29624], [availabilityDateTime#29629 ASC NULLS FIRST], row_number(), 1, Final

Sort [product#29623 ASC NULLS FIRST, plant#29624 ASC NULLS FIRST, availabilityDateTime#29629 ASC NULLS FIRST], false, 0

WholeStageCodegen (5)

Exchange hashpartitioning(product#29623, plant#29624, 25), REPARTITION_BY_NUM, [plan_id=9456]

ColumnarToRow

WholeStageCodegen (1)

FileScan parquet [product#29623,plant#29624,supplier#29625,sourceOfSupplyUuid#29626,orderDateTime#29627,deliveryDateTime#29628,availabilityDateTime#29629,supplierGroup#29630,nextAvailabilityDateTime#29631] Batched: true, DataFilters: [], Format: Parquet, Location: InMemoryFileIndex(1 paths)[hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanac..., PartitionFilters: [], PushedFilters: [], ReadSchema: struct<product:string,plant:string,supplier:string,sourceOfSupplyUuid:string,orderDateTime:timest...

BroadcastExchange HashedRelationBroadcastMode(List(input[1, string, true], input[0, string, true]),false), [plan_id=9487]

Exchange RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=9477]

Project [product#29598.internalRefUUID AS product#29613, plant#29597.internalRefUUID AS plant#29614, productionAspect#29600.productPlanningPlants.mrpController.internalRefUUID AS mrpController#29615]

Filter (isnotnull(plant#29597.internalRefUUID) AND isnotnull(product#29598.internalRefUUID))

ColumnarToRow

WholeStageCodegen (2)

FileScan parquet [plant#29597,product#29598,productionAspect#29600,systemId#29595] Batched: true, DataFilters: [isnotnull(plant#29597.internalRefUUID), isnotnull(product#29598.internalRefUUID)], Format: Parquet, Location: PreparedDeltaFileIndex(1 paths)[hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20...., PartitionFilters: [], PushedFilters: [IsNotNull(plant.internalRefUUID), IsNotNull(product.internalRefUUID)], ReadSchema: struct<plant:struct<internalRefUUID:string>,product:struct<internalRefUUID:string>,productionAspe...

BroadcastExchange HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=9711]

Project [internalUUID#29727 AS sourceOfSupplyUuid#29774, supplierSubrange#29742 AS subrange#29775, _extract_internalRefUUID#29948 AS purchasingGroup#29776, _extract_internalRefUUID#29949 AS purchasingOrg#29777]

WholeStageCodegen (4)

Exchange RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=9499]

Project [internalUUID#29727, purchasingGroup#29733.internalRefUUID AS _extract_internalRefUUID#29948, purchasingOrganization#29734.internalRefUUID AS _extract_internalRefUUID#29949, supplierSubrange#29742]

Filter isnotnull(internalUUID#29727)

ColumnarToRow

WholeStageCodegen (3)

FileScan parquet [internalUUID#29727,purchasingGroup#29733,purchasingOrganization#29734,supplierSubrange#29742,systemId#29726] Batched: true, DataFilters: [isnotnull(internalUUID#29727)], Format: Parquet, Location: PreparedDeltaFileIndex(1 paths)[hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20...., PartitionFilters: [], PushedFilters: [IsNotNull(internalUUID)], ReadSchema: struct<internalUUID:string,purchasingGroup:struct<internalRefUUID:string>,purchasingOrganization:...

Details

== Physical Plan ==
AdaptiveSparkPlan (58)
+- == Final Plan ==
   ObjectHashAggregate (35)
   +- AQEShuffleRead (34)
      +- ShuffleQueryStage (33), Statistics(sizeInBytes=445.4 KiB, rowCount=1.03E+3)
         +- Exchange (32)
            +- ObjectHashAggregate (31)
               +- * Project (30)
                  +- * BroadcastHashJoin LeftOuter BuildRight (29)
                     :- * Project (19)
                     :  +- * BroadcastHashJoin LeftOuter BuildRight (18)
                     :     :- * Project (9)
                     :     :  +- * Filter (8)
                     :     :     +- Window (7)
                     :     :        +- WindowGroupLimit (6)
                     :     :           +- * Sort (5)
                     :     :              +- ShuffleQueryStage (4), Statistics(sizeInBytes=252.3 KiB, rowCount=1.20E+3)
                     :     :                 +- Exchange (3)
                     :     :                    +- * ColumnarToRow (2)
                     :     :                       +- Scan parquet  (1)
                     :     +- BroadcastQueryStage (17), Statistics(sizeInBytes=2.1 MiB, rowCount=1.87E+3)
                     :        +- BroadcastExchange (16)
                     :           +- ShuffleQueryStage (15), Statistics(sizeInBytes=221.3 KiB, rowCount=1.87E+3)
                     :              +- Exchange (14)
                     :                 +- * Project (13)
                     :                    +- * Filter (12)
                     :                       +- * ColumnarToRow (11)
                     :                          +- Scan parquet  (10)
                     +- BroadcastQueryStage (28), Statistics(sizeInBytes=2.1 MiB, rowCount=3.80E+3)
                        +- BroadcastExchange (27)
                           +- * Project (26)
                              +- ShuffleQueryStage (25), Statistics(sizeInBytes=455.7 KiB, rowCount=3.80E+3)
                                 +- Exchange (24)
                                    +- * Project (23)
                                       +- * Filter (22)
                                          +- * ColumnarToRow (21)
                                             +- Scan parquet  (20)
+- == Initial Plan ==
   ObjectHashAggregate (57)
   +- Exchange (56)
      +- ObjectHashAggregate (55)
         +- Project (54)
            +- BroadcastHashJoin LeftOuter BuildRight (53)
               :- Project (47)
               :  +- BroadcastHashJoin LeftOuter BuildRight (46)
               :     :- Project (41)
               :     :  +- Filter (40)
               :     :     +- Window (39)
               :     :        +- WindowGroupLimit (38)
               :     :           +- Sort (37)
               :     :              +- Exchange (36)
               :     :                 +- Scan parquet  (1)
               :     +- BroadcastExchange (45)
               :        +- Exchange (44)
               :           +- Project (43)
               :              +- Filter (42)
               :                 +- Scan parquet  (10)
               +- BroadcastExchange (52)
                  +- Project (51)
                     +- Exchange (50)
                        +- Project (49)
                           +- Filter (48)
                              +- Scan parquet  (20)


(1) Scan parquet 
Output [9]: [product#29623, plant#29624, supplier#29625, sourceOfSupplyUuid#29626, orderDateTime#29627, deliveryDateTime#29628, availabilityDateTime#29629, supplierGroup#29630, nextAvailabilityDateTime#29631]
Batched: true
Location: InMemoryFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-workload-determination-service/out/workload-determination-result/10000000096/results]
ReadSchema: struct<product:string,plant:string,supplier:string,sourceOfSupplyUuid:string,orderDateTime:timestamp,deliveryDateTime:timestamp,availabilityDateTime:timestamp,supplierGroup:string,nextAvailabilityDateTime:timestamp>

(2) ColumnarToRow [codegen id : 1]
Input [9]: [product#29623, plant#29624, supplier#29625, sourceOfSupplyUuid#29626, orderDateTime#29627, deliveryDateTime#29628, availabilityDateTime#29629, supplierGroup#29630, nextAvailabilityDateTime#29631]

(3) Exchange
Input [9]: [product#29623, plant#29624, supplier#29625, sourceOfSupplyUuid#29626, orderDateTime#29627, deliveryDateTime#29628, availabilityDateTime#29629, supplierGroup#29630, nextAvailabilityDateTime#29631]
Arguments: hashpartitioning(product#29623, plant#29624, 25), REPARTITION_BY_NUM, [plan_id=9456]

(4) ShuffleQueryStage
Output [9]: [product#29623, plant#29624, supplier#29625, sourceOfSupplyUuid#29626, orderDateTime#29627, deliveryDateTime#29628, availabilityDateTime#29629, supplierGroup#29630, nextAvailabilityDateTime#29631]
Arguments: 0

(5) Sort [codegen id : 5]
Input [9]: [product#29623, plant#29624, supplier#29625, sourceOfSupplyUuid#29626, orderDateTime#29627, deliveryDateTime#29628, availabilityDateTime#29629, supplierGroup#29630, nextAvailabilityDateTime#29631]
Arguments: [product#29623 ASC NULLS FIRST, plant#29624 ASC NULLS FIRST, availabilityDateTime#29629 ASC NULLS FIRST], false, 0

(6) WindowGroupLimit
Input [9]: [product#29623, plant#29624, supplier#29625, sourceOfSupplyUuid#29626, orderDateTime#29627, deliveryDateTime#29628, availabilityDateTime#29629, supplierGroup#29630, nextAvailabilityDateTime#29631]
Arguments: [product#29623, plant#29624], [availabilityDateTime#29629 ASC NULLS FIRST], row_number(), 1, Final

(7) Window
Input [9]: [product#29623, plant#29624, supplier#29625, sourceOfSupplyUuid#29626, orderDateTime#29627, deliveryDateTime#29628, availabilityDateTime#29629, supplierGroup#29630, nextAvailabilityDateTime#29631]
Arguments: [row_number() windowspecdefinition(product#29623, plant#29624, availabilityDateTime#29629 ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) AS rank#29786], [product#29623, plant#29624], [availabilityDateTime#29629 ASC NULLS FIRST]

(8) Filter [codegen id : 6]
Input [10]: [product#29623, plant#29624, supplier#29625, sourceOfSupplyUuid#29626, orderDateTime#29627, deliveryDateTime#29628, availabilityDateTime#29629, supplierGroup#29630, nextAvailabilityDateTime#29631, rank#29786]
Condition : (rank#29786 = 1)

(9) Project [codegen id : 6]
Output [9]: [product#29623, plant#29624, supplier#29625, sourceOfSupplyUuid#29626, orderDateTime#29627, deliveryDateTime#29628, availabilityDateTime#29629, supplierGroup#29630, nextAvailabilityDateTime#29631]
Input [10]: [product#29623, plant#29624, supplier#29625, sourceOfSupplyUuid#29626, orderDateTime#29627, deliveryDateTime#29628, availabilityDateTime#29629, supplierGroup#29630, nextAvailabilityDateTime#29631, rank#29786]

(10) Scan parquet 
Output [4]: [plant#29597, product#29598, productionAspect#29600, systemId#29595]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-productplant]
PushedFilters: [IsNotNull(plant.internalRefUUID), IsNotNull(product.internalRefUUID)]
ReadSchema: struct<plant:struct<internalRefUUID:string>,product:struct<internalRefUUID:string>,productionAspect:struct<productPlanningPlants:struct<mrpController:struct<internalRefUUID:string>>>>

(11) ColumnarToRow [codegen id : 2]
Input [4]: [plant#29597, product#29598, productionAspect#29600, systemId#29595]

(12) Filter [codegen id : 2]
Input [4]: [plant#29597, product#29598, productionAspect#29600, systemId#29595]
Condition : (isnotnull(plant#29597.internalRefUUID) AND isnotnull(product#29598.internalRefUUID))

(13) Project [codegen id : 2]
Output [3]: [product#29598.internalRefUUID AS product#29613, plant#29597.internalRefUUID AS plant#29614, productionAspect#29600.productPlanningPlants.mrpController.internalRefUUID AS mrpController#29615]
Input [4]: [plant#29597, product#29598, productionAspect#29600, systemId#29595]

(14) Exchange
Input [3]: [product#29613, plant#29614, mrpController#29615]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=9477]

(15) ShuffleQueryStage
Output [3]: [product#29613, plant#29614, mrpController#29615]
Arguments: 1

(16) BroadcastExchange
Input [3]: [product#29613, plant#29614, mrpController#29615]
Arguments: HashedRelationBroadcastMode(List(input[1, string, true], input[0, string, true]),false), [plan_id=9487]

(17) BroadcastQueryStage
Output [3]: [product#29613, plant#29614, mrpController#29615]
Arguments: 3

(18) BroadcastHashJoin [codegen id : 6]
Left keys [2]: [plant#29624, product#29623]
Right keys [2]: [plant#29614, product#29613]
Join type: LeftOuter
Join condition: None

(19) Project [codegen id : 6]
Output [10]: [plant#29624, product#29623, supplier#29625, sourceOfSupplyUuid#29626, orderDateTime#29627, deliveryDateTime#29628, availabilityDateTime#29629, supplierGroup#29630, nextAvailabilityDateTime#29631, mrpController#29615]
Input [12]: [product#29623, plant#29624, supplier#29625, sourceOfSupplyUuid#29626, orderDateTime#29627, deliveryDateTime#29628, availabilityDateTime#29629, supplierGroup#29630, nextAvailabilityDateTime#29631, product#29613, plant#29614, mrpController#29615]

(20) Scan parquet 
Output [5]: [internalUUID#29727, purchasingGroup#29733, purchasingOrganization#29734, supplierSubrange#29742, systemId#29726]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-sourceofsupply-sourceofsupply]
PushedFilters: [IsNotNull(internalUUID)]
ReadSchema: struct<internalUUID:string,purchasingGroup:struct<internalRefUUID:string>,purchasingOrganization:struct<internalRefUUID:string>,supplierSubrange:string>

(21) ColumnarToRow [codegen id : 3]
Input [5]: [internalUUID#29727, purchasingGroup#29733, purchasingOrganization#29734, supplierSubrange#29742, systemId#29726]

(22) Filter [codegen id : 3]
Input [5]: [internalUUID#29727, purchasingGroup#29733, purchasingOrganization#29734, supplierSubrange#29742, systemId#29726]
Condition : isnotnull(internalUUID#29727)

(23) Project [codegen id : 3]
Output [4]: [internalUUID#29727, purchasingGroup#29733.internalRefUUID AS _extract_internalRefUUID#29948, purchasingOrganization#29734.internalRefUUID AS _extract_internalRefUUID#29949, supplierSubrange#29742]
Input [5]: [internalUUID#29727, purchasingGroup#29733, purchasingOrganization#29734, supplierSubrange#29742, systemId#29726]

(24) Exchange
Input [4]: [internalUUID#29727, _extract_internalRefUUID#29948, _extract_internalRefUUID#29949, supplierSubrange#29742]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=9499]

(25) ShuffleQueryStage
Output [4]: [internalUUID#29727, _extract_internalRefUUID#29948, _extract_internalRefUUID#29949, supplierSubrange#29742]
Arguments: 2

(26) Project [codegen id : 4]
Output [4]: [internalUUID#29727 AS sourceOfSupplyUuid#29774, supplierSubrange#29742 AS subrange#29775, _extract_internalRefUUID#29948 AS purchasingGroup#29776, _extract_internalRefUUID#29949 AS purchasingOrg#29777]
Input [4]: [internalUUID#29727, _extract_internalRefUUID#29948, _extract_internalRefUUID#29949, supplierSubrange#29742]

(27) BroadcastExchange
Input [4]: [sourceOfSupplyUuid#29774, subrange#29775, purchasingGroup#29776, purchasingOrg#29777]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=9711]

(28) BroadcastQueryStage
Output [4]: [sourceOfSupplyUuid#29774, subrange#29775, purchasingGroup#29776, purchasingOrg#29777]
Arguments: 4

(29) BroadcastHashJoin [codegen id : 6]
Left keys [1]: [sourceOfSupplyUuid#29626]
Right keys [1]: [sourceOfSupplyUuid#29774]
Join type: LeftOuter
Join condition: None

(30) Project [codegen id : 6]
Output [6]: [supplierGroup#29630, supplier#29625, plant#29624, deliveryDateTime#29628, subrange#29775, struct(product, product#29623, orderDateTime, orderDateTime#29627, availabilityDateTime, availabilityDateTime#29629, nextAvailabilityDateTime, nextAvailabilityDateTime#29631, mrpController, mrpController#29615, sourceOfSupplyUuid, sourceOfSupplyUuid#29626, purchasingOrg, purchasingOrg#29777, purchasingGroup, purchasingGroup#29776) AS struct#29829]
Input [14]: [plant#29624, product#29623, supplier#29625, sourceOfSupplyUuid#29626, orderDateTime#29627, deliveryDateTime#29628, availabilityDateTime#29629, supplierGroup#29630, nextAvailabilityDateTime#29631, mrpController#29615, sourceOfSupplyUuid#29774, subrange#29775, purchasingGroup#29776, purchasingOrg#29777]

(31) ObjectHashAggregate
Input [6]: [supplierGroup#29630, supplier#29625, plant#29624, deliveryDateTime#29628, subrange#29775, struct#29829]
Keys [5]: [supplierGroup#29630, supplier#29625, plant#29624, deliveryDateTime#29628, subrange#29775]
Functions [1]: [partial_collect_list(struct#29829, 0, 0)]
Aggregate Attributes [1]: [buf#30799]
Results [6]: [supplierGroup#29630, supplier#29625, plant#29624, deliveryDateTime#29628, subrange#29775, buf#30800]

(32) Exchange
Input [6]: [supplierGroup#29630, supplier#29625, plant#29624, deliveryDateTime#29628, subrange#29775, buf#30800]
Arguments: hashpartitioning(supplierGroup#29630, supplier#29625, plant#29624, deliveryDateTime#29628, subrange#29775, 25), ENSURE_REQUIREMENTS, [plan_id=9930]

(33) ShuffleQueryStage
Output [6]: [supplierGroup#29630, supplier#29625, plant#29624, deliveryDateTime#29628, subrange#29775, buf#30800]
Arguments: 5

(34) AQEShuffleRead
Input [6]: [supplierGroup#29630, supplier#29625, plant#29624, deliveryDateTime#29628, subrange#29775, buf#30800]
Arguments: coalesced

(35) ObjectHashAggregate
Input [6]: [supplierGroup#29630, supplier#29625, plant#29624, deliveryDateTime#29628, subrange#29775, buf#30800]
Keys [5]: [supplierGroup#29630, supplier#29625, plant#29624, deliveryDateTime#29628, subrange#29775]
Functions [1]: [collect_list(struct#29829, 0, 0)]
Aggregate Attributes [1]: [collect_list(struct#29829, 0, 0)#29842]
Results [6]: [supplierGroup#29630, supplier#29625, plant#29624, deliveryDateTime#29628, subrange#29775, collect_list(struct#29829, 0, 0)#29842 AS productOrderSchedule#29843]

(36) Exchange
Input [9]: [product#29623, plant#29624, supplier#29625, sourceOfSupplyUuid#29626, orderDateTime#29627, deliveryDateTime#29628, availabilityDateTime#29629, supplierGroup#29630, nextAvailabilityDateTime#29631]
Arguments: hashpartitioning(product#29623, plant#29624, 25), REPARTITION_BY_NUM, [plan_id=9358]

(37) Sort
Input [9]: [product#29623, plant#29624, supplier#29625, sourceOfSupplyUuid#29626, orderDateTime#29627, deliveryDateTime#29628, availabilityDateTime#29629, supplierGroup#29630, nextAvailabilityDateTime#29631]
Arguments: [product#29623 ASC NULLS FIRST, plant#29624 ASC NULLS FIRST, availabilityDateTime#29629 ASC NULLS FIRST], false, 0

(38) WindowGroupLimit
Input [9]: [product#29623, plant#29624, supplier#29625, sourceOfSupplyUuid#29626, orderDateTime#29627, deliveryDateTime#29628, availabilityDateTime#29629, supplierGroup#29630, nextAvailabilityDateTime#29631]
Arguments: [product#29623, plant#29624], [availabilityDateTime#29629 ASC NULLS FIRST], row_number(), 1, Final

(39) Window
Input [9]: [product#29623, plant#29624, supplier#29625, sourceOfSupplyUuid#29626, orderDateTime#29627, deliveryDateTime#29628, availabilityDateTime#29629, supplierGroup#29630, nextAvailabilityDateTime#29631]
Arguments: [row_number() windowspecdefinition(product#29623, plant#29624, availabilityDateTime#29629 ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) AS rank#29786], [product#29623, plant#29624], [availabilityDateTime#29629 ASC NULLS FIRST]

(40) Filter
Input [10]: [product#29623, plant#29624, supplier#29625, sourceOfSupplyUuid#29626, orderDateTime#29627, deliveryDateTime#29628, availabilityDateTime#29629, supplierGroup#29630, nextAvailabilityDateTime#29631, rank#29786]
Condition : (rank#29786 = 1)

(41) Project
Output [9]: [product#29623, plant#29624, supplier#29625, sourceOfSupplyUuid#29626, orderDateTime#29627, deliveryDateTime#29628, availabilityDateTime#29629, supplierGroup#29630, nextAvailabilityDateTime#29631]
Input [10]: [product#29623, plant#29624, supplier#29625, sourceOfSupplyUuid#29626, orderDateTime#29627, deliveryDateTime#29628, availabilityDateTime#29629, supplierGroup#29630, nextAvailabilityDateTime#29631, rank#29786]

(42) Filter
Input [4]: [plant#29597, product#29598, productionAspect#29600, systemId#29595]
Condition : (isnotnull(plant#29597.internalRefUUID) AND isnotnull(product#29598.internalRefUUID))

(43) Project
Output [3]: [product#29598.internalRefUUID AS product#29613, plant#29597.internalRefUUID AS plant#29614, productionAspect#29600.productPlanningPlants.mrpController.internalRefUUID AS mrpController#29615]
Input [4]: [plant#29597, product#29598, productionAspect#29600, systemId#29595]

(44) Exchange
Input [3]: [product#29613, plant#29614, mrpController#29615]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=9367]

(45) BroadcastExchange
Input [3]: [product#29613, plant#29614, mrpController#29615]
Arguments: HashedRelationBroadcastMode(List(input[1, string, true], input[0, string, true]),false), [plan_id=9387]

(46) BroadcastHashJoin
Left keys [2]: [plant#29624, product#29623]
Right keys [2]: [plant#29614, product#29613]
Join type: LeftOuter
Join condition: None

(47) Project
Output [10]: [plant#29624, product#29623, supplier#29625, sourceOfSupplyUuid#29626, orderDateTime#29627, deliveryDateTime#29628, availabilityDateTime#29629, supplierGroup#29630, nextAvailabilityDateTime#29631, mrpController#29615]
Input [12]: [product#29623, plant#29624, supplier#29625, sourceOfSupplyUuid#29626, orderDateTime#29627, deliveryDateTime#29628, availabilityDateTime#29629, supplierGroup#29630, nextAvailabilityDateTime#29631, product#29613, plant#29614, mrpController#29615]

(48) Filter
Input [5]: [internalUUID#29727, purchasingGroup#29733, purchasingOrganization#29734, supplierSubrange#29742, systemId#29726]
Condition : isnotnull(internalUUID#29727)

(49) Project
Output [4]: [internalUUID#29727, purchasingGroup#29733.internalRefUUID AS _extract_internalRefUUID#29948, purchasingOrganization#29734.internalRefUUID AS _extract_internalRefUUID#29949, supplierSubrange#29742]
Input [5]: [internalUUID#29727, purchasingGroup#29733, purchasingOrganization#29734, supplierSubrange#29742, systemId#29726]

(50) Exchange
Input [4]: [internalUUID#29727, _extract_internalRefUUID#29948, _extract_internalRefUUID#29949, supplierSubrange#29742]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=9373]

(51) Project
Output [4]: [internalUUID#29727 AS sourceOfSupplyUuid#29774, supplierSubrange#29742 AS subrange#29775, _extract_internalRefUUID#29948 AS purchasingGroup#29776, _extract_internalRefUUID#29949 AS purchasingOrg#29777]
Input [4]: [internalUUID#29727, _extract_internalRefUUID#29948, _extract_internalRefUUID#29949, supplierSubrange#29742]

(52) BroadcastExchange
Input [4]: [sourceOfSupplyUuid#29774, subrange#29775, purchasingGroup#29776, purchasingOrg#29777]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=9391]

(53) BroadcastHashJoin
Left keys [1]: [sourceOfSupplyUuid#29626]
Right keys [1]: [sourceOfSupplyUuid#29774]
Join type: LeftOuter
Join condition: None

(54) Project
Output [6]: [supplierGroup#29630, supplier#29625, plant#29624, deliveryDateTime#29628, subrange#29775, struct(product, product#29623, orderDateTime, orderDateTime#29627, availabilityDateTime, availabilityDateTime#29629, nextAvailabilityDateTime, nextAvailabilityDateTime#29631, mrpController, mrpController#29615, sourceOfSupplyUuid, sourceOfSupplyUuid#29626, purchasingOrg, purchasingOrg#29777, purchasingGroup, purchasingGroup#29776) AS struct#29829]
Input [14]: [plant#29624, product#29623, supplier#29625, sourceOfSupplyUuid#29626, orderDateTime#29627, deliveryDateTime#29628, availabilityDateTime#29629, supplierGroup#29630, nextAvailabilityDateTime#29631, mrpController#29615, sourceOfSupplyUuid#29774, subrange#29775, purchasingGroup#29776, purchasingOrg#29777]

(55) ObjectHashAggregate
Input [6]: [supplierGroup#29630, supplier#29625, plant#29624, deliveryDateTime#29628, subrange#29775, struct#29829]
Keys [5]: [supplierGroup#29630, supplier#29625, plant#29624, deliveryDateTime#29628, subrange#29775]
Functions [1]: [partial_collect_list(struct#29829, 0, 0)]
Aggregate Attributes [1]: [buf#30799]
Results [6]: [supplierGroup#29630, supplier#29625, plant#29624, deliveryDateTime#29628, subrange#29775, buf#30800]

(56) Exchange
Input [6]: [supplierGroup#29630, supplier#29625, plant#29624, deliveryDateTime#29628, subrange#29775, buf#30800]
Arguments: hashpartitioning(supplierGroup#29630, supplier#29625, plant#29624, deliveryDateTime#29628, subrange#29775, 25), ENSURE_REQUIREMENTS, [plan_id=9407]

(57) ObjectHashAggregate
Input [6]: [supplierGroup#29630, supplier#29625, plant#29624, deliveryDateTime#29628, subrange#29775, buf#30800]
Keys [5]: [supplierGroup#29630, supplier#29625, plant#29624, deliveryDateTime#29628, subrange#29775]
Functions [1]: [collect_list(struct#29829, 0, 0)]
Aggregate Attributes [1]: [collect_list(struct#29829, 0, 0)#29842]
Results [6]: [supplierGroup#29630, supplier#29625, plant#29624, deliveryDateTime#29628, subrange#29775, collect_list(struct#29829, 0, 0)#29842 AS productOrderSchedule#29843]

(58) AdaptiveSparkPlan
Output [6]: [supplierGroup#29630, supplier#29625, plant#29624, deliveryDateTime#29628, subrange#29775, productOrderSchedule#29843]
Arguments: isFinalPlan=true

SQL / DataFrame Properties

Name	Value
spark.activity.hdlfsEnabled	true
spark.activity.isolateSession	true
spark.sql.parquet.fieldId.read.enabled	true
spark.sql.parquet.fieldId.write.enabled	true
temporal.activity.id	e8bbd528-e182-393f-b737-ebededb77cbf
temporal.activity.type	EnhanceWorklistByAttributes