crp-order-qty-opt-service - Details for Query 5

Submitted Time: 2026/02/06 06:54:20
Duration: 3 s
Succeeded Jobs: 9 11 12 13 14 15 16

Show the Stage ID and Task ID that corresponds to the max metric

digraph G { 0 [labelType="html" label=" AdaptiveSparkPlan "]; 1 [labelType="html" label="ObjectHashAggregate number of output rows: 179 time in aggregation build total (min, med, max (stageId: taskId)) 194 ms (28 ms, 34 ms, 36 ms (stage 24.0: task 277)) spill size total (min, med, max (stageId: taskId)) 0.0 B (0.0 B, 0.0 B, 0.0 B (stage 24.0: task 275)) number of sort fallback tasks: 0"]; 2 [labelType="html" label="AQEShuffleRead number of partitions: 6 partition data size total (min, med, max (stageId: taskId)) 346.9 KiB (51.3 KiB, 57.9 KiB, 65.6 KiB (driver)) number of coalesced partitions: 6"]; 3 [labelType="html" label="Exchange shuffle records written: 1,032 local merged chunks fetched: 0 shuffle write time total (min, med, max (stageId: taskId)) 57 ms (1 ms, 2 ms, 3 ms (stage 21.0: task 259)) remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 1,032 local bytes read total (min, med, max (stageId: taskId)) 165.8 KiB (19.3 KiB, 32.4 KiB, 37.5 KiB (stage 24.0: task 280)) fetch wait time total (min, med, max (stageId: taskId)) 0 ms (0 ms, 0 ms, 0 ms (stage 24.0: task 275)) remote bytes read total (min, med, max (stageId: taskId)) 162.9 KiB (14.8 KiB, 33.1 KiB, 41.6 KiB (stage 24.0: task 277)) merged fetch fallback count: 0 local blocks read: 258 remote merged chunks fetched: 0 remote blocks read: 248 data size total (min, med, max (stageId: taskId)) 445.4 KiB (13.9 KiB, 17.6 KiB, 21.8 KiB (stage 21.0: task 250)) local merged bytes read: 0.0 B number of partitions: 25 remote reqs duration total (min, med, max (stageId: taskId)) 67 ms (8 ms, 12 ms, 15 ms (stage 24.0: task 277)) remote bytes read to disk: 0.0 B shuffle bytes written total (min, med, max (stageId: taskId)) 328.7 KiB (10.9 KiB, 12.9 KiB, 16.0 KiB (stage 21.0: task 257))"]; 4 [labelType="html" label="ObjectHashAggregate number of output rows: 1,032 time in aggregation build total (min, med, max (stageId: taskId)) 475 ms (2 ms, 24 ms, 39 ms (stage 21.0: task 252)) spill size total (min, med, max (stageId: taskId)) 0.0 B (0.0 B, 0.0 B, 0.0 B (stage 21.0: task 253)) number of sort fallback tasks: 0"]; subgraph cluster5 { isCluster="true"; label="WholeStageCodegen (6)\n \nduration: total (min, med, max (stageId: taskId))\n748 ms (2 ms, 24 ms, 76 ms (stage 21.0: task 250))"; 6 [labelType="html" label=" Project "]; 7 [labelType="html" label="BroadcastHashJoin number of output rows: 1,196"]; 8 [labelType="html" label=" Project "]; 9 [labelType="html" label="BroadcastHashJoin number of output rows: 1,196"]; 10 [labelType="html" label=" Project "]; 11 [labelType="html" label="Filter number of output rows: 1,196"]; } 12 [labelType="html" label="Window spill size total (min, med, max (stageId: taskId)) 0.0 B (0.0 B, 0.0 B, 0.0 B (stage 21.0: task 253))"]; 13 [labelType="html" label="WindowGroupLimit number of output rows: 1,196"]; subgraph cluster14 { isCluster="true"; label="WholeStageCodegen (5)\n \nduration: total (min, med, max (stageId: taskId))\n1.6 s (8 ms, 34 ms, 153 ms (stage 21.0: task 250))"; 15 [labelType="html" label="Sort sort time total (min, med, max (stageId: taskId)) 10 ms (0 ms, 0 ms, 2 ms (stage 21.0: task 252)) peak memory total (min, med, max (stageId: taskId)) 51.6 MiB (2.1 MiB, 2.1 MiB, 2.1 MiB (stage 21.0: task 253)) spill size total (min, med, max (stageId: taskId)) 0.0 B (0.0 B, 0.0 B, 0.0 B (stage 21.0: task 253))"]; } 16 [labelType="html" label="Exchange shuffle records written: 1,196 local merged chunks fetched: 0 shuffle write time total (min, med, max (stageId: taskId)) 31 ms (0 ms, 1 ms, 1 ms (stage 11.0: task 132)) remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 1,196 local bytes read total (min, med, max (stageId: taskId)) 92.8 KiB (2.0 KiB, 3.8 KiB, 5.3 KiB (stage 21.0: task 264)) fetch wait time total (min, med, max (stageId: taskId)) 8 ms (0 ms, 0 ms, 2 ms (stage 21.0: task 265)) remote bytes read total (min, med, max (stageId: taskId)) 73.9 KiB (867.0 B, 2.9 KiB, 4.4 KiB (stage 21.0: task 262)) merged fetch fallback count: 0 local blocks read: 96 remote merged chunks fetched: 0 remote blocks read: 77 data size total (min, med, max (stageId: taskId)) 252.3 KiB (3.2 KiB, 10.3 KiB, 20.7 KiB (stage 11.0: task 123)) local merged bytes read: 0.0 B number of partitions: 25 remote reqs duration total (min, med, max (stageId: taskId)) 112 ms (3 ms, 4 ms, 6 ms (stage 21.0: task 253)) remote bytes read to disk: 0.0 B shuffle bytes written total (min, med, max (stageId: taskId)) 166.7 KiB (2.6 KiB, 6.7 KiB, 11.9 KiB (stage 11.0: task 123))"]; subgraph cluster17 { isCluster="true"; label="WholeStageCodegen (1)\n \nduration: total (min, med, max (stageId: taskId))\n15.8 s (533 ms, 619 ms, 812 ms (stage 11.0: task 130))"; 18 [labelType="html" label="ColumnarToRow number of output rows: 1,196 number of input batches: 200"]; } 19 [labelType="html" label="Scan parquet number of files read: 200 scan time total (min, med, max (stageId: taskId)) 15.5 s (524 ms, 607 ms, 791 ms (stage 11.0: task 130)) metadata time: 2 ms size of files read: 757.7 KiB number of output rows: 1,196"]; 20 [labelType="html" label="BroadcastExchange time to broadcast: 4 ms time to build: 33 ms time to collect: 118 ms number of output rows: 1,867 data size: 2.1 MiB"]; 21 [labelType="html" label="Exchange shuffle records written: 1,867 local merged chunks fetched: 0 shuffle write time: 2 ms remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 1,867 local bytes read total (min, med, max (stageId: taskId)) 93.4 KiB (3.5 KiB, 3.7 KiB, 4.0 KiB (stage 17.0: task 203)) fetch wait time total (min, med, max (stageId: taskId)) 0 ms (0 ms, 0 ms, 0 ms (stage 17.0: task 200)) remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 25 remote merged chunks fetched: 0 remote blocks read: 0 data size: 221.3 KiB local merged bytes read: 0.0 B number of partitions: 25 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written: 93.4 KiB"]; subgraph cluster22 { isCluster="true"; label="WholeStageCodegen (2)\n \nduration: 157 ms"; 23 [labelType="html" label=" Project "]; 24 [labelType="html" label="Filter number of output rows: 1,867"]; 25 [labelType="html" label="ColumnarToRow number of output rows: 1,867 number of input batches: 1"]; } 26 [labelType="html" label="Scan parquet number of files read: 1 scan time: 149 ms dynamic partition pruning time: 0 ms metadata time: 906 ms size of files read: 108.6 KiB number of output rows: 1,867 number of partitions read: 1"]; 27 [labelType="html" label="BroadcastExchange time to broadcast: 3 ms time to build: 12 ms time to collect: 131 ms number of output rows: 3,797 data size: 2.1 MiB"]; subgraph cluster28 { isCluster="true"; label="WholeStageCodegen (4)\n \nduration: total (min, med, max (stageId: taskId))\n0 ms (0 ms, 0 ms, 0 ms (stage 19.0: task 208))"; 29 [labelType="html" label=" Project "]; } 30 [labelType="html" label="Exchange shuffle records written: 3,797 local merged chunks fetched: 0 shuffle write time: 2 ms remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 3,797 local bytes read total (min, med, max (stageId: taskId)) 154.2 KiB (6.0 KiB, 6.2 KiB, 6.3 KiB (stage 19.0: task 245)) fetch wait time total (min, med, max (stageId: taskId)) 0 ms (0 ms, 0 ms, 0 ms (stage 19.0: task 208)) remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 25 remote merged chunks fetched: 0 remote blocks read: 0 data size: 455.5 KiB local merged bytes read: 0.0 B number of partitions: 25 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written: 154.2 KiB"]; subgraph cluster31 { isCluster="true"; label="WholeStageCodegen (3)\n \nduration: 167 ms"; 32 [labelType="html" label=" Project "]; 33 [labelType="html" label="Filter number of output rows: 3,797"]; 34 [labelType="html" label="ColumnarToRow number of output rows: 3,797 number of input batches: 1"]; } 35 [labelType="html" label="Scan parquet number of files read: 1 scan time: 158 ms dynamic partition pruning time: 0 ms metadata time: 0 ms size of files read: 263.3 KiB number of output rows: 3,797 number of partitions read: 1"]; 1->0; 2->1; 3->2; 4->3; 6->4; 7->6; 8->7; 9->8; 10->9; 11->10; 12->11; 13->12; 15->13; 16->15; 18->16; 19->18; 20->9; 21->20; 23->21; 24->23; 25->24; 26->25; 27->7; 29->27; 30->29; 32->30; 33->32; 34->33; 35->34; }

AdaptiveSparkPlan isFinalPlan=true

ObjectHashAggregate(keys=[supplierGroup#135, supplier#130, plant#129, deliveryDateTime#133, subrange#280], functions=[collect_list(struct#334, 0, 0)])

AQEShuffleRead coalesced

Exchange hashpartitioning(supplierGroup#135, supplier#130, plant#129, deliveryDateTime#133, subrange#280, 25), ENSURE_REQUIREMENTS, [plan_id=1018]

ObjectHashAggregate(keys=[supplierGroup#135, supplier#130, plant#129, deliveryDateTime#133, subrange#280], functions=[partial_collect_list(struct#334, 0, 0)])

Project [supplierGroup#135, supplier#130, plant#129, deliveryDateTime#133, subrange#280, struct(product, product#128, orderDateTime, orderDateTime#132, availabilityDateTime, availabilityDateTime#134, nextAvailabilityDateTime, nextAvailabilityDateTime#136, mrpController, mrpController#120, sourceOfSupplyUuid, sourceOfSupplyUuid#131, purchasingOrg, purchasingOrg#282, purchasingGroup, purchasingGroup#281) AS struct#334]

BroadcastHashJoin [sourceOfSupplyUuid#131], [sourceOfSupplyUuid#279], LeftOuter, BuildRight, false

Project [plant#129, product#128, supplier#130, sourceOfSupplyUuid#131, orderDateTime#132, deliveryDateTime#133, availabilityDateTime#134, supplierGroup#135, nextAvailabilityDateTime#136, mrpController#120]

BroadcastHashJoin [plant#129, product#128], [plant#119, product#118], LeftOuter, BuildRight, false

Project [product#128, plant#129, supplier#130, sourceOfSupplyUuid#131, orderDateTime#132, deliveryDateTime#133, availabilityDateTime#134, supplierGroup#135, nextAvailabilityDateTime#136]

Filter (rank#291 = 1)

WholeStageCodegen (6)

Window [row_number() windowspecdefinition(product#128, plant#129, availabilityDateTime#134 ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) AS rank#291], [product#128, plant#129], [availabilityDateTime#134 ASC NULLS FIRST]

WindowGroupLimit [product#128, plant#129], [availabilityDateTime#134 ASC NULLS FIRST], row_number(), 1, Final

Sort [product#128 ASC NULLS FIRST, plant#129 ASC NULLS FIRST, availabilityDateTime#134 ASC NULLS FIRST], false, 0

WholeStageCodegen (5)

Exchange hashpartitioning(product#128, plant#129, 25), REPARTITION_BY_NUM, [plan_id=544]

ColumnarToRow

WholeStageCodegen (1)

FileScan parquet [product#128,plant#129,supplier#130,sourceOfSupplyUuid#131,orderDateTime#132,deliveryDateTime#133,availabilityDateTime#134,supplierGroup#135,nextAvailabilityDateTime#136] Batched: true, DataFilters: [], Format: Parquet, Location: InMemoryFileIndex(1 paths)[hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanac..., PartitionFilters: [], PushedFilters: [], ReadSchema: struct<product:string,plant:string,supplier:string,sourceOfSupplyUuid:string,orderDateTime:timest...

BroadcastExchange HashedRelationBroadcastMode(List(input[1, string, true], input[0, string, true]),false), [plan_id=575]

Exchange RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=565]

Project [product#103.internalRefUUID AS product#118, plant#102.internalRefUUID AS plant#119, productionAspect#105.productPlanningPlants.mrpController.internalRefUUID AS mrpController#120]

Filter (isnotnull(plant#102.internalRefUUID) AND isnotnull(product#103.internalRefUUID))

ColumnarToRow

WholeStageCodegen (2)

FileScan parquet [plant#102,product#103,productionAspect#105,systemId#100] Batched: true, DataFilters: [isnotnull(plant#102.internalRefUUID), isnotnull(product#103.internalRefUUID)], Format: Parquet, Location: PreparedDeltaFileIndex(1 paths)[hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20...., PartitionFilters: [], PushedFilters: [IsNotNull(plant.internalRefUUID), IsNotNull(product.internalRefUUID)], ReadSchema: struct<plant:struct<internalRefUUID:string>,product:struct<internalRefUUID:string>,productionAspe...

BroadcastExchange HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=799]

Project [internalUUID#232 AS sourceOfSupplyUuid#279, supplierSubrange#247 AS subrange#280, _extract_internalRefUUID#455 AS purchasingGroup#281, _extract_internalRefUUID#456 AS purchasingOrg#282]

WholeStageCodegen (4)

Exchange RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=587]

Project [internalUUID#232, purchasingGroup#238.internalRefUUID AS _extract_internalRefUUID#455, purchasingOrganization#239.internalRefUUID AS _extract_internalRefUUID#456, supplierSubrange#247]

Filter isnotnull(internalUUID#232)

ColumnarToRow

WholeStageCodegen (3)

FileScan parquet [internalUUID#232,purchasingGroup#238,purchasingOrganization#239,supplierSubrange#247,systemId#231] Batched: true, DataFilters: [isnotnull(internalUUID#232)], Format: Parquet, Location: PreparedDeltaFileIndex(1 paths)[hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20...., PartitionFilters: [], PushedFilters: [IsNotNull(internalUUID)], ReadSchema: struct<internalUUID:string,purchasingGroup:struct<internalRefUUID:string>,purchasingOrganization:...

Details

== Physical Plan ==
AdaptiveSparkPlan (58)
+- == Final Plan ==
   ObjectHashAggregate (35)
   +- AQEShuffleRead (34)
      +- ShuffleQueryStage (33), Statistics(sizeInBytes=445.4 KiB, rowCount=1.03E+3)
         +- Exchange (32)
            +- ObjectHashAggregate (31)
               +- * Project (30)
                  +- * BroadcastHashJoin LeftOuter BuildRight (29)
                     :- * Project (19)
                     :  +- * BroadcastHashJoin LeftOuter BuildRight (18)
                     :     :- * Project (9)
                     :     :  +- * Filter (8)
                     :     :     +- Window (7)
                     :     :        +- WindowGroupLimit (6)
                     :     :           +- * Sort (5)
                     :     :              +- ShuffleQueryStage (4), Statistics(sizeInBytes=252.3 KiB, rowCount=1.20E+3)
                     :     :                 +- Exchange (3)
                     :     :                    +- * ColumnarToRow (2)
                     :     :                       +- Scan parquet  (1)
                     :     +- BroadcastQueryStage (17), Statistics(sizeInBytes=2.1 MiB, rowCount=1.87E+3)
                     :        +- BroadcastExchange (16)
                     :           +- ShuffleQueryStage (15), Statistics(sizeInBytes=221.3 KiB, rowCount=1.87E+3)
                     :              +- Exchange (14)
                     :                 +- * Project (13)
                     :                    +- * Filter (12)
                     :                       +- * ColumnarToRow (11)
                     :                          +- Scan parquet  (10)
                     +- BroadcastQueryStage (28), Statistics(sizeInBytes=2.1 MiB, rowCount=3.80E+3)
                        +- BroadcastExchange (27)
                           +- * Project (26)
                              +- ShuffleQueryStage (25), Statistics(sizeInBytes=455.5 KiB, rowCount=3.80E+3)
                                 +- Exchange (24)
                                    +- * Project (23)
                                       +- * Filter (22)
                                          +- * ColumnarToRow (21)
                                             +- Scan parquet  (20)
+- == Initial Plan ==
   ObjectHashAggregate (57)
   +- Exchange (56)
      +- ObjectHashAggregate (55)
         +- Project (54)
            +- BroadcastHashJoin LeftOuter BuildRight (53)
               :- Project (47)
               :  +- BroadcastHashJoin LeftOuter BuildRight (46)
               :     :- Project (41)
               :     :  +- Filter (40)
               :     :     +- Window (39)
               :     :        +- WindowGroupLimit (38)
               :     :           +- Sort (37)
               :     :              +- Exchange (36)
               :     :                 +- Scan parquet  (1)
               :     +- BroadcastExchange (45)
               :        +- Exchange (44)
               :           +- Project (43)
               :              +- Filter (42)
               :                 +- Scan parquet  (10)
               +- BroadcastExchange (52)
                  +- Project (51)
                     +- Exchange (50)
                        +- Project (49)
                           +- Filter (48)
                              +- Scan parquet  (20)


(1) Scan parquet 
Output [9]: [product#128, plant#129, supplier#130, sourceOfSupplyUuid#131, orderDateTime#132, deliveryDateTime#133, availabilityDateTime#134, supplierGroup#135, nextAvailabilityDateTime#136]
Batched: true
Location: InMemoryFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-workload-determination-service/out/workload-determination-result/10000000095/results]
ReadSchema: struct<product:string,plant:string,supplier:string,sourceOfSupplyUuid:string,orderDateTime:timestamp,deliveryDateTime:timestamp,availabilityDateTime:timestamp,supplierGroup:string,nextAvailabilityDateTime:timestamp>

(2) ColumnarToRow [codegen id : 1]
Input [9]: [product#128, plant#129, supplier#130, sourceOfSupplyUuid#131, orderDateTime#132, deliveryDateTime#133, availabilityDateTime#134, supplierGroup#135, nextAvailabilityDateTime#136]

(3) Exchange
Input [9]: [product#128, plant#129, supplier#130, sourceOfSupplyUuid#131, orderDateTime#132, deliveryDateTime#133, availabilityDateTime#134, supplierGroup#135, nextAvailabilityDateTime#136]
Arguments: hashpartitioning(product#128, plant#129, 25), REPARTITION_BY_NUM, [plan_id=544]

(4) ShuffleQueryStage
Output [9]: [product#128, plant#129, supplier#130, sourceOfSupplyUuid#131, orderDateTime#132, deliveryDateTime#133, availabilityDateTime#134, supplierGroup#135, nextAvailabilityDateTime#136]
Arguments: 0

(5) Sort [codegen id : 5]
Input [9]: [product#128, plant#129, supplier#130, sourceOfSupplyUuid#131, orderDateTime#132, deliveryDateTime#133, availabilityDateTime#134, supplierGroup#135, nextAvailabilityDateTime#136]
Arguments: [product#128 ASC NULLS FIRST, plant#129 ASC NULLS FIRST, availabilityDateTime#134 ASC NULLS FIRST], false, 0

(6) WindowGroupLimit
Input [9]: [product#128, plant#129, supplier#130, sourceOfSupplyUuid#131, orderDateTime#132, deliveryDateTime#133, availabilityDateTime#134, supplierGroup#135, nextAvailabilityDateTime#136]
Arguments: [product#128, plant#129], [availabilityDateTime#134 ASC NULLS FIRST], row_number(), 1, Final

(7) Window
Input [9]: [product#128, plant#129, supplier#130, sourceOfSupplyUuid#131, orderDateTime#132, deliveryDateTime#133, availabilityDateTime#134, supplierGroup#135, nextAvailabilityDateTime#136]
Arguments: [row_number() windowspecdefinition(product#128, plant#129, availabilityDateTime#134 ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) AS rank#291], [product#128, plant#129], [availabilityDateTime#134 ASC NULLS FIRST]

(8) Filter [codegen id : 6]
Input [10]: [product#128, plant#129, supplier#130, sourceOfSupplyUuid#131, orderDateTime#132, deliveryDateTime#133, availabilityDateTime#134, supplierGroup#135, nextAvailabilityDateTime#136, rank#291]
Condition : (rank#291 = 1)

(9) Project [codegen id : 6]
Output [9]: [product#128, plant#129, supplier#130, sourceOfSupplyUuid#131, orderDateTime#132, deliveryDateTime#133, availabilityDateTime#134, supplierGroup#135, nextAvailabilityDateTime#136]
Input [10]: [product#128, plant#129, supplier#130, sourceOfSupplyUuid#131, orderDateTime#132, deliveryDateTime#133, availabilityDateTime#134, supplierGroup#135, nextAvailabilityDateTime#136, rank#291]

(10) Scan parquet 
Output [4]: [plant#102, product#103, productionAspect#105, systemId#100]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-productplant]
PushedFilters: [IsNotNull(plant.internalRefUUID), IsNotNull(product.internalRefUUID)]
ReadSchema: struct<plant:struct<internalRefUUID:string>,product:struct<internalRefUUID:string>,productionAspect:struct<productPlanningPlants:struct<mrpController:struct<internalRefUUID:string>>>>

(11) ColumnarToRow [codegen id : 2]
Input [4]: [plant#102, product#103, productionAspect#105, systemId#100]

(12) Filter [codegen id : 2]
Input [4]: [plant#102, product#103, productionAspect#105, systemId#100]
Condition : (isnotnull(plant#102.internalRefUUID) AND isnotnull(product#103.internalRefUUID))

(13) Project [codegen id : 2]
Output [3]: [product#103.internalRefUUID AS product#118, plant#102.internalRefUUID AS plant#119, productionAspect#105.productPlanningPlants.mrpController.internalRefUUID AS mrpController#120]
Input [4]: [plant#102, product#103, productionAspect#105, systemId#100]

(14) Exchange
Input [3]: [product#118, plant#119, mrpController#120]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=565]

(15) ShuffleQueryStage
Output [3]: [product#118, plant#119, mrpController#120]
Arguments: 1

(16) BroadcastExchange
Input [3]: [product#118, plant#119, mrpController#120]
Arguments: HashedRelationBroadcastMode(List(input[1, string, true], input[0, string, true]),false), [plan_id=575]

(17) BroadcastQueryStage
Output [3]: [product#118, plant#119, mrpController#120]
Arguments: 3

(18) BroadcastHashJoin [codegen id : 6]
Left keys [2]: [plant#129, product#128]
Right keys [2]: [plant#119, product#118]
Join type: LeftOuter
Join condition: None

(19) Project [codegen id : 6]
Output [10]: [plant#129, product#128, supplier#130, sourceOfSupplyUuid#131, orderDateTime#132, deliveryDateTime#133, availabilityDateTime#134, supplierGroup#135, nextAvailabilityDateTime#136, mrpController#120]
Input [12]: [product#128, plant#129, supplier#130, sourceOfSupplyUuid#131, orderDateTime#132, deliveryDateTime#133, availabilityDateTime#134, supplierGroup#135, nextAvailabilityDateTime#136, product#118, plant#119, mrpController#120]

(20) Scan parquet 
Output [5]: [internalUUID#232, purchasingGroup#238, purchasingOrganization#239, supplierSubrange#247, systemId#231]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-sourceofsupply-sourceofsupply]
PushedFilters: [IsNotNull(internalUUID)]
ReadSchema: struct<internalUUID:string,purchasingGroup:struct<internalRefUUID:string>,purchasingOrganization:struct<internalRefUUID:string>,supplierSubrange:string>

(21) ColumnarToRow [codegen id : 3]
Input [5]: [internalUUID#232, purchasingGroup#238, purchasingOrganization#239, supplierSubrange#247, systemId#231]

(22) Filter [codegen id : 3]
Input [5]: [internalUUID#232, purchasingGroup#238, purchasingOrganization#239, supplierSubrange#247, systemId#231]
Condition : isnotnull(internalUUID#232)

(23) Project [codegen id : 3]
Output [4]: [internalUUID#232, purchasingGroup#238.internalRefUUID AS _extract_internalRefUUID#455, purchasingOrganization#239.internalRefUUID AS _extract_internalRefUUID#456, supplierSubrange#247]
Input [5]: [internalUUID#232, purchasingGroup#238, purchasingOrganization#239, supplierSubrange#247, systemId#231]

(24) Exchange
Input [4]: [internalUUID#232, _extract_internalRefUUID#455, _extract_internalRefUUID#456, supplierSubrange#247]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=587]

(25) ShuffleQueryStage
Output [4]: [internalUUID#232, _extract_internalRefUUID#455, _extract_internalRefUUID#456, supplierSubrange#247]
Arguments: 2

(26) Project [codegen id : 4]
Output [4]: [internalUUID#232 AS sourceOfSupplyUuid#279, supplierSubrange#247 AS subrange#280, _extract_internalRefUUID#455 AS purchasingGroup#281, _extract_internalRefUUID#456 AS purchasingOrg#282]
Input [4]: [internalUUID#232, _extract_internalRefUUID#455, _extract_internalRefUUID#456, supplierSubrange#247]

(27) BroadcastExchange
Input [4]: [sourceOfSupplyUuid#279, subrange#280, purchasingGroup#281, purchasingOrg#282]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=799]

(28) BroadcastQueryStage
Output [4]: [sourceOfSupplyUuid#279, subrange#280, purchasingGroup#281, purchasingOrg#282]
Arguments: 4

(29) BroadcastHashJoin [codegen id : 6]
Left keys [1]: [sourceOfSupplyUuid#131]
Right keys [1]: [sourceOfSupplyUuid#279]
Join type: LeftOuter
Join condition: None

(30) Project [codegen id : 6]
Output [6]: [supplierGroup#135, supplier#130, plant#129, deliveryDateTime#133, subrange#280, struct(product, product#128, orderDateTime, orderDateTime#132, availabilityDateTime, availabilityDateTime#134, nextAvailabilityDateTime, nextAvailabilityDateTime#136, mrpController, mrpController#120, sourceOfSupplyUuid, sourceOfSupplyUuid#131, purchasingOrg, purchasingOrg#282, purchasingGroup, purchasingGroup#281) AS struct#334]
Input [14]: [plant#129, product#128, supplier#130, sourceOfSupplyUuid#131, orderDateTime#132, deliveryDateTime#133, availabilityDateTime#134, supplierGroup#135, nextAvailabilityDateTime#136, mrpController#120, sourceOfSupplyUuid#279, subrange#280, purchasingGroup#281, purchasingOrg#282]

(31) ObjectHashAggregate
Input [6]: [supplierGroup#135, supplier#130, plant#129, deliveryDateTime#133, subrange#280, struct#334]
Keys [5]: [supplierGroup#135, supplier#130, plant#129, deliveryDateTime#133, subrange#280]
Functions [1]: [partial_collect_list(struct#334, 0, 0)]
Aggregate Attributes [1]: [buf#1331]
Results [6]: [supplierGroup#135, supplier#130, plant#129, deliveryDateTime#133, subrange#280, buf#1332]

(32) Exchange
Input [6]: [supplierGroup#135, supplier#130, plant#129, deliveryDateTime#133, subrange#280, buf#1332]
Arguments: hashpartitioning(supplierGroup#135, supplier#130, plant#129, deliveryDateTime#133, subrange#280, 25), ENSURE_REQUIREMENTS, [plan_id=1018]

(33) ShuffleQueryStage
Output [6]: [supplierGroup#135, supplier#130, plant#129, deliveryDateTime#133, subrange#280, buf#1332]
Arguments: 5

(34) AQEShuffleRead
Input [6]: [supplierGroup#135, supplier#130, plant#129, deliveryDateTime#133, subrange#280, buf#1332]
Arguments: coalesced

(35) ObjectHashAggregate
Input [6]: [supplierGroup#135, supplier#130, plant#129, deliveryDateTime#133, subrange#280, buf#1332]
Keys [5]: [supplierGroup#135, supplier#130, plant#129, deliveryDateTime#133, subrange#280]
Functions [1]: [collect_list(struct#334, 0, 0)]
Aggregate Attributes [1]: [collect_list(struct#334, 0, 0)#347]
Results [6]: [supplierGroup#135, supplier#130, plant#129, deliveryDateTime#133, subrange#280, collect_list(struct#334, 0, 0)#347 AS productOrderSchedule#348]

(36) Exchange
Input [9]: [product#128, plant#129, supplier#130, sourceOfSupplyUuid#131, orderDateTime#132, deliveryDateTime#133, availabilityDateTime#134, supplierGroup#135, nextAvailabilityDateTime#136]
Arguments: hashpartitioning(product#128, plant#129, 25), REPARTITION_BY_NUM, [plan_id=446]

(37) Sort
Input [9]: [product#128, plant#129, supplier#130, sourceOfSupplyUuid#131, orderDateTime#132, deliveryDateTime#133, availabilityDateTime#134, supplierGroup#135, nextAvailabilityDateTime#136]
Arguments: [product#128 ASC NULLS FIRST, plant#129 ASC NULLS FIRST, availabilityDateTime#134 ASC NULLS FIRST], false, 0

(38) WindowGroupLimit
Input [9]: [product#128, plant#129, supplier#130, sourceOfSupplyUuid#131, orderDateTime#132, deliveryDateTime#133, availabilityDateTime#134, supplierGroup#135, nextAvailabilityDateTime#136]
Arguments: [product#128, plant#129], [availabilityDateTime#134 ASC NULLS FIRST], row_number(), 1, Final

(39) Window
Input [9]: [product#128, plant#129, supplier#130, sourceOfSupplyUuid#131, orderDateTime#132, deliveryDateTime#133, availabilityDateTime#134, supplierGroup#135, nextAvailabilityDateTime#136]
Arguments: [row_number() windowspecdefinition(product#128, plant#129, availabilityDateTime#134 ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) AS rank#291], [product#128, plant#129], [availabilityDateTime#134 ASC NULLS FIRST]

(40) Filter
Input [10]: [product#128, plant#129, supplier#130, sourceOfSupplyUuid#131, orderDateTime#132, deliveryDateTime#133, availabilityDateTime#134, supplierGroup#135, nextAvailabilityDateTime#136, rank#291]
Condition : (rank#291 = 1)

(41) Project
Output [9]: [product#128, plant#129, supplier#130, sourceOfSupplyUuid#131, orderDateTime#132, deliveryDateTime#133, availabilityDateTime#134, supplierGroup#135, nextAvailabilityDateTime#136]
Input [10]: [product#128, plant#129, supplier#130, sourceOfSupplyUuid#131, orderDateTime#132, deliveryDateTime#133, availabilityDateTime#134, supplierGroup#135, nextAvailabilityDateTime#136, rank#291]

(42) Filter
Input [4]: [plant#102, product#103, productionAspect#105, systemId#100]
Condition : (isnotnull(plant#102.internalRefUUID) AND isnotnull(product#103.internalRefUUID))

(43) Project
Output [3]: [product#103.internalRefUUID AS product#118, plant#102.internalRefUUID AS plant#119, productionAspect#105.productPlanningPlants.mrpController.internalRefUUID AS mrpController#120]
Input [4]: [plant#102, product#103, productionAspect#105, systemId#100]

(44) Exchange
Input [3]: [product#118, plant#119, mrpController#120]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=455]

(45) BroadcastExchange
Input [3]: [product#118, plant#119, mrpController#120]
Arguments: HashedRelationBroadcastMode(List(input[1, string, true], input[0, string, true]),false), [plan_id=475]

(46) BroadcastHashJoin
Left keys [2]: [plant#129, product#128]
Right keys [2]: [plant#119, product#118]
Join type: LeftOuter
Join condition: None

(47) Project
Output [10]: [plant#129, product#128, supplier#130, sourceOfSupplyUuid#131, orderDateTime#132, deliveryDateTime#133, availabilityDateTime#134, supplierGroup#135, nextAvailabilityDateTime#136, mrpController#120]
Input [12]: [product#128, plant#129, supplier#130, sourceOfSupplyUuid#131, orderDateTime#132, deliveryDateTime#133, availabilityDateTime#134, supplierGroup#135, nextAvailabilityDateTime#136, product#118, plant#119, mrpController#120]

(48) Filter
Input [5]: [internalUUID#232, purchasingGroup#238, purchasingOrganization#239, supplierSubrange#247, systemId#231]
Condition : isnotnull(internalUUID#232)

(49) Project
Output [4]: [internalUUID#232, purchasingGroup#238.internalRefUUID AS _extract_internalRefUUID#455, purchasingOrganization#239.internalRefUUID AS _extract_internalRefUUID#456, supplierSubrange#247]
Input [5]: [internalUUID#232, purchasingGroup#238, purchasingOrganization#239, supplierSubrange#247, systemId#231]

(50) Exchange
Input [4]: [internalUUID#232, _extract_internalRefUUID#455, _extract_internalRefUUID#456, supplierSubrange#247]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=461]

(51) Project
Output [4]: [internalUUID#232 AS sourceOfSupplyUuid#279, supplierSubrange#247 AS subrange#280, _extract_internalRefUUID#455 AS purchasingGroup#281, _extract_internalRefUUID#456 AS purchasingOrg#282]
Input [4]: [internalUUID#232, _extract_internalRefUUID#455, _extract_internalRefUUID#456, supplierSubrange#247]

(52) BroadcastExchange
Input [4]: [sourceOfSupplyUuid#279, subrange#280, purchasingGroup#281, purchasingOrg#282]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=479]

(53) BroadcastHashJoin
Left keys [1]: [sourceOfSupplyUuid#131]
Right keys [1]: [sourceOfSupplyUuid#279]
Join type: LeftOuter
Join condition: None

(54) Project
Output [6]: [supplierGroup#135, supplier#130, plant#129, deliveryDateTime#133, subrange#280, struct(product, product#128, orderDateTime, orderDateTime#132, availabilityDateTime, availabilityDateTime#134, nextAvailabilityDateTime, nextAvailabilityDateTime#136, mrpController, mrpController#120, sourceOfSupplyUuid, sourceOfSupplyUuid#131, purchasingOrg, purchasingOrg#282, purchasingGroup, purchasingGroup#281) AS struct#334]
Input [14]: [plant#129, product#128, supplier#130, sourceOfSupplyUuid#131, orderDateTime#132, deliveryDateTime#133, availabilityDateTime#134, supplierGroup#135, nextAvailabilityDateTime#136, mrpController#120, sourceOfSupplyUuid#279, subrange#280, purchasingGroup#281, purchasingOrg#282]

(55) ObjectHashAggregate
Input [6]: [supplierGroup#135, supplier#130, plant#129, deliveryDateTime#133, subrange#280, struct#334]
Keys [5]: [supplierGroup#135, supplier#130, plant#129, deliveryDateTime#133, subrange#280]
Functions [1]: [partial_collect_list(struct#334, 0, 0)]
Aggregate Attributes [1]: [buf#1331]
Results [6]: [supplierGroup#135, supplier#130, plant#129, deliveryDateTime#133, subrange#280, buf#1332]

(56) Exchange
Input [6]: [supplierGroup#135, supplier#130, plant#129, deliveryDateTime#133, subrange#280, buf#1332]
Arguments: hashpartitioning(supplierGroup#135, supplier#130, plant#129, deliveryDateTime#133, subrange#280, 25), ENSURE_REQUIREMENTS, [plan_id=495]

(57) ObjectHashAggregate
Input [6]: [supplierGroup#135, supplier#130, plant#129, deliveryDateTime#133, subrange#280, buf#1332]
Keys [5]: [supplierGroup#135, supplier#130, plant#129, deliveryDateTime#133, subrange#280]
Functions [1]: [collect_list(struct#334, 0, 0)]
Aggregate Attributes [1]: [collect_list(struct#334, 0, 0)#347]
Results [6]: [supplierGroup#135, supplier#130, plant#129, deliveryDateTime#133, subrange#280, collect_list(struct#334, 0, 0)#347 AS productOrderSchedule#348]

(58) AdaptiveSparkPlan
Output [6]: [supplierGroup#135, supplier#130, plant#129, deliveryDateTime#133, subrange#280, productOrderSchedule#348]
Arguments: isFinalPlan=true

SQL / DataFrame Properties

Name	Value
spark.activity.hdlfsEnabled	true
spark.activity.isolateSession	true
spark.sql.parquet.fieldId.read.enabled	true
spark.sql.parquet.fieldId.write.enabled	true
temporal.activity.id	b191ef00-edf7-38ea-9e23-db676456ad33
temporal.activity.type	EnhanceWorklistByAttributes