crp-order-qty-opt-service - Details for Query 110

Submitted Time: 2026/02/06 10:33:51
Duration: 2 s
Succeeded Jobs: 231 233 235

Show the Stage ID and Task ID that corresponds to the max metric

digraph G { 0 [labelType="html" label=" AdaptiveSparkPlan "]; 1 [labelType="html" label="Execute InsertIntoHadoopFsRelationCommand task commit time total (min, med, max (stageId: taskId)) 539 ms (256 ms, 283 ms, 283 ms (stage 451.0: task 3084)) number of written files: 2 job commit time: 783 ms number of output rows: 179 number of dynamic part: 0 written output: 5.5 KiB"]; 2 [labelType="html" label=" WriteFiles "]; subgraph cluster3 { isCluster="true"; label="WholeStageCodegen (3)\n \nduration: total (min, med, max (stageId: taskId))\n218 ms (51 ms, 167 ms, 167 ms (stage 451.0: task 3084))"; 4 [labelType="html" label="HashAggregate spill size: 0.0 B time in aggregation build total (min, med, max (stageId: taskId)) 57 ms (22 ms, 35 ms, 35 ms (stage 451.0: task 3084)) peak memory total (min, med, max (stageId: taskId)) 4.5 MiB (2.3 MiB, 2.3 MiB, 2.3 MiB (stage 451.0: task 3082)) number of output rows: 179 number of sort fallback tasks: 0 avg hash probes per key (min, med, max (stageId: taskId)): (1, 1, 1 (stage 451.0: task 3082))"]; } 5 [labelType="html" label="AQEShuffleRead number of partitions: 2 partition data size total (min, med, max (stageId: taskId)) 132.4 KiB (52.2 KiB, 80.2 KiB, 80.2 KiB (driver)) number of coalesced partitions: 2"]; 6 [labelType="html" label="Exchange shuffle records written: 1,032 local merged chunks fetched: 0 shuffle write time total (min, med, max (stageId: taskId)) 44 ms (1 ms, 1 ms, 2 ms (stage 446.0: task 2990)) remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 1,032 local bytes read total (min, med, max (stageId: taskId)) 125.7 KiB (49.5 KiB, 76.2 KiB, 76.2 KiB (stage 451.0: task 3084)) fetch wait time total (min, med, max (stageId: taskId)) 0 ms (0 ms, 0 ms, 0 ms (stage 451.0: task 3082)) remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 497 remote merged chunks fetched: 0 remote blocks read: 0 data size total (min, med, max (stageId: taskId)) 126.3 KiB (4.0 KiB, 5.0 KiB, 6.1 KiB (stage 446.0: task 3012)) local merged bytes read: 0.0 B number of partitions: 25 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written total (min, med, max (stageId: taskId)) 125.7 KiB (4.1 KiB, 5.0 KiB, 5.9 KiB (stage 446.0: task 3002))"]; subgraph cluster7 { isCluster="true"; label="WholeStageCodegen (2)\n \nduration: total (min, med, max (stageId: taskId))\n177 ms (4 ms, 5 ms, 16 ms (stage 446.0: task 2984))"; 8 [labelType="html" label="HashAggregate spill size: 0.0 B time in aggregation build total (min, med, max (stageId: taskId)) 5 ms (0 ms, 0 ms, 1 ms (stage 446.0: task 2992)) peak memory total (min, med, max (stageId: taskId)) 56.3 MiB (2.3 MiB, 2.3 MiB, 2.3 MiB (stage 446.0: task 2992)) number of output rows: 1,032 number of sort fallback tasks: 0 avg hash probes per key (min, med, max (stageId: taskId)): (1, 1, 1 (stage 446.0: task 2992))"]; 9 [labelType="html" label=" Project "]; } 10 [labelType="html" label="Exchange shuffle records written: 1,196 local merged chunks fetched: 0 shuffle write time: 1 ms remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 1,196 local bytes read total (min, med, max (stageId: taskId)) 66.3 KiB (2.2 KiB, 2.7 KiB, 3.0 KiB (stage 446.0: task 3003)) fetch wait time total (min, med, max (stageId: taskId)) 0 ms (0 ms, 0 ms, 0 ms (stage 446.0: task 2992)) remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 25 remote merged chunks fetched: 0 remote blocks read: 0 data size: 193.0 KiB local merged bytes read: 0.0 B number of partitions: 25 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written: 66.3 KiB"]; subgraph cluster11 { isCluster="true"; label="WholeStageCodegen (1)\n \nduration: 111 ms"; 12 [labelType="html" label="ColumnarToRow number of output rows: 1,196 number of input batches: 1"]; } 13 [labelType="html" label="Scan parquet number of files read: 1 scan time: 98 ms metadata time: 0 ms size of files read: 37.4 KiB number of output rows: 1,196"]; 1->0; 2->1; 4->2; 5->4; 6->5; 8->6; 9->8; 10->9; 12->10; 13->12; }

AdaptiveSparkPlan isFinalPlan=true

Execute InsertIntoHadoopFsRelationCommand hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-order-qty-opt-service/out/supply-network-list/10000000096/shardId=0_1_10000000096, false, Parquet, [path=hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-order-qty-opt-service/out/supply-network-list/10000000096/shardId=0_1_10000000096], Overwrite, [supplierGroup, supplier, plant, deliveryDateTime, subrange]

WriteFiles

HashAggregate(keys=[plant#41161, supplierGroup#41159, subrange#41163, supplier#41160, deliveryDateTime#41162], functions=[])

WholeStageCodegen (3)

AQEShuffleRead coalesced

Exchange hashpartitioning(plant#41161, supplierGroup#41159, subrange#41163, supplier#41160, deliveryDateTime#41162, 25), ENSURE_REQUIREMENTS, [plan_id=12211]

HashAggregate(keys=[plant#41161, supplierGroup#41159, subrange#41163, supplier#41160, deliveryDateTime#41162], functions=[])

Project [supplierGroup#41159, supplier#41160, plant#41161, deliveryDateTime#41162, subrange#41163]

WholeStageCodegen (2)

Exchange hashpartitioning(product#41164, plant#41161, 25), REPARTITION_BY_NUM, [plan_id=12098]

ColumnarToRow

WholeStageCodegen (1)

FileScan parquet [supplierGroup#41159,supplier#41160,plant#41161,deliveryDateTime#41162,subrange#41163,product#41164] Batched: true, DataFilters: [], Format: Parquet, Location: InMemoryFileIndex(1 paths)[hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanac..., PartitionFilters: [], PushedFilters: [], ReadSchema: struct<supplierGroup:string,supplier:string,plant:string,deliveryDateTime:timestamp,subrange:stri...

Details

== Physical Plan ==
AdaptiveSparkPlan (20)
+- == Final Plan ==
   Execute InsertIntoHadoopFsRelationCommand (12)
   +- WriteFiles (11)
      +- * HashAggregate (10)
         +- AQEShuffleRead (9)
            +- ShuffleQueryStage (8), Statistics(sizeInBytes=126.3 KiB, rowCount=1.03E+3)
               +- Exchange (7)
                  +- * HashAggregate (6)
                     +- * Project (5)
                        +- ShuffleQueryStage (4), Statistics(sizeInBytes=193.0 KiB, rowCount=1.20E+3)
                           +- Exchange (3)
                              +- * ColumnarToRow (2)
                                 +- Scan parquet  (1)
+- == Initial Plan ==
   Execute InsertIntoHadoopFsRelationCommand (19)
   +- WriteFiles (18)
      +- HashAggregate (17)
         +- Exchange (16)
            +- HashAggregate (15)
               +- Project (14)
                  +- Exchange (13)
                     +- Scan parquet  (1)


(1) Scan parquet 
Output [6]: [supplierGroup#41159, supplier#41160, plant#41161, deliveryDateTime#41162, subrange#41163, product#41164]
Batched: true
Location: InMemoryFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-order-qty-opt-service/out/order-proposal-boundary-snapshot/10000000096/shardId=0_1_10000000096]
ReadSchema: struct<supplierGroup:string,supplier:string,plant:string,deliveryDateTime:timestamp,subrange:string,product:string>

(2) ColumnarToRow [codegen id : 1]
Input [6]: [supplierGroup#41159, supplier#41160, plant#41161, deliveryDateTime#41162, subrange#41163, product#41164]

(3) Exchange
Input [6]: [supplierGroup#41159, supplier#41160, plant#41161, deliveryDateTime#41162, subrange#41163, product#41164]
Arguments: hashpartitioning(product#41164, plant#41161, 25), REPARTITION_BY_NUM, [plan_id=12098]

(4) ShuffleQueryStage
Output [6]: [supplierGroup#41159, supplier#41160, plant#41161, deliveryDateTime#41162, subrange#41163, product#41164]
Arguments: 0

(5) Project [codegen id : 2]
Output [5]: [supplierGroup#41159, supplier#41160, plant#41161, deliveryDateTime#41162, subrange#41163]
Input [6]: [supplierGroup#41159, supplier#41160, plant#41161, deliveryDateTime#41162, subrange#41163, product#41164]

(6) HashAggregate [codegen id : 2]
Input [5]: [supplierGroup#41159, supplier#41160, plant#41161, deliveryDateTime#41162, subrange#41163]
Keys [5]: [plant#41161, supplierGroup#41159, subrange#41163, supplier#41160, deliveryDateTime#41162]
Functions: []
Aggregate Attributes: []
Results [5]: [plant#41161, supplierGroup#41159, subrange#41163, supplier#41160, deliveryDateTime#41162]

(7) Exchange
Input [5]: [plant#41161, supplierGroup#41159, subrange#41163, supplier#41160, deliveryDateTime#41162]
Arguments: hashpartitioning(plant#41161, supplierGroup#41159, subrange#41163, supplier#41160, deliveryDateTime#41162, 25), ENSURE_REQUIREMENTS, [plan_id=12211]

(8) ShuffleQueryStage
Output [5]: [plant#41161, supplierGroup#41159, subrange#41163, supplier#41160, deliveryDateTime#41162]
Arguments: 1

(9) AQEShuffleRead
Input [5]: [plant#41161, supplierGroup#41159, subrange#41163, supplier#41160, deliveryDateTime#41162]
Arguments: coalesced

(10) HashAggregate [codegen id : 3]
Input [5]: [plant#41161, supplierGroup#41159, subrange#41163, supplier#41160, deliveryDateTime#41162]
Keys [5]: [plant#41161, supplierGroup#41159, subrange#41163, supplier#41160, deliveryDateTime#41162]
Functions: []
Aggregate Attributes: []
Results [5]: [supplierGroup#41159, supplier#41160, plant#41161, deliveryDateTime#41162, subrange#41163]

(11) WriteFiles
Input [5]: [supplierGroup#41159, supplier#41160, plant#41161, deliveryDateTime#41162, subrange#41163]

(12) Execute InsertIntoHadoopFsRelationCommand
Input: []
Arguments: hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-order-qty-opt-service/out/supply-network-list/10000000096/shardId=0_1_10000000096, false, Parquet, [path=hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-order-qty-opt-service/out/supply-network-list/10000000096/shardId=0_1_10000000096], Overwrite, [supplierGroup, supplier, plant, deliveryDateTime, subrange]

(13) Exchange
Input [6]: [supplierGroup#41159, supplier#41160, plant#41161, deliveryDateTime#41162, subrange#41163, product#41164]
Arguments: hashpartitioning(product#41164, plant#41161, 25), REPARTITION_BY_NUM, [plan_id=12073]

(14) Project
Output [5]: [supplierGroup#41159, supplier#41160, plant#41161, deliveryDateTime#41162, subrange#41163]
Input [6]: [supplierGroup#41159, supplier#41160, plant#41161, deliveryDateTime#41162, subrange#41163, product#41164]

(15) HashAggregate
Input [5]: [supplierGroup#41159, supplier#41160, plant#41161, deliveryDateTime#41162, subrange#41163]
Keys [5]: [plant#41161, supplierGroup#41159, subrange#41163, supplier#41160, deliveryDateTime#41162]
Functions: []
Aggregate Attributes: []
Results [5]: [plant#41161, supplierGroup#41159, subrange#41163, supplier#41160, deliveryDateTime#41162]

(16) Exchange
Input [5]: [plant#41161, supplierGroup#41159, subrange#41163, supplier#41160, deliveryDateTime#41162]
Arguments: hashpartitioning(plant#41161, supplierGroup#41159, subrange#41163, supplier#41160, deliveryDateTime#41162, 25), ENSURE_REQUIREMENTS, [plan_id=12080]

(17) HashAggregate
Input [5]: [plant#41161, supplierGroup#41159, subrange#41163, supplier#41160, deliveryDateTime#41162]
Keys [5]: [plant#41161, supplierGroup#41159, subrange#41163, supplier#41160, deliveryDateTime#41162]
Functions: []
Aggregate Attributes: []
Results [5]: [supplierGroup#41159, supplier#41160, plant#41161, deliveryDateTime#41162, subrange#41163]

(18) WriteFiles
Input [5]: [supplierGroup#41159, supplier#41160, plant#41161, deliveryDateTime#41162, subrange#41163]

(19) Execute InsertIntoHadoopFsRelationCommand
Input: []
Arguments: hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-order-qty-opt-service/out/supply-network-list/10000000096/shardId=0_1_10000000096, false, Parquet, [path=hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-order-qty-opt-service/out/supply-network-list/10000000096/shardId=0_1_10000000096], Overwrite, [supplierGroup, supplier, plant, deliveryDateTime, subrange]

(20) AdaptiveSparkPlan
Output: []
Arguments: isFinalPlan=true

SQL / DataFrame Properties

Name	Value
spark.activity.hdlfsEnabled	true
spark.activity.isolateSession	true
spark.sql.parquet.fieldId.read.enabled	true
spark.sql.parquet.fieldId.write.enabled	true
temporal.activity.id	961dcc42-bffb-3b54-9460-e86e2549e19e
temporal.activity.type	DetermineSupplyNetworkList