3.5.5
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
Structured Streaming
crp-order-qty-opt-service
application UI
Details for Stage 10 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
5 s
Locality Level Summary:
Node local: 15; Process local: 35
Shuffle Read Size / Records:
14.8 KiB / 24
Associated Job Ids:
8
DAG Visualization
digraph G { subgraph clusterstage_10 { label="Stage 10"; 60 [labelType="html" label="SQLExecutionRDD [60] [Unordered]<br>$anonfun$recordDeltaOperationInternal$1 at DatabricksLogging.scala:128"]; 64 [labelType="html" label="SQLExecutionRDD [64] [Unordered]<br>$anonfun$recordDeltaOperationInternal$1 at DatabricksLogging.scala:128"]; subgraph cluster155 { label="Exchange"; 55 [labelType="html" label="ShuffledRowRDD [55] [Unordered]<br>$anonfun$recordDeltaOperationInternal$1 at DatabricksLogging.scala:128"]; } subgraph cluster150 { label="WholeStageCodegen (3)"; 56 [labelType="html" label="MapPartitionsRDD [56] [Unordered]<br>$anonfun$recordDeltaOperationInternal$1 at DatabricksLogging.scala:128"]; } subgraph cluster149 { label="DeserializeToObject"; 57 [labelType="html" label="MapPartitionsRDD [57] [Unordered]<br>$anonfun$recordDeltaOperationInternal$1 at DatabricksLogging.scala:128"]; } subgraph cluster148 { label="MapPartitions"; 58 [labelType="html" label="MapPartitionsRDD [58] [Unordered]<br>$anonfun$recordDeltaOperationInternal$1 at DatabricksLogging.scala:128"]; } subgraph cluster145 { label="WholeStageCodegen (4)"; 59 [labelType="html" label="MapPartitionsRDD [59] [Unordered]<br>$anonfun$recordDeltaOperationInternal$1 at DatabricksLogging.scala:128"]; } subgraph cluster156 { label="map"; 61 [labelType="html" label="Delta Table State #143 - hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-sourceofsupply-sourceofsupply/_delta_log [61] [Cached] [Unordered]<br>$anonfun$recordDeltaOperationInternal$1 at DatabricksLogging.scala:128"]; } subgraph cluster158 { label="WholeStageCodegen (1)"; 62 [labelType="html" label="MapPartitionsRDD [62] [Unordered]<br>$anonfun$recordDeltaOperationInternal$1 at DatabricksLogging.scala:128"]; } subgraph cluster157 { label="Project"; 63 [labelType="html" label="MapPartitionsRDD [63] [Unordered]<br>$anonfun$recordDeltaOperationInternal$1 at DatabricksLogging.scala:128"]; } subgraph cluster165 { label="map"; 65 [labelType="html" label="Delta Table State with Stats #143 - hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-sourceofsupply-sourceofsupply/_delta_log [65] [Cached] [Unordered]<br>$anonfun$recordDeltaOperationInternal$1 at DatabricksLogging.scala:128"]; } subgraph cluster168 { label="Scan ExistingRDD Delta Table State with Stats #143 - hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-sourceofsupply-sourceofsupply/_delta_log"; 66 [labelType="html" label="MapPartitionsRDD [66] [Unordered]<br>$anonfun$recordDeltaOperationInternal$1 at DatabricksLogging.scala:128"]; } subgraph cluster167 { label="Filter"; 67 [labelType="html" label="MapPartitionsRDD [67] [Unordered]<br>$anonfun$recordDeltaOperationInternal$1 at DatabricksLogging.scala:128"]; } subgraph cluster166 { label="Project"; 68 [labelType="html" label="MapPartitionsRDD [68] [Unordered]<br>$anonfun$recordDeltaOperationInternal$1 at DatabricksLogging.scala:128"]; } subgraph cluster169 { label="mapPartitionsInternal"; 69 [labelType="html" label="MapPartitionsRDD [69] [Unordered]<br>$anonfun$recordDeltaOperationInternal$1 at DatabricksLogging.scala:128"]; } } 55->56; 56->57; 57->58; 58->59; 59->60; 60->61; 61->62; 62->63; 63->64; 64->65; 65->66; 66->67; 67->68; 68->69; }
54,55
61
65
Event Timeline
Enable zooming
Tasks: 50. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time