3.5.5
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
Structured Streaming
crp-order-qty-opt-service
application UI
Details for Stage 609 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
0.2 s
Locality Level Summary:
Process local: 25
Input Size / Records:
3.0 KiB / 23
Associated Job Ids:
325
DAG Visualization
digraph G { subgraph clusterstage_609 { label="Stage 609"; subgraph cluster4289 { label="Exchange"; 1617 [labelType="html" label="ShuffledRowRDD [1617] [Unordered]<br>save at FileStorageAdapterImpl.java:290"]; } subgraph cluster4284 { label="WholeStageCodegen (7)"; 1618 [labelType="html" label="MapPartitionsRDD [1618] [Unordered]<br>save at FileStorageAdapterImpl.java:290"]; } subgraph cluster4306 { label="Exchange"; 1619 [labelType="html" label="ShuffledRowRDD [1619] [Unordered]<br>save at FileStorageAdapterImpl.java:290"]; } subgraph cluster4291 { label="WholeStageCodegen (8)"; 1620 [labelType="html" label="MapPartitionsRDD [1620] [Unordered]<br>save at FileStorageAdapterImpl.java:290"]; } subgraph cluster4280 { label="WholeStageCodegen (9)"; 1621 [labelType="html" label="ZippedPartitionsRDD2 [1621] [Unordered]<br>save at FileStorageAdapterImpl.java:290"]; 1622 [labelType="html" label="MapPartitionsRDD [1622] [Unordered]<br>save at FileStorageAdapterImpl.java:290"]; } subgraph cluster4279 { label="ObjectHashAggregate"; 1623 [labelType="html" label="MapPartitionsRDD [1623] [Unordered]<br>save at FileStorageAdapterImpl.java:290"]; } subgraph cluster4278 { label="ObjectHashAggregate"; 1624 [labelType="html" label="MapPartitionsRDD [1624] [Unordered]<br>save at FileStorageAdapterImpl.java:290"]; } subgraph cluster4307 { label="mapPartitionsInternal"; 1625 [labelType="html" label="MapPartitionsRDD [1625] [Unordered]<br>save at FileStorageAdapterImpl.java:290"]; } subgraph cluster4308 { label="mapPartitionsInternal"; 1626 [labelType="html" label="AdaptiveSparkPlan isFinalPlan=false\n+- ObjectHashAggregate(keys=[orderProposalId#46726L], functions=[collect_list(struct(optGroupId, optGroupId#51603, minOrderProfile, minOrderProfile#52159, assignmentDetails, assignmentDetails#52161, minOrderGoal, minOrderGoal#51907), 0, 0)], output=[orderProposalId#46726L, optimizationGroups#52187])\n +- ObjectHashAggregate(keys=[orderProposalId#46726L], functions=[partial_collect_list(struct(optGroupId, optGroupId#51603, minOrderProfile, minOrderProfile#52159, assignmentDetails, assignmentDetails#52161, minOrderGoal, minOrderGoal#51907), 0, 0)], output=[orderProposalId#46726L, buf#54891])\n +- Project [orderProposalId#46726L, optGroupId#51603, minOrderProfile#52159, assignmentDetails#52161, minOrderGoal#51907]\n +- SortMergeJoin [orderProposalId#46726L], [orderProposalId#51960L], LeftOuter\n :- Sort [orderProposalId#46726L ASC NULLS FIRST], false, 0\n : +- HashAggregate(keys=[orderProposalId#46726L], functions=[], output=[orderProposalId... [1626] [Cached] [Unordered]<br>save at FileStorageAdapterImpl.java:290"]; } subgraph cluster4406 { label="InMemoryTableScan"; 1651 [labelType="html" label="MapPartitionsRDD [1651] [Unordered]<br>save at FileStorageAdapterImpl.java:290"]; 1652 [labelType="html" label="MapPartitionsRDD [1652] [Unordered]<br>save at FileStorageAdapterImpl.java:290"]; 1653 [labelType="html" label="MapPartitionsRDD [1653] [Unordered]<br>save at FileStorageAdapterImpl.java:290"]; } subgraph cluster4397 { label="WholeStageCodegen (2)"; 1654 [labelType="html" label="MapPartitionsRDD [1654] [Unordered]<br>save at FileStorageAdapterImpl.java:290"]; } subgraph cluster4396 { label="Project"; 1655 [labelType="html" label="MapPartitionsRDD [1655] [Unordered]<br>save at FileStorageAdapterImpl.java:290"]; } subgraph cluster4395 { label="ObjectHashAggregate"; 1656 [labelType="html" label="MapPartitionsRDD [1656] [Unordered]<br>save at FileStorageAdapterImpl.java:290"]; } subgraph cluster4394 { label="Exchange"; 1657 [labelType="html" label="MapPartitionsRDD [1657] [Unordered]<br>save at FileStorageAdapterImpl.java:290"]; } } 1617->1618; 1619->1620; 1618->1621; 1620->1621; 1621->1622; 1622->1623; 1623->1624; 1624->1625; 1625->1626; 1626->1651; 1651->1652; 1652->1653; 1653->1654; 1654->1655; 1655->1656; 1656->1657; }
1602,1617
1607,1619
1626
Event Timeline
Enable zooming
Tasks: 25. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time