3.5.5
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
Structured Streaming
crp-order-qty-opt-service
application UI
Details for Stage 287 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
1 s
Locality Level Summary:
Node local: 23; Process local: 2
Shuffle Read Size / Records:
66.3 KiB / 1358
Associated Job Ids:
155
DAG Visualization
digraph G { subgraph clusterstage_287 { label="Stage 287"; subgraph cluster2054 { label="Exchange"; 775 [labelType="html" label="ShuffledRowRDD [775] [Unordered]<br>save at FileStorageAdapterImpl.java:290"]; } subgraph cluster2049 { label="WholeStageCodegen (7)"; 776 [labelType="html" label="MapPartitionsRDD [776] [Unordered]<br>save at FileStorageAdapterImpl.java:290"]; } subgraph cluster2071 { label="Exchange"; 777 [labelType="html" label="ShuffledRowRDD [777] [Unordered]<br>save at FileStorageAdapterImpl.java:290"]; } subgraph cluster2056 { label="WholeStageCodegen (8)"; 778 [labelType="html" label="MapPartitionsRDD [778] [Unordered]<br>save at FileStorageAdapterImpl.java:290"]; } subgraph cluster2045 { label="WholeStageCodegen (9)"; 779 [labelType="html" label="ZippedPartitionsRDD2 [779] [Unordered]<br>save at FileStorageAdapterImpl.java:290"]; 780 [labelType="html" label="MapPartitionsRDD [780] [Unordered]<br>save at FileStorageAdapterImpl.java:290"]; } subgraph cluster2044 { label="ObjectHashAggregate"; 781 [labelType="html" label="MapPartitionsRDD [781] [Unordered]<br>save at FileStorageAdapterImpl.java:290"]; } subgraph cluster2043 { label="ObjectHashAggregate"; 782 [labelType="html" label="MapPartitionsRDD [782] [Unordered]<br>save at FileStorageAdapterImpl.java:290"]; } subgraph cluster2072 { label="mapPartitionsInternal"; 783 [labelType="html" label="MapPartitionsRDD [783] [Unordered]<br>save at FileStorageAdapterImpl.java:290"]; } subgraph cluster2073 { label="mapPartitionsInternal"; 784 [labelType="html" label="AdaptiveSparkPlan isFinalPlan=false\n+- ObjectHashAggregate(keys=[orderProposalId#17266L], functions=[collect_list(struct(optGroupId, optGroupId#22161, minOrderProfile, minOrderProfile#22717, assignmentDetails, assignmentDetails#22719, minOrderGoal, minOrderGoal#22465), 0, 0)], output=[orderProposalId#17266L, optimizationGroups#22745])\n +- ObjectHashAggregate(keys=[orderProposalId#17266L], functions=[partial_collect_list(struct(optGroupId, optGroupId#22161, minOrderProfile, minOrderProfile#22717, assignmentDetails, assignmentDetails#22719, minOrderGoal, minOrderGoal#22465), 0, 0)], output=[orderProposalId#17266L, buf#25449])\n +- Project [orderProposalId#17266L, optGroupId#22161, minOrderProfile#22717, assignmentDetails#22719, minOrderGoal#22465]\n +- SortMergeJoin [orderProposalId#17266L], [orderProposalId#22518L], LeftOuter\n :- Sort [orderProposalId#17266L ASC NULLS FIRST], false, 0\n : +- HashAggregate(keys=[orderProposalId#17266L], functions=[], output=[orderProposalId... [784] [Cached] [Unordered]<br>save at FileStorageAdapterImpl.java:290"]; } } 775->776; 777->778; 776->779; 778->779; 779->780; 780->781; 781->782; 782->783; 783->784; }
760,775
765,777
784
Event Timeline
Enable zooming
Tasks: 25. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time