3.5.5
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
Structured Streaming
crp-order-qty-opt-service
application UI
Details for Stage 298 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
0.2 s
Locality Level Summary:
Process local: 25
Input Size / Records:
3.0 KiB / 23
Associated Job Ids:
159
DAG Visualization
digraph G { subgraph clusterstage_298 { label="Stage 298"; subgraph cluster2054 { label="Exchange"; 775 [labelType="html" label="ShuffledRowRDD [775] [Unordered]<br>save at FileStorageAdapterImpl.java:290"]; } subgraph cluster2049 { label="WholeStageCodegen (7)"; 776 [labelType="html" label="MapPartitionsRDD [776] [Unordered]<br>save at FileStorageAdapterImpl.java:290"]; } subgraph cluster2071 { label="Exchange"; 777 [labelType="html" label="ShuffledRowRDD [777] [Unordered]<br>save at FileStorageAdapterImpl.java:290"]; } subgraph cluster2056 { label="WholeStageCodegen (8)"; 778 [labelType="html" label="MapPartitionsRDD [778] [Unordered]<br>save at FileStorageAdapterImpl.java:290"]; } subgraph cluster2045 { label="WholeStageCodegen (9)"; 779 [labelType="html" label="ZippedPartitionsRDD2 [779] [Unordered]<br>save at FileStorageAdapterImpl.java:290"]; 780 [labelType="html" label="MapPartitionsRDD [780] [Unordered]<br>save at FileStorageAdapterImpl.java:290"]; } subgraph cluster2044 { label="ObjectHashAggregate"; 781 [labelType="html" label="MapPartitionsRDD [781] [Unordered]<br>save at FileStorageAdapterImpl.java:290"]; } subgraph cluster2043 { label="ObjectHashAggregate"; 782 [labelType="html" label="MapPartitionsRDD [782] [Unordered]<br>save at FileStorageAdapterImpl.java:290"]; } subgraph cluster2072 { label="mapPartitionsInternal"; 783 [labelType="html" label="MapPartitionsRDD [783] [Unordered]<br>save at FileStorageAdapterImpl.java:290"]; } subgraph cluster2073 { label="mapPartitionsInternal"; 784 [labelType="html" label="AdaptiveSparkPlan isFinalPlan=false\n+- ObjectHashAggregate(keys=[orderProposalId#17266L], functions=[collect_list(struct(optGroupId, optGroupId#22161, minOrderProfile, minOrderProfile#22717, assignmentDetails, assignmentDetails#22719, minOrderGoal, minOrderGoal#22465), 0, 0)], output=[orderProposalId#17266L, optimizationGroups#22745])\n +- ObjectHashAggregate(keys=[orderProposalId#17266L], functions=[partial_collect_list(struct(optGroupId, optGroupId#22161, minOrderProfile, minOrderProfile#22717, assignmentDetails, assignmentDetails#22719, minOrderGoal, minOrderGoal#22465), 0, 0)], output=[orderProposalId#17266L, buf#25449])\n +- Project [orderProposalId#17266L, optGroupId#22161, minOrderProfile#22717, assignmentDetails#22719, minOrderGoal#22465]\n +- SortMergeJoin [orderProposalId#17266L], [orderProposalId#22518L], LeftOuter\n :- Sort [orderProposalId#17266L ASC NULLS FIRST], false, 0\n : +- HashAggregate(keys=[orderProposalId#17266L], functions=[], output=[orderProposalId... [784] [Cached] [Unordered]<br>save at FileStorageAdapterImpl.java:290"]; } subgraph cluster2171 { label="InMemoryTableScan"; 809 [labelType="html" label="MapPartitionsRDD [809] [Unordered]<br>save at FileStorageAdapterImpl.java:290"]; 810 [labelType="html" label="MapPartitionsRDD [810] [Unordered]<br>save at FileStorageAdapterImpl.java:290"]; 811 [labelType="html" label="MapPartitionsRDD [811] [Unordered]<br>save at FileStorageAdapterImpl.java:290"]; } subgraph cluster2162 { label="WholeStageCodegen (2)"; 812 [labelType="html" label="MapPartitionsRDD [812] [Unordered]<br>save at FileStorageAdapterImpl.java:290"]; } subgraph cluster2161 { label="Project"; 813 [labelType="html" label="MapPartitionsRDD [813] [Unordered]<br>save at FileStorageAdapterImpl.java:290"]; } subgraph cluster2160 { label="ObjectHashAggregate"; 814 [labelType="html" label="MapPartitionsRDD [814] [Unordered]<br>save at FileStorageAdapterImpl.java:290"]; } subgraph cluster2159 { label="Exchange"; 815 [labelType="html" label="MapPartitionsRDD [815] [Unordered]<br>save at FileStorageAdapterImpl.java:290"]; } } 775->776; 777->778; 776->779; 778->779; 779->780; 780->781; 781->782; 782->783; 783->784; 784->809; 809->810; 810->811; 811->812; 812->813; 813->814; 814->815; }
760,775
765,777
784
Event Timeline
Enable zooming
Tasks: 25. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time