3.5.5
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
Structured Streaming
crp-order-qty-opt-service
application UI
Details for Stage 291 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
7 s
Locality Level Summary:
Process local: 25
Input Size / Records:
3.0 KiB / 23
Output Size / Records:
78.9 KiB / 59
Associated Job Ids:
156
DAG Visualization
digraph G { subgraph clusterstage_291 { label="Stage 291"; subgraph cluster2054 { label="Exchange"; 775 [labelType="html" label="ShuffledRowRDD [775] [Unordered]<br>save at FileStorageAdapterImpl.java:290"]; } subgraph cluster2049 { label="WholeStageCodegen (7)"; 776 [labelType="html" label="MapPartitionsRDD [776] [Unordered]<br>save at FileStorageAdapterImpl.java:290"]; } subgraph cluster2071 { label="Exchange"; 777 [labelType="html" label="ShuffledRowRDD [777] [Unordered]<br>save at FileStorageAdapterImpl.java:290"]; } subgraph cluster2056 { label="WholeStageCodegen (8)"; 778 [labelType="html" label="MapPartitionsRDD [778] [Unordered]<br>save at FileStorageAdapterImpl.java:290"]; } subgraph cluster2045 { label="WholeStageCodegen (9)"; 779 [labelType="html" label="ZippedPartitionsRDD2 [779] [Unordered]<br>save at FileStorageAdapterImpl.java:290"]; 780 [labelType="html" label="MapPartitionsRDD [780] [Unordered]<br>save at FileStorageAdapterImpl.java:290"]; } subgraph cluster2044 { label="ObjectHashAggregate"; 781 [labelType="html" label="MapPartitionsRDD [781] [Unordered]<br>save at FileStorageAdapterImpl.java:290"]; } subgraph cluster2043 { label="ObjectHashAggregate"; 782 [labelType="html" label="MapPartitionsRDD [782] [Unordered]<br>save at FileStorageAdapterImpl.java:290"]; } subgraph cluster2072 { label="mapPartitionsInternal"; 783 [labelType="html" label="MapPartitionsRDD [783] [Unordered]<br>save at FileStorageAdapterImpl.java:290"]; } subgraph cluster2073 { label="mapPartitionsInternal"; 784 [labelType="html" label="AdaptiveSparkPlan isFinalPlan=false\n+- ObjectHashAggregate(keys=[orderProposalId#17266L], functions=[collect_list(struct(optGroupId, optGroupId#22161, minOrderProfile, minOrderProfile#22717, assignmentDetails, assignmentDetails#22719, minOrderGoal, minOrderGoal#22465), 0, 0)], output=[orderProposalId#17266L, optimizationGroups#22745])\n +- ObjectHashAggregate(keys=[orderProposalId#17266L], functions=[partial_collect_list(struct(optGroupId, optGroupId#22161, minOrderProfile, minOrderProfile#22717, assignmentDetails, assignmentDetails#22719, minOrderGoal, minOrderGoal#22465), 0, 0)], output=[orderProposalId#17266L, buf#25449])\n +- Project [orderProposalId#17266L, optGroupId#22161, minOrderProfile#22717, assignmentDetails#22719, minOrderGoal#22465]\n +- SortMergeJoin [orderProposalId#17266L], [orderProposalId#22518L], LeftOuter\n :- Sort [orderProposalId#17266L ASC NULLS FIRST], false, 0\n : +- HashAggregate(keys=[orderProposalId#17266L], functions=[], output=[orderProposalId... [784] [Cached] [Unordered]<br>save at FileStorageAdapterImpl.java:290"]; } subgraph cluster2076 { label="InMemoryTableScan"; 785 [labelType="html" label="MapPartitionsRDD [785] [Unordered]<br>save at FileStorageAdapterImpl.java:290"]; 786 [labelType="html" label="MapPartitionsRDD [786] [Unordered]<br>save at FileStorageAdapterImpl.java:290"]; 787 [labelType="html" label="MapPartitionsRDD [787] [Unordered]<br>save at FileStorageAdapterImpl.java:290"]; } subgraph cluster2074 { label="WriteFiles"; 788 [labelType="html" label="MapPartitionsRDD [788] [Unordered]<br>save at FileStorageAdapterImpl.java:290"]; } } 775->776; 777->778; 776->779; 778->779; 779->780; 780->781; 781->782; 782->783; 783->784; 784->785; 785->786; 786->787; 787->788; }
760,775
765,777
784
Event Timeline
Enable zooming
Tasks: 25. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time