Divide and Conquer
- General Strategy:
- Divide problem into smaller pieces
- Recursively solve smaller pieces
- Combine solutions of smaller pieces
- Two fast divide-and-conquer sorting algorithms:
- Merge Sort: O(n log n) worst case running time
- Quicksort: O(n log n) average case running time
Merge Sort
- Divide & Conquer strategy:
- "Break" array into two equal halves
- Recursively sort first half
- Recursively sort second half
- Merge two sorted halves into a single sorted array
- Recursion depth O(log n) ⇒ running time is O(n log n).
- MUCH faster than Bubble Sort, Insertion Sort & Selection
Sort which take O(n^{2}) time.
Quicksort
- Choose a pivot item x (often the first item)
- Partition array into two parts:
- First part: all items ≤ x
- Second part: all items > x
- Recursively sort first part
- Recursively sort second part
- No combining step
Quicksort Running Time
- IF the partition step breaks the array in half,
we have O(log n) levels of recursion and O(n log n) running time.
- Worst case running time is O(n^{2}).
For example, when the array is already sorted each
partition divides the array into parts with size 1 and n-2.
- We don't need to break up the array exactly in half.
Even a partition into parts with size ratio of 1:10 will
give O(log n) depth.
- Pick pivot element randomly.
Then probability that we have a 1:10 or 10:1 split is ~9/11.
Randomized Quicksort Running Time
- Cheating Method:
- Randomly pick a pivot element.
- Partition.
- If partition doesn't give a 1:3 or 3:1 or better split,
try again with another randomly chosen pivot element.
- Recursively randomly Quicksort the parts.
- Probability of getting a 1:3 or 3:1 or better split is 1/2.
- Expected number of times we have to pick a pivot element is 2.
- Recursion depth ≤ 2 log n.
Running Times
n = |
20k |
40k |
80k |
160k |
320k |
1,280k |
2,560k |
Insertion |
3s |
12s |
54s |
4.2m |
19.5m |
> 5h? |
> 20h? |
Selection |
6s |
25s |
1.9m |
9m |
38m |
> 10h? |
> 40h? |
QuickSort |
0.03s |
0.05s |
0.11s |
0.23s |
0.49s |
2.10s |
4.34s |
MergeSort |
0.05s |
0.11s |
0.22s |
0.46s |
0.95s |
4.12s |
8.57s |
s = seconds, m = minutes, h = hours
red numbers are estimates
Tweaking MergeSort
We can change MergeSort so it calls Insertion Sort, instead of recursive
calls to itself, for small arrays.
These running times were collected for MergeSort on 1,000,000 items
with different switch over points. All times in seconds.
switchover = |
5 |
10 |
15 |
20 |
30 |
50 |
100 |
500 |
MergeSort |
2.319 |
2.144 |
2.084 |
2.074 |
2.081 |
2.083 |
2.222 |
5.171 |