For functions f(n) and g(n), we say that "f(n) is Big-O of g(n)" if

There exists a constant c > 0 there exists a constant n_{0}such that

for all n ≥ n_{0}, 0 ≤ f(n) ≤ c * g(n)

We write f(n) = O(g(n)), but the "=" is not the usual meaning.

The intention is to allow us to say

- 3*n
^{2}+ 14*n - 7 = O(n^{2}) - 0.0007 * n
^{3}is*not*O(n^{2})

Technically, 10^{1,000,000}*n is O(n), but programs with that
running time is still very slow.

Asymptotic running times for n items:

Insert | Search | Delete* | |
---|---|---|---|

Arrays | O(1) | O(n) | O(n)/O(1) |

Sorted Arrays | O(n) | O(log n) | O(n) |

Linked Lists | O(1) | O(n) | O(n)/O(1) |

Sorted Linked Lists | O(n) | O(n) | O(n)/O(1) |

*Running time for an unsorted linked list is O(1) if you are given the
index of the item to be deleted (replace with last item).
Running time for deletion from a linked list is O(1) if you
already have a pointer for the node to be deleted. Otherwise,
deletion is O(n).

Suppose we have n items inserted and do n searches:

- Arrays: n * O(1) + n * O(n) = O(n) + O(n
^{2}) = O(n^{2}) - Sorted Arrays: n * O(n) + n * O(log n) = O(n
^{2}) + O(n log n) = O(n^{2}) - Linked List: n * O(1) + n * O(n) = O(n) + O(n
^{2}) = O(n^{2}) - Sorted Linked List: n * O(n) + n * O(n) = O(n
^{2}) + O(n^{2}) = O(n^{2})

If we have all n items before hand, we can sort them all at the same time in O(n log n), plus n * O(log n) for n searches for O(n log n) time total.

If not, we need binary search trees.

For each node in a binary search tree, its left child (if any) holds
a smaller number and its right child (if any) holds a larger number.

There are several schemes for maintaining a O(log n) height for a binary search tree n nodes: AVL trees, Red-Black trees, B-trees, 2-3 trees, ...

For such trees:

- Insert: O(log n) time
- Find: O(log n) time
- Delete: O(log n) time

- Stores items in an array.
- Uses a
*hash function*to compute the array index of an item. - Ideally the size of the array is close to the number of items.
- A
*collision*occurs when the hash function assigns the same index to two items. -
*Open hashing*places items with the same index in a linked list. -
*Closed hashing*finds some other place in the array for the second item that has the same index.

Using a "good" hash function:

- Insert: O(1) time (average)
- Find: O(1) time (average)
- Delete: O(1) time (average)

In the worst case, all items have the same hash index and a closed hash table degenerates into an unsorted array.

There are provably good hash functions.