1 Oct 2013 - 
Greedy Algorithms
Spanning Trees, Disjoint Sets, Path Compression in Union Find

Greedy algorithms: "myopic" - only looking nearby
- repeatedly pick the next part of the solution that looks best
- not in GENERAL an optimal strategy
	eg: not good for shortest paths
- BUT: for many problems it is optimal.
	
Minimum spanning trees
- example: finding least expensive way to network a set of computers
[picture]









What is a possible solution?  [cost should be 16]











Property of a solution:
- Cannot contain cycles -- because removing a cycle edge cannot disconnect a graph

Krustal's algorithm:
- Repeatedly add next lightest edge that does not create a cycle.
// NOTE: theh partial solution is not necessarily connected until the end! //

Run example 




WHY does Krustal's algorithm find an optimal solution?

Def: CUT: partition of a vertices into two groups, S and V-S:

Cut property:
Suppose S is part of a MST T.  Then a lowest cost edge e connecting S to V-S is part of SOME MST T'
case 1: T=T'. Done!
case 2: T =/= T'.  Must be some edge e' that connects S and V-S.  Replace e' with e.  
	Since cost(e) <= cost(e'), cost(T') <= cost(T) so T' is a MST as well.

The Cut Property is the inductive step in an inductive proof on |S| of correctness of Krustal's algorithm.
Base case: |S|=1, so S contains one node v and no edges.  By def it is part of a MST.
Inductive case: Cut property.

How to implement Krustal's algorithm efficiently?

Let's rephrase the algorithm as follow:

Krustal, Take 2:
Start with every vertice in G in its own component (no edges).
Repeat:
	Select lightest edge that connects two unconnected components
	Merge the two components

Thus the state of the algorithm at any point is a collection of DISJOINT SETS.
The operations we need to perform are UNION and FIND.

procedure Krustal(G,w)
	for all u in V: makeset(u)
	S = {}
	sort edges E by weight
	for all edges (u,v) in E in increasing order of weight:
		if find(u) =/= find(v) then	
			add edge (u,v) to S
			union(u,v)
	return S
end

Have you seen a data structure before for disjoint sets?

UP TREES:
Tree where children point to parents.
Examples:







	
procedure makeset(x)
	parent(x) = x
	rank(x) = 0 // will be height of tree rooted at x
end

function find(x)
	while (x != parent(x)):	 x = parent(x)
	return x
end

union: idea: make root shorter tree point to root of taller tree.

procedure union(x,y)
	rx = find(x)
	ry = find(y)
	if rx = ry then return
	if rank(rx) > rank(ry) then parent(ry) = rx
	else if rank(ry) > rank(rx) then parent(rx) = ry
	else // they have same rank, so will increase	
		parent(ry) = x
		rank(x) = rank(x) + 1
end

Example:























Properties:
- rank(x) < rank(parent(x)) for all x
- any root of rank k has at LEAST 2^k nodes in its tree
- if there are n elements, there can be at most n/2^k nodes of rank k
all imply: maximum rank is log n  
	THEREFORE 
find(x) = O(log n)
union(x) = O(log n)

Overall running time:
makeset(x) = O(1), done |V| times, so O(|V|)
sorting edges: O(|E|log|E|) = O(|E|log|V|)
	why?  in worst case, |E| = |V|^2
		log|E| = log|V|^2 = 2 log|V|
Two finds and a union repeated O(|E|) times
	O(|E|log|V|)

So overall: O(|E|log|V|)


Improving FIND in union/find: Path Compression

function find(x)
	if x =/= parent(x) then parent(x) = find(parent(x))
	return parent(x)
end

Claim: Although the worst case for ONE find is unchanged, TOTAL time for 
|V| finds is O(|V| log*(|V|))
	where log*(n) = "inverse Ackerman's function"
		= number of times you need to take the log of
		the argument to get a value <= 1
		<= 5 in practice
So can do a find in O(1) amortized time.

If edges come in pre-sorted, then what is running time?
O(|E|)