Today’s lecture introduces a completely new topic: the Lovász Local Lemma (LLL). This is an important method for analyzing events that are not independent, but have some restricted sort of dependencies. It is not as widely applicable as many of the other the techniques we have seen so far, but from time to time one encounters scenarios in which the LLL is the only technique that works.
1. The Lovász Local Lemma
Suppose are a collection of “bad” events. We would like to show that there is positive probability that none of them occur. If the events are mutually independent then this is simple:
(assuming that for every ). The LLL is a method for proving that when the ‘s are not mutually independent, but they can have some sort of limited dependencies.
Formally, we say that an event does not depend on the events if
So, regardless of whether some of the events in occur, the probability of occurring is unaffected.
Theorem 1 (The “Symmetric” LLL) Let be events with for all . Suppose that every event does not depend on at least other events. If then .
We will not prove this theorem. Instead, we will illustrate the LLL by considering a concrete application of it in showing satisfiability of -CNF Boolean formulas. Recall that a -CNF formula is a Boolean formula, involving any finite number of variables, where the formula is a conjunction (“and”) of any number of clauses, each of which is a disjunction (“or”) of exactly distinct literals (a variable or its negation).
For example, here is a -CNF formula with three variables and eight clauses.
This formula is obviously unsatisfiable. One can easily generalize this construction to get an unsatisfiable -CNF formula with variables and clauses. Our next theorem says: the reason this formula is unsatisfiable is that we allowed each variable to appear in too many clauses.
Theorem 2 There is a universal constant such that the following is true. Let be a -CNF formula where each variable appears in at most clauses. Then is satisfiable. Moreover, there is a randomized, polynomial time algorithm to find a satisfying assignment.
The theorem is stronger when is small. The proof that we will present can be optimized to get . By applying the full-blown LLL one can achieve .
Let be the number of variables and be the number of clauses in . Each clause contains variables, each of which can appear in only other clauses. So each clause shares a variable with less than other clauses.
The algorithm proving the theorem is perhaps the most natural algorithm that one could imagine. However it took more than 30 years from the introduction of the LLL for this algorithm to be provably analyzed.
- Set each variable in to either or randomly and independently.
- While there is an unsatisfied clause
- Set each variable in to either or randomly and independently.
- While there is an unsatisfied clause sharing some variable with (possibly )
Claim 3 Suppose every call to Fix terminates. Then Solve calls Fix at most times, and terminates with a satisfying assignment.
Proof: For any call to Fix, we claim that every clause that was satisfied before the call is still satisfied after the call completes. This follows by induction, starting at the deepest level of recursion. So, for every call from Solve to Fix() the number of satisfied clauses increases by one, since must now be satisfied when Fix() terminates.
So it remains to show that, with high probability, every call to Fix terminates.
Theorem 4 Let where is a sufficiently large constant. Then the probability that the algorithm makes more than calls to (including both the top-level and recursive calls) is at most .
The proof proceeds by considering the interactions between two agents: the “CPU” and the “Debugger”. The CPU runs the algorithm, periodically sending messages to the Debugger (we describe these messages in more detail below). However, if Fix gets called more than times the CPU interrupts the execution and halts the algorithm.
The CPU needs bits of randomness to generate the initial assignment in Solve, and needs bits to regenerate variables in each call to Fix. Since the CPU will not execute Fix more than times, it might as well generate all its random bits at the very start of the algorithm. So the first step performed by the CPU is to generate a random bitstring of length to provide all the randomness used in executing the algorithm.
The messages sent from the CPU to the Debugger are as follows.
- Every time the CPU runs Fix(), he sends a message containing the identity of the clause , and an extra bit indicating whether this is a top-level Fix (i.e., a call from Solve) or a recursive Fix.
- Every time Fix() finishes the CPU sends a message stating “recursive call finished”.
- If Fix gets called times, the CPU sends a message to the Debugger containing the current assignment of all variables.
Because the Debugger is notified when every call to Fix starts or finishes, he always knows which clause is currently being processed by Fix. A crucial detail is to figure out how many bits of communication are required to send these messages.
- For a top-level Fix, bits suffice because there are only clauses in .
- For a recursive Fix, bits suffice because the Debugger already knows what clause is currently being fixed, and that clause shares variables with only other clauses, so only possible clauses could be passed to the next call to Fix.
- When each call to Fix() finishes, the corresponding message takes bits.
- When Fix gets called times, the corresponding message takes bits.
The main point of the proof is to show that, if Fix gets called times, then these messages reveal the random string to the Debugger.
Since each clause is a disjunction (an “or” of literals), there is exactly one assignment to those variables that does not satisfy the clause. So, whenever the CPU tells the Debugger that he is calling Fix(), the Debugger knows exactly what the current assignment to is. So, starting from the assignment that the Debugger received in the final message, he can work backwards and figure out what the previous assignment was before calling Fix. Repeating this process, he can figure out how the variables were set in each call to Fix, and also what the initial assignment was. Thus the Debugger can reconstruct the random string .
The total number of bits sent by the CPU are
- bits for all the messages sent when Solve calls Fix.
- for all the messages sent in the recursive calls.
- bits to send the final assignment.
So has been compressed from bits to
This is an overall shrinking of
bits, assuming that and are sufficiently big constants.
We have argued that, if Fix gets called times, then can be compressed by bits. The next claim argues that this happens with probability at most .
Claim 5 The probability that can be compressed by bits is at most .
Proof: Consider any deterministic algorithm for encoding all bit strings of length into bit strings of arbitrary length. The number of bit strings that are encoded into bits is at most . So, a random bit string has probability of being encoded into bits. (One can view this as a simple special case of the Kraft inequality.)