In this tutorial, we’ll present the Akra-Bazzi method. It’s a theorem we use to determine the complexity class of a divide-and-conquer algorithm.
2. The Complexity of Divide-and-Conquer Algorithms
Divide-and-conquer is an algorithm design technique that solves a problem by breaking it down into smaller sub-problems and combining their solutions. Each sub-problem is treated the same way: we divide it into smaller parts, solve them recursively, and unify the sub-solutions.
2.1. The Master Theorem
If the sub-problems are of the same size, we use the Master Theorem. Those are the ones where , the number of steps an algorithm makes while solving a problem of size , is a recurrence of the form:
where is a real-valued function. More specifically, the recurrence describes the complexity of an algorithm that divides a problem of size into sub-problems each of size . It performs steps to break down the problem and combine the solutions to its sub-problems.
3. The Akra-Bazzi Method
We can determine many algorithms’ complexities using the Master Theorem. For example, we can use it to show that Merge Sort has the log-linear worst-case complexity. Unfortunately, the Master Theorem isn’t always applicable.
Let’s say that the recurrence is:
That is, we perform steps to divide a problem of size into sub-problems of sizes and and combine their solutions. Since the sub-problems are uneven, we can’t use the Master Theorem.
3.1. The Akra-Bazzi Theorem
Instead, we use the more general Akra-Bazzi Theorem. It’s applicable to recurrences of the form:
where is bounded, and the following holds:
- is large enough so that is well-defined
- For each :
- the constant
- the constant
- is a constant
- is a non-negative polynomial-growth function
We say that satisfies the polynomial-growth condition if there exist such that for all , and we have . This condition ensures that doesn’t change too fast. It holds if is bounded by a polynomial.
If conditions 1-4 are satisfied and is the unique real solution to the equation:
then the Akra-Bazzi Theorem guarantees that:
We should note that it requires an additional technical condition to hold. The integral in Equation (3) should be finite:
3.2. The Akra-Bazzi Theorem in the Context of Analyzing Algorithms
The theorem holds for real numbers and general recurrences. But, in complexity analysis, we’ll deal with recurrent functions of a particular kind.
First of all, will apply to positive integers only, so in our applications. That’s because the input size is an integer.
The value of represents the number of steps the algorithm does to divide the input of size into smaller sub-problems and combine their solutions after solving them recursively, plus possibly some extra work on processing the input. Passing it to an algorithm’s implementation counts as 1 step at least, so will be strictly positive. For the same reasons, will always be positive.
3.3. Floor and Ceiling
Since is integer in analysis of algorithms, we should rather use or instead of . Luckily, we can ignore the ceiling and floor functions because they don’t affect asymptotic growth. That’s what the functions are for. To work around , we set and rewrite the term as:
That’s why we’ll usually formulate our recurrences without the s in practice. We’ll get the correct results nevertheless but work with a simpler formula:
3.4. Boundary Conditions
If we take a closer look at the theorem’s result, we’ll see that doesn’t appear there. That may seem somewhat counter-intuitive at first. After all, if we double all the values in the base cases, we expect the final value of to double for every . However, scaling the base cases only multiplies the function by a constant, and the multiplicative constants don’t affect the asymptotic behavior. In turn, this implies that the asymptotic behavior of an algorithm is independent of its base cases.
We also say that should be large enough so that is well defined. To see why, let’s consider . If , then we solve recursively:
which is a contradiction. So, must be large enough to accommodate all such cases.
3.5. On Calculating
Finally, from Equation (2) may not be easy to calculate precisely and by hand. If that’s the case, we can approximate its value numerically. For example, we can use Newton’s method.
However we compute or approximate , it always exists and is unique. The conditions on and ensure that the function is continuous with the range . By the intermediate value theorem, will always have a solution, and since the function is also strictly monotone, that solution will be unique.
3.6. The Akra-Bazzi Theorem for Polynomial
In some cases, we can derive the complexity directly without evaluating the definite integral. The method is a corollary of the Akra-Bazzi Theorem:
If we can bound , then
Luckily for us, the corollary applies to many algorithms. Most of the time, will be of the form:
where , the highest-order coefficient, will be strictly positive. To see why , let’s imagine that is negative. In that case, will become negative for a large enough , which is impossible, as it would mean we make a negative number of steps.
3.7. The and Versions
In general, if , where , we’ll have:
The and cases are of particular interest to us. If we can’t determine exactly but can bound it asymptotically from below () or above (), we can get the lower and upper bounds for .
Let’s work out an example:
We see that the recurrence satisfies all the conditions of the Akra-Bazzi Theorem. So, the first step is to find .
Since , , and , we should solve:
Since , the equation becomes:
There are two possible values for : and . We can discard the latter since it’s negative and must be positive. So, we get by solving:
From there, we get:
4.2. Applying the Akra-Bazzi Theorem
First, let’s calculate the integral:
Then, we get:
Would we get the same result if we used the shortcut for a polynomial ? Since , its degree is . As , we conclude that , which is the correct answer.
In this article, we presented the Akra-Bazzi method. We use it to determine the complexity divide-and-conquer algorithms for which the Master Theorem fails.