Randomized Algorithms | Set 1 (Introduction and Analysis)

What is a Randomized Algorithm?

An algorithm that uses random numbers to decide what to do next anywhere in its logic is called a Randomized Algorithm. For example, in Randomized Quick Sort, we use a random number to pick the next pivot (or we randomly shuffle the array). And in Karger’s algorithm, we randomly pick an edge. 

How to analyse Randomized Algorithms?

Some randomized algorithms have deterministic time complexity. For example, this implementation of Karger’s algorithm has time complexity is O(E). Such algorithms are called Monte Carlo Algorithms and are easier to analyse for worst case. 
On the other hand, time complexity of other randomized algorithms (other than Las Vegas) is dependent on value of random variable. Such Randomized algorithms are called Las Vegas Algorithms. These algorithms are typically analysed for expected worst case. To compute expected time taken in worst case, all possible values of the used random variable needs to be considered in worst case and time taken by every possible value needs to be evaluated. Average of all evaluated times is the expected worst case time complexity. Below facts are generally helpful in analysis os such algorithms. 
Linearity of Expectation
Expected Number of Trials until Success.
For example consider below a randomized version of QuickSort. 
A Central Pivot is a pivot that divides the array in such a way that one side has at-least 1/4 elements. 

// Sorts an array arr[low..high]
randQuickSort(arr[], low, high)

1. If low >= high, then EXIT.

2. While pivot 'x' is not a Central Pivot.
  (i)   Choose uniformly at random a number from [low..high]. 
        Let the randomly picked number number be x.
  (ii)  Count elements in arr[low..high] that are smaller 
        than arr[x]. Let this count be sc.
  (iii) Count elements in arr[low..high] that are greater 
        than arr[x]. Let this count be gc.
  (iv)  Let n = (high-low+1). If sc >= n/4 and
        gc >= n/4, then x is a central pivot.

3. Partition arr[low..high] around the pivot x.

4. // Recur for smaller elements
   randQuickSort(arr, low, sc-1) 

5. // Recur for greater elements
   randQuickSort(arr, high-gc+1, high) 

The important thing in our analysis is, time taken by step 2 is O(n). 
How many times while loop runs before finding a central pivot?
The probability that the randomly chosen element is central pivot is 1/n. 
Therefore, expected number of times the while loop runs is n (See this for details) 
Thus, the expected time complexity of step 2 is O(n). 
What is overall Time Complexity in Worst Case?
In worst case, each partition divides array such that one side has n/4 elements and other side has 3n/4 elements. The worst case height of recursion tree is Log _3/4 n which is O(Log n). 

T(n) < T(n/4) + T(3n/4) + O(n)
T(n) < 2T(3n/4) + O(n)

Solution of above recurrence is O(n Log n) 

Note that the above randomized algorithm is not the best way to implement randomized Quick Sort. The idea here is to simplify the analysis as it is simple to analyse. 
Typically, randomized Quick Sort is implemented by randomly picking a pivot (no loop). Or by shuffling array elements. Expected worst case time complexity of this algorithm is also O(n Log n), but analysis is complex, the MIT prof himself mentions same in his lecture here. 
Example : 

C

#include <stdio.h>
#include <stdlib.h>
#include <time.h>

int find_solution(int n) {
  // seed the random number generator with the current time
  srand(time(0));
  // randomly select a number between 1 and n and return it as the solution
  return rand() % n + 1;
}

int main() {
  int n = 10;  // the range of possible solutions is 1 to n
  printf("Solution: %d\n", find_solution(n));
  return 0;
}

C++

#include <iostream>
#include <stdlib.h>
#include <time.h>

int find_solution(int n) {
  // seed the random number generator with the current time
  srand(time(0));
  // randomly select a number between 1 and n and return it as the solution
  return rand() % n + 1;
}

int main() {
  int n = 10;  // the range of possible solutions is 1 to n
  std::cout << "Solution: " << find_solution(n) << std::endl;
  return 0;
}

Java

import java.util.Random;

public class Solution {
  // method to find a random solution within a given range
  public static int findSolution(int n) {
    // create a Random object to generate random numbers
    Random rand = new Random();
    // use the nextInt method to generate a random number between 0 and n-1
    int solution = rand.nextInt(n) + 1;
    return solution;
  }

  public static void main(String[] args) {
    int n = 10; // the range of possible solutions is 1 to n
    System.out.println("Solution: " + findSolution(n));
  }
}

Python3

import random
import time

def find_solution(n):
  # seed the random number generator with the current time
  random.seed(time.time())
  # randomly select a number between 1 and n and return it as the solution
  return random.randint(1, n)

def main():
  n = 10  # the range of possible solutions is 1 to n
  print("Solution:", find_solution(n))

if __name__ == '__main__':
  main()

Javascript

function findSolution(n) {

  // seed the random number generator with the current time
  const random = Math.random() * (new Date().getTime());

  // randomly select a number between 1 and n and return it as the solution
  return Math.floor((random % n) + 1);
}

function main() {
  const n = 10; // the range of possible solutions is 1 to n
  console.log(`Solution: ${findSolution(n)}`);
}

main();

C#

// C# equivalent
using System;

public class Solution
{
  // method to find a random solution within a given range
  public static int findSolution(int n)
  {
    // create a Random object to generate random numbers
    Random rand = new Random();
    // use the Next method to generate a random number between 0 and n-1
    int solution = rand.Next(n) + 1;
    return solution;
  }

  public static void Main(string[] args)
  {
    int n = 10; // the range of possible solutions is 1 to n
    Console.WriteLine("Solution: " + findSolution(n));
  }
}

Output

Solution: 10

Randomized Algorithms | Set 2 (Classification and Applications)
References:
http://www.tcs.tifr.res.in/~workshop/nitrkl_igga/randomized-lecture.pdf
This article is contributed by Ashish Sharma. Please write comments if you find anything incorrect, or you want to share more information about the topic discussed above