Integrals are Fun: Illustrated Riemann-Stieltjes Integral

Integrals are more than just the sum of its parts! Well, let's not exaggerate. In their most fundamental definition, they are only a sum of infinitely many rectangles under the curve of some function \( f \) over some interval \( [a, b] \) for \( a, b \in \mathbb{R} \). However, solving and numerically approximating them is lots of fun and besides that, they are quite useful in applied sciences.

In the previous blog dedicated to the integrals, we went through the most basic form of integration, namely the Riemann Integral. This time we take a step forward and learn about its more general extension, the Riemann-Stieltjes Integral, the precursor of the great Lebesgue Integral.

In the following few minutes of reading, first, we give intuitive reasoning why the Riemann Integral is limited and how to escape those restrictions. This paves the way to the formal definition of the Riemann-Stieltjes Integration. Taking the theoretical foundations for granted we provide a straightforward Python implementation. Finally, we make an illustration using Matplotlib to facilitate the understanding of this subject. Stay tuned!

Intuition: How to extend the Riemann Integral?

The Riemann Integral is quite straightforward. It allows us to take any real-valued function \( f \) and calculate the area of the surface bounded by the intersection of the same function \( f \) with the two vertical lines \( x = a \) and \( x = b \), for \(a, b \in \mathbb{R}\). We have to fit infinitely many rectangles (or right trapezoids) inside this body and sum their areas as depicted on the left in Fig. 1, something we already explained in the previous blog on Riemann Integration.

Fig. 1: Extending the Riemann Integral

But why being constrained on the interval \( [a, b] \) which gives all rectangles more or less the same width? Can we scale and shift this interval as shown on the right in Fig. 1, such that some rectangles would be less and adequately more important than others? In fact, this is the main limitation of the Riemann Integral, we have no option, but to use only the interval \( [a, b] \) without the possibility to assign different weights to different parts.

The point here is not to reinvent the wheel, probably Bernhard Riemann and Thomas Stieltjes had the same questions more than a century ago for which they came up with an excellent solution. The goal is to gain a common sense of how we can easily twist the problems we already solved to get new ones, thus opening new horizons.

All of this brings us to the basic principle of the Riemann-Stieltjes Integration, which we define formally in the next section.

Theoretical Foundations of the Riemann-Stieltjes Integration

Time to be more serious and formal now. Let \( f: [a, b] \rightarrow \mathbb{R} \) be a non-negative and continuous function over the interval \( [a, b] \). In order to fit infinitely many rectangles, we need to divide the input space into \( N \) sub-intervals for some \( N \in \mathbb{N} \). In other words, we need a partition of the interval \( [a, b] \) which is a sequence of numbers in the form:

Without loss of generality, we can assume that all sub-intervals are equidistant. As we already saw in the previous section, the surface under the curve \( f \) is not always bounded on the interval \( [a, b] \), i.e. we can shift it or scale it. This shift is formally defined as applying a real-to-real and monotone (it preserves the order) function \( g \) on the interval \( [a, b] \). The interval is now transformed into \( [g(a), g(b)] \), and respectively all members of the partition.

Fig. 2: Trapezoidal Rule to approximate the area under the curve

Having the partition set and transformed, it divides the space into infinitely many parts whose areas we need to sum. Thus, by using the Trapezoidal Rule, the Riemann-Stieltjes Integral is defined as:

Typically, the functions \( f \) and \( g \) are called the integrand and the integrator, because the integral of \( f \) is calculated with respect to \( g \).

Python Implementation

Once we have a neat equation, we can easily transcribe it to a Python implementation, as given in the code snippet below:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

def derivative(f, a, h=0.01):
    '''Approximates the derivative of the function f in a
    
    :param function f: function to differentiate
    :param float a: the point of differentiation
    :param float h: step size
    :return float: the derivative of f in a
    '''
    return (f(a + h) - f(a - h))/(2*h)


def stieltjes_integral(f, g, a, b, n):
    '''Calculates the Riemann-Stieltjes integral based on the composite trapezoidal rule
    relying on the Riemann Sums.
    
    :param function f: integrand function
    :param function f: integrator function 
    :param int a: lower bound of the integral
    :param int b: upper bound of theintergal
    :param int n: number of trapezoids of equal width
    :return float: the integral of the function f between a and b
    '''
    eps = 1e-9
    h = (b - a)/(n + eps)  # width of the rectangle
    dg = lambda x: derivative(g, x, h=1e-8)  # derivative of the integrator function
    result = 0.5*f(a)*dg(a) + sum([f(a + i*h)*dg(a + i*h) for i in range(1, n)]) + 0.5*f(b)*dg(b)
    result *= h
    return result

In order to fully understand the process of Riemann-Stieltjes Integration, we make an illustration using Matplotlib. For this purpose let's take some linear integrand function \( f \) and let the integrator function be \( g(x) = 3x \).

In the image below, the standard Riemann Integration is depicted with the blueish rectangles on the left. In this case, the actual width of the underlying rectangles is kept. To transform this to a Riemann-Stieltjes Integral we must plot the graph of the curve \( (x, y) = (g(x), f(x)) \), which is depicted with the greenish rectangles on the right. As we can see their widths are 3 times bigger since \( g(x) = 3x \), while still preserving the same height \( f(x) \).

Illustration: Transforming Riemann Integral to Riemann-Stieltjes Integral

The full source code related to all we have discussed during this blog can be found on GitHub. If you have any suggestions or remarks please let me know by commenting below. I would be happy to discuss this.

Applications of the Riemann-Stieltjes Integral

One might ask what is so special about this integral? Well, this mechanism is indispensable and lays down the foundations for many fields.

First of all, the so-called law of the unconscious statistician would not be possible. This theorem is used to calculate the expected value of the random variable obtained by applying some arbitrary function \( f \) on some other random variable \( X \), for which we only know its cumulative distribution function \( g(x) \). That means we don't have to know at all the distribution of \( f(X) \), for which the expected value is given by the Riemann-Stieltjes Integral:

More importantly, the Riemann-Stieltjes Integral is one of the cornerstones of the Stochastic Calculus. Specifically, the Itô Integral for Elementary Random Process \( X \) is defined as:

where \( W \) is a Brownian Motion. The details about this integral are intentionally left because it is a complex topic that could be covered in several blog posts.

If this is something you like and would like to receive similar posts, please subscribe to the mailing list below. For more information, please follow me on Twitter or LinkedIn.

Conclusion

Finally, we reached the end of this blog that covered in short the Riemann-Stieltjes Integral. It is a generalization of the Riemann Integral such that it provides means to transform the input space. We started with a general intuition and gradually continued towards a theoretical definition and illustrated examples for a complete overview. In the end, we saw the most important applications of this integral.

References

[1] Svein Linge, Hans Petter Langtangen, "Programming for Computations - Python" (2016), Springer Open
[2] Arturo Fernandez, “Brownian Motion and An Introduction to Stochastic Integration” (2011), Statistics 157: Topics In Stochastic Processes Seminar