This article by Max Slater explores the powerful concept of representing functions as infinite-dimensional vectors, which allows the tools of linear algebra to be applied to problems in computer graphics, signal processing, and machine learning. The central thesis is that by formalizing this analogy, complex operations on functions can be simplified through techniques like diagonalization.

Functions as Infinite-Dimensional Vectors

The foundation of this concept lies in reinterpreting what a vector is. A standard N-dimensional vector can be seen as a map from a finite set of indices (e.g., {1, 2, ..., N}) to a set of values. By extending this idea, a function defined on the natural numbers, like a sequence, can be viewed as a vector with countably infinite dimensions. The crucial leap is to consider functions defined on the real numbers, which correspond to vectors with an uncountably infinite number of dimensions, where each real number serves as an index. This perspective, rigorously defined in functional analysis, allows for a powerful intuitive bridge from finite-dimensional linear algebra.

Formalizing the Analogy: Vector Spaces and Linear Operators

To treat functions as vectors formally, they must be shown to form a vector space. For the set of real-valued functions, this is achieved by defining vector addition and scalar multiplication in a pointwise manner:

• Vector Addition: (f + g)[x] = f[x] + g[x]

• Scalar Multiplication: (αf)[x] = αf[x]The zero vector is the function that is zero everywhere. These definitions satisfy all the necessary vector space axioms (commutativity, associativity, etc.), confirming that functions can indeed be treated as vectors.

Just as matrices are linear transformations on finite vectors, linear operators are linear transformations on functions. A prime example is the differentiation operator, d/dx, which is linear because the derivative of a linear combination is the linear combination of the derivatives. By considering a specific basis, such as the power basis (1, x, x², ...) for the space of polynomials, differentiation can be represented as an infinite-dimensional matrix that transforms the vector of a polynomial's coefficients.

Diagonalization and Eigenfunctions

A core technique in linear algebra is diagonalization, where a matrix A is decomposed into UΛU⁻¹. The columns of U are the eigenvectors of A—vectors that are only scaled by the transformation (Av = λv). The diagonal matrix Λ contains the corresponding scaling factors, or eigenvalues.

This concept extends to functions, where an eigenfunction of a linear operator L is a function f such that Lf = ψf, where ψ is the eigenvalue. The article attempts to diagonalize the differentiation operator by finding its eigenfunctions. Solving the differential equation df/dx = ψf yields exponential functions of the form Ce^(ψx). However, the differentiation operator cannot be fully diagonalized in the space of real functions because its eigenfunctions (real exponentials) do not form a basis capable of representing all analytic functions (e.g., polynomials like f[x] = x).

Inner Products and the Spectral Theorem

To find a more useful diagonalization, the concept of an inner product is introduced, which endows the vector space with geometric notions of length and orthogonality. The Euclidean dot product is generalized for functions via integration: ⟨f, g⟩ = ∫f[x]g[x] dx.

This leads to the Spectral Theorem, a cornerstone result which states that symmetric matrices (where A = Aᵀ) can be diagonalized using an orthonormal basis of eigenvectors. In the realm of functions, the equivalent of a symmetric matrix is a self-adjoint operator (L = L*). The Spectral Theorem guarantees that such operators admit an orthonormal eigenbasis, making them cleanly diagonalizable.

The Laplacian Operator and the Fourier Transform

While differentiation is not self-adjoint, the Laplacian operator (Δ = d²/dx²) is, provided the functions satisfy certain boundary conditions (such as being periodic on the integration domain). The eigenfunctions of the Laplacian are sines, cosines, and, more compactly, complex exponentials (e^(iψx)).

By selecting eigenfunctions that are periodic on a given interval (e.g., e^(2πξix) for integer ξ on [0,1]), one can construct an orthonormal basis. The process of changing a function from its standard representation into this eigenbasis is precisely the Fourier Transform. The inverse transform reconstructs the original function from its basis components. Therefore, the Fourier transform is fundamentally a change of basis that diagonalizes the Laplacian operator.

Applications

This framework has profound practical applications, as diagonalizing the Laplacian provides a natural "frequency" decomposition for functions on various domains.

• Fourier Series and Image Compression: The 1D Fourier series decomposes a periodic function into a sum of sine and cosine waves. This allows for operations like low-pass filtering by simply discarding high-frequency coefficients. The concept extends to 2D, where the Laplacian's eigenfunctions are 2D waves. This 2D Fourier transform is a core component of compression algorithms like JPEG, which store images efficiently by representing them with a small number of basis function coefficients.

• Geometry Processing: The Laplacian can be defined on more complex domains. On the surface of a sphere, its eigenfunctions are the Spherical Harmonics, which are widely used in computer graphics to compress lighting information (environment maps). Furthermore, a discrete version of the Laplacian can be defined for 3D meshes. Its eigenfunctions provide a natural basis for functions on the mesh, enabling algorithms for smoothing, feature detection, and compressing geometric data.