Matrices that appear in computational mathematics are so often of low rank. Since random ("average") matrices are almost surely of full rank, mathematics needs to explain the abundance of low rank structures. We will give a characterization of certain low rank matrices using Sylvester equations and show that the decay of singular values can be understood via an extremal rational problem. We will use it to explain why low rank matrices appear in galaxy simulations, polynomial interpolation, Krylov methods, and fast transforms.