Learning the governing equations in dynamical systems from time-varying measurements is of great interest across different scientific fields. When the underlying system exhibits chaotic behaviors, such as sensitivity to initial conditions, it is crucial to recover the governing equations with high precision. In this work, we bring together connections between compressed sensing, splitting optimization methods, sparse representations of the governing equations, and the statistical properties of chaotic systems -- to provide exact recovery guarantees for classes of dynamical systems. As our main theoretical result, we show that if a system is sufficiently ergodic, thus its data satisfies a strong central limit theorem (as is known to hold for chaotic Lorenz systems), then the governing equations can be exactly recovered as the solution to an L1 minimization problem -- even if a large percentage of the data is corrupted by outliers. Numerically, we apply the alternating minimization method to solve the corresponding constrained optimization problem. We illustrate the power, generality, and efficiency of our model through several examples of 3D chaotic systems and higher dimensional hyper-chaotic systems.