The capacity of non-coherent stationary Gaussian fading channels with memory under a peak-power constraint is studied in the asymptotic weak-signal regime. It is assumed that the fading law is known to both transmitter and receiver but that neither is cognizant of the fading realization. A connection is demonstrated between the asymptotic behavior of channel capacity in this regime and the asymptotic behavior of the prediction error incurred in predicting the fading process from very noisy observations of its past. This connection can be viewed as the low signal-to-noise ratio (SNR) analog of recent results by Lapidoth & Moser and by Lapidoth demonstrating connections between the high SNR capacity growth and the noiseless or almost-noiseless prediction error. We distinguish between two families of fading laws: the "slowly forgetting'' and the "quickly forgetting''. For channels in the former category the low SNR capacity is achieved by IID inputs, whereas in the latter such inputs are typically sub-optimal. Instead, the asymptotic capacity can be approached by inputs with IID phase but block-constant magnitude.