The Spectral Form Factor is an important diagnostic of level repulsion Random Matrix Theory (RMT) and quantum chaos. The short-time behavior of the SFF as it approaches the RMT result acts as a diagnostic of the ergodicity of the system as it approaches the thermal state. In this work we observe that for systems without time-reversal symmetry, there is a second break from the RMT result at late times: specifically at the Heisenberg Time $T_H=2\pi \rho$. That is to say that after agreeing with the RMT result to exponential precision for an amount of time exponential in the system size, the spectral form factor of a large system will very briefly deviate in a way exactly determined by its early time thermalization properties. The conceptual reason for this is the Riemann-Siegel Lookalike formula, a resummed expression for the spectral determinant relating late time behavior to early time spectral statistics. We use the lookalike formula to derive a precise expression for the late time SFF for semiclassical systems, and then confirm our results numerically. We find that at late times, the various modes act on the SFF via repeated, which may give hints as to the analogous behavior for systems with time-reversal symmetry.