The tree-level soft theorems were recently shown to arise from the conservation of infinite towers of charges extracted from the asymptotic Einstein equations. There is evidence this tower promotes the extended BMS algebra to an infinite higher-spin symmetry algebra. In this talk I will introduce towers of canonically conjugate memory and Goldstone operators, highlighting their role in parameterizing the gravitational phase space. I will discuss the conditions under which these towers provide a complete set of scattering states and demonstrate that they are the building blocks of both soft and hard charges. I will finally show that the tower of tree level soft symmetries can be used to extend the Dirac (Faddeev-Kulish) dressings to include the infinite towers of Goldstones and comment on their implications for the gravitational S-matrix.