We show that the Wald-Zoupas prescription for gravitational charges is valid in the presence of anomalies and field-dependent diffeomorphism, but only if these are related to one another in a specific way. The geometric interpretation of the allowed anomalies is exposed looking at the example of BMS symmetries: They correspond to soft terms in the charges. We determine if the Wald-Zoupas prescription coincides with an improved Noether charge. The necessary condition is a certain differential equation, and when it is satisfied, the boundary Lagrangian of the resulting improved Noether charge contains in general a non-trivial corner term that can be identified a priori from a condition of anomaly-freeness. Our results explain why the Wald-Zoupas prescription works in spite of the anomalous behaviour of BMS transformations, and should be helpful to relate different branches of the literature on surface charges.