We argue that the temporal asymmetry of influence is not merely the result of thermodynamics: it is a consequence of the fact that modal structure of the universe must admit only processes which cannot give rise to contradictions. We appeal to the process matrix formalism developed in the field of quantum foundations to characterise processes which are compatible with local free will whilst ruling out contradictions, and argue that this gives rise to ‘consistent chaining’ requirements that explain the temporal asymmetry of influence. We compare this view to the perspectival account of causation advocated by Price and Ramsey.