Text this: The critical role of hot carrier cooling in optically excited structural transitions