How can I stop followups from being sent when I'm on vacation?

You just have to update the out-of-office hours under the Followup Settings.

Log into your Agent Legend account. Click the profile icon in the upper right-hand corner and go to Settings. Click on Followup at the top. Scroll down to the Working Hours box. Click the Out of the Office toggle so that it says On. 

Messages that were scheduled for the days you're out of the office will now be postponed.

Remember to log back in and turn the Out of the Office toggle "Off" when you get back so that your messages and followups will resume. They will continue to be postponed indefinitely until this setting is manually turned off again.