Withdrawals from your savings pot will have significant negative consequences for your long-term financial security.