Once you start working with a mortgage broker, you may be tempted to stay the course no matter how tough it gets. You may think that all that you've gained will be lost if you change brokers midway through. This is simply not the case.