If there were no such thing as gravity, then, yeah, get it off! Seriously, that's why you keep the lower retainer, is because of the curvature of your lower jaw. Everything is just always pushing forward. One thing you could do, however, is have a removable retainer made that you could just wear at night. If you have a nice dentist, or orthodontist, they can make a clear plastic stent for practically nothing, and that would work.