I doubt it...this sounds illegal. It is in my state. No sale=No pay. They get paid for sales, not efforts. Don't give up a dime until you check your local law. At best, this is unethical. Around here, it's illegal and could result in loss of license.
I'd tell them Tough Crap...you should have done a better job marketing my house! Some nerve. Oh, and even if this is legal somewhere, which I doubt, check your contact. if you didn't agree in advance, they are out of luck. I'd report this to the state board of real estate immediately!
Realtor of 3 years.