He cant, unless the damage is over the amount of the initial deposit, and interest associated with it (if any)
But, if you dont have PROOF of that, the chances are he can take advantage of that, if you went to small claims court, unless you requires it for ALL of his tenants, then you have background records of him charging that amount.
A pet deposit was to protect you, and the landlord, from minor damage like door clawing and minor carpet staining that can be shampooed out, anything past that is pretty excessive. Most landlords that allow pets take a HUGE risk by letting animals on their property.
Get an direct estimate of the MOST cost the damage would be to fix. Not just the minor, this can include carpet repair, stain lifting, door replacement, Flea infestation, any of those, are common pet damages.
When you get your estimate and your proof of your deposit, bring it to small claims court.
If you aren't moving out, and it's an annual inspection. Then you have to pay for the damage, his deposit was only for a move out, repairs, not current repairs.
i hope i helped.
But, dont rush it, your deposit may have covered the damage, you need him to show you the reciept or get an estimate on the damage. If your deposit that you gave him,