Small single-phase motors usually have bi-metallic switches inside the motor whic are actuated by the heat produced by the current flowing to the motor. These switches have to cool before they can be reset, or reset by themselves in the case of automatic resetiing devices.
Ver large motors (three-phase0 normally have thermistors embedded in the coils of the motor. These thermistors will experience an increase in resistance as the temperature of the coils rise. These thermistors are in a circuit which controls a realy within the motor starter cabinet of the motor. The control circuit of the motor has a normally-open contact in the control circuit. When the motor temperature is normal, the relay will be energized and this allows the motor to be started. When the resistance of the thermistor reaches a point where it drops enough voltage to drop the relay out, this will stop the motor. A motor contro circuit must be locked-in in order for the motor to run. A push button must be pushed to reset the thermistor relay.
The majority of three-phase motors have overload relays in the motor starter cubicle. These relays are of the bi-metallic or melting type whcih are activated by small resistance heaters installed directly in the motor leads of the motor. These relay contacts are also in the motor control circuit.
This is a basic description of the motor protection devices.
There are more schemes, but this will give you an idea.
Retired electrical and Control Systems engineer