Yes, it's easy for any one of us to be indifferent toward a person or people and not care about what they say or do. Is being this way beneficial for us?
Jesus summarized what Jehovah [God's personal name] require of those who worship him, in a few simple and profound words, the greatest commandment, he said, is to love Jehovah with our whole heart, soul, mind, and strength. (Matthew 22:37)
The second greatest commandment, Jesus said, is linked to the first: “You must love your neighbor as yourself.” (Matthew 22:39) It is to this commandment that we now give attention, and for good reason. The times in which we live are marked by a selfish, distorted form of love. In his inspired description of “the last days,” the apostle Paul wrote that people would love, not one another, but themselves, money, and pleasures. Many would have “no natural affection,” or as one Bible translation puts it, they would “lack normal affection for their families.” (2 Timothy 3:1-4) Jesus Christ foretold: “Many . . . will betray one another and will hate one another. . . . The love of the greater number will cool off.” (Matthew 24:10, 12)
Notice, however, that Jesus did not say that the love of everyone would cool off. There have always been and will always be those who display the sort of love that Jehovah both requires and deserves. Those who truly love Jehovah will strive to view others as he does.
Lastly, to sum up my answer to your question, love of God and neighbor will require you to care to some degree about what others say and do, just don't go overboard with being concerned to the point that it hinders you showing love for them.