Pot is not harmless as many believe. Sure it won't kill you like meth or heroin, but it is not without some issues. Physically, it is much less dangerous than most drugs including alcohol and tobacco. There has never been a test that proved it causes cancer, quite the opposite. It is used by modern medical professionals around the globe for over 400 different known medical problems with clinically proven results. But that is medical, not recreational use which is what you are asking about.
Psychologically, recreational pot use can be another story. Everybody is physiologically a bit different from each other. That in itself will cause marijuana to effect people differently. The effects of course also have a lot to do with frequency and quantity of use, and yes, even the quality of the pot. A daily user will experience more side effects than the occasional user. There are many highly intelligent (no pun intended) people who smoke daily. Pot doesn't make you stupid. That doesn't mean that there aren't plenty of people who smoke pot who are stupid. But that's because they were stupid anyway, hence the large number of stupid people who do not smoke pot. But I can tell you some psychological issues that frequently happen to many but not all long term daily smokers.
1.) You already know about the short term memory loss, but it also tends to reduce the attention span.
2.) Reduction of drive, ambition and motivation, often to levels that prevent many people from every truly reaching their goals.
3.) A large percentage of high quality employers drug test, resulting in less than ideal income levels for those who can't pass.
4.) Many get really stoned jut before bed. This greatly reduces dreaming and the necessary REM sleep we need to not go bonkers.
5.) I know this sounds funny, but pot gives you the munchies and it never seems to be for food that is nutricious. The munchies can lead to being overweight or even obese.
6.) Spend a few hours around a daily smoker who is out of pot and can't find any and you'll see how it affects some peoples attitudes. Not every daily user, but many.
7.) Often hard to get to sleep if you don't have any pot.
8.) The worst one of all...life went by, and you missed it.