Warm water can cause some shrinkage, but in reality, any temperature of water can cause shrinkage in a cotton shirt. Cotton fiber is very absorbent and the fibers swell or expand when wet and then when the moisture is removed, the fiber pulls in a great deal. (That is the simplest way to describe it.)
Another thing related to 100% cotton tees is that most likely it is a knit fabric. Knit fabrics shrink more than a woven fabric. Knits are items like sweaters, t-shirts, sweats, etc. Wovens are items like jeans, dress pants, button up dress shirt.
I suggest that you go with the larger size because if the arms are already tight, they are only going to get tighter and bind your arms. The size 4 will be your best bet because it will likely shrink a little due to both the cotton content and the knit structure. PLUS, if you find that the item did not shrink enough for you in warm water, then wash with hot water and dry a little longer the next time. Just don't over do it.
Textile (fibers and fabrics) instructor