Well, you can't really class David's love for Agnes and Emily with his infatuations with Miss Shepherd and Miss Larkins. His only infatuations were with those two (Shepherd & Larkins), and Dickens used them to describe David's rather foolish "teen" years. He meant to convey the message that David was just like every other young man, suffering from the same idle crushes. His love for Emily went back to his boyhood love of beautiful things. David says that his babyish affection for Emily was the purest of his life--and it wasn't short-lived, so you can't really say it was infatuation. His love for Agnes was "founded on a rock" to use Mrs. Strong's words. It lasted a lifetime, so that definitely wasn't infatuation either.
I have read the book many, many times.