Like most plantation children of colonial times, Madison received his earliest schooling at home, probably largely from his grandmother, Mrs. Frances Taylor Madison. When he was about 12, he was enrolled in the school of Donald Robertson in King and Queen County. After “three or four years” with Robertson, he studied for “a year or two” under the Reverend Thomas Martin and in 1769 enrolled in the College of New Jersey.
Madison attended the College of New Jersey, finishing its four-year course in two years, 1769–1771; and continued to study with John Witherspoon, the College's president at that time, for a year after graduating. Madison has been called America's first graduate student.
Adair, Douglass G., The Intellectual Origins of Jeffersonian Democracy
Dennis F. Thompson, "The Education of a Founding Father”