In the U.S., VFR aircraft need only contact ATC in certain circumstances. Before entering Class C or Class D airspace, VFR aircraft must establish two-way communication with ATC (this means that the VFR aircraft calls ATC, and ATC responds with the aircraft's call sign). Before entering Class B airspace, VFR aircraft must establish communication with ATC and obtain a clearance into the Class B airspace.
No clearance is required to enter Class C or Class D, but the aircraft must heed ATC instructions, if any ("remain clear" means that the aircraft cannot enter).
TRSAs do not require contact with ATC, but it is recommended.
If a VFR aircraft remains clear of all these airspaces, it doesn't have to talk to ATC at all, except for local towers.
VFR aircraft can contact ATC to request flight following. ATC will assign a transponder code in many cases. Otherwise, VFR aircraft always squawk 1200.
The rules are different in other countries.