The first steam engines were very primitive and slow. They had boilers with single or double flues and mainly vertical cylinders, although the very first one, built by Richard Trevthick in 1804, had a horizontal one.
The main advance in 19th century steam locomotive design came with Stephenson's Rocket of 1829, which had a multi-tube boiler, thus greatly improving its ability to make steam. Stephenson didn't invent this btw - it had been developed by a marine engineer and Stephenson adapted the design.
Steam engines then followed the design principle of the draughting boiler. At the rear is a firebox. Tubes run the length of the boiler, to a hollow chamber at the front known as the smokebox. Steam heated in the boiler goes under pressure into cylinders, where it pushes the pistons back and forth. These in turn are connected by rods and cranks to the driving wheels. Nineteenth century engines could have one, two or three driving axles.
When the steam has done its work in driving the piston it is exhausted. This is what produces the chuffing sound. The steam passes up through an exhaust pipe known as the blast pipe, which is inside the smokebox, and the force of steam going up through the chimney sucks air out of the smokebox.
Air then rushes in through the firebox, making the fire burn brighter and sucking the hot air and gases through the tubes, distributing the heat all over the boiler.
The main controls on a steam locomotive are the regulator, which lets steam into the cylinders, and the reverser, which allows the driver to adjust the inlet and exhaust valve timing so the engine can change direction. It is also known as the cut off, because it allows the driver to cut the supply of steam before the piston has completed its stroke. The piston will continue to move, because the steam will continue to expand.
There is also a brake. By the late nineteenth century, engines designed for passenger work had a steam brake for the engine and a vacuum or air brake for the train (this was a legal requirement), also a hand brake.
Other controls included a whistle, cylinder cocks for letting waste steam out. pressure gauge, and a blower - a steam jet to induce draughting while the engine was standing still.
There was also a safety valve on top of the boiler, which prevented excess steam from building up.
Early 19th century steam engines didn't have cabs - the driver and fireman stood in the open in all weathers. Later on cabs started to be built. These gave some protection from the weather.
Link shows a late Victorian loco (c.1880) known as the Dean Goods, after the man who designed it, William Dean, who worked for the Great Western Railway.
The engine is passing the station with the regulator open and also has steam to spare, because the safety valve has lifted.
Steam locomotive design continued to be developed into the 20th century, in 1904 the locomotive City of Truro clocked 100mph and in 1938, the high speed steam loco Mallard reached 126mph, which remains a world record for a steam locomotive.