First, head toward Klang, then Kapar, Jeram, Kg. Asam Jawa, Kuala selangor, Tanjung Karang, Sekinchan [ you may like to stop for some jagung or mango ], Sungai Besar, Sabak Bernam, Simpang Empat [ turn right at the traffic light ] going at the direction of Teluk Intan but about 7 km before Teluk Intan, turn left at another traffic light where you would be passing through a bridge [ across Sungai Perak ].
Go straight all the way, where many Malay kampung [ air nira is a very refreshing drink, try it ] dotted your journey, don't miss the signage of Lumut / Pulau Pangkor. The last major town would be Sitiawan but because of the new road, you need not have to pass through the busy town centre. You know you are in Lumut once you smell the sea !
Don't forget to rent a motorbike to explore the whole of Pangkor ! Take good care not to get sunburn too !