Actually all those three words are from Latin. Hippopotamus is late Latin, after the Greek 'hippo-potamos', 'horse of the river'; cactus is a Latin word after the Greek 'kaktos'; census is a pure Latin word from the verb 'censere', 'to rate, assess'.
Nouns in Latin ending in -us may be in one of three groups, called 'declensions': the second, the third-neuter or the fourth declension. Those in the 2nd have plurals in -i (dominus, domini); those in the 3rd-neuter have plurals in -a (vulnus, vulnera); and those in the 4th stay the same in the plural, although the U becomes a long vowel (exercitus, exercitus).
The first two nouns you mention are in the second declension, and the last is in the fourth. So in Latin the plurals would be 'hippopotami', 'cacti', 'census'.
In English both the forms 'hippopotami' and 'hippopotamuses' are used, and both 'cactuses' and 'cacti' are correct. 'Census' as a plural is not often found because it might be confused with the singular form. 'Censi' is incorrect. That leaves 'censuses' as the only available plural.
PS Latin words ending in -um all belong to the second-neuter declension, which forms plurals in -a: stadium, stadia; forum, fora. Normally in English, you have the choice whether to pluralize in -a or -ums: so both 'stadia' and 'stadiums' are correct.
PPS Not all words that end in -us are Latin! The word 'octopus' is Greek (via modern Latin), and its plural is not 'octopi' but 'octopuses'. In Greek the plural would be 'octopodes', but not in English.
As a general rule, I recommend pluralising words in -us in the English way (by adding -es), which is always correct, unless you are sure that the plural may also be -i or something else. That is the direction that English usage is taking in any case
(Oxford English Dictionary; Lewis & Short)