Move the unit into a city that has a barracks. If the unit is upgradeable, an upgrade button will appear over the unit action icons at the bottom middle of the screen (I think the keyboard shortcut is "u"). You'll get a dialog asking you if you want to pay for the upgrade.
You can also upgrade units in the city screen by right-clicking on a unit in the garrison and selecting upgrade. You can also do this from the military advisor page.
What I'd like to know is if there's any way to upgrade a group of units at once (like all the units in a city or all of the units of a particular type). Even going through the military advisor page selecting the upgrade menu option is a bit on the tedious side, particularly given how large some of my standing armies get.