When travelling abroad make sure you find out if a visa will be required prior to your arrival. Trying to get a visa in the last minute can get expensive. Visa requirements vary greatly depending on your country of citizenship and your destination. A good place to start your research is at http://www.visahq.com.
Visa on Arrival
If possible you should get your visa on arrival as this is usually the cheapest way. Nevertheless use caution since there is a very small chance of not getting a visa on arrival. A friend of mine was actually denied entry to Indonesia even with a US passport. She had to board the next flight back to Malaysia. If you had any problems obtaining of visas in the past you might want to go the safe route to get visas in advance.
Embassy or Consulate
If a visa on arrival is not available get the visa directly from the embassy or consulate of the country you are planning to visit. If you are lucky a consulate might be in the city you live and you can stop by there in person. You can usually find the locations on their website. Make sure to check their website to find out whether any specific paper work or photos are required (they will send you away if you don't bring everything). Most embassies also allow you to get a visa through the mail (usually you'll have to send your passport through the mail and pay for return shipping in addition to the visa fees. If you need multiple visas make sure to to start the process early to have enough time if visa processing takes longer or if you your passport gets lost in the mail.
The most expensive way to get a visa is usually through a visa agency They tend to charge another $50-$100 on top of the regular visa fees. Nevertheless they usually offer additional benefits of checking your paperwork before sending it over to the consulate and probably know the latest changes in the visa application process for a specific country. This might save you some unnecessary back and forth with the consulate if your documents aren't in order. Some visa applications can be lengthy and challenging to understand. One good starting point is VisaHQ (http://www.visahq.com). Make sure you are using a reputable visa agency as you are entrusting them with your passport after all.