You purchase a domain name from a company (I used registerfly, and now I prefer namecheap). When you purchase you can choose for however many years you want to register, from 1 to 10 years. You probably only want to start out with only one year to start.
When you register the most important part about actually running a website is customizing the DNS to point to your website. Many registrars offer services such as email or forwards (ex forwarding
you@yourdomain.tld to
you@gmail.com).
You should start off with a simple hosting account from a provider like 1&1, or GoDaddy (incidently both of these providers have very bad reviews, just some major names that came to my mind), you can host
http://webhostingtalk.com to find a good hosting provider.
When you purchase hosting usually you'll have a choice of a reseller or shared package, a reseller allows you to become a webhost, a regular hosting package is probably all you need. Most providers come with a control panel, the most popular is cPanel, then Plesk, and then DirectAdmin. A control panel allows you to setup emails, ftps (easiest way to upload files to your website), etc. You can setup an email client (like thunderbird) to interface with your new emails, upload files via FTP, etc.
If you'd like I could set up a tiny hosting account (on Plesk) for you to experiment on.