What is my PATH?

I may have missed this lesson in my Operating Systems class (which was admittedly a long time ago). Or, perhaps more likely, I didn't know I'd need it and forgot. Either way, here's everything no one told you about your PATH variable, what it does, why it's important, and how to modify it.