Click here for a Mac OS X fix.
This problem, while originally predominately for Mac OS X, seems to be occurring a lot more on computers running Windows 7.
Update: Thanks to Uroboch and Ken in the comments for verifying that this procedure does in fact work! 
Disclaimer: This procedure involves using regedit.exe to edit your steam registry files. There’s a bit of a taboo against doing stuff like this, so I just want to state that if you screw something up it’s your fault, not mine. You take responsibility if you mess something up, it’s your risk. 
I’ve written this out in a very basic way so hopefully even a more “novice” user can follow these steps. If you know what you’re doing with regedit, all you want to do is navigate to Steam and change the Offline value to 0 instead of 1. That’s basically the plan in a nutshell.
Proposed Procedure:
1. Open up regedit.exe. The way I did this was to click the windows logo/start button on the bottom left and search for it (see image below).
2. You might get a “User Account Control” warning pop up asking you if you want the program to make changes to your computer. Click yes (of course).
3. Expand (click the arrow to the left of) “HKEY_CURRENT_USER”.
4. Expand the “Software” folder (inside of “HKEY_CURRENT_USER”).
5. Expand “Valve” (inside of “Software”).
6. CLICK (not expand) “Steam” (inside of “Valve”).
7. On the right panel of regedit you’ll see a bunch of things listed (see figure below).
8. Double-click “Offline” (not “OfflineAFS”). A box should pop up.
9. Change the “Value data:” value to 0 instead of 1 (see figure below).
