With the exception of new and heavily-dyed clothing, I don't really sort laundry by colors anymore. I sort by "effort required to fold". When I'm doing multiple loads, the first load will be the fiddly, annoying stuff - mostly my shirts and the girls' clothes. I usually have more energy for folding at the beginning of a laundry day, so I try to get the annoying stuff out of the way first. Next, I tackle the bulky stuff that folds up quickly and easily - like bath towels and Jeremy's shirts. Finally, I wash the things that don't really need to be folded at all, like socks and underwear, so that when laundry day bleeds into laundry night, I can just stick them in the dryer and go to bed, without worrying about the inevitable wrinkles in the morning. (This also works out nicely because the socks and underwear tend to sink to the bottom of the hamper, anyhow.)
I use the same logic when I'm doing just one load of laundry, and not tackling the whole pile. I start by throwing in the things I need right away, and as I throw them in, I assess the time required to fold them. Then I throw in towels or shirts or socks to fill the washer the rest of the way, based on how much remaining folding energy I expect to have.