I totally understand how he felt in the boutique, though. That's how I get when I go to Hot Topic (^_^b...)
So, yeah, during this whole episode I kept getting this feeling, and I think there's no denying it now. Disheartened nerdly glasses guys are my moe. Of course, that phrase means little to you if you don't know what moe means. Unfortunately, moe is one of the hardest anime-fandom-related concepts to describe, so this is only my interpretation. The generally idea of moe is a feeling of wanting to protect a character, and it usually applies to young girl characters (the kanji for moe means "to bud," so there's the idea of freshness, purity), but it has been used in all kinds of situations. To me, moe is that feeling I get that can only be described as "awwww..." with any kind of inflection.
Now, I knew that glasses were a turn-on for me (ask my husband; it was his being too lazy to wear contacts that played a major part in getting us together), but the disheartened part and definitely the nerd part are only moe to me. I got the same feeling when I saw the president of the newspaper club in the same volume of Kujibiki Unbalance, and back further, I definitely felt it for the main character of Train Man (the version released by Viz).
I mean, look at that! How can you not feel all moe for him?
While watching the episode again to take screenshots, I decided to listen to the audio commentary on a whim (I don't usually listen to series audio commentaries, although I do like to listen to them for movies), and the seiyuu for Sakaki mentioned that during the voiceover sessions, she was starting to fall for Madarame (the character, not the seiyuu). Seriously, I am not suprised. And then Madarame's seiyuu mentioned that Madarame is very popular with the female seiyuus ^_~