Well, because he's quite good at listening, to begin with.

[ That is a voice that is, briefly, absolutely thrumming with adoration. ]

And because he has many of the same sorts of frustrations. Freemen - commoners - are treated like they're below notice in many places, but especially in Orlais. So he had to grow up with utter disdain, despite his very real and very remarkable talents. Much as you did.