If his answer is a little smug, and a little coy - well, it just is.

"That is one way to describe the foolishness I encountered, yes." Skellig will wait his turn among those who are mingling about, but continue the conversation with her alone (for now). "They wished to speak ill of a friend, and their judgement was...clouded as to when they should have returned home to rest, versus pick fights."