Wear what is comfortable. It is not uncommon to see people wearing jeans sitting next to someone in a suit.