Swim shorts with logo patch