I’d go with the duck out early option personally. Showing your face and socializing with others can make a real impact on how your coworkers think about you. They won’t remember you left early, but they may remember you didn’t go.

Now of course, if you don’t care about that, just politely decline the invitation.