Am I being unreasonable in this argument?

You should have listened to him but not at the expense of your work. If he can't understand that then screw him. Also let him know tgatw you thought his comments were mean Andy don't appreciate it. If nothing more than for your own self-respect.