Not so, sir, I do care for something; but in my conscience, sir, I do not care for you. If that be to care for nothing, sir, I would it would make you invisible. Apparently the Clown's fool-logic is that he does care for something, so if he does not care for Viola/Cesario, that makes her/him not something, but nothing.

In his joking, the Clown seems to express the wish that Viola/Cesario would just disappear. Why? He could easily guess that Viola/Cesario has come to Olivia with another troubling message of love from Orsino. Or maybe the Clown senses that the appearance of this handsome young gentleman can only mean trouble for Olivia.