"No, it's all right. You don't have to apologize." He has to focus on remaining... solid? Perceptible? So any panic would cause that to falter, and it's not as though anyone can control their panic. It makes sense, even if it is unsettling for her. "I'm sorry that I made you panic. Would it help if I didn't look at you?" Does he mean visually noticed, or does the simple fact that she is aware that he exists make him nervous?