I'm friends with a number of models on CB and chat offline a few times a week with most of them. I don't do C2C and normally not using a VPN.
Sometimes I go into their room while logged on, but have to leave quickly, or they are busy and I won't say anything, as they will normally start a PM. I'll often get an offline message soon after asking why I didn't stay, or why I didn't say hi.
Other times I go into their room, while not logged in, to see whether they are busy, or see the mood of the room. I have never had them contact me when I have done this, so I doubt they can tell I was in the room anonymously.
Double check you are logged out, not logged in on another tab in the browser. When you are in their room, then the Follow icon should be showing if you are logged out, if you are following them normally.