It sounds like you need to stop, or at least cut down if lovense is causing you issues. If you keep going, you'll end up having to quit one way or another anyway, so don't be afraid to switch it up, because doing something that keeps you healthy and happy is what will help you long-term.
Sorry to sound like a broken record in these threads but what you offer in your content will attract that kind of audience. I've never used a lovense, and am fortunate enough to do well when I get on cam. Since 2016, I've only had maybe 15 people ask if I had a lovense, even during the big lovense craze. They would leave when I said no. Is it income loss? Yes, for that particular thing. But there's still so much more that I can offer and get paid for in turn.
Yes, there might be income loss in the short term as the people who came to you for lovense-only content leave. Long term, you'll begin attracting the kind of viewers who are happy not to see a pink tail.
Don't be afraid about the changes. Try a few streams without lovense and see what happens.