Marketing and finding an audience for Visual Novels in the west was the biggest struggle preventing them from making any kind of impact here. Non-targeted advertisement for almost anything on the spectrum of anime has been a colossal failure in all but memetic status. (See the old Sakura-con commercial and Crunchyrolls newer TV advertisements for some great examples.)
Even with Manga and Anime taking off in the west stronger than ever, people still have trouble detaching Visual Novels from the “lonely person dating simulator” stigma. With such a niche medium already, not many people are there to clear up these misgivings. It’s just currently not feasible for visual novels to break into the more mainstream without some kind of puzzle game elements or mass marketable gimmicks. I wonder which two visual novels I’m talking about. I much prefer puzzles if you catch my drift.
There’s also the unusual debate of whether these even count as games at all, but that’s probably not worth getting into