I honestly think that in more recent times they have widened the gap between the player bases. Heroic raiding now is as difficult as the hardcore spectrum has ever been, while LFR much easier than the casual spectrum has ever been. For example, in DoD we held ourselves to pretty high standards with our progression, especially through BC and Wrath, because even on our short schedule it was possible to be competitive. Nowadays, a more casual guild on 2 nights a week isn't going to get that deep into heroics. They're going to find it much more difficult.
The real issue that some hardcore players have is with accessibility and other things the casual players get, such as LFR. They get to see the content. I just wonder why that offends these hardcore types so much though. Even today, there is excruciatingly hard content that only a small percentage of players get to experience. Coming with that content are exclusive mounts, titles, and other things. So they have their challenge, their have their prestige, and they have their rewards that set them apart. What else is needed? For the game's population to be cut in half without those players altogether? Even more exclusivity?
Maybe it took some iteration to get to this point (read: a LOT), but I feel the game is currently in a pretty good state for both sides of the coin. Sure, LFR diminishing the excitement of seeing a raid to the end still kind of irks me, but that progression and prestige are still present when you start getting into heroics. Maybe the normal mode raiders are the ones stuck in limbo now. These are, after all, the bulk of the true raiding population in the game. I think the current tier pushed us a bit closer to the right direction in that sense because normal difficulty was pretty damn challenging to go through for the average non-LFR raider this time around.
So if the game is in a pretty good place right now, in my opinion, why are subscribers dipping even lower? Simply put, I think it's because the game is getting old. Many of the casual players who would try the game probably have already, and either still play or have moved on. On top of that, you have other raiders and PvPers whose numbers have thinned out over time and they are only getting some of these players back in cycles, like Kedavra mentioned. I don't think it's as simple as casual/hardcore --> killing the game. I just think it's a case of "this game is starting to age."
Interestingly enough, from a casual standpoint, I actually got my mother to start playing this after seeing her playing some awful browser-based MMOs and she's been pretty hooked. LFR each week, dailies, archaeology, pet battling, and so forth. Generally just lots to do for players now and I think that is a generally positive thing. Maybe it took Blizzard too long to get enough things right, but I just really don't get the hardcore arguments anymore. I question whether the people making these arguments are even doing heroic raiding right now. It would seem some people may only be arguing in principle without being a part of it.
Pandaria has brought a solid raiding experience so far, in my opinion. That exclusive content is as hard as its even been and all of the nice things that have come with clearing it sets you apart from the crowd as much as it ever has.