What an absorbing work of 'good' fantasy like Lord of the Rings does is to create other possibilities of life. It opens your mind to other kinds of existence, and that's instantly threatening to those who wish to maintain the status quo.
I liken their hatred to two things.
One stems from fear, and the best way I can describe that is my own refusal to accept that there may be aliens, because at heart, I don't want there to be aliens as they are more advanced and would simply wipe us out (history shows that this always happens, just as the native Americans and the Aborigines were destroyed). So some do not want the possibility of alternate existence put before them because they fear it.
The other thing is control. We (or at least, some of us) live in an ordered world where we know what will happen from one day to another. We don't go to extremes, we accept our lot. But fantasy offers chaos and anarchy simply by its very existence. That's A Not Very Good Thing to some, so they want it to go away. They either tell us its bad for us or ban it if that doesn't work.
The human imagination is a terrible thing. Far better to shut the door, switch on the soaps and peruse nothing more challenging than the Argos book
The alternative is to be like Bilbo and be swept into things too big for you, or be like Lord Asriel and want more of it all.