Oh definitely. The issue wasn’t that it was actually implied - it was that the audience reading it had never read anything like it before (and indeed, arguably there had never been anything quite like it before), and so made attempts to contextualize it in a way that made sense to them, which was, in that case, as a post-apocalyptic SF novel (which were already not uncommon and often had fantasy elements).
And yes I don’t buy ASoIaF as sci-fi, it’s pretty clearly epic fantasy - though again kind of its own thing, because no epic fantasy prior to it had really done what it did, and particularly not with such a small number of actual fantasy elements. Tad Williams as discussed upthread is perhaps the closest that pre-dates ASoIaF. The Assassin series of novels by Robin Hobb have some very similar elements, but came out at the exact same time (1995 onwards - GRRM is a big fan), and a much smaller-scale focus, with lower but somehow more important stakes. Lois McMaster Bujold is also worth comparing, though she was writing sci-fi (but does have a character who feels like a potential inspiration for Tyrion, in that he’s sauve, politically astute, erudite, keen on the ladies, and so on, but has an off-putting physical appearance and physical disabilities, including being short - also both had domineering fathers who wanted to disinherit them at birth - but Miles’ one came to respect him in childhood). She didn’t really write fantasy until later but it’s not dissimilar in tone.
Anyway I’m way off-track - I think when people are exposed to something new they try to contextualize it as something they understand, and for a lot of people, ASoIaF was pretty new, so the old deal with “What if it’s sci-fi really?” came back.
Re: Shannara, it was bad enough that when I was only slightly older than you (somewhere in the 12-15 age range), the first or second book in the series was the first book I really remember putting down not because it was dull in a grown-ups yammering at each other way, or a “for kids” way, but because it was clearly badly and clumsily written, and the characters didn’t make sense. I remember I had initially enjoyed the post-apocalyptic stuff and a fight with a cyborg dinosaur or rhino or something, but I was sitting reading it in some airport and thought “Damn, this is actually bad… I’m going to need another book for the plane…”. Memory does not relate whether I did get another book.
(Of course the fates made up for that years later, when I randomly grabbed China Mieville’s Perdido Street Station from a book rack at an airport, and it completely rekindled my interest in fantasy just in time for the post-2000 fantasy explosion, which books like The Eye of the World (the first Wheel of Time one), or much worse, Wizard’s First Rule (the first Sword of Truth one), had managed to strangle into unconsciousness by being so terrible.)