Sephiroth is an overrated character but there's a reason for it and not because of the length of his sword. To me, what FFVII does so well with Sephiroth is that he's always portrayed as an enigma. You only ever learn about him for the first time when you see the main villian at that time, President Shinra, had died and you only found his sword sticking out of him. It's a strong opening you thought Shinra was going to be the main villian for the whole game when you played it. You then learn about Sephiroth through some backstory and he's even a playable character for a time, showing you how badass he is, killing monsters in one hit, using materia you don't get until much later in the game. And after Cloud's (Zach's) backstory is told, which explains more of his personality, the game then just slowly builds him up afterwards.
For example, you find the corpse of a giant snake, a monster which you just cannot kill at that point in the game, impaled on a tree. Your party cannot take one out and this single guy does with ease. It's little things that keep building up this idea in your head of what this villian can do and makes you excited (and somewhat tentative) for the time when you eventually do. And on top of that, he goes and pulls a dick move by killing Aerith, now making it personal to you, the player.