it felt like the producers wanted to go down the “bad route” to make clannad so emotionally strong and give the story depth , but then revert back to the “happy route” to make everyone happy
(VN spoilers) That’s how it worked in the VN too. You had to get the bad end before you could go back and get the good end
I kind of excuse this by using (CLANNAD everything)the mutli-world idea brought up in Kotomi’s route. Every “route” is a different world. In some of them he gets an orb of light. All of the Tomoya are connected through Ushio (I’m going to drop this here because I don’t feel like going into detail again about this. I basically say that Ushio, who is the girl in the illusionary world chose Tomoya to help Nagisa.). So Tomoyo’s bad end exists in one world. Kyou’s good end exists in another. And there is a world were Ushio and Nagisa are dead. But there is one world where Nagisa, Tomoya, and Ushio get their happy ending. The progress Tomoya went through isn’t nullified, at least not completely, because nothing was “undone.”
The reason why “Key magic” exists in CLANNAD is so that they can explore these themes of family and allow Tomoya to grow as a person by experiencing these hardships while still leaving room for a happy ending without nullifying anything.