Why you do this to me! God now you've got me thinking.
It would make sense In a way and it explains why up until episode 16, all the narration Tomoya's done was in past tense as if he's done it all before, such as
"After a while I felt like my heart couldn't take it anymore, I felt like I ran out of hope" episode 16 of After Story.
however when Tomoya was on the hill with Nagisa, when he's life didn't involve Nagisa (first episode of season 1) and when he contemplated her not being in his life (episode 16 and episode 22 of After Story) the colour of the scenes were bluish, but when Nagisa was involved, it was full of colour. So this again could have symbolised how Nagisa brought colour into his life (as you pointed out) rather than it just being a flashback.
That being said, it could very well mean the scene on the hill represented both Nagisa bringing colour into his life, and was expressed by the use of flashbacks to the events that determined where Nagisa was going to be In Tomoya's life. So honestly I don't know
Sorry if it's a bit hard to follow and confusing, I haven't watched Clannad in a while and I only watched After Story once (despite it's reputation, I found it a bit boring and preferred the first season and the school orientated episodes of the second season, but that's a different topic) and I'm not the best at the abstract side of English, hence why I'm just passing in English ATAR, so I might have said things that make absolutely no sense.