He left when he was six, the age he was in the flashback from last week. Spencer never said he was only gone a few years. He said in the pilot that he doesn’t have a father cause he’s been gone that long. Spencer is just in high school 16-17 so that time lines up. Dylan would have been a baby or she was pregnant with him as I mentioned in the prior ask message edit. Billy is from the neighborhood and knew Grace prior to him coming to see Spenser play and as we found out last night they (he and Grace) were an item in high school. That’s pre- Spencer, which no one but he and Grace knew we can assume for now.

In some stuff I’ve read, April, who was the previous showrunner and still show creator said it’s not meant to drag out the entire season and Jordan is gonna start looking into things- which got set off by his find last night it seems

I posted this article over here but here’s the quote from her in tvguide about that. This was after the pilot

. When are we going to find out what “the truth” is Coach Baker is referring to at the end of the episode?
April Blair: It will come. We’re not gonna draw it out too much. I mean, there are layers of secrets that Billy and Grace are keeping, but the kind of big one, [the] answer about his father, will come within the first few episodes. Because I hate that, when someone just draws out a thing forever… In Episode 3, Jordan starts to find clues into that. Jordan becomes the audience. He’s really gonna tell the story of getting under the hood of, “Why does my dad seem so invested in this guy?” and starts to piece together clues that leads him on the path to helping us discover Spencer’s parents question. It won’t go on. It won’t drag on. It’s not like an episode-y kind of situation. (x)