This passage is usually referenced as proof that believers go to heaven, and, as you have done, verse 4 is left out of consideration even though it is key to understanding what Jesus meant.
There is no logic in using this passage to support heaven as the reward for the faithful. For one thing the word heaven is not mentioned in this passage. If we are to be taken to heaven, why does Jesus come to earth? Why don't we just go to him? Jesus said, "I will come again.... that where I am there you may be also". He did not say "where I was". Where is Jesus when he gathers us to himself? On earth. Not heaven.
It is interesting that the phrase "house of God" and "My Father's House" [when God is the Father being referred to - see John 14:2 - only ever refers to the temple of God on earth. Jesus is referring to the dwelling of God among men which he goes to prepare. While he is away that edifice will be a spiritual one into which we as believers find our place and hope and home - the church. Once this spiritual edifice has been completed, he will return to establish it physically on the earth. See 1 Corinthians 3:16; 6:19; 2 Corinthians 6:16; Ephesians 2:14-22; Hebrews 3:6; 1 Peter 2:5.
Here is a link to information about this topic which you will find useful: In My Father's House
I hope you have found this helpful.