This is one of those things that you have to be a long time player and know the fluff from earlier editions to fully understand.
I can't remember the book, (I think it's a 2e book) but it has this long discussion between a Mage and spirit he has summoned. In it the mage is asking the spirit all sorts of questions and the Spirit is answering it.
At one point, the Mage asks the spirit "How old are you anyways?" to which the Spirit replies, "As old as time". The mage asks the Spirit a few more questions before asking
"If you are as old as time, where were you before the awakening?" and the spirit says "I was on the plane of Air, My home, with all the other Spirits of Air."
The story goes on, and ends with the mage going. "Thank you for your answers, It's refreshing to speak to someone who always tells the truth"
and the Sprit Responds "We don't always tell the truth...."
So.... was the spirit lying? Or telling the truth? That is up to the reader and the GM to decide.... (Basically wiggle room for tables to change things to fit their needs.)
So if you really them coming from "earth" for your table, so be it....
but the canon answer is they come from the meta-planes (3e magic book, 4e magic book, 5e aetherology) With a meta-plane of each type of Spirit (meta-plane of Air for Air spirits, Meta-plane of Earth for Earth spirits.. etc).
These are not empty places either! They are filled with objects and items, and spirits.... Very VERY powerful spirits!
(If you know someone with the 4e Street magic book, they do a good job of describing the Crystal palace of the "Queen of Air" of the Air meta-plane... A Great Form named Spirit)