We don't remember what we did until a while after we were born, though. My first memory is from when i was 5.
This also reminds me of a quote from Babylon 5 S4E2:"You can't create language without thought, and you can't conceive a thought without language"