Imagine if you will, this common scenario.
A web site builder puts together his site. He is very meticulous about every detail. He's got the perfect domain name, his H1 tag just so, his anchor text perfect and enough content to choke a horse. He's gotten himself backlinks from some great sites. His site, after just a few days, is number 3 on Google's hit parade. He is on top of the world.
And then it happens...the dreaded Google dance, or as I call it, "The Google Shuffle." Maybe I should write a song about it. But I digress.
Anyway, the Google dance takes place and this site is now nowhere to be found. The site owner is in an absolute panic. So he starts tinkering with the site. He starts slamming it with content, some of it which is not so good, and goes about doing some other things that I won't mention here because I don't want to give anybody any bad ideas. He does everything he can to save his site...only to end up making matters worse and getting it sandboxed for life.
If only he had left things alone.
Truth be told, most new sites get a pretty good boost up the rankings the first day or two. This is common. Then, as Google gets a feel for your site, it then settles down to where it should be AT THAT POINT IN TIME. Then, when you've settled in, THAT is when you SLOWLY start to work on your site. Maybe add one or two pieces of content a day. Don't go crazy. And don't go crazy with getting backlinks either. Too many, too fast is trouble.
But don't tinker with the structure of the site unless there are really some major things wrong with it, in which case you wouldn't have hit number three on the hit parade in the first place. Your site, with many flaws, would have been nowhere to be found and you wouldn't be in a panic in the first place. Well, you'd be in a panic...just a different kind of panic. At least then, you'd have something to panic about.
SEO is a time intensive task. It doesn't happen overnight and it isn't permanent. There will always be new sites coming up and trying to take your place in the rankings. Therefore, you will always have to add content and get backlinks. But don't do it all at once and don't panic when Google does its dance.
It's just part of the process.