The celebration of the new year is the oldest of all holidays. It was first observed in ancient Babylon about 4000 years ago. In the years around 2000 BC, the Babylonian New Year began with the first New Moon (actually the first visible cresent) after the Vernal Equinox (first day of spring).
The beginning of spring is a logical time to start a new year. After all, it is the season of rebirth, of planting new crops, and of blossoming. January 1, on the other hand, has no astronomical nor agricultural significance. It is purely arbitrary.
Full page here.
I think I learned in school that New Year's is the day when the Earth completes one full rotation around the sun. As this site points out, however, that really might as well be any day at all.
Technically, each person's specific "New Year" would be their birthday, but I don't begrudge the holiday and its given meaning on any of theses counts.