Yes, yes, yes, yes, yes!!!! And pleeeeeeeeeeeeeeeeeeeeeeease do your best to USE the seat on the plane too! You can avoid the issue of "will there be an empty seat" by buying a seat for the baby, or you can hope for the best. Trust me, having baby use the seat in the air will make EVERYONE on that flight, especially baby, happier and safer. If you have any questions about using a seat on the plane, please check out
car-seat.org. It's an amazingly informative site.
Do NOT check the seat, no matter what. The baggage handlers don't take the care you would, and the knocking around under the plane can cause the same sort of damage a collision would. Seats need to be replaced after collisions, so I wouldn't chance having that sort of damage happen when I NEED the seat at the other end of my destination.
All that said, if there is no space for you to use the seat once you're at the gate, they will gate check it. Very few seats fit in the overhead compartments on planes. The only one I know of is the Sunshine Kids Radian. Fab seat, rear-faces to 33lbs and forward faces to 65 or 80 pounds depending on the model you purchase.