Renee,
The average winter temperature in Ocean City is listed at 41 degrees. I'm not sure where you are from, but it is very similar weather/temperature to that of Harrisburg, PA and Baltimore, MD.... the thing that could impact how it feels outside though is the wind off the ocean... that can bring what it feels like down significantly.
Answered: July 13, 2010