This could be a few different reasons. If you don’t have a chimney cap, it could be dead animals stuck in your chimney. The smell could also be from creosote deposits in the chimney, a natural byproduct of wood burning. The odor is usually worse in the summer when the humidity is high and the air conditioner is turned on. A good sweeping will help but usually won't solve the problem completely. The real problem is the air being drawn down the chimney, a symptom of overall pressure problems in the house. One temporary solution is commercial chimney deodorants. They work fairly well, and many people have good results with baking soda or even kitty litter set in the fireplace. However, some make-up air should be introduced somewhere else in the house to solve the air pressure issues. A tight-sealing, top-mounted damper will also reduce this air flow coming down the chimney.
