Removing Wood Filler

I used wood filler to fill some trim nail holes around my windows. Typically this works just fine, but the wood filler dries very hard and is nearly impossible to sand if you wait too long. Too long in my case was about a week, but I think if you don't start sanding in about 45 minutes it can be problematic. I tried sanding it but the filler was so hard it would just sand the soft pine trim around the filler, ugh. The solution I found was to use acetone. This works like magic. I used a scotch brite pad, dipping the pad into some acetone in a yogurt cup. It comes off with ease, but also takes the primer with it. Stinks like hell -- use a mask and ventilate!