I think it's contextual, right up there with it being okay if black people call each other "the N word" but it's strictly off limits for Caucasians. I can think of a ton of situations where you run into that, where a term is okay to use if you're in that group (Italians calling each other guidos, gays calling each other faggot or mo) but if you are an outsider saying the same thing is a huge faux pas. Women can refer to each other as bitches affectionately but let a man say it even jokingly and look out, that sort of thing.
My father grew up in TN, he actually referred to his favorite horse as a peckerwood. I'm in SE PA and terms like hillbilly, redneck and cracker are insults, usually meaning the same basic thing: White trash.
Peckerwood I never hear unless spoken with a drawl.