1) It's "guilt free" because it's not bad for you - it's not high in fat or useless calories. Fish is often a choice for people with very health-conscious diets.
2) It's "guilt free" because relatively safe fishing methods are used. No dolphins or turtles were trapped in the nets and killed while harvesting the fish.
Remember those advertisements for canned tuna? "Charlie, they don't want tunas with great taste - they want great-tasting tuna." (As the tuna is showing off his collection of fine art.)
Right, it should be 'Guilt-free' not 'Guilty-free', I found that article from the "dining and wine" column at Nytimes.com. The title is: Holy Mackerel and Other Guilt-Free Fish.