Eggs can be washed in warm water with no consequence to the embryo.  As the warm water begins to loosen the excrement, take a smooth cloth or sponge and gently wipe the rest away, dry and replace the egg in sequence, within the nest box.

Additional reading: Are the eggs fertile? and Can I touch the eggs with my hands?