Eggs can be washed in warm water with no consequence to the embryo. As the warm water begins to loosen the excrement, take a smooth cloth or sponge and gently wipe the rest away, dry and replace the egg in sequence, within the nest box.
Additional reading: Are the eggs fertile? and Can I touch the eggs with my hands?