Although there is no harm in using a single email address for all the sites added, it would be advisable not to do so especially since you could expect to get a large number of incoming mails from the directories and it could pose a problem in managing all of them from a single account.

Directories usually send out emails to acknowledge a submission, request a confirmation to complete the submission, or to confirm whether a site has been approved or rejected. The larger the order, the more incoming emails you could expect. Hence, in order to manage these emails better, it is recommended that you use a different email address for submission purposes for each site.