I verified a strange bug related to attachments and the deleting of the parent record chain.
- If you delete an Email with an attachment it deletes the attachment.
- If you delete a Contact with emails that have attachments the email is deleted but not the attachment.
For on-premise CRM administrators, you can use this script to find the orphans:
SELECT t1.* FROM Attachment t1 LEFT JOIN ActivityMimeAttachment t2 ON t2.AttachmentId = t1.AttachmentId WHERE t2.AttachmentId IS NULL
My assumption is that you would need to clean these up manually.
If you are using CRM Online, then you will have a much different story and one for which I do not have an answer.
In a nutshell, you need to use the CRM SDK write a small application that will compare the IDs in the Attachment table to those in the ActivityMimeAttachment table and produce a list of Attachments that are not connected to ActivityMimeAttachments, then delete those records.