No, diamonds are one of the few materials harder than an anodized finish. When you brush your hand or fingernail against a brushed anodized surface, the ridges in the surfaces will remove skin or fingernail cells and material, leaving something that looks like a scratch, which is why I paraphrased the word 'scratch' as I am doing here. Of course its really a streak. This can happen with metal tools too.
Of course, if you break through the finish, the aluminum beneath is rather soft. I hope this clears this up.