Out of the box templates
SharePoint Foundation includes Pre-defined alert templates for e-mail and SMS messages
Pre-defined Alert Template Location: %ProgramFiles%\Common Files\Microsoft Shared\web server extensions\14\TEMPLATE\XML
Email Template: AlertTemplates.xml
SMS Template: AlertTemplates_SMS.xml
Template includes information such as the format, contents and properties for the alert email and SMS messages.
Customize alerts
(1) Create a copy of the xml template file that you need to change - AlertTemplates.xml and AlertTemplates_SMS.xml
(2) Modify the copy of the alert template file - (The original template file should not be modified)
You can make changes to the template such as add a button/link, include an image or privacy information, modify the look of the alert etc.
To customize the appearance of alerts, modify the css by using the Format element.
Resource variables listed within the template definition file, such as $Resources:Alerts_anything_filter_shortname, can be found in core.resx (in the local_drive\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\Resources folder)
If you are modifying any element other than the Format element, both the email and the SMS templates should be modified, in parallel - the 2 templates should be in sync except for the Format element.
(3) Load the custom templates by using the command: stsadm -o updatealerttemplates. I haven't found a direct equivalent Powershell command.
If both, the email and the SMS template are updated, the UpdateAlertTemplates command needs to be run twice, once for each of the two files.
(4) Restart IIS