Solaris Troubleshooting Sendmail: Investigating Sendmail Errors from Syslog
Step 1 - Verify where syslogd will log sendmail syslog records to.
After a Solaris install you will find following entries in /etc/syslogd.conf:
…
mail.debug ifdef(`LOGHOST’, /var/log/syslog, @loghost)
…
…
*.err;kern.debug;daemon.notice;mail.crit /var/adm/messages
This has the effect that:
- syslog will log critical sendmail messages to /var/adm/messages
- if loghost can be resolved, syslogd will log sendmail syslog record to the /var/log/syslog file of that host.
Resolve the ‘loghost’ hostname:
$ getent hosts loghost
172.16.1.1 e450 loghost
$
If loghost points to the same system where the sendmail service your want to troubleshoot is running on, you will find the sendmail syslog records in /var/log/syslog.
If loghost points to a different system, login into this system and verify the contents of /var/log/syslog. If neither is the case, verify the contents of syslod.conf and verify to which host and/or file mail.debug records points to.
Step 2 - Verify the contents of /var/adm/messages.
Syslog messages with priotity ‘mail.crit’ will by default logged to /var/adm/messages.
example:
The following will be logged when the writing permissions to /var/spool/clientmqueue are not sufficient.
... May 20 04:01:12 db7 sendmail[1872]: [ID 801593 mail.crit] NOQUEUE:SYSERR(oracle): can not write to queue directory /var/spool/clientmqueue/(RunAsGid=0, required=1): Permission denied May 20 05:00:01 db7 sendmail[1961]: [ID 801593 mail.crit] NOQUEUE:SYSERR(sys): can not write to queue directory /var/spool/clientmqueue/(RunAsGid=0, required=1): Permission denied ...
Step 3 - Verify the contents of the /var/log/syslog file or equivalent file.
Whenever the delivery status of a mail message changes, sendmail will record this event via syslog.
Example:
…
Apr 16 10:58:56 mymailhost sendmail[24234]: [ID 801593 mail.info] m3G8wu8g024234: to=<joe.foe@extdomain.com>, ctladdr=<john@mymailhost.mydomain.com> (22960/117), delay=00:00:00,
xdelay=00:00:00, mailer=relay, pri=30920, relay=mailhost.extdomain.com. [172.17.1.4], dsn=2.0.0, stat=Sent (m3G8wug1009950 Message accepted for delivery)
…
Each line of information logged looks like this:
date host sendmail[pid]: qid: field1=value,field2=value,field3=value, …
- The date is the month, day, and time that the line of information was logged.
- The host is the name of the host that produced this information.
This can be different from the name of the host on which the logfiles are kept (see above).
- The pid (process id) of the sendmail processes that produced the output.
- The qid (queue identifier) that uniquely identifies each message on a given host.
- The remainder is a list of fields which define values of who the sender or the recipient is
and whether delivery succeeded, failed, or was deferred.
The following is a list of some of these fields.
to= The final recipient
from= The envelope sender
ctladdr= The controlling user
delay= Total time to deliver
xdelay= Transaction delay for this address only
mailer= The delivery agent used
pri= The initial priority
relay= The host that sent or accepted the message
dsn= The DSN status code
stat= The status of delivery
size= The size of the message
ntries= The number of delivery attempts
With the above list you can identify which record(s) points to a local delivery attempt,
a relay attempt, a forwarding attempt, etc…
Step 4 - Verify the stat= field of this record.
This can be in the state queued when the recipient’s mailhost was not accessible. After the same mailhost is accessible again, and after re-queuing of mail messages,
the same message can be in the state Sent as mail was successful delivered. In the case of bounced mail the stat= field will show the reason for the failure.
Step 5 – Verify the dsn= field of this record (when available).
When a mail server for some reason bounces a message, it may notify the envelope sender of the problem with a DSN error code.
The meaning of the DSN error codes are documented in RFC1893 (Enhanced Mail System Status Codes).
example:
…
Apr 11 10:25:29 e450 sendmail[16860]: [ID 801593 mail.info] m3B8PSVM016855: to=<joe@wrongdomain.org>, ctladdr=<john@mydomain.org> (2031/2001), delay=00:00:01, xdelay=00:00:00, ma
iler=esmtp, pri=150549, relay=wrongdomain.org.be, dsn=5.1.2, stat=Host unknown (Name server: wrongdomain.org: host not found)
…
Here you can see that an attempt was made by john@mydomain.org to send out a mail to a user
joe on a non-resolvable domainname.
dsn=5.1.2 is decipherd according to RFC1893
…
5.X.X Permanent Failure
A permanent failure is one which is not likely to be resolved by resending the message in the current form. Some change to the message or the destination must be made for successful delivery.
…
X.1.2 Bad destination system address
The destination system specified in the address does not exist or is incapable of accepting mail. For Internet mail names, this means the address portion to the right of the “@” is invalid for mail. This codes is only useful for permanent failures.
…
Step 6 - Verify the reject= field of this record (when available).
When a mail message is processed by sendmail it will go through some check_rules used of Anti-spam control. (see the ANTI-SPAM CONFIGURATION CONTROL section of the /usr/lib/mail/README file) If one of these rules rejects a mail message, sendmail will log this.
Example:
…
Apr 16 15:38:34 e450 sendmail[17707]: [ID 801593 mail.notice] m3GDcYW8017707: ru leset=check_mail, arg1=<nekkipfa1981@54sales.com>, relay=to1-84-91-48-146.netvis ao.pt [84.91.48.
146], reject=553 5.1.8 <nekkipfa1981@54sales.com>… Domain of sender address nekkipfa1981@54sales.com does not exist
…
and
…
Apr 16 16:50:54 e450 sendmail[18581]: [ID 801593 mail.notice] m3GEorFQ018581: ru leset=check_rcpt, arg1=<a286e4184@yahoo.com.tw>, relay=203-73-236-153.adsl.dynam ic.seed.net.tw [
203.73.236.153], reject=550 5.7.1 <a286e4184@yahoo.com.tw>… Relayin Solaris 9 Operating System
Solaris 8 Operating System







4 Comments on “Solaris Troubleshooting Sendmail: Investigating Sendmail Errors from Syslog”
Thanks Ram . Nice article.
Can you post more on SMTP configurations and troubleshooting techniques.
@Babaji –
Sure, I am adding it to writing Queue
Thank You Ram, learned about it .Yesterday faced issue where we are getting system undeliverable message when run mailx with -r option .It was working for the user until day before yestrday but stopped working.Any idea that why such error comes. Exchange team says that they have no idea about .
@manish – below posts might help you for the sendmail troubleshooting. http://wp.me/p1EO9J-2y & http://wp.me/p1EO9J-2L