..a dose of zero-day know-hows ..

5/21/2009

Camiro-CMS: 1st Beta Release

5/17/2009

Using PHPMailer and Crontab in Enqueuing/Sending System Generated Emails

This article covers how to use PHPMailer and Crontab to send Emails via an SMTP server without having to directly trigger the actual email sending task via the web process. This implementation gives a performance gain especially to the systems wherein most actions need to trigger email sending (eg. Notification systems in ERP's, CRM's and CMS's).


Scenario
Most of the systems I have dealt with trigger the sending of Emails from the Web server through the web request. An example of this scenario are Notification Emails being sent to the recipients when a certain action is triggered/accomplished.

Although this is a common approach for most systems eg. CMS, CRM etc., I find it inefficient as the execution speed of the script which sends the actual email is dependent to the SMTP server set to be used, in such events, if the SMTP serves lags/fails on responding, the script which invokes fsckopen will wait until the STMP server answers which might cause the script to timeout after some moments of waiting.

Solution:
Instead of attempting to send the email directly along the web process, it is wise to enqueue the email by storing it on a database, and let crontab pick those entries up and send the emails using the php cli by accessing a PHP Email Class such as PHPMailer. The sent emails in the database can then be flagged as 'sent' so the query which picks the emails up can exclude sent emails via "WHERE `sent` != '1' LIMIT ...".

This is a sample implementation (PHPMailer)

A. Setup the Database Table

The first thing that needs to be done is to create a table where the enqueued emails will be stored. This sample table is generic enough for most purposes, but feel free to expand as deem fit.
CREATE TABLE  `outgoing_emails` (
`id` int(9) NOT NULL auto_increment,
`recepient` text,
`subject` text,
`body` text,
`mode` int(1) default '0',
`date_entered` datetime default NULL,
`sent` int(1) default '0',
PRIMARY KEY (`id`)
) ENGINE=MyISAM AUTO_INCREMENT=1 DEFAULT CHARSET=utf8
In the table above, the basic parts of an Email has columns which correspond to arguments that can be passed to the phpmailer Send() method.

B. Setup the PHPMailer Class
Download PHPMailer (PHPMailer 2.0.4 is used for this example, although other versions may be used), and uncompress it as appropriate.

C. Setup the actual email enqueuing and sending script
This script will be used to access the PHPMailer class, enqueue and send the emails (in short, it will be the all-in-one script which will do all the job). This script will be accessed by cron and execute it via PHP cli.

Note: This is a modified version of the functions used by Mambo CMS to send out emails and is found in all versions up to 4.6.x series. This is a very good implementation anyways and I am lazy to write my own for the sake of demo-ing:



Copy and paste the code above and save as enqueue_emails.php. You may need to modify lines 1-14 to set proper MySQL and SMTP credentials, as well as the path where you extracted the PHPMailer package.

D. Setup the Cron job
Next is set the crontab to run enqueue_emails.php, eg. if you intend to send the email in batches every 2 minutes, you would need to set the cron to the following:

*/2 * * * * php -f /var/www/mail/enqueue_emails.php


E. Reroute the email sending lines in your script to use the 'enqueue_emails()' function
Take it for a spin, An example would be:

require_once( 'enqueue_emails.php' );
enqueue_email('test@example.com', 'email subject', 'email body');

Wait for 2 minutes and let cron pick the email up and send it to the recipient, wait until it arrives in the inbox and And you're done.

For any questions, follow-ups, feel free to leave a comment, I hope this has been helpful.

5/01/2009

Fix: SugarCRM 5.1.2x Inbound Email -> Plain Text attachments getting Discarded

In SugarCRM 5.1.2c and up, you will notice that the Inbound Email functionality fails to pickup plain text attachments, this affects files with csv, txt extenstions and other files with plain text mime type. You should be able reproduce this issue by sending an email to an address that is monitored by SugarCRM, once cron invokes cron.php, the emails will be fetched but the plain text attachments are nowhere in the email.

It took me awhile fixing our company instance only to figure out the culprit was only one line which was added upon my upgrade (from 5.1.2a or 5.1.2b or from 5.1.2b to 5.1.2c I cannot exactly pinpoint)

The fix was simple, simply change line 3385 (approx) of /modules/InboundEmails/InboundEmail.php from

if(strtolower($part->disposition) == 'attachment' || 
((strtolower($part->disposition) == 'inline'))
&& $part->type != 0) {
to
if(strtolower($part->disposition) == 'attachment' || 
((strtolower($part->disposition) == 'inline'))) {
You will notice that with the unchanged code, there is no way for attachments whose $part->type is '0' (plain text) to get parsed and be attached in the email object.

I hope this spares someone form hair pulling I had to go through

12/06/2008

splitting large sql files using split

The split command have a couple of essential switches, one may either split per line,per bytes or line per bytes. Lets see which is best applicable for SQL dumps.

Splitting per line:
Splitting a very large SQL file generated from mysqldump will not be optimal and effective to be splitted using the per line switch, simply because data from one table will take up one line only regardless how large it is making the split almost useless.

Splitting per byte:
Splitting an SQL file per byte can be effective at times, but there is no guarantee that the the input file will be splitted in the proper string positions. SQL errors is likely to happen in situations where the splitted sql files contain incomplete query at the end because it is terminated in between a line.

Splitting line per byte:
This is the best way to split a huge sql file. It splits the SQL file in bytes specified but takes into account the line so it does not split the SQL file in between. Ex:


split -C 40312832 data.sql 


In the example above, data.sql is splitted into several files (xaa, xab xac... xaz..) and tries to fit as much line to the specified bytes. Although the output splitted files will not have the exact size as the byte specified, split will try to fill as much but making sure that no line is truncated between, thus assuring that the query will be valid upon execution.

Availability:
split command is part of the GNU coreutils and is available in almost all linux distributions and *nix based operating systems. In some OS's like Solaris 9 Sparc, you may need to upgrade the coreutils package that is prebundled with the system to have access to split binary that supports the -C switch.

11/20/2008

php fsockopen problems when connecting to ssl and tls mail servers

Few weeks ago I was stumped by some error that took me sometime to resolve concerning PHP Mailer - having it fetch emails from Gmail. After grepping through logs, I discovered few fsockopen errors stating wrong function parameter issues. The logs states:


Warning: fsockopen() expects parameter 2 to be long

I have gone and checked if the PHP Mailer version I use supports connecting via SSL, and it appeared it didn't, in effect I upgraded PHP Mailer to version 2.03. Everything worked fine in my local development server, but behold, yet another error only occurring in our live server, it was:


Warning: fsockopen() [function.fsockopen]:
unable to connect to ssl://smtp.gmail.com:465
(Unable to find the socket transport "ssl" -
did you forget to enable it when you configured PHP?)

The error message was obvious enough for me to realize that I did not compile the php on our server with ssl support. The final resolution was I re-compiled PHP with the following params:


'./configure'
'--with-apxs2=/usr/local/apache2/bin/apxs'
'--with-mysql=/usr/local/mysql/'
'--enable-mbstring'
'--with-gd'
'--with-jpeg-dir=/usr/local/lib'
'--with-png-dir=/usr/local/lib'
'--with-zlib' '--with-curl'
'--with-gettext'
'--with-ncurses'
'--with-imap=/usr/local/imap/imap-2006c1/'
'--with-imap-ssl=/usr/local/ssl/'
'--with-openssl'
'--enable-sockets'

The '--with-openssl' part was just what I needed. After recompiling and restarting httpd, everything worked fine.