dompdf too slow for serious use

[lathos]

I've got a client who is seeing lots and lots of timeouts, memory exceed etc. generating a PDF form letter for 150 people. There are a few thousand people in the database, so this is a very small subset, and we'll need to do bigger PDF letter runs than this one.

I've tried upping the memory and allowed run time. Memory usage maxes out at about 140M, but runtime is very difficult to set correctly.

Figuring that the problem is with dompdf, I upgraded dompdf to the latest nightly. Still no improvement.

Then I decided to cut out CiviCRM entirely, and run a small timed test which generates a few plain paragraphs of text and feeds them directly to dompdf. Here are the timings per number of paragraph:

1 paragraphs      0.4611s
5 paragraphs      0.944s
10 paragraphs     1.604s
100 paragraphs    14.162s
200 paragraphs    30.938s
400 paragraphs    60.08s

So it's O(n), which I guess isn't too catastrophic. But each form letter has about 20 paragraphs. 20x150 = 3000 paragraphs, so we should expect the print generation to take about 9 minutes. No wonder Apache is timing out.

If I want to print a form letter for the whole database, should I set the Apache timeout to an hour and a half? Can it even go that high?

Given these timings I'm not sure it makes sense for CiviCRM to use dompdf, even the very latest version. It's just way too slow for large print runs. Others have had similar complaints. (CRM-8459, http://forum.civicrm.org/index.php?topic=10511.0 etc.) The suggest "fix" for other dompdf issues within CiviCRM has been to turn PDF generation off! (http://forum.civicrm.org/index.php?topic=19801.15) But obviously this will not be much of a fix for generating PDF form letters...

It would be good if there would be at least an option to use a different PDF rendering engine. By way of comparison, wkhtmltopdf did 3000 paragraphs in 3.11 seconds.

Is it worth my trying to write a patch to detect if wkhtmltopdf is installed and if so to use it in preference to dompdf for generating form letters?

[xavier]

Sounds like an interesting option.

Have you thought about generating and printing directly html? Something I did recently to print badges, could get super good positioning and damn good speed (was for registration desks, speed is key).

Used the wonderful jsPrintSetup extension for firefox (allows to set margins and so on).

Another solution could be to separate the order "print pdf" and the processing. On 4.1 that's way easier to create a new job that would generate the pdf from the cli and send an email with the pdf ready to be downloaded to the user.

X+

[lathos]

Well, here's a patch anyway. It uses the snappy library.

[Donald Lobo]

can u please file an issue for this and attach the patch. Can you please link to this forum topic in the issue

thanx

lobo

[lathos]

Have an issue report: http://issues.civicrm.org/jira/browse/CRM-9334

[jamie]

I'm really glad to find this post .

If anyone gets the error on a Debian/Ubuntu server:

wkhtmltopdf: cannot connect to X server

You don't have to follow the directions on the net that tell you to download a static copy of the wkhtmltopdf binary (which means you will not get updates).

Instead, install xvfb:

Code: [Select]

apt-get install xvfb
Then, create a file (can be anywhere and named anything, I suggest: /usr/local/bin/wkhtmltopdf-wrapper) with the contents:

Code: [Select]

#!/bin/sh

converter=$(which wkhtmltopdf)
fb=$(which xvfb-run)

if [ ! -x "$converter" ]; then
printf "Failed to find wkhtmltopdf binary. Try: apt-get install wkhtmltopdf\n" >&2
exit 1
fi
if [ ! -x "$fb" ]; then
printf "Failed to find xvfb-run binary. Try: apt-get install xvfb\n" >&2
exit 1
fi

"$fb" "$converter" "$@"
Ensure it is executable:

chmod 755 /usr/local/bin/wkhtmltopdf-wrapper

Then, in place of the path to wkhtmltopdf in your civicrm configuration, use: /usr/local/bin/wkhtmltopdf-wrapper

[509tyler]

Is this patch still current with 4.3.3+ ?

I am also running into the same timeout issue on a dedicated VPS (not shared hosting).

[509tyler]

This apparently has been implemented into core on this page:
http://drupal.demo.civicrm.org/civicrm/admin/setting/misc?reset=1

Just wanted to let any wayward forum user's find some answers that were not all the apparent to me. Thanks!

[mathieu]

Instead of the wrapper script, should we add support for "xvfb-run"?

In the CiviCRM admin settings, we would enter:

Code: [Select]

/usr/bin/xvfb-run -- wkhtmltopdf
but this requires patching CRM/Admin/Form/Setting/Miscellaneous.php, formRule(), otherwise CiviCRM will say that the binary could not be found.

This seems to be the recommended way documented in /usr/share/doc/wkhtmltopdf/README.Debian of the Debian package.

[Donald Lobo]

i think this just requires a small patch to the formrule on the admin page. I suspect the current form rule is expecting only one string (name of executable)

should be a real simple patch and we can add it to 4.4

lobo

[jamie]

Ah - that's much simpler.

Just patched it here: http://issues.civicrm.org/jira/browse/CRM-13292