CiviCRM Community Forums (archive)

*

News:

Have a question about CiviCRM?
Get it answered quickly at the new
CiviCRM Stack Exchange Q+A site

This forum was archived on 25 November 2017. Learn more.
How to get involved.
What to do if you think you've found a bug.



  • CiviCRM Community Forums (archive) »
  • Old sections (read-only, deprecated) »
  • Support »
  • Using CiviCRM »
  • Using Core CiviCRM Functions (Moderator: Yashodha Chaku) »
  • IDS problems when saving an Activity with lots of text
Pages: [1]

Author Topic: IDS problems when saving an Activity with lots of text  (Read 1367 times)

petednz

  • Forum Godess / God
  • I’m (like) Lobo ;)
  • *****
  • Posts: 4899
  • Karma: 193
    • Fuzion
  • CiviCRM version: 3.x - 4.x
  • CMS version: Drupal 6 and 7
IDS problems when saving an Activity with lots of text
January 24, 2012, 03:09:15 pm
I think this problem is occurring purely because a lot of text is being pasted in to fields that are used for an Activity.

I get "There is a validation error with your HTML input. Your activity is a bit suspicious, hence aborting"

But there is no HTML involved. Just lots of text being added to a bunch of NOTE fields.

After adding and saving a few fields, when I add some more text to other fields I get the above.

Debugging/backtrace doesn't spit out anything obviously helpful to me yet

Code: [Select]
/sites/fuzion7/sites/www.outdoorsnz.org.nz/modules/civicrm/CRM/Core/Error.php, backtrace, 296
/sites/fuzion7/sites/www.outdoorsnz.org.nz/modules/civicrm/CRM/Core/IDS.php, fatal, 231
/sites/fuzion7/sites/www.outdoorsnz.org.nz/modules/civicrm/CRM/Core/IDS.php, kick, 165
/sites/fuzion7/sites/www.outdoorsnz.org.nz/modules/civicrm/CRM/Core/IDS.php, react, 144
/sites/fuzion7/sites/www.outdoorsnz.org.nz/modules/civicrm/CRM/Core/Invoke.php, check, 88
/sites/fuzion7/sites/www.outdoorsnz.org.nz/modules/civicrm/drupal/civicrm.module, invoke, 447
, civicrm_invoke,
/sites/fuzion7/includes/menu.inc, call_user_func_array, 503
/sites/fuzion7/index.php, menu_execute_active_handler, 21

On dev site I have turned off IDS and succeeded in saving the Activity fine.

Not keen to have to leave IDS off.

Is this something to do with the sheer size of the POST request hitting a maximum number of characters for the form?
Sign up to StackExchange and get free expert advice: https://civicrm.org/blogs/colemanw/get-exclusive-access-free-expert-help

pete davis : www.fuzion.co.nz : connect + campaign + communicate

petednz

  • Forum Godess / God
  • I’m (like) Lobo ;)
  • *****
  • Posts: 4899
  • Karma: 193
    • Fuzion
  • CiviCRM version: 3.x - 4.x
  • CMS version: Drupal 6 and 7
Re: IDS problems when saving an Activity with lots of text
January 24, 2012, 03:40:09 pm
Saw this in the log in case it sheds light (unlikely)
[ip] => 192.168.0.102 [reaction] => 3 [impact] => 91 ) [6] => Array ( [name] => custom_38_20 [value]

Also with IDS off, I tried resubmitting the data from the Webform (which is the original source of the data in theory - but has not been in the testing above that showed the error was happening directly when editing an Activity)
And got this
PEAR_Exception: DB Error: unknown error in CRM_Core_Error::exceptionHandler() (line 532 of /sites/fuzion7/sites/www.outdoorsnz.org.nz/modules/civicrm/CRM/Core/Error.php).

Sign up to StackExchange and get free expert advice: https://civicrm.org/blogs/colemanw/get-exclusive-access-free-expert-help

pete davis : www.fuzion.co.nz : connect + campaign + communicate

petednz

  • Forum Godess / God
  • I’m (like) Lobo ;)
  • *****
  • Posts: 4899
  • Karma: 193
    • Fuzion
  • CiviCRM version: 3.x - 4.x
  • CMS version: Drupal 6 and 7
Re: IDS problems when saving an Activity with lots of text
January 24, 2012, 06:16:18 pm
so trying to replicate on demo with Activity with lots of fields and lots of text and getting different error - unrelated?
Code: [Select]
Database Error Code: Got error 139 from storage engine, 1030
Additional Details:
Array
(
    [callback] => Array
        (
            [0] => CRM_Core_Error
            [1] => handle
        )

    [code] => a515ac9c2796ca0e23adbe92c68fc9fc
    [message] => DB Error: a515ac9c2796ca0e23adbe92c68fc9fc
    [mode] => 16
    [debug_info] => UPDATE civicrm_value_fuzion_activity_test_4  SET fuzion1_7 = 'Lorem ipsum dolor sit amet,
snip
Code: [Select]
[type] => DB_Error
    [user_info] => UPDATE civicrm_value_fuzion_activity_test_4  SET fuzion1_7 = 'Lorem
SNIP
r\n',radio1_15 = 1,radio2_16 = 1,radio3_17 = 1,radio4_18 = 1,radio5_19 = 1,radio6_20 = 1,radio7_21 = 1,radio8_22 = 1,fuzion9_23 = 'Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Aenean commodo ligula eget dolor. Aenean massa. Cum
SNIP
 vitae, justo. Nullam dictum felis eu pede mollis pretium. Integer tincidunt. Cras dapibus. Vivamus elementum semper nisi. Aenean vulputate eleifem nulla eros, ultricies sit amet, nonummy id, imperdiet feugiat, pede. Sed lectus. Donec mollis hendrerit risus. Phasellus nec sem in justo pellen. Maecenas vestibulum mollis\r\n',fuzion10_24 = 'Lorem
SNIP
  WHERE  id = 1 [nativecode=1030 ** Got error 139 from storage engine]"]
)
Sign up to StackExchange and get free expert advice: https://civicrm.org/blogs/colemanw/get-exclusive-access-free-expert-help

pete davis : www.fuzion.co.nz : connect + campaign + communicate

petednz

  • Forum Godess / God
  • I’m (like) Lobo ;)
  • *****
  • Posts: 4899
  • Karma: 193
    • Fuzion
  • CiviCRM version: 3.x - 4.x
  • CMS version: Drupal 6 and 7
Re: IDS problems when saving an Activity with lots of text
January 24, 2012, 06:16:55 pm
Above from adding more txt and setting all Yes buttons here http://drupal.demo.civicrm.org/civicrm/activity/add?reset=1&atype=46&action=update&reset=1&id=544&cid=103&context=activity
Sign up to StackExchange and get free expert advice: https://civicrm.org/blogs/colemanw/get-exclusive-access-free-expert-help

pete davis : www.fuzion.co.nz : connect + campaign + communicate

torrance123

  • I post occasionally
  • **
  • Posts: 57
  • Karma: 3
  • CiviCRM version: 4.0
  • CMS version: Drupal 7
  • MySQL version: 5.0.91
  • PHP version: 5.3.3
Re: IDS problems when saving an Activity with lots of text
January 29, 2012, 05:34:30 pm
The error 139 appears to be due to us running up into mysql limit: we're exceeding the maximum row limit.

To quote two sources from mysql:

Quote
The maximum row length, except for variable-length columns (VARBINARY, VARCHAR, BLOB and TEXT), is slightly less than half of a database page. That is, the maximum row length is about 8000 bytes. LONGBLOB and LONGTEXT columns must be less than 4GB, and the total row length, including BLOB and TEXT columns, must be less than 4GB. (http://dev.mysql.com/doc/refman/5.0/en/innodb-restrictions.html)

And:

Quote
The maximum row length, except BLOB and TEXT columns, is slightly less than half of a
database page, that is, the maximum row length is about 8000 bytes. LONGBLOB and LONGTEXT
columns are allowed to be < 4 GB, and the total row length, including also BLOB and TEXT
columns, must be < 4 GB. Of a BLOB and TEXT column InnoDB stores the first 512 bytes in
the row, and the rest to separate pages. (http://bugs.mysql.com/bug.php?id=1611)

This suggests that if we were to have just 15 (or 16) text fields (let alone other fields), each which had at least 512bytes of data in, we'd exceed the maximum mysql row size. This is because, even though text fields are stored out of the database row, the first 512 bytes are stored in the row (presumably for indexing).  Given that this activity has 150 fields, it seems we'll easily come up against this limit.

Have other people come up against this limit? Should this limit of fields be made clearer in the documentation?
« Last Edit: January 29, 2012, 05:37:21 pm by torrance123 »

Donald Lobo

  • Administrator
  • I’m (like) Lobo ;)
  • *****
  • Posts: 15963
  • Karma: 470
    • CiviCRM site
  • CiviCRM version: 4.2+
  • CMS version: Drupal 7, Joomla 2.5+
  • MySQL version: 5.5.x
  • PHP version: 5.4.x
Re: IDS problems when saving an Activity with lots of text
January 30, 2012, 07:16:53 am

Yes this has come up on the forums a few times and folks have tripped against it. I suspect its less than 1% of the times, i.e. the 99% of the user base is not affected :)

In general when designing custom groups / fields, people should think of them as database tables / schema (which is exactly what they are) and hence follows the same limits as mysql tables.

lobo
A new CiviCRM Q&A resource needs YOUR help to get started. Visit our StackExchange proposed site, sign up and vote on 5 questions

Pages: [1]
  • CiviCRM Community Forums (archive) »
  • Old sections (read-only, deprecated) »
  • Support »
  • Using CiviCRM »
  • Using Core CiviCRM Functions (Moderator: Yashodha Chaku) »
  • IDS problems when saving an Activity with lots of text

This forum was archived on 2017-11-26.