CiviCRM Community Forums (archive)

*

News:

Have a question about CiviCRM?
Get it answered quickly at the new
CiviCRM Stack Exchange Q+A site
This forum was archived on 25 November 2017. Learn more.
How to get involved.
What to do if you think you've found a bug.



  • CiviCRM Community Forums (archive) »
  • Old sections (read-only, deprecated) »
  • Support »
  • Using CiviCRM »
  • Using Import (Moderator: Yashodha Chaku) »
  • File upload size
Pages: [1]

Author Topic: File upload size  (Read 2447 times)

ewanmclean

  • I post occasionally
  • **
  • Posts: 36
  • Karma: 0
File upload size
January 08, 2014, 08:36:49 pm
Hello

How do I increase the allowed CSV file import upload size limit? It is set by default to 2 MB. Please note this is not a PHP limit, it's in Civi somewhere. My PHP limit is 10MB as confirmed by phpinfo and drupal.

Thanks

Eileen

  • Forum Godess / God
  • I’m (like) Lobo ;)
  • *****
  • Posts: 4195
  • Karma: 218
    • Fuzion
Re: File upload size
January 08, 2014, 08:49:39 pm
http://drupal.demo.civicrm.org/civicrm/admin/setting/misc?reset=1
Make today the day you step up to support CiviCRM and all the amazing organisations that are using it to improve our world - http://civicrm.org/contribute

ewanmclean

  • I post occasionally
  • **
  • Posts: 36
  • Karma: 0
Re: File upload size
January 14, 2014, 12:36:42 pm
I thought this was the jackpot but changing the value here hasn't changed it on the import screen and its still rejecting larger uploads.

ewanmclean

  • I post occasionally
  • **
  • Posts: 36
  • Karma: 0
Re: File upload size
January 14, 2014, 07:02:23 pm
I've upgraded to 4.4 today with the result that the import limit has decreased to 1MB. I can trace this variable back to CRM/Core/Config/Variables.php but altering it doesn't seem to do anything. There is a //FIXME in CRM/Import/DataSource/CSV.php saying why limit it to 8MB if the config limit is different. I can play around with these lines and get the limit to change on screen, but then the upload fails.

ewanmclean

  • I post occasionally
  • **
  • Posts: 36
  • Karma: 0
Re: File upload size
January 15, 2014, 04:44:57 pm
Fixed this by adding an extra line like $size=$size*8 which gave me an 8MB limit. Same fix was needed across various Datasource.php files for activity, contribution, etc.

gharris

  • I post occasionally
  • **
  • Posts: 32
  • Karma: 0
  • CiviCRM version: new
  • CMS version: new
  • MySQL version: new
  • PHP version: new
Re: File upload size
February 05, 2014, 08:04:14 pm
YIKES!   :o  Did you report this as a bug?  I'm seeing the same behavior on a WP based install.

gharris

  • I post occasionally
  • **
  • Posts: 32
  • Karma: 0
  • CiviCRM version: new
  • CMS version: new
  • MySQL version: new
  • PHP version: new
Re: File upload size
February 10, 2014, 12:21:54 pm
AHA!  The math is wrong!  Instead of dividing by 1024 * 1024, it should be 1024 * 8.  Now to try and figure out how to get the bug reported and fixed.

gharris

  • I post occasionally
  • **
  • Posts: 32
  • Karma: 0
  • CiviCRM version: new
  • CMS version: new
  • MySQL version: new
  • PHP version: new
Re: File upload size
February 11, 2014, 09:14:50 am
Well, apparently I tested the wrong file.   ???  That only changed what's posted on the form to the correct value.  It did not change the actual upload size.

Interpolat

  • I post frequently
  • ***
  • Posts: 140
  • Karma: -1
    • Interpolat Solutions, LLC.
  • CiviCRM version: 4.4+
  • CMS version: Wordpress
  • MySQL version: 5.5+
  • PHP version: 5.4+
Re: File upload size
February 21, 2014, 06:40:25 pm
It seems to me that there may be an influence from the environment at play here.  So I downloaded CiviCRM 4.4.4 and installed on Joomla 3.2.  The default limit was set at 8MB in the contacts import ui.  I looked at CiviCRM 4.4.4 on Joomla 2.5.8, and there, the default is 1MB.  Also, another difference I noted between the two setups, which may or may not be the cause of the difference in the default limit size, was the version of PHP installed.  The Joomla 3.2 was running on an environment with PHP 5.4.22.  The Joomla 2.5.8 install was on PHP 5.2.19.  Hope this sheds some light.  A patch would be real handy right now, until 4.5 comes out.
Interpolat Solutions, LLC
Connecting the dots for you!
http://www.interpolat.com

Max Hunter

  • I’m new here
  • *
  • Posts: 21
  • Karma: 1
  • CiviCRM version: 4.4.x
  • CMS version: Drupal 7
Re: File upload size
March 04, 2014, 08:24:26 am
I'm also seeing this behavior after an upgrade to 4.4 -- the max upload size in /civicrm/admin/setting/misc?reset=1 and php.ini are both 8MB, but all uploads larger than 1MB are failing.  Has anyone found a workaround?

sonicthoughts

  • Ask me questions
  • ****
  • Posts: 498
  • Karma: 10
Re: File upload size
March 20, 2014, 10:47:55 am
I have tried changing several variables but can't even hack the code :).  I see this logged as: CRM-14199 but tagged as trivial.  I know most guru's use other import tools but this seems to be more that trivial - no? 

Jeremy Proffitt

  • I post occasionally
  • **
  • Posts: 63
  • Karma: 2
    • Mobius New Media
  • CiviCRM version: 4.4.x
  • CMS version: Joomla 2.5.x/3.x / D7
  • MySQL version: 5.1.x
  • PHP version: 5.3.10+, 5.4.x
Re: File upload size
March 20, 2014, 11:00:16 am
I don't know that I'd consider it trivial, but in the meantime, here are a couple of other methods:

- Create a DB table and import the CSV file to the table. You can then use the SQL import option in the importer.

- Create a custom script using the SPLFileObject to iterate through the CSV file and use the Civi API to import each record. This actually gives you a lot more flexibility than the importer does.

http://wiki.civicrm.org/confluence/display/CRMDOC/Using+the+API#UsingtheAPI-ExploringtheAPIanddocumentation
http://wiki.civicrm.org/confluence/display/CRMDOC/Bootstrap+Reference
Jeremy Proffitt
Mobius New Media
IRC: JP_EzoD

gharris

  • I post occasionally
  • **
  • Posts: 32
  • Karma: 0
  • CiviCRM version: new
  • CMS version: new
  • MySQL version: new
  • PHP version: new
Re: File upload size
March 24, 2014, 09:09:03 am
Jeremy,

I understand that it gives a great deal more flexibility, but the CSV routines seem to take care of a lot of behind-the-scenes mapping and auto-generation that I didn't get the idea that the API does.  Did I miss something?   ???  I'm thinking that the CSV routines seem to be much more comprehensive on the first pass and that the API would be better for filling in additional details.  Am I off?  Does the API provide everything that the CSV does and then some?

sonicthoughts

  • Ask me questions
  • ****
  • Posts: 498
  • Karma: 10
Re: File upload size
March 25, 2014, 08:16:48 am
Thanks - This is very helpful but it is really not a valid workaround.  I'm surprise more people haven't flagged it.  Perhaps it is specific to my environment.

There are big differences with the API method since the standard import workflow includes many related fields (like custom fields, address) that require multiple passes.  Also, many users expect the standard workflow to work, without requiring shell access.  Also, the error message does not actually say that it is related to the file size so it requires pulling hair out a bit :). 

It's no longer an issue for me, but If someone can point me to the right file, I'd be happy to help - hope it get attention in the issue queue for the sake of others....
Pages: [1]
  • CiviCRM Community Forums (archive) »
  • Old sections (read-only, deprecated) »
  • Support »
  • Using CiviCRM »
  • Using Import (Moderator: Yashodha Chaku) »
  • File upload size

This forum was archived on 2017-11-26.