CiviCRM Community Forums (archive)

lying imports using civicrm 2.1 right? and how many records are you importing? with many fields eachone (dimension in kb of imported files?)

I get this error as well.  I'm on a shared server (BlueHost).  I'm running Civi 2.2.5 under Drupal 6.12
I have php memory set to 128M, BlueHost assures me that full amount is available to use.

I've uploaded the CSV file using FireFox, IE, and Chrome, and the import still fails.
I know there is a relationship between the number of fields imported, and the number of records successfully imported.
The fewer number of fields, the greater number of records that will succeed.

No one has been able to reproduce the problem on the demo server.  I'm willing to bet the demo server is not on a shared host.
I believe the problem arises when the server decides that you've used your allotment of database time and switches context to another user and returns a 500 error to you.

Perhaps, we could intercept that 500 error - close and reopen the database insert and continue on without the program puking.
I'm a php and mysql nube, so I don't know.  Maybe Lobo does.

I think I agree, mostly.  For those people here that have the problem, splitting the input file into smaller chunks does solve the problem.  However - that's a work-around that can be really unpractical depending on the size of the input file, how often you need to upload data, the size of a chunk that can be successful, and who is doing the import.  We are a chapter of a national organization that needs to import 1300 or so records each month.  Our maximum safe chunk is about 250 records, and untechnical volunteers are doing the import.  I'm willing to bet that many such smaller organizations exist that must operate on shared servers. 

What about a throttle mechanism that you can use to specify the number of records that can be safely imported, and close/open the db connection before it dies?  In other words, have a means for civi to do the file split based upon a user-supplied number of records?

I'd love to, but I'm a php noob.  I haven't even found the place where all the magic takes place.  An alternative possibility might be a batch import process that bypasses the interactive screens and uploads all files with a specific prefix.  some_prefix_xxx.csv?

Yep, that's one of the problems of being a small non-profit org.  If we had the money to do stuff like that, we'd have the money to use a dedicated server - and not have the problem in the first place.  I'm willing to try the patch, if you're willing to point me to the appropriate chunks of code.