CiviCRM Community Forums (archive)

*

News:

Have a question about CiviCRM?
Get it answered quickly at the new
CiviCRM Stack Exchange Q+A site

This forum was archived on 25 November 2017. Learn more.
How to get involved.
What to do if you think you've found a bug.



  • CiviCRM Community Forums (archive) »
  • Old sections (read-only, deprecated) »
  • Support »
  • Using CiviCRM »
  • Using Import (Moderator: Yashodha Chaku) »
  • best practices re: cleaning temp tables out of db
Pages: [1]

Author Topic: best practices re: cleaning temp tables out of db  (Read 3921 times)

JoeMurray

  • Administrator
  • Ask me questions
  • *****
  • Posts: 578
  • Karma: 24
    • JMA Consulting
  • CiviCRM version: 4.4 and 4.5 (as of Nov 2014)
  • CMS version: Drupal, WordPress, Joomla
  • MySQL version: MySQL 5.5, 5.6, MariaDB 10.0 (as of Nov 2014)
best practices re: cleaning temp tables out of db
January 06, 2010, 12:52:31 pm
Each import generally takes at least a few attempts before all of the errors are corrected. For data sets that include individuals, organizations, memberships, participation, and so on, a fair number of temporary tables get created when importing from csv files. Shouldn't we be dropping these tables? Or is my db filled with a few dozen because we've doing something wrong?
Co-author of Using CiviCRM https://www.packtpub.com/using-civicrm/book

Donald Lobo

  • Administrator
  • I’m (like) Lobo ;)
  • *****
  • Posts: 15963
  • Karma: 470
    • CiviCRM site
  • CiviCRM version: 4.2+
  • CMS version: Drupal 7, Joomla 2.5+
  • MySQL version: 5.5.x
  • PHP version: 5.4.x
Re: best practices re: cleaning temp tables out of db
January 06, 2010, 07:28:43 pm

yes we should, but probably via a cleanup cron job etc. for now i think we should add something to the upgrade script to clean up all the leftover tables. can you please file an issue for this

thanx

lobo
A new CiviCRM Q&A resource needs YOUR help to get started. Visit our StackExchange proposed site, sign up and vote on 5 questions

Piotr Szotkowski

  • I live on this forum
  • *****
  • Posts: 1497
  • Karma: 57
Re: best practices re: cleaning temp tables out of db
January 11, 2010, 06:41:54 am
I’m hesitant to add a hard table cleanup to every failed import, as the tables’ contents might hold some clues why the import failed (and imports shouldn’t fail to begin with).

I’m definitely happy with Lobo’s idea to clean the import tables on upgrades; that makes a lot of sense – first, that’s a point when one’s either generally happy with their imports or wants to try from scratch with the new version; second, I can foresee us changing the tables’ schemas at some point, and having legacy import leftover tables would only add confusion.

I filed CRM-5645 to track this.
If you found the above helpful, please consider helping us in return – you can even steer CiviCRM’s future and help us extend CiviCRM in ways useful to you.

JoeMurray

  • Administrator
  • Ask me questions
  • *****
  • Posts: 578
  • Karma: 24
    • JMA Consulting
  • CiviCRM version: 4.4 and 4.5 (as of Nov 2014)
  • CMS version: Drupal, WordPress, Joomla
  • MySQL version: MySQL 5.5, 5.6, MariaDB 10.0 (as of Nov 2014)
Re: best practices re: cleaning temp tables out of db
January 12, 2010, 05:35:32 am
Thanks, Piotr. I agree with the suggested approach.
Co-author of Using CiviCRM https://www.packtpub.com/using-civicrm/book

Alan.Guggenheim

  • Guest
Re: best practices re: cleaning temp tables out of db
April 12, 2010, 07:17:01 pm
In the mean time, I had a question about this, as my DB is growing, I found these tables and was wondering if it is safe to delete them from the DB?

adixon

  • I post frequently
  • ***
  • Posts: 314
  • Karma: 19
    • Blackfly Solutions
Re: best practices re: cleaning temp tables out of db
April 13, 2010, 07:40:34 am
If you really have a lot of these tables from failed imports, then you probably want to do more tests before you import on your production environment. My recommendation is to never do an import on a production environment until you've tested it on your development/staging server. It can be really ugly to recover from a bad import.

Alan.Guggenheim

  • Guest
Re: best practices re: cleaning temp tables out of db
April 13, 2010, 08:53:55 am
We have been actually in production for over two months now, and the contact db seems fine. I think these were the result of fine tuning imports, in particular duplicate or invalid emails.

Piotr Szotkowski

  • I live on this forum
  • *****
  • Posts: 1497
  • Karma: 57
Re: best practices re: cleaning temp tables out of db
April 13, 2010, 04:43:22 pm
Quote from: Alan.Guggenheim on April 12, 2010, 07:17:01 pm
In the mean time, I had a question about this, as my DB is growing, I found these tables and was wondering if it is safe to delete them from the DB?

The way we’ll do it in CiviCRM 3.2 is shown in this patch – so if you just DROP TABLE civicrm_import_job_… for every outstanding civicrm_import_job_… table it should be ok.
If you found the above helpful, please consider helping us in return – you can even steer CiviCRM’s future and help us extend CiviCRM in ways useful to you.

Alan.Guggenheim

  • Guest
Re: best practices re: cleaning temp tables out of db
April 13, 2010, 04:58:07 pm
Thanks a lot Piotr. Did it and everything looks fine.

Pages: [1]
  • CiviCRM Community Forums (archive) »
  • Old sections (read-only, deprecated) »
  • Support »
  • Using CiviCRM »
  • Using Import (Moderator: Yashodha Chaku) »
  • best practices re: cleaning temp tables out of db

This forum was archived on 2017-11-26.