CiviCRM Community Forums (archive)

*

News:

Have a question about CiviCRM?
Get it answered quickly at the new
CiviCRM Stack Exchange Q+A site
This forum was archived on 25 November 2017. Learn more.
How to get involved.
What to do if you think you've found a bug.



  • CiviCRM Community Forums (archive) »
  • Old sections (read-only, deprecated) »
  • Developer Discussion (Moderator: Donald Lobo) »
  • Hardware and modifications needed for 1 million records
Pages: [1]

Author Topic: Hardware and modifications needed for 1 million records  (Read 1401 times)

roballen

  • Guest
Hardware and modifications needed for 1 million records
August 07, 2008, 04:39:18 am
Hi,

I am looking at a proposal where I may need to import, dedupe and work with 1 million contact records. I have seen some posts around this where people had to make modifications, remove Foriegn keys etc..

Is there an accepted route/methodology for handling large amounts of records in CiviCRM or is it more of a 'use the dart arts of a DBA' route.

Any ideas on the hardware needed if you run internally or the hosting set up you would require if running it through the web.

Many Thanks

Rob

Donald Lobo

  • Administrator
  • I’m (like) Lobo ;)
  • *****
  • Posts: 15963
  • Karma: 470
    • CiviCRM site
  • CiviCRM version: 4.2+
  • CMS version: Drupal 7, Joomla 2.5+
  • MySQL version: 5.5.x
  • PHP version: 5.4.x
Re: Hardware and modifications needed for 1 million records
August 07, 2008, 12:30:29 pm

I think there are quite a few other factors to consider besides 1 million records. There are lots of other tables involved (email/phone./groups etc) and those may be of similar / larger size based on org. Also max number of users accessing the db at any point is a consideration

There is no documented way on how to handle this, so would be great if you could document some of the things you did / tweaked while u get this up and running

note that the 2.x series has significant schema and performance changes over 1.x to allow large installs to scale better. 2.1 has some more efficiency and db improvements

lobo
A new CiviCRM Q&A resource needs YOUR help to get started. Visit our StackExchange proposed site, sign up and vote on 5 questions

roballen

  • Guest
Re: Hardware and modifications needed for 1 million records
August 08, 2008, 12:51:59 am
Thanks,

If we go ahead then I will do a basic report back. The information regarding the performance and DB changes in 2.x is very interesting.

Best Regards

Rob
Pages: [1]
  • CiviCRM Community Forums (archive) »
  • Old sections (read-only, deprecated) »
  • Developer Discussion (Moderator: Donald Lobo) »
  • Hardware and modifications needed for 1 million records

This forum was archived on 2017-11-26.