CiviCRM Community Forums (archive)

*

News:

Have a question about CiviCRM?
Get it answered quickly at the new
CiviCRM Stack Exchange Q+A site

This forum was archived on 25 November 2017. Learn more.
How to get involved.
What to do if you think you've found a bug.



  • CiviCRM Community Forums (archive) »
  • Old sections (read-only, deprecated) »
  • Support »
  • Using CiviCRM »
  • Using Core CiviCRM Functions (Moderator: Yashodha Chaku) »
  • Loads of civicrm_log went missing: old bug or issue with upgrade?
Pages: [1]

Author Topic: Loads of civicrm_log went missing: old bug or issue with upgrade?  (Read 503 times)

xavier

  • Forum Godess / God
  • I’m (like) Lobo ;)
  • *****
  • Posts: 4453
  • Karma: 161
    • Tech To The People
  • CiviCRM version: yes probably
  • CMS version: drupal
Loads of civicrm_log went missing: old bug or issue with upgrade?
March 24, 2013, 04:30:22 am
Hi,

I have an old civicrm that started in 2008 was upgraded up to 4.2 At one point, something did happen or someone did something, and now about 1/2 of the contacts don't have any record into civicrm_log (even the creation record).

Looking at the logs, I have still some records from 2008 and records from all years.

Code: [Select]
select year(modified_date),count(*) from civicrm_log group by year(modified_date);
+---------------------+----------+
| year(modified_date) | count(*) |
+---------------------+----------+
|                2008 |      291 |
|                2009 |     9326 |
|                2010 |     9099 |
|                2011 |    11438 |
|                2012 |     6016 |
|                2013 |      336 |
+---------------------+----------+

I've checked which contacts are without log, didn't notice a pattern, it's a mix of individual and organisations, recent and old contacts

Code: [Select]
select c.id, c.display_name from civicrm_contact as c where not exists (select * from civicrm_log where c.id=civicrm_log.entity_id and entity_table= "civicrm_contact");
I did run that query on several databases on several servers. It seems to be a common issue shared, but only on old install that have been upgraded.

It seems that "best groomed" databases (where users merge regularly contacts for instance) are more affected than others, comparatively to the size of their databases.

Could you try running the above query on your database, see if you are affected as well?

More importantly: is this a known bug and is this fixed already?

We discovered the issue as part of a spring cleaning, any suggestion to try to identify when a contact that has lost the log record has been changed last?
-Hackathon and data journalism about the European parliament 24-26 jan. Watch out the result

Donald Lobo

  • Administrator
  • I’m (like) Lobo ;)
  • *****
  • Posts: 15963
  • Karma: 470
    • CiviCRM site
  • CiviCRM version: 4.2+
  • CMS version: Drupal 7, Joomla 2.5+
  • MySQL version: 5.5.x
  • PHP version: 5.4.x
Re: Loads of civicrm_log went missing: old bug or issue with upgrade?
March 24, 2013, 07:09:23 am

taking a totally random guess here:

Seems like there is one (or more) workflow(s) within the code that does not create a contact log record and hence the below. I dont think this is reported before. Might be good to do the same analysis on civicrm.org and see if we can detect a pattern there

however, now that we are using triggers in core civicrm might be a lot easier and more accurate to remove the php code and replace them with triggers. Might also help to augment the log / schema to indicate created date records vs modiied dates (CREATE vs UPDATE). A good 4.4 project maybe?

lobo

A new CiviCRM Q&A resource needs YOUR help to get started. Visit our StackExchange proposed site, sign up and vote on 5 questions

xavier

  • Forum Godess / God
  • I’m (like) Lobo ;)
  • *****
  • Posts: 4453
  • Karma: 161
    • Tech To The People
  • CiviCRM version: yes probably
  • CMS version: drupal
Re: Loads of civicrm_log went missing: old bug or issue with upgrade?
March 25, 2013, 05:19:24 am
Hi,

moving the log to the trigger level seems to be the good approach indeed. Will put that on my "to dig more" pile for the next sprint.

Still on my quest, the query  below gives a list of when there is a group subcription (or removal), but nothing in the log, so far using the subscription history as a proxy to the log is the best I could find.

Code: [Select]
select distinct c.id,c.sort_name from civicrm_contact as c left join civicrm_subscription_history on contact_id= c.id where group_id != 344 and date >= date("2011-01-01") and not exists (select * from civicrm_log where c.id=civicrm_log.entity_id and entity_table= "civicrm_contact");
-Hackathon and data journalism about the European parliament 24-26 jan. Watch out the result

Pages: [1]
  • CiviCRM Community Forums (archive) »
  • Old sections (read-only, deprecated) »
  • Support »
  • Using CiviCRM »
  • Using Core CiviCRM Functions (Moderator: Yashodha Chaku) »
  • Loads of civicrm_log went missing: old bug or issue with upgrade?

This forum was archived on 2017-11-26.