Author Topic: Command-line geocoding script seemingly does nothing (Read 4942 times)

bcobin · May 25, 2008, 08:37:20 am

Version 2.0.2. I'm dealing with voter lists and have imported about 30,000 records. One some, mapping works, on others, I get just the world map. I've run the command-line geocoding script, but it gives me a result like "UpdateAddress.php?name=admin.3' saved [40/40]" and completes instantaneously. After digging around, I added &start=1&end=28847 - no difference. Many, if not most of the imported records will not map.

Do I need to delete all the records and re-import using "look up mapping info during import?" Is there a command-line script that works? Thank you in advance for your help!

Donald Lobo · May 25, 2008, 11:38:50 am

1. you might want to try debugging the UpdateAddress.php script and see what is happening and why. Most likely it does not seem to be processing any records (if it returns quickly)

2. we geocode only those addresses with null values, so you might have to reset those values in the db

3. you might also want to manually geocode a couple and see what google geocoder returns for those. we basically just store the value returned by the geocoder

4. i would not reimport, i suspect import will give u the same behavior as current.

lobo

petednz · May 25, 2008, 12:48:52 pm

Also worth double checking you have Country field inputted for all records.

bcobin · May 25, 2008, 02:08:13 pm

No records have null values; they all have addresses, zip codes and countries (United States), for that matter. Does this mean that the script would do nothing here anyway?

Some contacts map OK; some don't - there's no discernible difference in those that don't.

This makes using CiviCRM for voter walk lists highly problematic - I am at a loss here. Any other ideas?

Donald Lobo · May 25, 2008, 02:59:37 pm

1. the script only processes the records that have null geo_code_1 and geo_code_2 values. If you are getting a world map for a contact, most likely it does not have a null value

2. we pretty much package the address info and make a call to the yahoo/google geocoder (based on what you have set up). you will need to track down what changes are needed to make it resolve the contacts that dont return a geo code. since these are simple web urls, you can potentially debug this at your end. If you are not familiar with PHP, you might want to get some help from someone who is. The geocode function are in: CRM/Utils/Geocode/*.php

lobo

petednz · May 26, 2008, 02:54:41 am

I know Chris ran something to force a geocode update for us in case that is what you need - may be in this post http://forum.civicrm.org/index.php/topic,2249.0.html

bcobin · May 26, 2008, 09:25:37 am

Thanks - I tried running the "php UpdateAddress.php -n <username> -p <pw>" command instead of the "wget" command on the existing UpdateAddress.php script (all addresses are in the US, so I didn't need the modified script) - this is what I got:

Failed loading /usr/local/IonCube/ioncube_loader_lin_5.2.so: /usr/local/IonCube/ioncube_loader_lin_5.2.so: cannot open shared object file: No such file or directory
Addresses Evaluated: 0
Addresses Geocoded : 0

At least it looks like it's trying to do something - any ideas? And, as always - thanks!

Donald Lobo · May 26, 2008, 05:46:30 pm

This means that all addresses have some value in their geo_code_1 and geo_code_2 columns and hence none were reevaluated

I'm not sure of your technical skills, but you can reset those values to null and try geo-coding again. I dont think it will geocode any more than when u did via import, but might give u a better handle. You can also add debug statements to that script to get an idea of what addresses are not being geocoded properly and why

If the above seems to technical for you, please consider hiring someone from our professional services listing: http://civicrm.org/professional/

thanx

lobo

bcobin · May 27, 2008, 12:13:04 pm

Thank you, Donald.

Indeed, there are geo_code_1 and geo_code_2 values in all of the records I checked in PHPMyAdmin.

I did not geocode during import (as per the advice on the page), so I'm not sure:

a) where the data came from or,
b) why some contacts map and others don't.

In any event, insofar as it's OK if I need to reimport the data, should I try to:

a) reimport, but enable mapping on input this time or,
b) reset the geocode values to none and try using the UpdateAddress.php script?

If the latter, would you mind posting the appropriate SQL command to set the geocode values to null? Thank you very much for your help.

Donald Lobo · May 27, 2008, 12:27:09 pm

i would go with option b. however as mentioned previously, i think the issue is with google not geocoding the addresses properly and you will need to check the ones that are not resolved and figure out the tweaks manually, discover a pattern and if general, we'll change the code for a future release

To reset the data in civicrm_address, run the following sql:

update civicrm_address set geo_code_1 = null, geo_code_2 = null;

please ensure u have a backup of the db as a safety measure.

lobo

bcobin · May 27, 2008, 02:38:19 pm

I've cleared out the data (thank you) but neither script works. The wget script gives the following result:

Quote

HTTP request sent, awaiting response... 200 OK
Length: 40 [text/html]
Saving to: `UpdateAddress.php?name=admin.2'

100%[=============================================================================================================>] 40 --.-K/s in 0s

16:10:30 (6.48 MB/s) - `UpdateAddress.php?name=admin.2' saved [40/40]

It's worth noting here that when the script completes and you return to the command prompt, the wget command string comes up, but it's truncated after the name=<username> - right before the ampersand. Perhaps the password doesn't show as a security measure? Or perhaps the syntax isn't correct? (I'm on a Mac, BTW.)

Running the other script results in the same IonCube error message. I still think there's a problem with the wget script - at least on centOS 5 Linux based on RHEL, which is what my server is running.

Looks like the only option may be to reimport the records but enable geocoding on the way in after all - any ideas are greatly appreciated and thank you for your help!

bcobin · May 27, 2008, 08:47:06 pm

Got the script to work (I think) by dropping the wget command and pasting into the browser URL:

http://www.<domain_name>.com/sites/all/modules/civicrm/bin/UpdateAddress.php?name=<username>&pass=<password>

Did not work with start and end command. Checking in PHPMyAdmin I see geocode fields being populated; with almost 30,000 records it will take a while.

bcobin · May 28, 2008, 06:33:01 pm

Problem solved - I think: Anything other than the street address (i.e., apartment numbers) messes up the geocoding. As soon as apartment information is removed and put in the additional address field, the contact maps OK.

bcobin · June 01, 2008, 06:00:13 am

No joy. After spending the better part of two days on this (reformatting the original data to eliminate Apt. numbers and eliminate extra spaces), only about half the contacts mapped when I mapped during import. Non-working addresses came up with 0,0. There is no discernible difference between the contacts that map and those that don't; in fact, the identical non-working address strings, when copied into Google maps, return the address perfectly.

I've tried nulling out the geocode_1 and 2 values and running various versions of the UpdateAddress.php script in all possible variations - from command-line, browser, and invoking the script directly from the /bin directory (where I get the IonCube error); geocode_1 and 2 get set to 0.

This is the last major thing I need to do with this site and I am truly at a loss here... it seems clear that something is very wrong...

Donald Lobo · June 01, 2008, 11:20:08 am

rather than using the mapping service, use the geocoding service and see if you get the geocode for those addresses. If not, tweak the address and see what needs to be done

http://maps.google.com/maps/geo?q=1600+Amphitheatre+Parkway,+Mountain+View,+CA&output=xml&key=USE_YOUR_KEY

lobo

CiviCRM Community Forums (archive)

News:

Author Topic: Command-line geocoding script seemingly does nothing (Read 4942 times)

bcobin

Command-line geocoding script seemingly does nothing

Donald Lobo

Re: Command-line geocoding script seemingly does nothing

petednz

Re: Command-line geocoding script seemingly does nothing

bcobin

Re: Command-line geocoding script seemingly does nothing

Donald Lobo

Re: Command-line geocoding script seemingly does nothing

petednz

Re: Command-line geocoding script seemingly does nothing

bcobin

Re: Command-line geocoding script seemingly does nothing

Donald Lobo

Re: Command-line geocoding script seemingly does nothing

bcobin

Re: Command-line geocoding script seemingly does nothing

Donald Lobo

Re: Command-line geocoding script seemingly does nothing

bcobin

Re: Command-line geocoding script seemingly does nothing

bcobin

Re: Command-line geocoding script seemingly does nothing

bcobin

Re: Command-line geocoding script seemingly does nothing

bcobin

Re: Command-line geocoding script seemingly does nothing

Donald Lobo

Re: Command-line geocoding script seemingly does nothing