3

I am using the geocoder package in Python to obtain the coordinates for a set of addresses (around 30k). I get the following error:

Status code Unknown from https://maps.googleapis.com/maps/api/geocode/json: ERROR - HTTPSConnectionPool(host='maps.googleapis.com', port=443): Max retries exceeded with url: /maps/api/geocode/json?address=Rancho+Palos+Verdes%2CCA%2CUS&bounds=&components=&region=&language= (Caused by ProxyError('Cannot connect to proxy.', timeout('timed out',)))

The number of times I receive the error reduces if I add the time.sleep(x) function but it significantly increases the time taken to execute the code. Is there a more efficient way to run the code?

Following is a snippet of the code:

for add in clean_address: g=geocoder.google(add) time.sleep(7) if(g.ok==True): lat.append(str(g.lat)) lon.append(str(g.lng)) if(g.ok==False): lat.append("") lon.append("")
Improve this question
5
  • 3
    Google's geocode API is only free up to a certain amount, you're probably exceeding that and getting rate-limited. If you want more you'll have to sign up and pay for your volume. Commented Dec 7, 2017 at 16:06
  • Is there a reason to use google api? you could use arcgis api that haven't use limitation: g=geocoder.arcgis(add) Commented Dec 7, 2017 at 16:07
  • @Lupanoide - Thanks for the recommending me ArcGIS. Its more accurate and faster in getting the coordinates for my 31k records. Commented Dec 8, 2017 at 17:41
  • @Lupanoide - could you post your comment as an answer so that I can accept it? Commented Feb 9, 2018 at 17:46
  • @MazahirBhagat sure, thanks Commented Feb 9, 2018 at 18:42

3 Answers 3

Reset to default
3

Google API has a use limitation. However you could even use geocoder library that collects a lot of geocoding services. I suggest to you to use the ArcGis api that hasn't any limitation on usage and is very accurate. The usage is very simple:

g=geocoder.arcgis(add) lat.append(g.x) lon.append(g.y)
Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

arcgis still times out after several hundred records. I am a fair newbie at python and would like to know a method to catch this error and keep going
2

This is an error originating in the Python requests library used by the geocoder package. The error itself is likely a symptom of exceeding a quota limit configured on your project's enabled API. There are short term quotas (per 100 seconds) and long-term quotas (per day) that you might be running into.

An improvement to automatically retry with exponential backoff may help if the error is due to short-term quota being exceeded. To do this, explicitly create a Session with a custom Retry, and use that session with the geocoder library:

import requests from requests.adapters import HTTPAdapter from requests.packages.urllib3.util.retry import Retry ... # Retry for up to 65 seconds ssn = requests.Session() adp = HTTPAdapter(max_retries=Retry(total=16, backoff_factor=0.001)) ssn.mount('https://', adp) ssn.mount('http://', adp) ... geocoder.google(..., session=ssn)
Improve this answer

1 Comment

Great answer but a follow up: What is the way to add retry functionality in R? I have R code which is running into the same problem...
1

Your addresses are 30K and the limit is 2,5K per day.

As Google quotes in the relevant docs:

To use the Google Maps Geocoding API, you must register your app project on the Google API Console and get a Google API key which you can add to your app or website.

After doing so, these are the limits:

Users of the standard API:

  • 2,500 free requests per day, calculated as the sum of client-side and server-side queries.
  • 50 requests per second, calculated as the sum of client-side and server-side queries.
Improve this answer

3 Comments

Is there a downvoting police in S.O.? I need to sue somebody.
no police. but i'll balance it out for you. I believe this answers the OP's question
The downvoters are the police (part of it anyway). Don't worry, the average statistics will work out in your favor. +1

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.