Archive | GIS RSS feed for this section

ESRI REST Geocoding Service in Go

9 Jul

The first thing I wanted to do with Go was to connect to a REST Endpoint. This is an easy task, something I do a lot, and it provides some great possibilities for larger applications. The full code is on GitHub Code, but I will walk through it section by section below.

First, declare the package and import the needed libraries.

package main

import (
“fmt”
“net/http”
“io/ioutil”
“encoding/json”)

We us fmt to print out the result to the console, net/http to connect to the ESRI service, io/util for reading the data the page returns and encoding/json to read the results -which are returned as json.

Next, we need to define the structure of our results.

type geocoderesult struct{
Candidates []candidates
}
type candidates struct{
Address string
Location struct{
X float64
Y float64
}
}

The GeoJSON result from the geocoding service will have a bunch of data and the results are stored in an array named candidates. You need a struct that will grab that result set. The struct geocoderesult has a Candidate []candidates. We need to define candidates. The second struct defines the candidate as contianing an Address and a Location. Location is also a struct that contains and x and y value. The structs match the JSON response.

Now, we create our main function to run the program.

func main(){

response, err := http.Get(“http://server/Geocode/findAddressCandidates?Street=100+lomas&f=json”)
if err != nil {
fmt.Printf(“%s”, err)

}else{}

}

The function connects to the service and passes the parameters street and f. These are part of the ESRI REST API and I have explained them in other posts. The result of http.Get returns a response and an error. If there is an error, we Printf it. In the else statement, we can print the results.

defer response.Body.Close()
c, _ := ioutil.ReadAll(response.Body)
var data geocoderesult

json.Unmarshal(c,&data)
fmt.Println(data.Candidates[0].Address)
fmt.Println(data.Candidates[0].Location)

We close the Body when we are done with it – defer allows us to keep the close close to where we open it. Next, we read the body of the response. In Go, there are a lot of functions that return more than one value (a value and an error). If you do not want to use one of the values returned, you need to use an underscore. Otherwise, if you declare it with a name and don’t use it later, the application will not run.

The next line declares data as a new empty geocoderesult. We then unmarshal(decode) the json response – c – to data. Notice the ampersand before data? Go uses pointers. The ampersand says that we want to put the unmarhaled json data in to the location of data in memory. Basically, replace the current value of data with the json.

Lastly, we can grab the first result returned from the service and print it. Indexes start at 0. Data contains a slice of Candidates so we can index on it using data.candidates[x]. Then candidates has fields for address and location.

Now you can build the application and run it. To make it more useful, you could have it read a textfile of addresses and write out a new file with their locations. Or, reverse geocode a series of points. Then you could build the binary and run it all the time.

In my next post, I will show you how to wrap this code in to a library so we can call it from another program as a function ourlibrary.geocode().

Advertisements

Go Library for ESRI REST

9 Jul

In my last post, I showed how to consume an ESRI REST geocoding service using Go. In this post, I will consume a feature service and query it. This post will take it one step further. We will put the code in a library so that anyone can use it in their applications. The full code is on GitHub.

Usually, the first line is package main, but we will change it to the name for our new package. then we can import the same libraries as in our geocoding sample.

package esrirest

import (
“fmt”
“net/http”
“io/ioutil”
“encoding/json”

)

The structs that we will create will be different as well. In the geoocoding example we wanted candidates. When running a query, we want features.

type query struct{
Features []features
}

ESRI GeoJSON uses Attributes instead of properties, so we will create a map of type interface for the attributes in the feature. This allows us to grab any data types without having to hardcode them as we did in the geocoding example (address, location {x,y}).

type features struct{
Attributes map[string]interface{}
}

In case we need it later – I ever get around to adding more functionality – I will put the variable data outside the function.

var data query

Now, instead of writing a main() function, we will write a function Query – an not include a main in this program because it is a library, and not meant to be executed stand alone. The Query functino will take a string (URL with parameters) that points to the ESRI service. It returns a slice of features. The rest of the code looks exactly like the geocoding example.

func Query(s string) []features {

response, err := http.Get(s)
if err != nil {
fmt.Printf(“%s”, err)

} else {
defer response.Body.Close()
c, _ := ioutil.ReadAll(response.Body)
json.Unmarshal(c,&data)

}
return data.Features
}

The one difference is that the function returns the slice of features – data.Features. You now have a functioning library that you can use in any program.

Use the Library

I will now show you how to use the library.

Run the command:

go get github.com/PaulCrickard/Go/esrirest

This will grab the plugin and install it locally for you. Now write your application.

Declare the package to be main and import the needed libraries. In this case we only need fmt and our new library that we created, downloaded and installed.

package main

import (
“fmt”
“github.com/PaulCrickard/Go/esrirest”

)

If you look in your pkg directory, you will see a folder with your OS type (windows_amd64) then  github.com, PaulCrickard, Go and a file esrirest.a. That is the path you use for the import.

Create your main function and now just call the function in our library passing the URL.

func main(){
//Pass query parameters in URL. MUST HAVE —–> f=json
d :=esrirest.Query(“http://coagisweb.cabq.gov/arcgis/rest/services/public/PublicArt/MapServer/0/query?where=1=1&outFields=*&f=json”)
}

All of the data is now in d. We can now access features and attributes.

fmt.Println(d[0].Attributes[“TITLE”]

Or, we can iterate through all the features and print the titles.

for i:=0; i<len(d); i++{
fmt.Println(d[i].Attributes[“TITLE”])
}

What if you do not know what the attribute names (keys) are? You can iterate through and grab them.

for key, value := range d[0].Attributes{
fmt.Println(key,value)

}

This will print out the key and value for the first feature. Throw this inside your for loop above to go through each feature.

You can grab just the keys by using:

for key, _ := range d[0].Attributes{
fmt.Println(key)

}

Or print the values:

for key, _ := range d[0].Attributes{
fmt.Println(d[0].Attributes[key])

}

Now you know how to crate a library, how to install it and how to use it in another application.

 

Excel Geocoder

17 Apr

ESRI makes maps for Office. I thought this could be interesting and went to work on trying to insert a map in to a spreadsheet. I did not have much luck. Instead, I decided to throw together a quick macro that would geocode addresses in a spreadsheet and give back coordinates. I do not think VB Script can parse JSON, so the result is ugly, but you get the idea.

Start with a spreadsheet of addresses

sheet

 

Then create a Macro – I copied the code for reading the URL from Ryan Farley.

Sub Macro1()
Range(“A2”).Select
i = 2

Do Until IsEmpty(ActiveCell)

URL = “http://coagisweb.cabq.gov/arcgis/rest/services/locators/CABQ_NetCurr/GeocodeServer/findAddressCandidates?f=json&outSR=4326&street=&#8221; & ActiveCell.Value
Dim objHttp
Set objHttp = CreateObject(“Msxml2.ServerXMLHTTP”)
objHttp.Open “GET”, URL, False
objHttp.Send
Cells(i, 2).Value = objHttp.ResponseText
i = i + 1
ActiveCell.Offset(1, 0).Select
Set objHttp = Nothing
Loop

End Sub

The code starts at cell A2 and reads addresses until it reaches an empty cell. It takes the value and sends it to the ESRI REST endpoint for the City of Albuquerque Geocoding Service. It sets the cell next to it with the results. They should really be parsed, but I am too lazy and was just curious if it could be done. It can. The result is below.

results

I am still thinking of how to embed a web page in the sheet.

Historic Bus Location Data

27 Mar

Last night I was trying to think of interesting uses for a Rasberry Pi. One thing I came up with was a data logger. But what to log? Then I thought about a previous post on the Albuquerque Realtime Bus Data. Hmmm. What if I wanted to show the bus locations using a time slider. What if I want to see if they ever deviated from their routes, or if they deviated from their schedules? I can’t really do any analysis without the historic data, and the City does not give that out currently. So I think I find a use – logging Albuquerque Bus Data.

I don’t have a Rasberry Pi, yet, so I wrote a python script on my desktop to test the logger.

I am not going to post the code because I don’t know the impact on the Albuquerque server. I will give a brief explanation. The City has KML files for each bus route. Each route has multiple buses. I grabbed a single route – 766, and parsed the results. I initially sent the results to a csv – as you can see in the data below this post. Writing to CSV is not too helpful when the data gets large (I am not going to say BIG). Once I knew it worked, I sent the data to a MongoDB that was spatially indexed. In the database, I can now:

Get the total records

                         collection.count()

Get all the Records

                        for x in collection.find():
                                      print x

Get a specific bus number

                       for x in collection.find({‘number’:’6903′}):
                                           print x

Or find near a lat,lng

                       for x in collection.find({“loc”:{“$near”:[35.10341,-106.56711]}}).limit(3):
                                           repr(x)

With a database, multiple people can query it and perform operations on it. Lastly, if the data gets larger, Mongo can be split (sharding) across multiple machines to hold it all.

My MongoDB records look like:

{u’loc’: [35.08156, -106.6287], u’nextstop’: u’Central @ Cornell scheduled at 3:45 PM’, u’number’: u’6903′, u’time’: u’3:46:52 PM’,
u’_id’: ObjectId(‘5515cfd814cd2829e4c1b718′), u’speed’: u’20.5 MPH’}

Here is the results of my original run. I ran the script for 7 minutes and got the following results for Route 766.

Bus

Hard to see, but displays bus locations along route 766 over a 7 minute period.

 

6409,0.0 MPH,1:34:02 PM,Central @ San Mateo (Rapid) scheduled at 1:31 PM,-106.58642,35.07778
6904,1.9 MPH,1:34:03 PM,Central @ Edith scheduled at 1:31 PM,-106.64776,35.08401
6411,0.0 MPH,1:33:56 PM,CUTC Bay B scheduled at 1:39 PM,-106.72266,35.07726
6407,21.7 MPH,1:34:00 PM,Central @ 1st (across from A.T.C.) scheduled at 1:34 PM,-106.64317,35.08359
6410,35.4 MPH,1:33:53 PM,Central @ San Mateo (Rapid) scheduled at 1:36 PM,-106.58247,35.07749
6403,0.0 MPH,1:34:02 PM,Indian School @ Louisiana scheduled at 1:40 PM,-106.57089,35.10305
6903,29.8 MPH,1:33:56 PM,Central @ Atrisco scheduled at 1:36 PM,-106.69049,35.08443
6409,0.0 MPH,1:35:14 PM,Louisiana @ Central (Rapid) scheduled at 1:36 PM,-106.5852,35.07764
6904,1.9 MPH,1:35:14 PM,Central @ Edith scheduled at 1:31 PM,-106.6478,35.08413
6411,0.0 MPH,1:35:07 PM,Next stop is CUTC Bay B scheduled at 1:44 PM,-106.72525,35.07886
6407,0.0 MPH,1:35:11 PM,Copper @ 2nd scheduled at 1:34 PM,-106.64784,35.08417
6410,19.3 MPH,1:35:17 PM,Central @ Carlisle (Rapid) scheduled at 1:40 PM,-106.58734,35.07802
6403,0.0 MPH,1:35:14 PM,Indian School @ Louisiana scheduled at 1:40 PM,-106.57087,35.10306
6903,14.9 MPH,1:35:08 PM,Central @ Tingley (Rapid) scheduled at 1:38 PM,-106.68485,35.08576
6409,23.6 MPH,1:36:38 PM,Louisiana @ Central (Rapid) scheduled at 1:36 PM,-106.58084,35.07723
6904,18.0 MPH,1:35:53 PM,Central @ Edith scheduled at 1:31 PM,-106.647,35.08373
6411,0.0 MPH,1:36:31 PM,Next stop is CUTC Bay B scheduled at 1:44 PM,-106.72525,35.07885
6407,0.6 MPH,1:36:35 PM,Copper @ 5th scheduled at 1:35 PM,-106.64944,35.08541
6410,0.0 MPH,1:36:40 PM,Central @ Carlisle (Rapid) scheduled at 1:40 PM,-106.59512,35.07883
6403,0.0 MPH,1:36:42 PM,Indian School @ Louisiana scheduled at 1:40 PM,-106.57086,35.10306
6903,31.7 MPH,1:36:33 PM,Central @ Rio Grande (Rapid) scheduled at 1:40 PM,-106.67824,35.09252
6409,37.9 MPH,1:37:49 PM,Louisiana @ Central (Rapid) scheduled at 1:36 PM,-106.57203,35.07627
6904,0.6 MPH,1:37:55 PM,Central @ Cedar (Rapid) scheduled at 1:33 PM,-106.63771,35.08276
6411,0.0 MPH,1:37:55 PM,Central @ Coors scheduled at 1:47 PM,-106.72526,35.07885
6407,1.9 MPH,1:37:47 PM,Copper @ 5th scheduled at 1:35 PM,-106.6496,35.08487
6410,0.0 MPH,1:37:52 PM,Central @ Yale (UNM) scheduled at 1:44 PM,-106.60369,35.07979
6403,0.0 MPH,1:37:57 PM,Indian School @ Louisiana scheduled at 1:40 PM,-106.57087,35.10306
6903,0.0 MPH,1:37:56 PM,Central @ Rio Grande (Rapid) scheduled at 1:40 PM,-106.67159,35.09515
6409,0.0 MPH,1:39:14 PM,Louisiana @ Central (Rapid) scheduled at 1:36 PM,-106.56872,35.07595
6904,18.0 MPH,1:38:19 PM,Central @ Cedar (Rapid) scheduled at 1:33 PM,-106.63713,35.08265
6411,0.0 MPH,1:39:19 PM,Central @ Coors scheduled at 1:47 PM,-106.72527,35.07884
6407,3.1 MPH,1:39:10 PM,Central @ Rio Grande (Rapid) scheduled at 1:40 PM,-106.65295,35.086
6410,31.1 MPH,1:39:16 PM,Central @ Yale (UNM) scheduled at 1:44 PM,-106.61167,35.08084
6403,5.6 MPH,1:39:23 PM,Indian School @ Louisiana scheduled at 1:40 PM,-106.5707,35.10368
6903,23.0 MPH,1:39:20 PM,Gold @ 5th (Rapid) scheduled at 1:44 PM,-106.67011,35.09438
6409,26.1 MPH,1:40:37 PM,Louisiana @ Lomas scheduled at 1:38 PM,-106.56848,35.07723
6904,24.9 MPH,1:40:10 PM,Central @ Cedar (Rapid) scheduled at 1:33 PM,-106.63585,35.08255
6411,0.0 MPH,1:40:31 PM,Central @ Coors scheduled at 1:47 PM,-106.72526,35.07884
6407,23.0 MPH,1:40:36 PM,Central @ Rio Grande (Rapid) scheduled at 1:40 PM,-106.65675,35.0863
6410,34.2 MPH,1:40:40 PM,Central @ Yale (UNM) scheduled at 1:44 PM,-106.61567,35.0811
6403,0.0 MPH,1:40:42 PM,Indian School @ Louisiana scheduled at 1:40 PM,-106.56875,35.10221
6903,23.6 MPH,1:40:32 PM,Gold @ 5th (Rapid) scheduled at 1:44 PM,-106.66327,35.08892
6409,0.6 MPH,1:41:49 PM,Indian School @ Uptown Loop Road scheduled at 1:42 PM,-106.56849,35.08691
6904,24.9 MPH,1:41:55 PM,Central @ Cedar (Rapid) scheduled at 1:33 PM,-106.63585,35.08255
6411,0.0 MPH,1:41:55 PM,Central @ Coors scheduled at 1:47 PM,-106.72526,35.07884
6407,0.0 MPH,1:41:58 PM,Central @ Rio Grande (Rapid) scheduled at 1:40 PM,-106.65822,35.08648
6410,28.6 MPH,1:41:52 PM,Central @ Yale (UNM) scheduled at 1:44 PM,-106.6216,35.08113
6403,0.0 MPH,1:42:01 PM,Louisiana @ Lomas scheduled at 1:45 PM,-106.56779,35.10184
6903,23.0 MPH,1:41:55 PM,Gold @ 5th (Rapid) scheduled at 1:44 PM,-106.65819,35.08629
6409,41.0 MPH,1:43:13 PM,Indian School @ Uptown Loop Road scheduled at 1:42 PM,-106.56863,35.09282
6904,24.9 MPH,1:42:15 PM,Central @ Cedar (Rapid) scheduled at 1:33 PM,-106.6217,35.08093
6411,0.0 MPH,1:43:19 PM,Central @ Coors scheduled at 1:47 PM,-106.72528,35.07883
6407,9.9 MPH,1:43:10 PM,Central @ Rio Grande (Rapid) scheduled at 1:40 PM,-106.661,35.08784
6410,34.2 MPH,1:43:16 PM,Central @ Mulberry (Rapid) scheduled at 1:47 PM,-106.62524,35.08125
6403,16.8 MPH,1:43:22 PM,Louisiana @ Lomas scheduled at 1:45 PM,-106.56635,35.10156
6903,19.3 MPH,1:43:19 PM,Gold @ 5th (Rapid) scheduled at 1:44 PM,-106.6549,35.084

Python Wrapper for Leaflet

20 Mar

I recently stumbled upon Folium – a python wrapper for leaflet. I was excited and it seemed to work well. I slowly ran in to problems and the pages loaded slow. I probably did something wrong on my end, but decided to write a simple wrapper on my own.

My wrapper is a python function for different Leaflet features such as map and marker. When you call each function, it writes a string to a file to generate the HTML. Below is my python code (pyLeaflet.py).

class l(object):

def __init__(self,path):
self.path=path
self.f=open(self.path,”w+”)
self.f.write(‘<html><head><title>Map From Python</title><link rel=”stylesheet” href=”http://cdn.leafletjs.com/leaflet-0.7.2/leaflet.css&#8221; /></head><body><script src=”http://cdn.leafletjs.com/leaflet-0.7.2/leaflet.js”></script><div style=”height:900px; width:900px” id=”map”></div><script>\n’)

def map(self, lat,long,zoom):
self.lat=lat
self.long=long
self.zoom=zoom
self.f.write(“var map = L.map(‘map’, {center: [“+str(self.lat)+”,”+str(self.long)+”], zoom:”+str(self.zoom)+”});\n”)
self.f.write(“L.tileLayer(‘http://{s}.tile.osm.org/{z}/{x}/{y}.png’).addTo(map);\n”)
def marker(self,lat,long, popup=””):
self.x=lat
self.y=long
self.popup=popup
self.f.write(‘L.marker([‘+str(self.x) +’,’+str(self.y)+’]).bindPopup(“‘+str(self.popup)+'”).addTo(map);\n’)
def onclick():

def makeMap(self):
self.f.write(‘</script></body></html>’)
self.f.close()

To use the code, follow the example below.

>>> from pyLeaflet import l
>>> L=l(“Paul.html”)
>>> L.map(35,-106,8)
>>> L.marker(35,-106)
>>> L.marker(34,-106,”Hello from Python”)
>>> L.makeMap()

The output will be an HTML file called paul.html that displays a map with a maker.

Albuquerque Open Data

18 Mar

I have put together several posts on Albuquerque Open Data and have felt bad that the examples were not live. I shutdown my old website and tried to use JsBin for a few examples, but I was not consistent and couldn’t upload third party JavaScript files. Well, I found a solution.

GitHub Pages to the Rescue

I have a GitHub repo. A GitHub Repo is a place to put code where you can harness the full power of git – branching, committing and others can pull and push code to your repo. GitHub also has a thing called GitHub Pages where you can host sites. I have one. And I am now using it to host my presentations. I do not own Albuquerque Open Data, but the community does. The City puts the data out there for us to use. As a community member, I decided to create a GitHub repo for Albuquerque Data and a GitHub page for it. You can download the code for all the examples, or run them. You can also grab a copy and modify thm. Lastly, you can add your own code examples to the repo.

The Examples

There are only a few examples thus far.

  • Aerial – Zoom to a location in ABQ. Double click to get back an aerial image of the map window.
  • Crime – Click the map to see incidents near you.
  • Density
  • Elevation – Draw a line with lots of segments to see an elevation profile.
  • Game – Click the map to add a marker. Drag the marker to a park to win.

The Code for the Examples

You can get the code for each example by clicking the link.

  1. Aerial
  2. Crime
  3. Density
  4. Elevation
  5. Game

You can also grab the whole Repo as a ZIP file.

Aerial Imagery of Albuquerque and Surrounding Counties

10 Mar

When you need an aerial image, do you go to Google and take a screenshot? If you live in Albuquerque or the surrounding area, you no longer need to do that. The City of Albuquerque publishes aerial imagery of Albuquerque and the surrounding areas as open data. In this post, I will show you how to retrieve an image.

The Service

The Imagery Service provides a list of all the available years.

The historic data is a great addition.

The historic data is a great addition.

For this tutorial, click on Imagery/Aerials2014. At the bottom of the page, choose Export Image. You will see the full extent of the imagery and a form below.

Capture2

To get an image, you need to enter the lower left and upper right corner coordinates of a bounding box in NAD83(HARN) / New Mexico Central (ftUS). I have no clue what this would be. You can change it to WGS84 (4326) as we have done in previous examples, but still, how would I know the coordinates without clicking on a map and checking the points? The form is pretty much useless to me. Our application will provide a better GUI – map based – to the form.

The Application

Our application will be a Leaflet.js map that allows the user to zoom to their desired location. The map window will be used to select what is exported from the imagery service.

The Logic

The application is a basic Leaflet.js map. We will add an onClick function that grabs the image.

map.on(“click”,function(){

getImage();

});

The getImage() function will make an AJAX call to the service. It will pass the coordinates of the map window as parameters to get back an image in a popup. You can then right click and save image as…

function getImage(){

b=map.getBounds();
var bounds=b._southWest.lng+”,”+b._southWest.lat+”,”+b._northEast.lng+”,”+b._northEast.lat;

var url = “http://coagisweb.cabq.gov/arcgis/rest/services/Imagery/Aerials2014/ImageServer/exportImage&#8221;;
var params=”f=json&bbox=”+bounds+“&bboxSR=4326&size=1028,1028&imageSR=4326”;

http=new XMLHttpRequest();
http.open(“POST”, url, true);
http.setRequestHeader(“Content-type”, “application/x-www-form-urlencoded”);
http.onreadystatechange = function() {//Call a function when the state changes.
if(http.readyState == 4 && http.status == 200) {

result= JSON.parse(http.responseText);
window.open(result.href);

}}

http.send(params);

}

The above code starts by assigning the map bounds to a variable. We then create a bounds string in the format the service wants – lower left corner to top right corner. We pass the bounds string in to the parameters string and make the request.

The result contains an href property that is the URL to the image. We open a popup to the URL so the user can save the image.

Map on the left. Click. Image popup on the right.

Map on the left. Click. Image popup on the right.

Now the user can zoom to the desired extent and click the map to get back an image. Much easier than finding the coordinates and entering them in to the form.