Archive | January, 2015

City of Albuquerque 311 API

30 Jan

The City of Albuquerque has a 311 service that can be reached by calling 311. They have also use SeeClickFix for web and app based submitting.  This post will show you how to query the SeeClickFix API to retrieve Albuquerque Data.

The finished Application

The finished Application

The API

The SeeClickFix API is rather uncomplicated. You can find the details on their development site. To see issues, you will call the API by going to the issues service and passing parameters.

https://seeclickfix.com/api/v2/issues?page=1&per_page=1

The code above grabs a single issue on page 1. Change page to 2 and it will be a different issue. Change issues to 100 and you will hit the maximum of issues per page. Look through the JSON and make not of the fields and also that the addresses are for cities throughout the country – not just Albuquerque.

Using the API for Albuquerque Data

To narrow the data for Albuquerque, you could grab all the data and filter based on address but that would require reading TONS of data. Not the way to go. Using the API, we can filter by an envelope. The coordinates for a bounding box around the City of Albuquerque are (35.02212, -106.79672),(35.20636, -106.48911). Using the API, we can enter the URL below and get back some issues for Albuquerque.

https://seeclickfix.com/api/v2/issues?min_lat=35.02212&min_lng=-106.79672&max_lat=35.20636&max_lng=-106.48911

I will add that I want only open tickets on page 1 with 100 records on the page.

https://seeclickfix.com/api/v2/issues?min_lat=35.02212&min_lng=-106.79672&max_lat=35.20636&max_lng=-106.48911&status=open&page=1&per_page=100

Now you should see 100 open 311 issues in the City of Albuquerque.

Mapping the Data

With the URLs working for our requests, we can automate calling them using AJAX and parse out the response to display the points on a leaflet.js map. I have covered how to do this multiple times on this blog, but the code is below.

var url = “https://seeclickfix.com/api/v2/issues?min_lat=35.02212&min_lng=-106.79672&max_lat=35.20636&max_lng=-106.48911&status=open&page=1&per_page=100”;

http=new XMLHttpRequest();
http.open(“GET”, url, true);
http.setRequestHeader(“Content-type”, “application/x-www-form-urlencoded”);
http.onreadystatechange = function() {
if(http.readyState == 4 && http.status == 200) {
console.log(http.responseText);
var result= JSON.parse(http.responseText);
for(x=0;x<Object.keys(result.issues).length;x++){

markers.addLayer(L.marker([result.issues[x].lat,result.issues[x].lng]).bindPopup(“<ul><li>”+result.issues[x].summary+”</li><li><a href='”+result.issues[x].media.image_full+”‘>Image</a></li><li>”+result.issues[x].created_at+”</li></ul>”));

}//end for
markers.addTo(map);

}}
http.send();

The code above makes the request, parses the response as JSON then grabs the coordinates. When creating the marker, I went ahead and added some other fields in a popup so that we could have more information about the request.

In a future post, I will show you how to post to SeeClickFix to report a problem.

Advertisements

Revit REST API: More Detail

29 Jan

I have ideas in my head about why a Revit REST API is cool and what it can do. I also know that I fail to explain or show what I am thinking in the best possible way. Often, my examples are quick and dirty. I really love the Revit REST API idea and have built out a slightly more detailed example to show what can be done. The images below show a more complete site. I have sorted many of the tables by the field that makes the most sense. Remember, the data can be edited – without a Revit license I might add.  The model I used was the sampel that comes with Revit 2014.

The sample model

The sample model

walls

rooms

levels

sheets

views

 

CSV in JavaScript Using Albuquerque Open Data

28 Jan

In my last two posts, I showed you how to read KML (XML) over the web using AJAX. In this example, we will do the same with a tab delimited file from Albuquerque Open Data. The application will look like the image below.

The finished app. Choose a committee and see their financial information.

The finished app. Choose a committee and see their financial information.

 

Get the Data

On the Albuquerque Open Data site, we will use Campaign Finalized. You will see two options: XML or CSV. We will use the CSV – which is really tab delimited. Grab the data using AJAX.

var url = “http://data.cabq.gov/government/campaignfinalized/CampaignReportFinalizedVersionCABQ-en-us.csv&#8221;;

http=new XMLHttpRequest();
http.open(“GET”, url, true);
http.setRequestHeader(“Content-type”, “application/x-www-form-urlencoded”);
http.onreadystatechange = function() {//Call a function when the state changes.
if(http.readyState == 4 && http.status == 200) {
var results=http.responseText;

Parse the Data

The data is tab delimited in rows that have a newline character at the end. This file also has new line characters in fields – but they are wrapped in quotes. Handling this will be difficult on our own, so let’s use a library for it. Download PapaParse 4. This will simplify the process significantly. We can grab the data with:

p=Papa.parse(results,{
delimiter:”\t”,
newline:”\n” });

Now I have an array with each line as p.data[x] and each item as p.data[x][0-8]. For this example, we will use the committee name(p.data[x][0]) and the amount(p.data[x][8]).

Let’s populate the dropdown box. First, we need to create it in the HTML and add a default value.

<select id=’D1′></select>
var select = document.getElementById(“D1”);
var d= document.createElement(“option”);
d.value = -1;
d.id=-1;
d.textContent = “—Choose—“;
select.appendChild(d);

We can now iterate through our data and create an array of individual groups.

for(r=1;r<p.data.length;r++){
if(groups.indexOf(p.data[r][0].trim())>-1){
//do nothing
}
else{groups.push(p.data[r][0].trim());}
}

And then populate the combo box

for(g=0;g<groups.length-1;g++){
var opt = document.createElement(“option”);
opt.value = g;
opt.id=g;
opt.textContent = groups[g];
select.appendChild(opt);
}

Displaying the Data

We need to add an Event Listener on the combo box and execute a function.

document.getElementById(‘D1’).addEventListener(‘change’, money, false);

The function will grab the value of the item selected then get its text content (name). With this information we can pull all the data from the CSV for the financials of the committee selected. Lastly, update the DOM to display the results.

function money(m){
m=document.getElementById(this.value).textContent;
console.log(m);
for(y=0;y<p.data.length;y++){
if(p.data[y][0].trim()==m){
if(parseFloat(p.data[y][8])>0){
raised+=parseFloat(p.data[y][8]);
}
else{spent+=parseFloat(p.data[y][8]);}
}
else{}
}
document.getElementById(“output”).innerHTML='<h3>Raised:</h3>’+raised.toLocaleString()+'<h3>Spent:</h3>’+spent.toLocaleString()+'<h3>Difference</h3>’+(raised+spent).toLocaleString();

}

Without Papa Parse 4, this would have been a much more complicated task. There is no CSV standard, so dealing with individuals CSV files requires a lot of modification. In this example, having new line characters in a field wreaked havoc on my first attempts to parse it out manually. Also, I originally set the PapaParse option header=true, but the application complained. Instead of figuring out why it didn’t like it, I just started my loops at an index of 1 – dropping the headers at index 0.

Albuquerque Live Bus Data as JSON

27 Jan

In my last post on the City of Albuquerque live bus data I showed how to parse the KML straight from the server in JavaScript. In this example, I will show you how to create your own REST endpoint for Route 766.

Classic ASP

Using classic ASP, because why not, I made a request to the server and read in the KML file.

url = “http://data.cabq.gov/transit/realtime/route/route766.kml&#8221;
set xmlhttp = CreateObject(“MSXML2.ServerXMLHTTP”)
Set xml = Server.CreateObject(“Microsoft.XMLDOM”)
xmlhttp.open “GET”, url, false
xmlhttp.send “”
xmlDoc=xmlhttp.responseText

The xmlhttp.responseText will hand back the KML file (KML is just XML) as text formatted as XML.

<?xml version=”1.0″ encoding=”UTF-8″?>
<kml xmlns=”http://www.opengis.net/kml/2.2″&gt;
<Document>
<Name>Route 766</Name>
<Style id=”NoDir”>

To parse it by nodes, we need to convert it to XML (though you could parse it using split() or a combination of other methods.

xml.async = False
xml.loadXML(xmlDoc)

To make the service return JSON, we will need to loop through all the Placemark/Point nodes and pull out the coordinates. We grab the node then setup our counters.

Set nodes = xml.SelectNodes(“//Placemark/Point”)
count=0
total=nodes.length-1

Before entering the loop, we need to setup our JSON string

Response.ContentType = “application/json”
response.write(“[“)

Finally, we can loop through and print each coordinate value in {}. Then we close the JSON string and empty out our variables.

For Each node In nodes
response.write “{”
For Each child in node.ChildNodes
Response.Write “”””& child.nodeName & “””: “”” & child.Text & “”””
Next
if count=total Then
response.write “}”
else
response.write”},”
count=count+1
End if
Next
response.write(“]”)
Set nodes = Nothing
set xmlhttp = nothing

Here is what the service returns

[{“coordinates”: “-106.65785,35.08641”},{“coordinates”: “-106.58566,35.07783”},{“coordinates”: “-106.56848,35.07648”},{“coordinates”: “-106.64804,35.0838”},{“coordinates”: “-106.69363,35.08368”},{“coordinates”: “-106.70488,35.0813”},{“coordinates”: “-106.57089,35.10303”}]

The KML is still ugly – mostly because the <description> node has a table as a child node and getting each value out of the table is annoying. Please, City of Albuquerque, please change the KML to not have tables but nodes for <Vehicle Number>, <Speed., <Msg Time> and <Next Stop>.

Client Side Consumption of the REST Service

With the service up and running, we can query it using AJAX on the client side and draw a map. In the client side JavaScript, we can take httpresponseText and parse it to JSON. Iterating through the length, we grap the coordinates and split them on the “,”. Then add them to the map. We could improve our JSON service to split the coordinates for us and write out an X and Y attribute.

var url = “http://dmdview/paul/xml.asp&#8221;;
http=new XMLHttpRequest();
http.open(“GET”, url, true);
http.setRequestHeader(“Content-type”, “application/x-www-form-urlencoded”);
http.onreadystatechange = function() {//Call a function when the state changes.
if(http.readyState == 4 && http.status == 200) {
console.log(http.responseText);
var result= JSON.parse(http.responseText);
for(x=0;x<Object.keys(result).length;x++){
xy=result[x].coordinates;
coords=xy.split(“,”);
markers.addLayer(L.marker([coords[1],coords[0]]))
}//end for
markers.addTo(map);}}
http.send();

Now you have a JSON service that can be replicated for each bus route.

Albuquerque Live Bus Tracker

26 Jan

The City of Albuquerque has put out a lot of good data and they have done so in several different formats. There is one data set that I love and despise at the same time – the Real Time Bus data. I love it because I think it is an excellent example of open data. It is extremely useful as it is updated every minute. I despise it because it is in KML. I will give the City the benefit of the doubt and assume they have done this because their software for tracking the buses uses KML by default so it was an easy choice – just spit out what we already have.

As a developer, KML is worthless to me. I want a stream of data in JSON or CSV. I have struggled with what to do for some time now – ignore the bus data or figure out a way to build something with it? In 2013, I wrote a post using the real time bus data that required me to use some tools that I don’t usually pull out – OpenLayers and PHP. OpenLayers allows for the importing of a KML, however, it only did so via a local file. I had to find a way to get the file from the City Website to my server. PHP came to the rescue and the app downloaded the data then rendered it in an OpenLayers map. A fairly large amount of overhead for what should be a simple task – read bus data.

Two years later, almost to the day, I have revisited the bus data. This time I think I have defeated the dreaded KML. In this post, I will show you how to conquer KML files using JavaScript.

Route 766 Live in Leaflet.js

Route 766 Live in Leaflet.js

The Solution

The solution that came to mind was to use AJAX to read the KML file off the City webpage. The first thing I did was to write an AJAX request using the city URL and inspected the response.

XMLHttpRequest cannot load http://domain1 Origin http://domain2 is not allowed by Access-Control-Allow-Origin.

Already off to a great start. I was about ready to wait another two years before looking at the City Bus Data again. I looked in to Chrome and found that you can launch it without web security. I created a shortcut for it on my Desktop:

C:\Program Files(x86)\Google\Chrome\Application\chrome.exe” –args –disable-web-security

When I ran the code again, I got back a bunch of XML. Bingo! that is exactly what I wanted. Now I had to figure out how to parse it. Enter the JavaScript DOMParser().

var xmlDoc = new DOMParser().parseFromString(http.responseText,’text/xml’);

That is all it took to get the KML in to a parsable format. To add the buses to the map I grabbed the coordinates using

var coords = xmlDoc.getElementsByTagName(“coordinates”);

Grabbing the length of the coords object, I looped through mapping the coordinates.

for(i=0;i<Object.keys(coords).length-1;i++){
var xy=coords[i].childNodes[0].nodeValue.split(“,”);
markers.addLayer(L.marker([xy[1],xy[0]]));}

The coordinates are (Long,Lat) in KML and Leaflet.js expects (Lat,Long) so the indexes are switched. Also, the markers are added to a FeatureGroup so that I can easily clear them all and redraw them every 62 seconds.

The application is wrapped in a function getBus() and executed using

 window.setInterval(getBus,62000);

Now every 62 seconds the map will refresh and the buses will have moved.

More Information

There is more data in the KML: Vehicle Number, Speed, Msg Time and Next Stop. The KML file has unfortunately put these values in a table. You cannot grab them by Tag Name easily – they all have the same tag. You can see them all using:

var vehicleNumber=xmlDoc.getElementsByTagName(“td”);
for(i=0;i<Object.keys(vehicleNumber).length-1;i++)
{console.log(vehicleNumber[i].childNodes[0].nodeValue);}

This is where I really wish the data were cleaner – or in JSON. A little data wrangling is good for us, so that will be my next move.

JavaScript Classes and Events

23 Jan

Every so often I like to check out Microjs and browse the micro frameworks for JavaScript. I always stumble upon something new and interesting. Most recently, I found chic.js – a class like interface – and minivents.js – a small event system.

I started by following the examples for each framework and had a class example and a button that triggered events. Neat, but nothing special. I remembered a design pattern called observer – read Learning JavaScript Design Patterns free online – and thought I could combine the two to implement this pattern.

I started by writing a class, and extending it with another. The parent class has three function: init, eat and sleep. The child class overrides init and inherits eat and sleep. The init function takes the minivents.js Event object as a parameter.

var Class = chic.Class;
var Animal = Class.extend({
init: function (eventSender) {
this.info = “I am a 22 year old Animal. I like toys and chasing my tail”;
this.eventSender=eventSender;
this.eventSender.on(“talk”, function(){
alert(“WOOF!”);});
},
eat: function () { return “yummy” },
sleep: function () { return “zzzzzzz” }
});

var Cat = Animal.extend({
init: function (eventSender) {
this.eventSender=eventSender;
this.eventSender.on(“talk”, function(){alert(“MEOW!”);});}
});

Next, I hooked up each function to a button in HTML

<button onclick=”trigger()”>Trigger Event</button>
<button onclick=”e()”>Eat</button>
<button onclick=”s()”>Sleep</button>

function trigger(){changed.emit(“talk”);}
function e(){alert(“Fluffy says: “+fluffy.eat()+”\nPaws says: “+paws.eat());}
function s(){alert(“fluffy sleeps: “+fluffy.sleep()+”\nPaws sleeps: “+paws.sleep());}

Lastly, I created the Event object and two classes – passing the Event object to them.

var changed = new Events();
var fluffy = new Animal(changed);
var paws= new Cat(changed);

When an event is triggered, each animal responds differently. I do not need to tell them to respond, nor do I need to keep track of how many objects. When I call eat or sleep, I have to tell each object to do so – fluffy.eat(). With the observer pattern, the objects react on their own. The observer pattern is similar to the Pub/Sub pattern.

As your applications become more advanced, you may find yourself implementing existing patterns without even knowing. The patterns exist for a reason – they are the best solution to a common problem. I will spend more time this year learning design patterns.

Live Scenario Planning in Leaflet.js with Turf.js

22 Jan

When I started scenario planning, I had to run my data through a model to get some output. To try a new scenario, I had to rerun the model with new data and then compare the outputs. The process was slow. With the tools currently available, we can do better. I wrote a simpl live scenario planning application using Leaflet.js using Turf.js as the “model”. I put model in quotes because for the example we will simply sum a field in each marker.

The application showing the values of two fields in our data.

The application showing the values of two fields in our data.

The above image is my application. It takes points in Albuquerque and sums two different fields and displays the results in a bar chart below. The application starts as follows:

1. Create a Leaflet.js Map.

var map = L.map(‘map’, {
center: [35.10418, -106.62987],
zoom:15
});

L.tileLayer(‘http://{s}.tile.osm.org/{z}/{x}/{y}.png’).addTo(map);

2. Add my base scenario points and a polygon for the area of interest.

var p1 = turf.point(-106,35, {“marker-color”: “#6BC65F”,”title”: 100, “description”: “Not in Polygon”, “someOtherProperty”:3 });
var p2 = turf.point(-106.62987,35.10418, {“marker-color”: “#6BC65F”,”title”: 4, “description”: “In Polygon”, “someOtherProperty”:6 });
var p3 = turf.point(-106.64429,35.14125, {“marker-color”: “#6BC65F”,”title”: 1, “description”: “Also in Polygon”, “someOtherProperty”:5 });

var polygon = turf.polygon([ [
[-106.73355,35.21197],[-106.73355,35.04911],[ -106.51932,35.04911],[-106.49872,35.19177]
]], {
“fill”: “#6BC65F”,
“stroke”: “#6BC65F”,
“stroke-width”: 5,
“title”:”Polygon”,
“description”:”A sample polygon”
});

var p = turf.featurecollection([polygon]);
var t = turf.featurecollection([p1,p2,p3]);

3. Add the Turf.js points to the map as GeoJSON in Leaflet.js

function onEachFeature(feature, layer) {

layer.bindPopup(“<h3>Add this number: “+feature.properties.title+”</h3>”+feature.properties.description);
}

L.geoJson(t, {
onEachFeature: onEachFeature
}).addTo(map);

4. Using Turf.js, get the sum of the base scenario for two fields.

var sum = turf.sum(p,t,”title”,”titleOutput”);
var sum = turf.sum(p,t,”someOtherProperty”,”otherOutput”);

5. Setup both charts using the base scenario. The example only shows the code for the first chart.

var data={
labels: [“Title Value”],
datasets: [
{label: “Title Value”,
fillColor: “rgba(151,187,205,0.5)”,
strokeColor: “rgba(151,187,205,0.8)”,
highlightFill: “rgba(151,187,205,0.75)”,
highlightStroke: “rgba(151,187,205,1)”,
data: [sum.features[0].properties.titleOutput]}]};

var ctx = document.getElementById(“myChart”).getContext(“2d”);
var myBarChart = new Chart(ctx).Bar(data);

6. Hold the value we will change in a variable so we can change it later.

var x=sum.features[0].properties.titleOutput;

7. Setup an onclick method to modify the scenario. The method will add a marker and provide a textbox to set a value.

map.on(“click”,function(e){
L.marker(e.latlng).addTo(map).bindPopup(‘Add Title Value:<input type=”text” id=”v”><br><button onclick=”add()”>Add</button>’).openPopup();
});

8. Lastly, when the user clicks the button on the new marker, we update the scenario and redraw the chart.

function add(){
myBarChart.destroy();
newValue=Number(document.getElementById(“v”).value);
x+=newValue;
var d = {
labels: [“Title Value”],
datasets: [{
label: “Title Value”,
fillColor: “rgba(151,187,205,0.5)”,
strokeColor: “rgba(151,187,205,0.8)”,
highlightFill: “rgba(151,187,205,0.75)”,
highlightStroke: “rgba(151,187,205,1)”,
data: [x]}]};
myBarChart= new Chart(ctx).Bar(d);
}

If you add a point in the study are, you can add a new Title Value and the chart will update as in the image below.

Title Value is now 10.

Title Value is now 10.

This is a simplistic example but it shows the possibilities for live updating a dashboard as the data changes. This method allows for quick iterations in your models.