Archive | April, 2013

Auto Message Popups in ArcGIS Desktop Explorer From RabbitMQ

19 Apr
Message sent from terminal to ArcGIS Desktop Explorer Extension.

Message sent from terminal to ArcGIS Desktop Explorer Extension.

My previous post showed how to send messages to GIS from a terminal using Python-Pika, RabbitMQ, and C#. It required the user to click a button to receive them. I have modified code from several sources to make it run on a thread- as an extension to ArcGIS Desktop Explorer. Every 2 seconds, the extension checks for messages.

I modified code from:

LosTechies – For C#.

RabbitMQ Tutorial – For Python Producer.

And the ArcGIS Resource Center for the threading info.

I can send you the Addin .EAZ if you wish. Here is what I came up with:

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Windows.Forms;

using RabbitMQ.Client;

using ESRI.ArcGISExplorer;
using ESRI.ArcGISExplorer.Application;
using ESRI.ArcGISExplorer.Mapping;
using ESRI.ArcGISExplorer.Geometry;
using ESRI.ArcGISExplorer.Data;
using ESRI.ArcGISExplorer.Threading;
namespace RabbitMQgisExtension
{
public class Extension : ESRI.ArcGISExplorer.Application.Extension
{
string msg;
static IConnection connection;
static IModel channel;
System.Timers.Timer _t = new System.Timers.Timer(2000);

public override void OnStartup()
{
_t.Start();
_t.Elapsed += new System.Timers.ElapsedEventHandler(_t_Elapsed);

}

public override void OnShutdown()
{
_t.Stop();
channel.Close();
connection.Close();
}

void _t_Elapsed(object sender, System.Timers.ElapsedEventArgs e)
{
var connectionFactory = new ConnectionFactory();

connection = connectionFactory.CreateConnection();
channel = connection.CreateModel();
channel.ExchangeDeclare(“GIS”, ExchangeType.Direct);
channel.QueueDeclare(“GIS”, false, false, false, null);
channel.QueueBind(“GIS”, “GIS”, “PAUL”);
BasicGetResult result = channel.BasicGet(“GIS”, true);

if (result != null)
{
msg = Encoding.UTF8.GetString(result.Body);
MessageBox.Show(msg, “You’ve Got a Message!”);
//If set Message to send coords in form “35,-106” Next lines will draw points.
//string[] coords = msg.Split(‘,’);
// float x = float.Parse(coords[0]);
// float y = float.Parse(coords[1]);
//Note fromRabbit = new Note(“From RabbitMQ”, new ESRI.ArcGISExplorer.Geometry.Point(y, x));
//ESRI.ArcGISExplorer.Application.Application.ActiveMapDisplay.Map.ChildItems.Add(fromRabbit);
}
}

 

and the Python Producer – EXAMPLE: python send.py KEY TEXT

(python send.py PAUL message to you):

import pika
import sys

connection = pika.BlockingConnection(pika.ConnectionParameters(
host=’localhost’))
channel = connection.channel()

channel.exchange_declare(exchange=’GIS’,
type=’direct’)

to = sys.argv[1]
message = ‘ ‘.join(sys.argv[2:])

channel.basic_publish(exchange=’GIS’,
routing_key=to,
body=message)
print “Sent %r:%r” % (to, message)
connection.close()

Send Message to GIS or Revit from a Terminal using RabbitMQ, Python, Pika and C#

19 Apr

I have been playing with messaging in software applications. Using RabbitMQ, C# and Python, I got the bright idea to send messages to GIS from a python console – which will later be another program or windows form, but for demo purposes I just needed to verify the possibility. The SDK for my GIS is in C#, and so is the SDK for Revit.
I will show you what I did in GIS and then explain how this would work in Revit.
First, I sent a message to the GIS software.
20130419-081234.jpg
The DOS window is my command center that sends the messages. You can see that there are two commands: one has JOE and one has PAUL in it. You can send messages to every listener, or you can specify a keyword or word pattern – in this case PAUL and JOE are the keywords. My GIS app will only receive messages sent to PAUL. When you click the button, it grabs all waiting messages.
If I can send text, I can send data too. What if I send coordinates? I can draw them in the map.
20130419-081352.jpg
Now it takes the message and does something with it. This example has the sender and receiver on the same machine, but they do not have to be. So for Revit, what if we had a window that could update schedules in Revit with data. A user could input the data and the Revit user could press a button to update the model with the new data. Perfect for additional fields in a schedule. Or you could set it to run as a thread so when a message comes in the box pops up automatically. Here is the Code for the Plugin:
string msg = null;
var connectionFactory = new ConnectionFactory();
IConnection connection = connectionFactory.CreateConnection();
IModel channel = connection.CreateModel();
channel.ExchangeDeclare(“GIS”,ExchangeType.Direct);
channel.QueueDeclare(“GIS”,false,false,false,null);
channel.QueueBind(“GIS”,”GIS”,”PAUL”);
BasicGetResult result = channel.BasicGet(“GIS”, true);
if (result != null)
{
msg = Encoding.UTF8.GetString(result.Body);
MessageBox.Show(msg, “Status”);

}
else
{
MessageBox.Show(“No Messages Waiting.”, “Status”);
}
And the sender in python:
import pika
import sys
connection = pika.BlockingConnection(pika.ConnectionParameters(
host=’localhost’))
channel = connection.channel()
channel.exchange_declare(exchange=’GIS’,
type=’direct’)
to = sys.argv[1]
message = ‘ ‘.join(sys.argv[2:])
channel.basic_publish(exchange=’GIS’,
routing_key=to,
body=message)
print “Sent %r:%r” % (to, message)
connection.close()

Public Safety GIS 2: Python, MongoDB, C# and ArcGIS Desktop Explorer

17 Apr

I wrote a plugin for ArcGIS Desktop explorer that called a python script and passed the map center as an argument. The python script queried a MongoDB and wrote out a csv of closest points. The button then loads the csv as points. I had to do this because I didn’t know how to do it in C#. Well, I figured it out.

Now C# does all the work querying the database and drawing the points. I still have a python script that reads data from a csv export from sharepoint, geocodes it and creates the mongoDB.

Here is my code:

PYTHON TO CREATE THE DB:

from pymongo import Connection,GEO2D
import urllib, urllib2, simplejson
import csv

data = csv.reader(open(‘C:\Documents and Settings…\myfile.csv’))
fields=data.next()
db=Connection().rtcc
db.cnau.create_index([(“loc”,GEO2D)])

for row in data:
try:
address=row[0]
combinedaddress=address+”,albuquerque, NM”

param = {‘q’: combinedaddress,’output’: ‘json’, ‘oe’: ‘utf8’}
url = ‘http://maps.google.com/maps/geo?’ + urllib.urlencode(param)
rawreply = urllib2.urlopen(url).read()
reply = simplejson.loads(rawreply)
print reply[‘Placemark’][0][‘Point’][‘coordinates’][0]
long=reply[‘Placemark’][0][‘Point’][‘coordinates’][0]
lat=reply[‘Placemark’][0][‘Point’][‘coordinates’][1]
z={“loc”:[long,lat],”Activity”:row[1]}

db.cnau.insert(z)
except KeyError:
pass

 

C# TO PLUGIN ARCGIS DESKTOP EXLPORER:

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Windows.Forms;
using ESRI.ArcGISExplorer;
using ESRI.ArcGISExplorer.Application;
using ESRI.ArcGISExplorer.Mapping;
using ESRI.ArcGISExplorer.Geometry;
using ESRI.ArcGISExplorer.Data;
using ESRI.ArcGISExplorer.Threading;

using MongoDB;
using MongoDB.Bson;
using MongoDB.Driver.Linq;
using MongoDB.Driver;
using MongoDB.Driver.Builders;

namespace MongoC
{
public class Button : ESRI.ArcGISExplorer.Application.Button
{

public override void OnClick()
{

Folder folderMapItem = new Folder();
MapDisplay mdisp = ESRI.ArcGISExplorer.Application.Application.ActiveMapDisplay;

//use this to change the projections 35,-106
FeatureLayer proj = FeatureLayer.OpenShapefile(“C:\\Documents and Settings…\\forProjection.shp”);
// Path for Windows 7: FeatureLayer proj = FeatureLayer.OpenShapefile(“C:\\Users\\user\\…\\forProjection.shp”);
mdisp.CoordinateSystem2D = proj.CoordinateSystem;
string center = mdisp.Center.ToString();
folderMapItem.Name = “Close To Me”;
string[] c = center.Split(null);
String lon = c[3];
String lat = c[6];

MongoClient client = new MongoClient(); // connect to localhost. Deploy at: “mongodb://ip.of.host:27017”
MongoServer server = client.GetServer();
MongoDatabase db = server.GetDatabase(“rtcc”);

var collection = db.GetCollection<BsonDocument>(“cnau”);

var query = Query.Near(“loc”, double.Parse(c[3]), double.Parse(c[6]));
//var cursor = collection.Find(query); Put this in foreach so dont need it anymore.
foreach (BsonDocument item in collection.Find(query).SetLimit(5))
{
BsonElement loc = item.GetElement(“loc”);
string g = loc.Value.ToString();
string x = g.Trim(new Char[] { ‘[‘, ‘]’ });
String[] a = x.Split(‘,’);
// BsonElement name = item.GetElement(“name”); Work for all indexes in MongoDB
// Pass the other data to the notes popups below

ESRI.ArcGISExplorer.Geometry.Point mypoint = new ESRI.ArcGISExplorer.Geometry.Point(double.Parse(a[0]), double.Parse(a[1]));

// Create a Note using the Point geometry.
Note mypointNote = new Note(“MyNote”, mypoint, Symbol.Marker.Pushpin.Orange);

// Populate the Note popup with information about the Point geometry using the ToString method.
StringBuilder info = new StringBuilder(@”<p><a href=””http://google.com””>Google</a></p>&#8221;);
info.AppendLine(@”<p>” + mypoint.ToString() + @”</p>”);

//MapDisplay disp = ESRI.ArcGISExplorer.Application.Application.ActiveMapDisplay;
mypointNote.Popup.Content = info.ToString();

// Add the Note to the current Map.
//mdisp.Map.ChildItems.Add(mypointNote);
folderMapItem.ChildItems.Add(mypointNote);
}

mdisp.Map.ChildItems.Add(folderMapItem);
}
}
}

Messaging in Python with RabbitMQ and Pika

17 Apr

I just started using RabbitMQ and Pika and sent a message – “hello world!” Cool, but so what. I threw an IF statement in my code to simulate reading the message and doing different tasks based on the content of the message. This is better, but still interested in doing more. How about sending binary data – files?

Sure!

I modified the simple tutorial producer and consumer to send a JPG. The producer opens a jpg image in binary mode RB and reads it to a variable. It then sends that variable to the queue.

The consumer knows to expect a jpg – but you could code all this in assuming it doesn’t know. It opens a blank file, gets the message and writes it to the file in binary mode – wb.

This is cool. Now we can send tasks – files to be operated on. And these tasks can be sent to multiple workers.

Here is the code:

SEND.PY

import pika
import logging
f=open(“image.jpg”,”rb”)
i=f.read()

logging.basicConfig()
connection = pika.BlockingConnection(pika.ConnectionParameters(host=’localhost’))
channel = connection.channel()

channel.queue_declare(queue=’hello’)

channel.basic_publish(exchange=”,routing_key=’hello’,body=i)
print ” [x] Sent ‘Hello World!'”
connection.close()

 

CONSUME.PY

import pika

f=open(“outputimage.jpg”,”wb”)

connection = pika.BlockingConnection(pika.ConnectionParameters(
host=’localhost’))
channel = connection.channel()

channel.queue_declare(queue=’hello’)

print ‘ [*] Waiting for messages. To exit press CTRL+C’

def callback(ch, method, properties, body):
f.write(body)
f.close()
channel.basic_consume(callback,
queue=’hello’,
no_ack=True)

channel.start_consuming()

RabbitMQ, Python and C#

17 Apr
C# consuming a message from python producer via RabbitMQ.

C# consuming a message from python producer via RabbitMQ.

I don’t know much about software patterns. My applications tend to be more of the script variety – though I throw some OOP in for fun. I recently came across RabbitMQ – a message broker. It sits between producers and consumers of messages – it is the queue. I tested out the python hello world example and then got the bright idea to use C# as one piece. The thing about RabbitMQ is that it uses a standard Advanced Messaging Queuing Protocol. This means any language or application that implements that protocol can communicate – think HTTP or FTP protocols.

Here is the python producer taken from the RabbitMQ Tutorial. You will need pika.

import pika
import logging
logging.basicConfig()
connection = pika.BlockingConnection(pika.ConnectionParameters(host=’localhost’))
channel = connection.channel()

channel.queue_declare(queue=’hello’)

channel.basic_publish(exchange=”,routing_key=’hello’,body=’Hello World!’)
print ” [x] Sent ‘Hello World!'”
connection.close()

 

Simple enough. Now I wrote a C# form that consumes the messages onClick of a button and displays them in a text box – appending the new messages as they come in.

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Windows.Forms;
using RabbitMQ.Client;

 

namespace WindowsFormsApplication1
{
public partial class Form1 : Form
{
public Form1()
{
InitializeComponent();
}

private void button1_Click(object sender, EventArgs e)
{
var connectionFactory = new ConnectionFactory();
IConnection connection = connectionFactory.CreateConnection();
IModel channel = connection.CreateModel();
channel.QueueDeclare(“hello”, false, false, false, null);
BasicGetResult result = channel.BasicGet(“hello”, true);
if (result != null)
{
string message = Encoding.UTF8.GetString(result.Body);

textBox1.Text += message+”\r\n”;
}

channel.Close();
connection.Close();
}
}
}

 

C# Front-End to Python

12 Apr

I am best at coding in Python, but when it comes adding GUIs, I find myself lacking. My very first C# program was a plugin that had one line of code

System.Diagnostics.Process.Start("http://www.webpage.com");

That led to my second, which just replaced the URL with “python MyApp.py.” It finally clicked. I could just use the drag and drop GUI form builder in C# Visual Studio Express. Connect each python script to a button, pass text box values to the python script using sys.argv and grab the output as a string. The best part is I found a function for this on CodeProject.

So now I can write python scripts for my functionality and use a windows form for the GUI control.

Get Attachments from SharePoint SQL Server

11 Apr

I have been attempting to grab data from SharePoint to Geocode for a GIS Platform I was working on. I started by using Python to Parse the RSS of the List. Then I moved on to using the Web Service via SOAP. And lastly, I have switched to C# and am grabbing the SQL Server data for the SharePoint Backend. NOTE: If you can, run code on the server using SharePoint Service:

SPWeb web = new SPSite(“<Site URL>”).OpenWeb();
//Open List
SPList list = web.Lists[“<ListName>”];
//Get the item
SPListItem item = list.Items[1];
//Get the folder
SPFolder folder = web.Folders[“Lists”].SubFolders[strListName].SubFolders[“Attachments”].SubFolders[item.ID.ToString()];

foreach(SPFile file in folder.Files)
{
byte[] binFile = file.OpenBinary();
System.IO.FileStream fstream = System.IO.File.Create(“c:\\MyDownloadFolder\\” + file.Name);
fstream.Write(binFile, 0, binFile.Length);
}

I do not have access to the server, so I have had to go the difficult route. This method still requires a read only account on the database – which you are more likely to get than physical access to the server.

I have stated in previous posts how the SharePoint Database is structured. I usually stay in dbo.AllUserData. But to get the contents of an attachment, you need dbo.AllDocStreams. The Content field holds the data – in binary. This code will grab all the attachments and assign a filename. To do this right, you need to perform a few joins to figure out the real name. Pascal’s Blog may help you in this.  I got this far and moved on. All I really needed was the path to the file. I solved this by reading Pascal’s Blog and found that the attachments reside at:

 /MyRootWebsite/MyWebsite/Lists/MyList/Attachments/tp_ID/nvarchar14

This made my life easier. But still good to know how to grab the binary.  Here is the code to loop through the dbo.AllDocStreams and only grab the content. Then write it out as a JPG with an incremented filename.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Data.SqlClient;
using System.IO;
using System.Diagnostics;

namespace SharepointAttachmentCIT
{
    class Program
    {
        static void Main(string[] args)
        {
            StringBuilder myString = new StringBuilder();
            SqlConnection myConnection = new SqlConnection();
            myConnection.ConnectionString = “Data Source=ip.of.sharepoint.;” + “Initial Catalog=WSS_YourDB;” + “User id=Yourname;” + “Password=YourPasswd;”;

            try
            {
                myConnection.Open();
            }
            catch (Exception e)
            {
                Console.WriteLine(e.ToString());
            }

            try
            {
                SqlDataReader myReader = null;
                SqlCommand myCommand = new SqlCommand(“select dbo.AllDocStreams.Content as ‘C’from dbo.AllDocStreams”,
                                                         myConnection);
               myReader = myCommand.ExecuteReader();

                int i = 0;
               while (myReader.Read())
               {
                   byte[] buffer = (byte[])myReader[“C”];
                   using (FileStream fs = new FileStream(“test”+i.ToString()+”.jpg”, FileMode.Create))
                      
                {
                    fs.Write(buffer, 0, buffer.Length);
                    i = i + 1;
                 
                }

                }
            }
            catch (Exception e)
            {
                Console.WriteLine(e.ToString());
            }

            try
            {
             
                myConnection.Close();
              
            }
            catch (Exception e)
            {
                Console.WriteLine(e.ToString());
            }

        }
    }
}