Archive | March, 2013

Python: Dynamically Load Module and Call Functions

28 Mar

I was looking through the videos for PyCon 2013 and there was a talk on Dynamic Code Patterns that caught my attention – because I don’t know what they are. I love the idea of loading plugins but have no idea how to write code that would allow for it. So, without watching the video yet, I took a shot at trying to write some code.

I started at David Janes’ Code Weblog to learn how to dynamically load code. I took his function for my code.

Then I wrote a simple python script with two functions and dropped it in a folder:

def hello(x): print “Hello, ” +x
def add(x): d=x.split(“,”) print int(d[0])+int(d[1])

My first attemt at loading a module and calling the function looked like this:

for (path, dirs, files) in os.walk(‘C:\\Documents and Settings\\user\\Desktop\\test’):
i=0
while(i<len(files)):

#print files[i]
a=load_module(‘C:\\Documents and Settings\\user\\Desktop\\test\\’+str(files[i]))
i+=1
a.hello(“Paul”)
print dir(a)

I load the one file in the folder to variable a. This code is awful. For starters, it grabs all the files – even non .py files – and if there are non .py files it will crash. There will be a .pyc after it executes succesfully once – 2nd attempt will fail. It also counts all the files so the loop will be off (out of index range error). Lastly, I hard coded the function and parameters.

My last attempt was a program that prints out the functions in a module and lets you pick one. It then asks for parameters and then executes the function. It uses getattr() to allow the passing of the function name and method from strings.

for (path, dirs, files) in os.walk(‘C:\\Documents and Settings\\user\\Desktop\\test’):
i=0
while(i<len(files)):

#print files[i]
a=load_module(‘C:\\Documents and Settings\\user\\Desktop\\test\\’+str(files[i]))
i+=1

z=dir(a)
i=0
while(i<len(z)):
if z[i][0]==”_”:
i+=1
else:
print z[i]+”\n”
i+=1

var = raw_input(“Choose a Function: “)
name=raw_input(“parameters: “)
getattr(a,str(var))(str(name))

Here is the full code to my simple attempt to dynamically load a module (plugin) and have it exposed automatically to the user.

import md5
import os.path
import imp
import traceback
import sys

try:
os.remove(‘C:\\Documents and Settings\\user\\Desktop\\test\\extone.pyc’)
except:
pass

def load_module(code_path):
try:
try:
code_dir = os.path.dirname(code_path)
code_file = os.path.basename(code_path)

fin = open(code_path, ‘rb’)

return imp.load_source(md5.new(code_path).hexdigest(), code_path, fin)
finally:
try: fin.close()
except: pass
except ImportError, x:
traceback.print_exc(file = sys.stderr)
raise
except:
traceback.print_exc(file = sys.stderr)
raise

for (path, dirs, files) in os.walk(‘C:\\Documents and Settings\\user\\Desktop\\test’):
i=0
while(i<len(files)):

#print files[i]
a=load_module(‘C:\\Documents and Settings\\user\\Desktop\\test\\’+str(files[i]))
i+=1

z=dir(a)
i=0
while(i<len(z)):
if z[i][0]==”_”:
i+=1
else:
print z[i]+”\n”
i+=1

var = raw_input(“Choose a Function: “)
name=raw_input(“parameters: “)
getattr(a,str(var))(str(name))

 

 

Advertisements

C# and SharePoint Backend

17 Mar

I have been using sharepoint lists for my GIS data. My only problem is I’m using different methods in an attempt to get the data out in the fastest and easiest way. I am using haufe.sharepoint and feedparser to use webservices and RSS feeds to parse the data. I have been trying to figure out how to get in to the SQL Server that holds all the goodies. Well, I finally got setup with the info I needed-like an account. Tried pyodbc and had trouble querying – converting uuid to GUID and a few other issues. I switched to C# and the solution was simple. The structure-to get list data-of the sharepoint backend is database>site name>dbo.AllUserData.
Here is some simple code:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Data.SqlClient;
using System.IO;

namespace sp
{
class Program
{
static void Main(string[] args)
{
SqlConnection myConnection = new SqlConnection();
myConnection.ConnectionString = “Data Source=123.45.678;”+ “Initial Catalog=Sitedb name;” + “User id=id;” + “Password=pwd;”;

try
{
myConnection.Open();
}
catch (Exception e)
{
Console.WriteLine(e.ToString());
}

try
{
SqlDataReader myReader = null;
SqlCommand myCommand = new SqlCommand(“select * from dbo.AllUserData where tp_ListId=’2264655-7575-484f-b2576-4d546643esv465dgg'”,
myConnection);
myReader = myCommand.ExecuteReader();

while (myReader.Read())
{

//Do something with data
//where [] is field name.
myReader[‘nvchar14’].ToString();

}
}
catch (Exception e)
{
Console.WriteLine(e.ToString());
}

try
{
myConnection.Close();
}
catch (Exception e)
{
Console.WriteLine(e.ToString());
}
}
}
}

Calling Python From C#

11 Mar

I have been programming ArcGIS Desktop Explorer in C#. My C# is not very good but my python is. I figured out that I can do minimal work in C# by calling python and just returning the results. I use Process.Start().

Why use python? Besides being better at it, because it allows access to all my ArcPy tools. That’s right, I can have a button in Desktop Explorer that allows the user to click it and run analysis on layers in their map.

I have several machines with Desktop Explorer. They are loading layers from remote machine on the network. This machine also has ArcMap and python. The buttons in Desktop Explorer call the scripts on this machine. Requiring only one install of python with all the additional libraries loaded. And one license of ArcMap to call ArcPy.

This works great for real time GIS too. Desktop Explorer can refresh a layer on command or at intervals of 1 minute. I link all the Desktops to a layer on the remote machine and set the refresh to 5 minutes. I have a python script that is running as a scheduled task that reads data from remote sources and writes out the shapefile every 3 minutes. The user and application dont know anything about how the shapefile is created and dont care, they just load the newest data as it comes in. And where I need temporary data, I link straight to the database and load it as notes.

Lastly, if I need to fix or change anything, instead of modifying the plugin and sending it to each machine I can just change the python file and all machines are automatically updated.

Public Safety GIS

11 Mar
My Custom Buttons

My Custom Buttons

ESRI has been promoting ArcGIS Online but I think their best, and least appreciated, product is ArcGIS Desktop Explorer. Besides being free, redistributable, and having a lot of built in functionality, it ships with a .NET SDK.

I have been playing with writing buttons to load multiple files from remote URLS, a button to launch websites, and a button to connect to an FTP site and grab the newest shapefile. My newest button is a data connection to MongoDB. It is written completely in C# and just displays all the data from a collection as notes.

I took this one step further, and because I had trouble figuring out $near in C# I wrote it in python, by only selecting the points near the current map views center, with a limit of 20 points.

The button onClick calls a python script and passes the current map center as an argument. The python script takes the sys.argv and queries the MongoDB and writes out a CSV. The button reads in the CSV and loads the points. I am really working to get the $near figured out in C# but had to make it work quickly.

I’m writing this on my phone and don’t have the code handy but if you are interested leave a comment and I will update this post with the C# and python. Think I’ll also write a post on calling python from C#.