Tips and tricks for ArcGIS, Excel and R

Liz Law reports on the Wilson Conservation Ecology Lab meeting for Aug 5th 2011

Why? Because it’s a Bustard! (Photo of Australian Bustard near Morven, QLD, by Liz Law)

How many times have we spend a good few hours (even days) trying to do something in ArcGIS/Excel only to find out, after we finished, that we could have done it in a fraction of time another way?

For me, this has occurred quite a lot. So, I decided to run our weekly lab meeting on tips, tools and functions that can save us time. Here is a quick summary of what we talked about:

Model builder and other geoprocessing tools

Making data with interpolation

Editing shapefile layers

Getting help with ESRI stuff



Model builder and other tools: It took me ages to find out that you can add custom toolboxes, in which you can drag in copies of your favourite tools, and also create your own using model builder.

I think model builder is a really useful tool within ArcGIS toolbox. Essentially it is a space where you can build and visualise geoprocessing models (i.e. a series of tools, functions, scripts, etc.). I find it really useful to record, communicate, and repeat your model (analysis process). I also find it a little more intuitive to batch process, or to run several processes using the output of the prior one as the input of the next (without collecting masses of intermediate files). More information on model builder is available on the ESRI website, and they also provide a free online training seminar.

Luis Verde noted that large models can get a bit buggy and can give a generic error message. When this happens, he recommends using the “make feature layer” to make a temporary copy of your inputs prior to each tool. For some reason this works.

There are also lots of geoprocessing models that already exist for a number of different tasks. Some of these can be found and downloaded at the ESRI “geoprocessing model and script tool gallery”. Other packages of tools operate as plug-ins, for example ET Geowizards, and Hawths tools (aka GME). Ayesha Tulloch says some of these are great: for example, the ET Geowizards “Explode multipart polygon” tool is way less buggy than the one provided by ArcGIS (and actually does the job right the first time). However, Ayesha also cautions that while the old Hawths tools was pretty awesome, the newer version (GME) is not. It doesn’t even work with any ArcGIS prior to 10.0, which is funny, because a lot of the functions that we use to use it for are apparently available in 10.0 anyway…

Making data with interpolation:  Jane MacDonald has not been working much with ArcGIS, but her co-workers have. And she is worried about an emerging trend to use interpolation to make data layers from point observations. While it is really easy to get results using this technique, the basic principles of models apply: junk in = junk out. You really need to question whether you are getting accurate maps, for example by validating using reserved point data, and/or comparing with GLM outputs.

Editing shapefile layers: Karen Mustin has been going through the joys of editing shapefiles. There are about a billion ways to do editing things in ArcGIS. I recommend checking out the ESRI online training seminars of which there are about 4 ones on “editing tips and tricks”. In particular, if you don’t know what “snapping” and “sticky move tolerance” are, or how to modify them, I highly recommend you seek advice BEFORE editing your layers.

Getting help with ESRI stuff: If you have spent ages trying to understand ArcGIS, gone through all the normal help and forums, and you still have unanswered questions, you can always give ESRI a call. However, many large institutions have people that are designated ESRI gateways, says Jude Keyes. Our UQ people are Jürgen Overheu (GPEM), Gai Trewinnard-McNeill (ITS), and Steven Clark (ITS).

Vlookup: Excel is probably one of the most commonly used spreadsheet programs, but probably one of the most poorly utilised ones as well. Many people turn to proper database software (and for good reason) if they have large databases that they have to run many query variations on, but you can also create a database in Excel, or do pretty basic query-like tasks. For example, Angela Guerrero suggests if you have a lookup table that you want to use to append values to a list you can use functions like VLOOKUP. If you don’t know what this is, look it up!

SDMTools: Saving the best till last, Luke Shoo blew us out of the water with his “SDMTools”. This R-package, developed by Luke and his colleagues, provides a set of tools for post processing the outcomes of species distribution models: comparing models, tracking changes in distributions over time, visualising outcomes, selecting thresholds, calculating measures of accuracy and landscape fragmentation statistics, and more. Absolutely amazing, extremely useful, and supremely beautiful.

Thanks everyone for a fantastic and very useful meeting!