I committed to testing this a long time ago, however, a number of other projects intervened, so I have only just got around to writing up this short tutorial. One of the exciting things from the ESRI Developers Conference this year was the launch of the R-ArcGIS bridge. In simple terms, this enables you to run R scripts from within ArcGIS and share data between the software. In fact, this is all explained in a nice interview here.

I won’t go into detail about the R script itself, and the code can be found on github. If I am honest, this is pretty rough, and was written to demonstrate what could be done – that said, it should be usable (I hope… but don’t complain if it isn’t!). ESRI have also provided a nice example which can be found here, and was the basis of my code.

Preparing R

Before you can link ArcGIS Pro to R, you need to install and load the ‘arcgisbinding’ package, which is unfortunately not on CRAN. There are instructions about how to do this here using a Python toolbox; however, I preferred a more manual approach.

Open up R and run the following commands which installs the various packages used by the toolbox. You might also need to install the Rtools utilities as you will be compiling on Windows (available here). Although the TwitteR and httr packages are available on CRAN, for some reason I have been having issues with the latest versions failing to authenticate with Twitter; as such, links to some older versions are provided.

#Install the arcgisbinding package
install.packages("https://4326.us/R/bin/windows/contrib/3.2/arcgisbinding_1.0.0.111.zip", repos=NULL, method="libcurl")

#Install older versions of the TwitteR and httr packages
install.packages("https://cran.r-project.org/src/contrib/Archive/twitteR/twitteR_1.1.8.tar.gz", repos=NULL, method="libcurl")
install.packages("https://cran.r-project.org/src/contrib/Archive/httr/httr_0.6.0.tar.gz", repos=NULL, method="libcurl")

#Load the arcgisbinding package and check license
library(arcgisbinding)
arc.check_product()

Creating a Twitter Search App

Before you can use the Twitter Search Tool in ArcGIS Pro, you first need to register an app with Twitter, which gives you a series of codes that are required to access their API.

1. Visit https://apps.twitter.com/ and log in with your Twitter username and password.
2. Click the “Create New App” button where you will need to specify a number of details about the application. I used the following
3. Name: ArcGIS Pro Example
4. Description: An application testing R integration with ArcGIS Pro and Twitter
5. Website: http://www.alex-singleton.com
6. I left the callback URL blank, then checked the “Yes, I agree” to the developer agreement, and clicked the “Create your Twitter application” button.
7. On the page that opens, you then need to click on the “Keys and Access Tokens” tab. You need four pieces of information that enable the Toolbox to link up with Twitter. The first two are displayed – “Consumer Key (API Key)” and the “Consumer Secret (API Secret)”. You then need to authorize this application for your account. You do this my clicking the “Create my access token” button at the base of the page. This creates two new codes which are now displayed – “Access Token” and “Access Token Secret”. You now have the 4 codes required to run a Twitter search in ArcGIS Pro.

R Script

I created an R script that:
1. Authenticates a session with Twitter
2. Performs a search query for a user specified term within a proximity (10 miles) of a given lat / lon location
3. Outputs the results as a Shapefile in a folder specified

The inputs to the script include the various access codes, a location, a search term and an output file location. These variables are all fed into the script based on Toolbox inputs. Getting the inputs is relatively simple – they appear in the order that they are added to the Toolbox, and are acquired via in_params[[x]] where x is the order number; thus search_term = in_params[[1]] pulls a search term into a new R object called “search_term”. The basic structure of a script are as follows (code snippet provided by ESRI):

tool_exec <- function(in_params, out_params) {
        # the first input parameter, as a character vector
        input.dataset <- in_params[[1]]
        # alternatively, can access by the parameter name:
        input.dataset <- in_params$input_dataset

        print(input.dataset)
        # ... do analysis steps

        out_params[[1]] <- results.dataset
        return(out_params)
      }

For more details about the functions available in arcgisbinding, see the documentation located here

How to use the Twitter Search Tool

The Twitter Search Tool was run within ArcGIS Pro and requires you to add a new toolbox. The toolbox should be downloaded along with the R script and placed in a folder somewhere on your hard drive. The files can be found on github here.

1. Open ArcGIS Pro and created a new blank project called Twitter Map.
2. Create a new map from the insert menu
3. From the map tab, click the “basemap” button and select the OpenStreetMap tile layer
4. Zoom into Liverpool on the map using the navigation wheel
5. Find the latitude and longitude of map centre. These are recorded just under the map on the window border. The centre of Liverpool is approximately -2.95 (longitude), 53.4 (latitude) (although displayed as 002.95W, 53.40N)
6. Click on the “Insert” menu, the “Toolbox” and then “Add Toolbox” buttons. Navigate to the folder where you have the Toolbox and R script. Click on the Twitter.tbx file and press the “Select” button.
7. If you don’t see a side bar called “Geoprocessing”, then click on the “Analysis” tab and press the “Tools” button. After this is visible, under the search box there is a “Toolboxes” link. Click this and you will see the Twitter toolbox listed. If you look inside the toolbox you will see the Twitter Search script – click on this to open.
8. Enter a search term (I used “Beatles” – hey we are in Liverpool), the Twitter authentication details, the location and where you want the output Shapefile stored. This defaults to the geodatabase associated with the project; however, you can browse to a folder and specify a Shapefile name – e.g. Twitter_Beatles.shp.
9. Press the “Run” button and with luck you should now have a Shapefile created in the folder specified.
10. Add the results to your map by clicking on the “Map” tab, then the “Add Data” button. Browse to where you saved the Shapefile and click the “Select” button.

The following screenshot is of the Shapefile shown on an OpenStreetMap basemap; with the attribute table also shown – you will see that the full Tweet details are displayed as attributes associated with each point.

Anyway, I hope this is of use and can assist people getting started linking R to ArcGIS.

View the site here.

HEFCE have released funding prior to a full loan system being introduced that provides bursaries to students wishing to pursue postgraduate study. The ways in which this scheme are being operated by the different university are d…

My new book (co-edited with Chris Brunsdon) is now out.

Many thanks to all the chapter authors for their hard work; and if not arrived already, a very brightly coloured book should be in the post!

Thanks also go to Sage for a really nice production job.

About the book…

Geocomputation is the intersection of advanced computational methods and geographical analysis and modelling. Geocomputation is applied and often interdisciplinary, with methodological developments typically embedded in applications seeking to address real world problems.

Geocomputation excels as a framework for researching many contemporary social science problems associated with large volumes of dynamic and spatio-temporal ‘big data’, such as those generated in ‘smart city’ contexts or from crowdsourcing.
This text:

1. provides a selection of practical examples of geocomputation techniques and ‘hot topics’ written by world leading practitioners
2. Integrates selected supporting materials, such as code and data so that readers can work through some examples themselves
3. Chapters provide highly applied and practical discussions of: Visualisation and exploratory spatial data analysis / space time modelling / spatial algorithms / spatial regression and statistics / open geographic information systems and science
4. All chapters are uniform in design, and each includes an introduction, case study and conclusion – drawing together both the generalities of the chapter topic and illustration through the case study application. Guidance for further reading is also provided.This accessible text, published in full colour, has been specifically designed for those readers who are new to Geocomputation as an area of research, showing how complex real-world problems can be solved through the integration of technology, data, and geocomputational methods. This is the key primer for applied Geocomputation in the social sciences.

Quite a long time ago I spent considerable hours trawling through QMRG reports that featured in old issues of Area and Transactions of the Institute of British Geographers to examine how the committee had evolved over time. In these pre R days, I assem…

Yesterday I needed to simplify a shapefile quite substantially to get the size down enough that it could be loaded into CartoDB. Using QGIS this tended to leave sliver or gaps between polygons, but I came across Mapshaper. This is primarily a command l…

Paul Longley , James Cheshire and I supervised the PhD of Chris Gale over the past couple of years who developed the 2011 Census ONS Output Area Classification.

During development, Paul and I were approached by the Greater London Authority (GLA) to…

The Transport Map Books are available for each local authority district in England and present a series of maps related to commuting behaviour. The data are derived from multiple sources including: the 2011 Census, Department for Transport estimates and the results of a research project looking at carbon dioxide emissions linked to the school commute.

All the maps are available to download HERE; and the R code used to create them and the emissions model is on Github.

Travel to work flows

These data relate to Middle Layer Super Output Area (MSOA) level estimates of travel to work flows by transport mode. The raw data are available from the ONS. For the maps, the flows have been limited to those both originating and terminating within each local authority district.

Accessibility to Services

The Department of Transport provide a range of statistics at Lower Layer Super Output Area level about accessibility and connectivity to a series of key services. A subset of variables were mapped.

Emissions associated with the school commute

These data were generated as part of an ESRC funded project investigating emissions associated with the school commute. The model provides an estimate of the carbon dioxide emitted at Lower Layer Super Output Area level. For full details of the methodology, see the open access paper:

Singleton, A. (2013) A GIS Approach to Modelling CO2 Emissions Associated with the Pupil-School Commute. International Journal of Geographical Information Science, 28(2):256–273.

Car availability and travel to work mode choice

These attributes were extracted from the 2011 census data provided by Nomis at Output Area level.

Distance and mode of travel to work

Workplace zones are a new geography for the 2011 census for the dissemination of daytime population statistics. A number of attributes were selected related to transport, and also were downloaded from Nomis.