Automate and customize GIS workflows with R and Python

[2.] R Programming

R programming is a free, open-source language for statistical computing and data visualization. Statisticians use it for everything from exploratory analysis, and data mining to graphing. Despite ArcGIS and Q-GIS being mostly Python-based, both have extensions to work with R.

R programming and GIS

R is a statistical programming language that can be used for data analysis, visualization, and modeling. R can also be used for GIS applications, such as spatial statistics, geostatistics, spatial econometrics, and spatial machine learning. R can integrate with GIS software in different ways, such as importing and exporting spatial data, calling GIS functions, or embedding R scripts. Some of the common R packages and functions for GIS are sp, sf, raster, rgdal, maptools, and rgeos. To use R with GIS software, you need to install the appropriate R version and packages, load the libraries, and write or run the code in an editor or a console.

Integration of R libraries

The usage of R in GIS is growing because of its enhanced capabilities for statistics, data visualization, and spatial analytics. Here are some important fields that often require the use of R.

Data Visualization – By leveraging packages like ggplot2, GIS users use R mainly for statistical analyses and plotting data. There are various mapping and data visualization packages like tmap and ggplot2. If you’re already familiar with these tools, they’re fairly straightforward for visualizations.

Table Operations – You can perform some powerful table operations with both Python and R. But you can’t underestimate some of the packages available in R. For example, tools like dplyr are intuitive to use and give flexibility for data manipulation.

Data Support – Not only does R support spatio-temporal arrays (data cubes), but it also supports tools like tidycensus to obtain census bureau data. For anyone working with government data, R offers a package to help with these types of trivial tasks.

Why not use Python?

While Python can do most of what R can do, we typically can use a two-pronged approach in GIS. Because you can do most work in both languages, it usually comes down to whatever you feel most comfortable using. While R is good at visualization and statistical analysis, Python is particularly good at working with file systems, networks, web scraping, and automation. This is why Python is the default programming language for Q-GIS and ArcGIS, instead of R.

While matplotlib is an alternative to ggplot2, some data analysts prefer one over the other. You can use both R and Python to make maps. Albeit, most are rudimentary without capabilities to customize them as you can in GIS software. But they’re still functional maps. For more advanced spatial analysis, there are libraries like PySal such as detecting clusters, outliers, and hot spots.

How can you use R in GIS?

It’s becoming more common practice to use R within your GIS workflow. Whether it’s for computational analysis or data visualization, there always seems to be a usage of R.

i) R-ArcGIS Bridge

The purpose of R-ArcGIS Bridge is that you can store your vector and raster data in ArcGIS. Then, you can directly access it in R and return R objects back into ArcGIS native data types. At the same time, you can also use powerful spatial analysis and visualization tools in ArcGIS and seamlessly go back and forth to R. This makes it ideal for R and ArcGIS users to use in R Notebooks.

ii) Processing R Provider (Q-GIS)

If you’re looking for an open-source GIS software solution to leveraging the open-source statistical language of R, then the Processing R Provider is likely what you’d be looking for in Q-GIS. This plugin allows you to write and run R scripts natively inside Q-GIS. In order to use this plugin, you will have to install R on your machine with the correct R packages added beforehand.

——————————————————————————————————————————————————————————

Training course – Introduction to GIS and R programming

R has a full library of tools for working with spatial data. This includes tools for both vector and raster data, as well as interfacing with data from other sources (like ArcGIS) and making maps. These tutorials — which build off Claudia Engel’s excellent GIS in R tutorials — are designed for users with some familiarity with R, but require no knowledge of spatial analysis. If you aren’t used to working with R, you will probably want to spend some little time familiarizing yourself with the language before starting this series. (Here’s a good set of R tutorials if you need them).

Each tutorial is divided into several parts (to be done in sequence), and include a zipped folder of data for exercises.  In addition, cheatsheets are provided which may be of help in remembering the various commands you will frequently use. Each tutorial is meant to take ~1.5 hours, not including software installation. These tutorials have a share-and-share-alike Creative Commons license, so please feel free to use and modify as you see fit. You can find rmarkdown source files on github here. There is a variety of important R command cheatsheets. In fact, learning GIS in R involves learning both concepts and vocabulary. Here are some cheatsheets to help with the later.

• Vector Data Cheatsheet
• Raster Data Cheatsheet
• Third-party Cheatsheet

Tutorial 1: Introduction to GIS Data Types

The aim of Tutorial 1 is to provide users with a solid understanding of how R thinks about spatial data. It covers some material that users will not need to deal with on a day to day basis, but by providing an understanding of the organization of each library, it is my hope they will help users avoid problems, solve problems when they arise, and to know how to identify the sources of problems when they need to ask others for guidance. Here is the data for Tutorial 1 with parts 1.1, 1.2 and 1.3 as detailed below.

• Part 1.1: Setup Instruction
• Part 1.2: Vector Data in R
• Part 1.3: Raster Data in R

Tutorial 1.5: Background on Projections, Coordinate Reference Systems, and Geographic Coordinate Systems

Concepts like “projections” and “coordinate systems” are tricky on their own, and this trickiness is made all the worse by the fact there’s incredible inconsistency in how these terms are used. In this handout, I lay out the two main concepts you need to know, then provide an overview of what different programs and communities actually mean when they use terms like “projection”.

• How Computers Relate x-y Coordinates to Real Locations on Earth

Tutorial 2: Combining Multiple Data Sources

Rarely can the answers we seek be found in a single data-set. Here are a set of tools for combining multiple datasets through methods like spatial joins, distance calculations, etc. Here is the Data for Tutorial 2 with parts 2.0, 2.1 and 2.2 as detailed below.

• Part 2.0: Setup instructions
• Part 2.1: Spatial Joins (Here is also a supplementary Doc on lapply and sapply).
• Part 2.2: Geometric Manipulations: Buffers, Distances, Intersections, etc.

Tutorial 3: Making Cartographic Maps

Visualization is central to sharing our results. This tutorial covers the basics of making visually appealing and informative maps in R. Here is Data for Tutorial 3 with parts 3.1 and 3.2 as detailed below.

• 3.1 Making Maps from Vector Data
• 3.2 Making Maps from Raster Data

Tutorial 4: Geocoding (Very Short) and Background on APIs

How to convert addresses and location names to latitudes and longitudes from R using the Google Maps API. Also some optional background on what an API is, and how most web APIs work. Here is Data for Tutorial 4 with part 4 as detailed below.

• 4 Geocoding with the Google Maps API

Tutorial 5: Spatial Statistics and Surface Interpolation

Libraries for spatial statistics like Ripley’s K, Moran’s I, Spatial Auto-Regressive  Lag regressions, Error Auto-Regressive regressions, etc, as well as libraries for surface interpolation via IDW and Kriging. Here is Data for Tutorial 5 with part 5 as detailed below.

• 5 Spatial Statistics and Surface Interpolation

Tutorial 6: Combining Network Analysis with GIS

This tutorial introduces network analysis concepts, and provides an overview of how to combine a specific type of network analysis — community detection — with GIS to visualize community structures in space. Data for Tutorial 6 with part 6 as detailed below.

•  6 Network Analysis and GIS

Tutorial 7: Need more performance?

A brief overview of some tools for getting better performance. For example, tutorial includes an example of how to find the nearest neighbor in one DataFrame for each observation in a second DataFrame without calculating every single distance and taking the minimum.

• 7 Speeding Up GIS

If you want to create interactive maps, ipyleaflet is a fusion of Jupyter notebook and Leaflet. You can control an assortment of customizations like loading basemaps, geojson, and widgets. It also gives a wide range of map types to pick from including choropleth, velocity data, and side-by-side views. Just like ipyleaflet, Folium allows you to leverage leaflet to build interactive web maps. It gives you the power to manipulate your data in Python, then you can visualize it with the leading open-source JavaScript library.