The guest for this episode is Peter Petrik, the CPO of Lutra Consulting. He previously worked in the automotive industry before transitioning to geospatial about eight years ago. Ever since joining Lutra Consulting, he has made use of his programming knowledge and skills, while collaborating with others to develop products and contributions to the open geospatial community. One of their recent developments is Mergin Maps, an open-source software that allows users to take QGIS to the field.
What Is Mergin Maps?
Mergin Maps is an integration that allows users to collect geodata using their mobile devices or tablets. Teams working together can use Mergin Maps to synchronize their data seamlessly to the cloud, or a server. The QGIS plugin enables users to easily access their data in the cloud, and utilize QGIS to post-process or analyse their project.
How Mergin Maps Works
In order to use Mergin Maps to conduct field surveys for a QGIS project, the first step is installing the Mergin Maps Plugin for QGIS. The plugin will ‘package’ your whole project into a folder, and make all of its layers available for use offline. If the project uses an online map, QGIS processing tools can be used to generate its layers for offline use. The plugin also runs several validation checks to ensure that the tool will work correctly on a mobile device. Once everything is set up, a single click on the synchronise button will take the project to the cloud. The project can then be downloaded to a mobile device, and used offline within the Mergin Maps Input app.
Every time a user clicks on the sync button in the app, any new changes are pushed to the cloud, and a new version of the project containing those changes is created in the cloud. It is easy to track the changes made to the project and identify the user that made them.
Mergin Maps also has an automatic merging feature. Users working offline on the same project will have their data automatically merged by Mergin Maps once they sync it to the cloud. This removes the need to do it manually, and can save a lot of time if there are many users in the field. Mergin Maps does not have restrictions on the number of users that can be added to a project.
Data Storage When Using Mergin Maps
Mergin Maps is open source, so users are not restricted to using just its cloud. Users may opt to deploy their own servers to store their data.
The Mergin Maps Cloud is an option for users who do not wish to have the weight of server infrastructure and maintenance on their shoulders. There are various Mergin Maps subscriptions for users depending on their storage needs. For academia and personal use, Mergin Maps is free to use for up to 100 MB of storage.
Mergin Maps has features that help users reduce the size of their data, and cut down their storage needs. For example, there is a QGIS tool where users can set the resolution of all the photos captured via the app, i.e. to 1 MB. This helps to avoid storing high resolution photos when they are not necessary.
Mergin Maps User Management
Mergin Maps allows a project owner to manage the users that can participate in the project. A user can be added to a Mergin Maps project as a reader, writer, or administrator.
Readers can only access the project and view the data, while writers can create new versions of the project, either in the mobile application, or from QGIS. Administrators can invite other users, as well as change user permissions.
Each user will need to create a Mergin Maps account before they can be added to a project. Once they have an account, they can use the same log in credentials to sign into the mobile application and download the project from the cloud onto their device. They can then start working in the project (i.e. digitise, edit data, syncing etc.)
What Expertise Do You Need to Use Mergin Maps?
For the Mergin Maps QGIS plugin, some knowledge of GIS is required to be able to set up a project and push it to the cloud, and later access the data and do analyses and generate reports.
No GIS knowledge is required for users of the Mergin Maps Input app. The developers acknowledge that the users of the app are likely from different fields, with varying levels of knowledge. Therefore, they kept the design simple and easy to use. On average, a new user can learn how to use the Mergin Maps Input app in about 15 minutes!
Mergin Maps is flexible and has plenty of users from various industries and applications including agriculture, utilities and fibre optic companies, consulting firms, and many more.
Mergin Maps Integration with External GPS Tools
If the location from a mobile device is not very accurate, a user can connect to an external GPS. Mergin Maps supports most of the common external GPS tools which a user can plug in or simply connect to. They will then be able to use the external GPS’s precision for their data collection. Only Android is supported for this functionality at this time.
Controlling Data Inputs on Mergin Maps
Mergin Maps offers a number of widgets that can be used to control the values accepted by the Mergin Maps Input app.
For instance, if an input field is set to only accept integer values, an integer widget will appear on the data collector’s mobile device as a slider, or text box where they can only input a number.
Further limits can be set to define whether the number must be positive, or may not exceed a certain figure. For dates, a widget will pop up on the mobile device as a calendar. These functionalities are great for helping to manage data quality and data assurance standards.
There are also widgets for lists, checkboxes, and other common widgets and constraints. Users can also use the QGIS Expression Engine to write calculations for derived fields.
There are lots of possibilities supported on both the QGIS plugin, and Mergin Maps Input app, where users can apply some basic quality assurance methodologies, and help with capturing quality data in the desired format. After data has been pushed to the cloud, users can make use of the Python integration to write scripts in QGIS, or on their server to further validate the data from an input field.
Mergin Maps Community Edition
Mergin Maps are advocates for open-source GIS and that reflects in their products. Their users are not vendor locked within Mergin Maps infrastructure. If a user is tech-savvy enough, they can set up Mergin Maps, and use it on their own. There is a public GitHub repository with a Docker compose file that users can run for themselves for their personal projects. In case they encounter problems, they can join the Slack chat and seek help.
Mergin Maps Enterprise Edition
The Enterprise edition is best suited for companies that need to hold data in their servers due to legal or security reasons, or internal rules. Mergin Maps facilitates this by offering their knowledge from their years of experience running cloud services, to help in the deployment and maintenance of each company’s server, making sure everything works smoothly and efficiently.
Recommended Podcast Episodes
In Conversation
From the Automotive Industry to Open Source GIS
Daniel: Peter, welcome to the podcast. You’re with Lutra Consulting, and today I want to talk about Mergin Maps. Would you introduce yourself and tell us how you got involved in geospatial?
Peter: Thank you for having me. I’m Peter Petrik, and I got into geospatial about eight years ago at Lutra — it’s a lot of fun to do open source and GIS together. Before that I was in the automotive industry, developing numerical solvers for combustion systems. It sounds a long way from maps, but I studied physics and numerical methods, and I worked in QGIS on the mesh data layer, where you use those simulations for modelling things like floods or meteorological data. The physics is the same everywhere — it just depends where you use it. I develop mostly in C++, Python and C.
What Mergin Maps Is
Daniel: What is Mergin Maps?
Peter: Mergin Maps is a way to collect field data with your mobile or tablet easily and collaboratively with your team, and then synchronize it seamlessly to the cloud or your server, and from there to QGIS, where you can use all the QGIS features to post-process or analyze the project. It starts with a QGIS project. If you already have a QGIS project you want to use in a field survey, you install the Mergin Maps plugin for QGIS and use the packaging system — it packages everything to a folder so all your layers can be used offline, and does a bunch of validation so you can be sure it’ll work correctly on your phone. Then you click synchronize, it goes to the cloud, and on your mobile you install the Mergin Maps Input app and synchronize back.
Daniel: What about online layers, like a base map from OpenStreetMap?
Peter: The mobile application uses the same rendering engine, the QGIS core library, as your desktop QGIS — so it’s really the same QGIS running on your mobile with a slightly different user interface. If you have an online map in your project, you can use it as-is on the mobile, but then you need to be online. Or you can use QGIS processing tools to generate an offline map from your online map, and then it can be used offline without an internet connection. You can use map themes to switch between online and offline — it’s up to you.
Versioning and Automatic Merging
Daniel: When you click sync, what actually happens, and what makes a new version?
Peter: The packaging is really a series of validations — it makes sure your survey layer (we prefer GeoPackage) is in the same folder and flags settings that won’t work on mobile. When you click sync, it takes the whole folder and pushes it to the server — either your own server, since we’re all open source and you can deploy the server Docker container yourself, or our cloud. The technical background is similar to git. Every time you change something — add a few points, modify attributes — and press sync, it makes a difference between the original file and your modified file, and that difference is sent to the server. One version is a set of changes sent in one batch. It also does automatic merging: if you have 10 users in the field, all offline, and they all synchronize in the evening, it automatically merges the data from all 10 sources. You can make an audit log to see who did what change, when. You can even think of it as a versioning system if you don’t want to set up PostGIS — just use Mergin Maps to version your QGIS project or GeoPackages.
Storage and the Input App
Daniel: Is there a size limit to projects in the cloud?
Peter: My project is a few gigabytes and a few thousand versions, and it works nicely. The only limit, if you use our cloud, is the storage in your subscription plan — and that storage is for the latest version, not the whole history. We also have tools to reduce storage size — for example, a tool where in QGIS you set that you don’t need full-resolution photos, so the mobile app automatically reduces photos to, say, 1 megabyte. Or you can run a Python Docker container that moves photos from the cloud to third-party storage like an Amazon S3 bucket and patches your QGIS project with a link. The Input app runs mainly on Android and iOS — phones and tablets — and we’re developing Windows and desktop versions. It has no strict hardware requirements and is optimized for small mobiles; the main limitation is having enough storage on your phone for the project.
User Management and Field Volunteers
Daniel: If I have 15 freelancers doing field work, do I just send them my login?
Peter: I would not advise that, because then you won’t see who did the change — it’s the same login. There’s no restriction on the number of users you can share a project with, so the best way is to ask them to create Mergin Maps accounts, then add them to your project as writers, readers, or administrators. Readers can view the data, writers can create new versions from QGIS or mobile, administrators can invite others and change permissions. We have many users doing it this way — for example, a GIS consultant maintains a project and has 20 or 40 volunteers who just install the app and go around the forest collecting data about trees or animals. There’s a case study from a wildlife trust in the UK doing exactly that, and a citizen project in the Netherlands where people fill in forms for archaeological discoveries. The Input app is a simplified version of QGIS on your mobile — we keep it very intuitive, so you can learn it in about 15 minutes without any training.
Controlling Data Quality
Daniel: Does my mobile device’s GPS limit accuracy, and can the app use an external GPS?
Peter: Yes, we support most external GPS units — you plug it in, connect to it, and it uses the external GPS precision. For many applications, though, the few-meters precision of the phone under clear sky is fine. In QGIS you set up the form for what’s editable and what values are accepted — for an integer value you use an integer widget, which appears as a slider or number field on the phone; for dates a calendar pops up. You can set constraints — a number must be positive, or not bigger than 100 — and use lists, checkboxes, and the QGIS expression engine for derived fields. You can use tabs so that additional info only shows if a checkbox is checked. It brings back quality data in the format you want, with basic quality assurance already, and afterwards you can use Python integration to write scripts that further validate the data.
Who Uses It, and What’s Next
Daniel: Who is using this commercially?
Peter: Plenty of users from various industries — electricity utilities, fibre optic companies, GIS consultants doing field surveys, agriculture. We have a case study from South Africa where a farmer records the weight of pecan nuts for each tree and soil composition. For a fibre optic company, they use the same QGIS project for high-level design, detailed design verification, recording deviations during implementation, and then asset management — it’s as flexible as QGIS itself. In the long term we want to integrate web map overviews into the Mergin Maps cloud, so when you get a new version you can instantly see a web map, the same graphics as desktop, and share it with your client to see progress in real time — that should arrive late this year or early next. We have a community Slack channel where we discuss new features with our users.
Daniel: Can I run all this myself?
Peter: Yes — there’s a public GitHub repository with a Docker compose file, so you can run it yourself for personal projects. For a production-ready company deployment you need some server and Docker knowledge. We’re very much in the open source GIS world — no vendor locking — so there’s always a Community Edition you can set up on your own. We also offer an Enterprise Edition for big companies that need data on their own servers for legal or security reasons, which comes with help with deployment and maintenance. To learn more, start at merginmaps.com, or join the community Slack channel — there are over 400 users there already.
