Brendan Ashworth the CTO and co-founder of https://buntinglabs.com/ focuses on integrating AI with QGIS, and today on the podcast we are talking about Autocomplete for vectorization.

Along the way Brendan will share with us why Bunting Labs chose to build this on top of QGIS, the Challenges in Map Digitization, what the development process was like and how this is different from tools like Segment Anything (from Meta)

Here’s what we discussed:

1. Introduction to Bunting Labs: Get to know more about Brendan and Bunting Labs, whose mission revolves around enhancing QGIS with AI, especially focusing on automating vectorization processes.
2. AI Autocomplete for Vectorization: We explored the AI autocomplete feature developed by Bunting Labs that simplifies the vectorization of maps in QGIS, streamlining the digitization process for better efficiency.
3. Brendan’s Background and Motivation: Brendan shared his journey from a software engineer to a pivotal player in the geospatial sector, spurred by a project that showcased the potential of merging geospatial data with machine learning.
4. Why Choose QGIS?: Discover why Bunting Labs opted for QGIS over other GIS platforms, with an emphasis on its open-source nature and vibrant community ecosystem.
5. Challenges in Map Digitization: Our conversation covered the technical challenges involved in developing AI capable of accurately understanding and digitizing maps.
6. Iterative Development and Learning: Brendan highlighted the evolutionary process of their AI model, which has significantly improved from its early versions.
7. AI vs. Segment Anything: Brendan explained how their AI autocomplete tool differs from existing solutions like Segment Anything, particularly in handling specific digitizing challenges.
8. The Future of AI in Geospatial Data Analysis: We discussed potential future applications of AI in geospatial data, including automatic georeferencing and metadata extraction.
9. Privacy Considerations: We also touched on the importance of privacy in the development and deployment of AI technologies in geospatial data analysis.
10. Changing the Geospatial Landscape: Brendan shared his vision for using geospatial data not just to map the current world but to plan and improve future landscapes.

Sponsored by https://www.scribblemaps.com/

Recommended Listening

• https://mapscaping.com/podcast/the-business-of-web-maps/
• https://mapscaping.com/podcast/the-business-of-qgis-development/
• https://mapscaping.com/podcast/qgis-offline-and-in-the-field/
• https://mapscaping.com/podcast/computer-vision-and-geoai/

In Conversation

From Node.js to Geospatial Machine Learning

Daniel: Who are you, and what do you do?

Brendan: I’m Brendan, CTO at Bunting Labs, and I’ve been building an AI autocomplete for QGIS for the past year. I wasn’t originally a GIS or geospatial person — I got my start in open-source software, with the Node.js project, the open-source platform for running JavaScript on servers, and that’s where I initially learned to engineer software and got my spark in computer science. Later I went to MIT and studied physics and computer science, getting more into machine learning — specifically Bayesian inference. Towards the later end of college I was working on a consulting project with a real estate developer, summarizing neighbourhoods by their characteristics with statistics and neural networks, and that’s the moment I realized geospatial data is extremely powerful — I was confused why other people weren’t using it more. That set me on the path, and that’s where Bunting Labs came from.

Why Build on QGIS

Daniel: Why did you decide to build on top of the QGIS platform?

Brendan: My familiarity has always been with open-source software — when I think of software I can contribute to and build an ecosystem around, I think of GitHub, repositories everyone can see, learn from, and contribute to. QGIS is in many ways closer to my comfort zone than other GIS software. It’s extremely easy to get started with — we built the first version of the plugin in about two weeks, and I could do that by learning from all the other open-source plugins available. QGIS isn’t just one app you download to your computer; it’s an ecosystem everyone is contributing software to and enjoying as a result.

What “Autocomplete for Vectorization” Means

Daniel: Explain what autocomplete for digitizing vectors means, and why it was a problem you wanted to solve.

Brendan: Another way to describe it is an autocomplete for tracing maps — an addition to the GIS pen tool. If you’re presented with raster imagery — a satellite image, or a rendered PDF of some construction or architecture asset — and need to convert it into a vector file with a projection and some accuracy, you use the pen tool to digitize it. In many conversations with GIS professionals we found they did this workflow extremely frequently: they’d get a PDF with no projection, georeference it to sub-metre resolution by adding control points until the raster lines up with its location in the real world, then get out the pen tool and digitize lines, polygons, and points representing semantic shapes, and add metadata — the diameter of a buried utility pipe, or when painted highway lines were last painted and when they’ll need repainting. Given my background, we started imagining what that workflow would look like with AI, and this is more or less the product of that.

The Pivot: From Full Vectorization to Autocomplete

Daniel: With your background, I’d be tempted to try converting every line in a rendered image to vectors at once, rather than tracing line by line. Did that cross your mind?

Brendan: It’s funny — my conception of how to solve this is totally different now than it was a year ago. When Michael and I decided to tackle map digitization, I went to the whiteboard and said it’s probably really easy to convert an entire map into its vector representation — there’s an entire academic field dedicated to it, lots of published research — surely a solved problem, I can build this in two weeks. I said that 13 or 14 months ago, so I take that back. The first six months were exactly that approach: I did a literature search, found the best-performing models for converting an image into its vector representation, and to your eye the output looks fantastic — perfectly recreating the image as an SVG. But when we showed it to our original design partners, it missed the mark — a priori it’s difficult to determine which assets in a map are important to have a vector representation of. We realized we needed a semantic understanding of every feature, which is akin to machine reading of maps, and a year ago that seemed impossible. And even then, you don’t trust a model’s first output for important assets — you’d load it into your GIS and review it to your standard. After six or seven months I went on a walk with Michael and concluded I was not on a trajectory of success. So I went back to the drawing board, and only then — with inspiration from generative AI models like ChatGPT — did I consider an AI autocomplete. It was never originally going to be a desktop plugin, but we realized that was the modality people wanted: access to it, and the ability to immediately review its output.

Daniel: They wanted to be in charge — make me more efficient at what I’m already doing, and that way I’ll trust the output more.

Brendan: Exactly.

How It Differs from Segment Anything

Daniel: How is this different from Segment Anything, which segments everything in an image?

Brendan: Segment Anything is extremely powerful, but in a few specific ways. It’s grounded in text descriptions of what you’re segmenting, and the best use case is really satellite and aerial imagery — if you load drone imagery you just took and ask it to segment out the lake boundary, that’s pretty much the best thing you can get. Our AI autocomplete is preferable when you’re digitizing geometries you understand semantically but are difficult to textually explain, or lines that are difficult to semantically extract from the map. We see a lot of very low-quality maps — JPEG artifacts, low resolution, artifacts from the original scan — where Segment Anything struggles. It’s much harder to get it to extract a pressure-sewer line where the line style is a dashed line interrupted intermittently by little “PS” labels — whereas our autocomplete, because it looks at the line you’ve already started, excels at just completing it.

Daniel: So I take the pen tool, start tracing the line, and it jumps ahead of me and follows it?

Brendan: That’s exactly right. It’s a drop-in replacement for the pen tool. Once you activate our plugin and draw two segments of, say, a utility line on a raster map, a small bit of that map is sent to our inference server, and a neural network we’ve specifically trained autocompletes the next 50 vertices of that line. It literally looks at the pixels you’ve already drawn to predict which parts of the map are semantically this same feature. The magic isn’t that it works on certain maps — it’s that it works on your map, generalizing from the hundreds of maps we’ve trained on, even one our AI has never seen.

1,854 Models and Why Lines Are Hard

Daniel: On your website there’s a video tracing a dashed line, and right next to it is an identical line, and another identical line crossing over it — yet the autocomplete knows which one you want. That seems like an incredibly hard problem.

Brendan: I underestimated that difficulty early on. If you follow our journey on social media it might seem as though we built AI in QGIS in a couple of weeks, but that’s a total simplification. I’ve looked at probably a thousand different maps building this, and run nearly 2,000 machine learning experiments. Experiment one did not perform well at all — if you download our plugin today, you’re actually running model 1,854, and the 1,853 models before it were bad in many ways, mostly that they weren’t able to generalize to the map you’re looking at. Lines have so much semantic meaning that lets you disambiguate between two lines when they intersect, and encoding that into the model has been pretty much the most important task I’ve completed.

The Next Problem: Automatic Georeferencing

Daniel: Digitizing lines is one step in the journey. Does any of what you’ve built overlap with the next step — georeferencing?

Brendan: I’ve been working on both models simultaneously. The autocomplete has crested this threshold of usefulness; I also have a model that can georeference maps — as-built documents from architecture and construction firms — automatically, but it hasn’t yet crested the threshold where it actually saves someone time. They’re actually totally different problems: the autocomplete is about understanding the semantic nature of maps, whereas georeferencing is a search problem. When you’re presented with a map and don’t know where it came from, you embark on a search using Google Maps, aerial and satellite imagery, OpenStreetMap. The possibilities are astronomical — comparable to the number of possible chess positions. If a map is just a picture of a building and two cross streets, there are a thousand roads in the United States with those street names. An AI georeferencing model has to do all of that plus more — synthesizing every road in the world, high-resolution aerial imagery to georeference an exact building outline. It’s surprisingly difficult, but I think it’s something we’ll be able to do in hopefully three months. One complexity people forget: things change over time — highway and road names change, buildings appear and disappear, rivers change direction as they flow — so a map is a description of how a place was at a certain point in time.

Metadata, Privacy, and the Future

Daniel: As people use the autocomplete, are they in a way creating labels and improving the model?

Brendan: Not by default. Our plugin doesn’t collect any unnecessary telemetry — we know when you’ve requested an autocomplete, but we don’t track whether you keep the result, delete it, or cut the completion at a particular point, so we don’t automatically add any maps to our training set. We do occasionally have conversations with users who request that we add their map, and our GIS staff hand-digitizes it. It is definitely in the interest of privacy — collecting that data could make a better model, but we’ve found we can collect really high-quality data without going into our users’ data.

Daniel: After you’ve solved autocomplete and georeferencing, how would you approach metadata extraction?

Brendan: Metadata is really interesting because it brings the semantic understanding to a new level — at that point it’s not about a particular map, it’s about how the map fits into your overall project and goal. The exciting applications aren’t extracting a legend, which is easy — it’s metadata hidden in the maps. Continuing the subsurface-utility example: if you’re going to “daylight” a buried utility line — dig carefully around it so you can see it — really important metadata is the depth of that line and the type of soil it’s buried in, things construction crews need to know to delicately dig around a gas line. Or, for geologists working with an older geologic map, matching the exact stylistic look of a polygon to a tiny patch in the legend that says “sediment deposit.” This is where multimodal models can accelerate things — combining the intelligence we generate about a map with the visual and textual understanding an LLM provides.

Daniel: Are you surprised at the lack of AI being used in GIS tools?

Brendan: There’s a big gap between what’s possible in desktop GIS software and what’s already happening — everyone’s talking about it and nobody is implementing it. OpenAI and the larger AI research companies drive most of the innovation, and Meta is a great example, creating Segment Anything and releasing a lot of groundbreaking research publicly. But most of the opportunity in machine learning today is in building domain-specific models and embedding them into professionals’ workflows. An LLM that’s memorized the entire QGIS documentation plus every mention of GIS online won’t immediately accelerate a professional’s workflow — but once you understand that these large models are doing much more than generate text, that they’re generating a semantic understanding of what you’re asking, you can embed that semantic understanding into all programs. It’s not just about autocomplete — it’s about augmentation. And if I weren’t working on this, I’d work on using geospatial data not to map the world’s current state but to imagine what’s possible — finding the optimal place for a wind or solar farm, or the best spot to plant street trees to create shade and improve nearby residents’ lives. Changing what’s actually in the map.