The main topics discussed during this episode include:
- Basics of LiDAR data and its applications.
- Differences between LiDAR and photogrammetry.
- Processing chain of LiDAR data.
- Challenges in classifying point clouds.
- Applications of LiDAR technology in vegetation mapping, terrain modelling, and infrastructure inspection.
- The future of LiDAR technology, including the potential for more affordable and widespread use.
- Importance of automated processing tools for handling large volumes of data.
Connect with Nejc Dougan here: https://www.linkedin.com/in/nejcdougan/ or at https://www.flai.ai/
Recommended listening
- Cloud Optimized Point Clouds
- PDAL – The Point Data Abstraction Library
- LiDAR From Drones
- Bathymetric LiDAR
In Conversation
From Aerial Data Acquisition to FLAI
Daniel: Nejc, welcome to the podcast. Could you introduce yourself and tell us how you got involved in LiDAR processing?
Nejc: I’m CTO and founder at FLAI. We’re a Slovenia-based startup that spun out from Flycom Technologies, an aerial data acquisition company that does a lot of remote sensing and LiDAR scanning jobs in the region. We started by optimizing their internal processing flows for LiDAR, and in doing so we decided to spin out and create our own company focused solely on classification and processing of point clouds using AI.
Daniel: I’ve been saying LiDAR — but you just said point clouds. Do you also process point clouds from photogrammetry?
Nejc: Our technology is able to process any kind of point cloud data, from photogrammetry to bathymetry and other sources, although we focus mainly on LiDAR point clouds. They’re quite similar, but there are some differences between the sources.
What LiDAR Data Is
Daniel: Let’s start at the bottom — what is LiDAR data?
Nejc: The main difference between LiDAR and photogrammetry is that LiDAR is an active sensor — it emits its own source of light and captures how many photons bounce back. This has important implications. We get penetration through the canopy, because the light emitted has a footprint of a few centimeters that gets scattered and bounces back from ground points as well, which makes it a really good sensor for mapping vegetation or terrain. And because it’s an active sensor, we always get the intensity with each point — a measure of how many photons bounced back from each echo.
Daniel: Should I think of LiDAR as a cone of light, with a footprint that gets bigger the further it is from the object?
Nejc: That’s correct. Although we imagine lasers as narrow and focused, there’s still some divergence in each beam, resulting in a larger footprint the farther away from the sensor we go. Scanning from around a thousand meters above ground level, depending on the sensor, we can get footprints of roughly 10 to 15 centimeters.
Wavelengths and Sensor Platforms
Daniel: Could you say something about the different wavelengths used in LiDAR?
Nejc: Most commonly, a near-infrared wavelength is used for topographic mapping. But there are also a lot of applications in bathymetric LiDAR — LiDAR that penetrates the water surface. Water absorbs quite a lot of the red wavelength, so red beams aren’t suitable for bathymetric LiDAR; green laser beams are used instead. In terms of XYZ coordinates the point cloud looks the same, but the intensity value changes because different wavelengths are absorbed differently by different surfaces — and that has to be taken into account if your classification algorithm relies heavily on intensity.
Daniel: Is there a difference in processing data from different platforms — spaceborne, airborne, drone, handheld?
Nejc: The GEDI mission was set up to do vegetation and biomass mapping across the globe, but a laser in space has a footprint in the tens of meters, resulting in quite a different point cloud — that data wouldn’t be directly suitable for our algorithms. Coming closer to Earth, from airborne to handheld sensors, there are differences in the point cloud itself, but in terms of processing we handle all those data sets. What usually differs is the use case — a large plane does nationwide topographic mapping, while a handheld sensor moving through a forest is used to classify trees or furniture.
The LiDAR Processing Chain
Daniel: What does the general processing chain look like?
Nejc: A LiDAR project starts with a client specification — they want a certain area captured at a certain point density. The acquisition company plans the flight mission, captures the laser scanning along with GNSS positioning and an inertial measurement unit. Then the first step is calculating the trajectory of the sensor, generating the point cloud from raw measurements, and georeferencing it. Usually you can’t cover the whole area with a single strip, so multiple overlapping strips are flown, and because sensor calibration is never perfect, the next step is matching those flight strips. Then the fun part begins — filtering, classification, and extraction of meaningful information — labeling points into categories like ground, vegetation, building, and noise. The last step is generating final products like digital terrain models and building footprints.
Classifying Point Clouds with AI
Daniel: When you label each point, do you also group them into objects?
Nejc: In the geospatial community we usually call this “classification,” but from a computer vision perspective there are two problems: semantic segmentation, where we associate a semantic label with each point, and instance segmentation, where we say all these points belong to an individual instance of a car or tree. Doing both in parallel is called panoptic segmentation, and it turns out doing both simultaneously can improve the quality of both results.
Daniel: Is there as mature a library of code for point clouds as there is for computer vision on images?
Nejc: There has been more research effort in the imagery domain, so those libraries are more mature, and there are far more training data sets available. A significant difference is that imagery data is structured — organized in a grid, so neighboring pixels are neighboring locations in space. Point clouds are unordered sets of points; records stored one behind the other don’t guarantee a spatial relationship. So we always have to calculate the spatial relationships, which makes those algorithms more computationally expensive. The Cloud Optimized Point Cloud format actually stores spatial indexing in the file itself, which helps, but either way you have to calculate those indexes at some point.
Pre-Trained Models and Use Cases
Daniel: Can you talk me through the point cloud classifiers you’ve developed?
Nejc: We offer three pre-trained models — for the geospatial domain, the mobile mapping domain, and the forestry domain. In aggregate they currently classify around 44 different categories, and for each real use case we fine-tune or select the right combination. If a client has their own training data, we can also retrain our models for a specific use case. The rule of thumb is: if a human annotator can see and distinguish an object in the point cloud, we can train a classifier to do it too. If you can’t visually see it, it’s practically impossible to create a training data set for it.
Daniel: Who are the big users of LiDAR data, and what are they doing with it?
Nejc: The main use case is large nationwide topographic mapping — national mapping agencies ordering scans, mainly to generate digital terrain models. The second is power line mapping: both inspecting the infrastructure itself — mapping each conductor to check its state — and vegetation management, which is getting more important because of wildfires caused by sparks jumping from power lines to nearby vegetation. Then there’s forestry, and the whole virtual reality and metaverse space, where you want semantically labeled point clouds to generate digital twins of the built environment.
Working with Cloud-Optimized Point Clouds
Daniel: With cloud-optimized formats, do you need direct access to the data, or can you stream it into your processing chain?
Nejc: We can read or stream data stored in blob storage in Cloud Optimized Point Cloud format. One consideration is that for large projects, data sizes get high, and cloud ingress and egress fees just for moving data around can get quite expensive. So if your data is already in a particular cloud provider and region, it makes sense to move the compute to that region rather than moving the data in and out — that reduces both cost and processing time.
The Future of LiDAR
Daniel: When you think about the future of LiDAR, what do you see?
Nejc: Two big trends are playing out. First, sensors are getting more affordable, resulting in more data being captured — a few years ago acquisition was expensive and only large missions were carried out, but now lightweight sensors can be mounted on UAVs, and more handheld devices capture LiDAR. Second is the automation of point cloud processing — AI and machine learning tools that automatically classify large volumes of data. Together those two drivers push the affordability of point cloud projects down and make space for new use cases.
Nejc: What would be really great is a systematic source of LiDAR point clouds. Currently different countries plan their data captures every several years. But in the Earth observation community you get Sentinel-2 data every five days — a systematic, continuous source. We don’t have that yet for LiDAR, but it would open up a whole new potential for time series of point cloud data and new use cases. That’s a large opportunity for someone to fill.
