Sentinel Hub – The Cloud API for Analysis Ready Satellite Data

Gregor Milcinski is the CEO and co-founder of Sentinel Hub and has worked in the geospatial field for about 20 years. He shares his knowledge of the Earth observation industry and gives us an in-depth explanation of what it is that the Sentinel Hub does.

Sentinel Hub is a cloud API for satellite imagery. It uses its APIs to enhance access to satellite data from missions such as Sentinel, Landsat, and other commercial satellite projects. Sentinel Hub users make use of its APIs to process the data they are interested in, and access it in a format that best provides the information they need.

This helps take off the operational load in terms of time and effort required to process the large, ever-changing volumes of satellite data. Sentinel Hub enables users to obtain Analysis Ready Data (ARD).

This means having data in a form that is ready for a particular workflow process and does not require additional cleaning. Sentinel Hub estimates that the capabilities they provide cover about 80% of what users may want to do as far as data processing is concerned.

Among others, these include Ortho-rectification, data transformations, rescaling, re-projection, as well as applying some machine learning models. Using a simple API request, users can get immediate access to full and global archives of the relevant satellite missions partnered in Sentinel Hub.

A Peek into How Sentinel Hub was Built

The idea that eventually transformed into Sentinel Hub was born from operational difficulties in a previous project. This triggered the realization that the technology which was already available for working with satellite data was not overly suitable for processing the ever-expanding, continually updating, large volumes of data collected by satellites. A prototype was developed and subsequently iterated to evolve into the now Sentinel Hub.

This journey of growth is an inspiration to those who may want to build something, but are unsure on how they should start. Do not get stuck in the idea that you have to start big straightaway. Start where you are confident, and once you figure out what is working, keep building on it.

How Sentinel Hub Works

Sentinel Hub works ‘on the fly’. It does not store pre-processed data that can be ordered off-the-shelf. Rather, the processing happens on the fly as requested by the user. This approach was chosen since it is quite impossible to accurately predict what a users’ needs will be in terms of geographical, temporal, and spectral aspects.

This is also one of the reasons why Sentinel Hub does not cache processed results (apart from privacy policies). Considering these aspects, there is a high risk of storing petabytes of data that may never be needed. Instead, Sentinel Hub worked to optimise their processing steps as much as possible for speed.

Currently, it only takes a couple of seconds for the most usual requests (like above) to process. This is typically fast enough that users can integrate these APIs directly in their applications in an interactive manner, without even having to store the data on their side.

Keeping operations to “on-the-fly” also allows Sentinel Hub to expose their new capabilities and improvements to their users by simply deploying a new version.

If data was pre-processed, there would be a fixed dataset which would need processing in order to introduce the new improvements. This could be expensive, time-consuming, and inconvenient.

How is Sentinel Hub Different?

The description of what Sentinel Hub does may have you thinking that it is the same as Google Earth Engine, or Microsoft’s Planetary Computer. Ideally, it actually falls somewhere in between the two. See the sections below to see how they compare.

Sentinel Hub vs Google Earth Engine

Google Earth Engine was designed as a search platform where people could do analyses, and then make use of the results. It was not primarily designed to power other applications.

Even though there are workarounds regarding this, Google Earth Engine was mainly designed for people to work with the results of its analyses. On the flip side, Sentinel Hub is designed to power other applications.

It offers a set of APIs which users can easily integrate into their procedures and workflows. It removes limitations on what users can do, and where they can do it. Building your workflow on top of Sentinel Hub, you can do just about everything, since it is possible to fine-tune what you’d like to get from Sentinel Hub APIs.

Sentinel Hub vs Microsoft’s Planetary Computer

Microsoft’s Planetary Computer can be viewed as a platform that is mainly for making geospatial data available in a cloud-native way. It does not abstract the complexity of satellite data, which makes it difficult to perform tasks like stitching scenes. Microsoft’s Planetary Computer provides APIs to access the metadata of the satellite data for users’ workflows being run on virtual machines in Azure. This in itself is very limiting for users who may want to run it on a different cloud. On the other hand, Sentinel Hub allows users to be more flexible as they can use the program in their own environments.

Businesses that have their own infrastructures and proprietary data that they do not want to use outside their environments can simply use Sentinel Hub as a data stream in their procedures from wherever they are.

Who Are the Users of Sentinel Hub?

The vast majority of Sentinel Hub users are application developers and data scientists. The application developers are mostly those working in the agricultural sector.

One reason for this is that Sentinel-2 provides a powerful collection of data with a resolution that is most useful for agriculture and vegetation monitoring. Typically, many of the Precision Agriculture applications are powered by Sentinel Hub.

The use of Sentinel Hub by data scientists leans towards agriculture primarily for monitoring activities and climate change analyses. Data scientists are the largest consumers of Sentinel Hub since they perform large scale operations using machine learning. Despite making up only 5% of the paid users of Sentinel Hub, data scientists consume about 80% of the volume of data.

Apart from the agricultural sector, Sentinel Hub also serves the energy, defence, environment, and mining sectors.

Partnering with Sentinel Hub

Satellite companies can partner with Sentinel Hub through its “Bring Your Own Data” service. This service allows satellite companies to plug in their data to Sentinel Hub in any cloud native format supported by Sentinel Hub (i.e. GeoTIFF, XAR), and basically any kind of raster data. Sentinel Hub does not copy this data, but merely stores the metadata in their database for faster access, like a catalog.

Users can then make use of the same set of features offered by Sentinel Hub to distribute data efficiently. The satellite providers are responsible for charging users for using their data as Sentinel Hub does not support this functionality at the moment.

Sentinel Hub’s EO browser

The EO browser is a Google Maps-type application that runs in a browser. It provides access to the satellite missions supported by Sentinel Hub, and is free for non-commercial use. In the EO browser, you can check for the most recent data in any part of the world and visualize it interactively (i.e. zooming in and out, viewing different band combinations, doing time lapses, switching between satellites), and much more. No account is needed to use these capabilities in the browser.

The EO browser showcases what Sentinel Hub’s APIs can do when integrated into workflows. It is a perfect display of how satellite data can be used. If you want to get familiar with or know more about satellite data, the EO browser is a useful resource for that.

An Overview of the Earth Observation (EO) Industry

The usefulness of satellite imagery has grown over the years as experts explore more applications where the use of satellite data would improve efficiency, and insights. Coupled with developments in IT and data capture technologies, the Earth Observation industry has exposed new opportunities but also faces some setbacks.

Opportunities in the EO industry

Spatial resolution in satellite imagery has improved in recent years and has unlocked new opportunities for using satellite data. Some satellite missions, such as Sentinel-2, have launched a revolution by providing very useful, good quality, free data set that can be used as the basis for a number of geospatial workflows.

Machine learning is also worth mentioning due to its contribution to the EO industry. It has revolutionized the interaction with the normally large volumes of satellite data, and processing it. There is still much to be done in terms of developing machine learning procedures for highly dynamic satellite data, and the wide variety of more niche uses that consumers crave.

Setbacks in the EO Industry

The complexity in the nature of satellite data, coupled with the unique traits of geospatial applications introduces difficulties in using machine learning to fully realize opportunities that emerge in the EO industry. This uniqueness hinders the development of off-the-shelf machine learning algorithms. There are a very limited set of applications for which machine learning procedures can be offered as a service (i.e. ship or building detection). In most other cases, there would still be a need to tailor the procedure to a specific workflow or niche.

In an effort to contribute to the development of machine learning in geospatial applications, Sentinel Hub have open-sourced all their internally developed machine learning procedures, and hosted them on GitHub. They allow other people to refine these procedures and use them as a starting point for their own machine learning projects.

Making the decision on how to balance temporal and spatial resolutions during data capture is also a defining line on whether the missions will generate sufficient value.

For instance, applications in the construction industry may require high spatial resolution (i.e. 1 meter) as well as a high temporal resolution (i.e. 1 day) to monitor construction progress. For applications like vegetation monitoring where there will be no significant change within a day, using these exact resolutions will be overkill.

It is highly likely that the value generated may not be sufficient to compensate for the cost used in processing.

Another challenge is the fact that satellite data is only periodically produced. It is unlikely that a user can find what they want in their area of interest at the particular time that they are interested in. A satellite can only be tasked for the future, not the past.

Advancing the Earth Observation Industry

Sharing knowledge has always been one of the pillars for new developments that can take any industry to the next level.

In the geospatial field, more and more companies are cultivating the culture of sharing knowledge. Sentinel Hub is already making strides in this by sharing a lot of what they do (i.e. openly sharing their machine learning procedures, which they could easily choose to make proprietary).

Sentinel Hub explained that they are doing this because they believe the Earth Observation field is in a phase where contributing to its growth is more important than growing one’s position within it.

The Future of the Earth Observation Industry

Growth is expected in the Earth Observation industry for the foreseeable future. With the introduction of the monitoring approach in almost all elements of our lives, satellite data will be needed since it is a rich, objective, and useful primary source.

The industry will grow since this data will be needed in a form that makes it easier to extract information, and be processed it in a smart way. Sentinel Hub acknowledges that they are only doing a tiny part in the industry. It will need effort from the geospatial community to grow all the tiny parts and realize growth across the entire industry.

Recommended Podcast Episodes

Sentinal Hub – Frequently Asked Questions

In Conversation

The Origins of Sentinel Hub

Daniel: Gregor, welcome to the podcast. You’re the CEO and co-founder of Sentinel Hub. Before we dive in, can you introduce yourself and describe how you got involved in Earth observation?

Gregor: I’ve been working in this field for about 20 years already. We still operate a company that has worked on geospatial data quite a bit, and about five years ago we tried to make use of the Sentinel data, which had just been announced. It looked really cool in one of our projects in Africa, and we realized that the technology available at the time to work with this data was simply not suitable for processing the large volumes of always-up-to-date data. In every IT company there are a few people who believe they can do it much better than anyone else has done in the past — that’s how Sentinel Hub appeared.

Daniel: Was that a big decision, or was it just obvious that the platform needed to be built?

Gregor: It was not a big decision, and it didn’t start as a very large project. It started as a thing that three people were working on part-time for a few months. When we had a working prototype, we felt it might actually be a good business. Previously our business model was custom-based applications for large governments — we didn’t have a product. Sentinel Hub was the first time we said this actually might be a product. We started to invest more and more time, and now more than two-thirds of the company works on things related to Sentinel Hub. It evolved naturally — we didn’t kind of say “let’s do something very big.” We just started small and iterated.

The Sentinel Hub Philosophy: On-the-Fly, No Cache

Daniel: How would you describe Sentinel Hub today?

Gregor: We see Sentinel Hub as the AWS of Earth observation. AWS always said they do the heavy lifting on top of the IT — we do the same with EO data. We have about 50 petabytes of data: Sentinel, Landsat, MODIS, commercial missions. All the complexities of multi-temporal and multi-spectral imagery are not easy to tackle, even with open source tools. We do this for our users so they don’t have to, and we expose the capabilities through a set of simple APIs. People can get analysis-ready data — and what’s analysis-ready depends on what you’re working on. Our APIs let users configure what’s ready for them: orthorectification of Sentinel-1, upscaling, reprojection. With one simple API request, you get immediate access to full global archives of just about every relevant satellite mission.

Daniel: Are you pre-processing all this data and storing different formats, or is that happening on the fly?

Gregor: It’s a question we asked ourselves five years ago. The answer was very quickly no, because of the sheer size of the data. Hosting is typically paid by someone else — the European Union pays for DIAS, Amazon offers it, Google Cloud offers it. As soon as we reprocess it, we’d suddenly have to store it ourselves. We’re a small company — 75 people, bootstrapped, no venture capital — we couldn’t afford millions for storage. Instead we invested heavily in optimising the processing so we can do everything on the fly. A request takes a couple of hundred milliseconds to a couple of seconds. If you ask for Sentinel-1 data, we fetch the data from one bucket, the digital elevation model from another, do the orthorectification, radiometric terrain correction, speckle filtering and backscatter coefficient — about half a second. That’s fast enough that our users integrate it directly in their applications interactively, without storing the data on their side.

Daniel: Do you cache results in case the user comes back?

Gregor: We had cache at the beginning. But when you’re working with a mission like Sentinel-2 that has new global data every few days, it’s very rare that people want exactly the same data at exactly the same location, time, and band combination. The cache was adding overhead but was rarely used, so we dropped it and put effort into making processes a tiny bit faster. We have no cache whatsoever — everything is on the fly. There’s also a benefit: when we deploy a new version, all new capabilities are available immediately without reprocessing everything. If you pre-process, every improvement means reprocessing a huge fixed dataset, which is extremely expensive.

Sentinel Hub vs Google Earth Engine vs Planetary Computer

Daniel: How is this different from Google Earth Engine or Microsoft’s Planetary Computer?

Gregor: Google Earth Engine and Microsoft Planetary Computer aren’t the same thing. Earth Engine was designed as a platform where people go and run analyses and use the results — it wasn’t primarily designed to power other applications. There are workarounds, but generally you run analyses and work with the results. Sentinel Hub was always built to power other applications. We don’t want to be the one-stop shop. We offer a set of APIs and people integrate them into their own procedures. That’s a bit more complicated than running things in a browser, but it removes limitations. With Earth Engine, if you can’t do something, you can’t do it. With Sentinel Hub and your own workflow on top, you can do just about everything because you fine-tune what you get from us. Earth Engine is also free for non-commercial use and Sentinel Hub is paid — but Earth Engine doesn’t have clear commercial pricing, and our pricing is much cleaner.

Gregor: The Planetary Computer feels more like “we’ll make the data available in a cloud-native way, provide some APIs for the metadata, and everything else is your own process running in your own VM.” It can run on Azure, but if you want to run it from Google Cloud — bad luck. Planetary Computer is closer to Amazon’s open data buckets with a few more APIs. Sentinel Hub APIs you can run from anywhere, and people appreciate that — they have their own infrastructure and proprietary data they don’t want to share, so they just use Sentinel Hub as a data stream in their procedures from wherever they are.

Daniel: So at one end you have the Planetary Computer, at the other Google Earth Engine, and Sentinel Hub is in the middle.

Gregor: I would agree with that.

Who Uses Sentinel Hub

Daniel: Who is building applications on top of Sentinel Hub?

Gregor: The vast majority of our users are application developers or data scientists, with a few end users who integrate the APIs into QGIS or ArcGIS. Application developers are mostly in agriculture, because Sentinel-2 is a powerful collection — the resolution is most useful for agriculture and vegetation monitoring. Many precision agriculture applications are powered by Sentinel Hub. You wouldn’t know it because we’re in the back end. Data scientists also lean heavily towards agriculture, monitoring at large scale: crop type, land cover, agriculture activities. We have about 1,500 paying users — roughly 95% are application developers, but the 5% who are data scientists consume 80% of the data volume, because machine learning consumes orders of magnitude more than just providing data to end users.

Gregor: Beyond agriculture, we have mapping — TomTom is an example, they processed the whole world for their medium resolution layer. Class in 365 in agriculture. Maxar integrates our data into SecureWatch. Oil and gas, raw minerals, environment monitoring, and defence (the smallest sector for us). Geographically the user base is global — Asia is the least represented, probably because of firewalls making it harder to use APIs from outside. Europe is currently heavy in consumption because of area monitoring activities for the Common Agricultural Policy, which is a huge driver of machine learning-based services on top of satellite data.

Bring Your Own Data

Daniel: If I had my own satellite constellation collecting large amounts of data, could I plug it into Sentinel Hub?

Gregor: Technically things are already there — we have a service called Bring Your Own Data. You get your data into a cloud-native format (we support Cloud Optimised GeoTIFF and Zarr), put it in a bucket close to one of our deployments, and trigger an API to tell Sentinel Hub where the data is. We collect the metadata and store it in our database to be able to process it fast — we don’t copy the data, just like we don’t copy Sentinel data. Then you can make use of the same set of services and features we have. What we don’t yet support is automatically charging other users for accessing your data — as a satellite provider, you’d have to take care of that yourself. It’s an important question because there are companies launching hundreds of satellites and not many are thinking about distributing the data efficiently.

Daniel: Could it be any raster data, not just satellite imagery?

Gregor: Yes — any raster data. We host derived products too, like ESA WorldCover, which is super useful for doing machine learning on top, and Copernicus services with phenology indices.

The Reality of Machine Learning in Earth Observation

Daniel: Do you consider creating object-detection algorithms and offering them as a service, so I get an answer instead of imagery?

Gregor: Surprisingly enough, machine learning in this field is quite immature. There are lots of questions, but most don’t have an operational procedure to provide answers. There are examples, prototypes, papers with nice images — but if you say “run this over that area,” most procedures don’t work. One reason is that the situation changes from field to field, use case to use case. Take change detection — everyone wants to do change detection. But what is change? Is a cornfield maturing a relevant change? We open-sourced our internally developed ML procedures on GitHub under MIT license so people could start from there and refine for their needs. Some, like our parcel delineation library, work well just about anywhere. But to generalise as a service — you can do that for a very limited set of things: airplanes, ships, maybe buildings or roads. In practice, the valuable problems require thinking on top of these basic tools.

Daniel: Machine learning is sometimes thrown around as a blanket statement that glosses over how hard the work really is.

Gregor: Couldn’t agree more. You can take the data and labels, throw it into TensorFlow, and get results that look good — but when the details matter you realise you’ve done maybe 20% of the work, and the hard part is still ahead. We use machine learning a lot, but typically the simplest things possible — we go into deep learning rarely because often random forest does the job. The work around the model is usually way more difficult than the model itself.

Fuzzy Pixels: Resolution, Cadence, and the EU Monitoring Story

Daniel: Earlier you mentioned that one of the big reasons we don’t see more successful EO businesses is “fuzzy pixels.” Can you unpack that?

Gregor: People often have a very wrong understanding of what satellite data really is. They expect Google Maps-like imagery on an ongoing basis. We get questions like “can you get me a 1-meter resolution image from two months ago between 9 and 9:30 a.m.?” — very unrealistic. Systematically available data is really just Sentinel, Landsat, and PlanetScope. Everything higher resolution is tasked — you have to be lucky to find something in your area and time window. With Sentinel-2 at 10 meters, identifying a single building is difficult even though buildings are larger than 10 meters. People aren’t yet experienced in using the multi-temporal and multi-spectral value of ongoing missions. That’s slowly changing — Sentinel-2 made a revolution by providing useful, free, good-quality data so people can build tools on top.

Daniel: Could you walk us through the European Common Agricultural Policy use case?

Gregor: About half the European budget goes to farmers to better manage their lands. That used to be controlled by on-the-spot field checks, but with systematic satellite acquisition over the whole of Europe, you can now monitor some practices remotely — was the field mown, what crop type is there, was a permanent grassland converted to arable land? Many of these rules can be monitored using Sentinel data, Landsat, or sometimes PlanetScope for systematic monitoring. They process tens of millions of agricultural parcels in Europe, check for these activities, and if something is off, farmers may have to return subsidies. We’ve thought about offering these as standard products, but mostly the answer is no — we focus on what we know best, which is data processing. The exception is agriculture, where we have 20 years of experience and have built an Area Monitoring Suite on top of Sentinel Hub.

Daniel: If I could snap my fingers and increase Sentinel-2’s resolution by an order of magnitude, would the value increase by the same amount?

Gregor: Short answer: yes. Most challenges require seeing an object, and 10 meters isn’t enough unless your object is a field or a forest. Half-meter to 1-meter resolution would change things drastically. But for systematic monitoring, the cadence matters as much. Weekly is probably the minimum useful frequency. You don’t need every 15 minutes — startups are building constellations for that, but I have a hard time understanding why you’d need more than daily at high resolution at large scale. The cost of processing tens of petabytes of data daily to find 100 relevant changes wouldn’t generate enough value to compensate.

EO Browser and the Future of Earth Observation

Daniel: Can you describe the EO Browser?

Gregor: EO Browser is a Google Maps-like application that runs in a browser and provides access to just about all the missions we support — Sentinel, Landsat, MODIS, Copernicus services, even commercial data. It’s free for non-commercial use. You go to your favourite place, check the most recent data, zoom in and out, do different band combinations, go back decades on Landsat, do time lapses, indices, water body detection — all in a browser, no account needed. We built it as a showcase for what Sentinel Hub can do, and to raise awareness of remote sensing data. It now has 140,000 monthly visitors and processes 50 million requests a month. The unfortunate side effect is that some people think Sentinel Hub IS the visualisation — but Sentinel Hub is the back end, and EO Browser is just one front end on top of the APIs.

Daniel: Where do you see growth coming from over the next five years?

Gregor: One part is our current users becoming end users themselves and consuming more data. But the real change will be introducing a monitoring approach to just about all elements of our lives — proactively processing vast amounts of data in a smart, cost-efficient way to identify places where we need a deeper look. Deforestation, reforestation, carbon sequestration claims that nobody has been verifying — all of these need satellite data because it’s objective, systematic, and rich. We think this is probably 1,000 times more than what we see today, and I’d guess in two years’ time we’ll have a completely different world in terms of how much machine learning is used daily on top of satellite data, often without people even knowing. The example I like: I’d like to walk into a fast-fashion shop, look at a t-shirt, and understand the impact my purchase has on the rivers in Indonesia where the fabric was made. All of that is technically possible with satellite data. We’re doing a tiny piece, but once we crack it as a community, it’ll be used at a scale we can’t even imagine right now.

Daniel: Gregor, this has been a great conversation. Where can people go to learn more about Sentinel Hub?

Gregor: Start with EO Browser to see what the data can do, then our Medium blog at medium.com/sentinel-hub for in-depth knowledge, our Twitter account @sentinel_hub, and the website sentinel-hub.com.