Where does one field end and the next one begin? It sounds trivial right up until you try to answer it.
In this episode, I’m joined by Hannah Kerner — Assistant Professor at Arizona State University, AI Lead for NASA Harvest and NASA Acres, and Research Advisor for Taylor Geospatial — to talk about Fields of the World.
Thank you to the people making this episode possible: the Cloud Native Geospatial Forum.
https://2026.cloudnativegeo.org/
In Conversation
From Computer Science To NASA Harvest
Daniel: Hannah, you are an assistant professor at Arizona State University, the AI lead for NASA Harvest, and a research advisor for Taylor Geospatial. Welcome to the show. What does it mean to be the AI lead for NASA Harvest?
Hannah: Thank you for having me on the show. NASA Harvest is NASA’s applied sciences, or Earth action, program on agriculture and food security. The whole program works to advance the use of satellite data for agricultural applications and decision-making around the world. It also has a sister program called NASA Acres that is similar, but focuses only on the United States, and I’m the AI lead for that program too. What it means to be the AI lead for both is that I keep up with, lead and advance the different ways we can move the entire field forward in terms of developing AI capabilities that are helpful for agricultural decision-making.
Daniel: My guess is that has a lot to do with Earth observation?
Hannah: It has everything to do with Earth observation, and in particular satellite Earth observation.
Daniel: A lot of people understand what it means to be an assistant professor. What does the research advisor role at Taylor Geospatial involve?
Hannah: As research advisor I work with the team there to help identify what the highest impact research directions might be, and where the organisation might focus its efforts in terms of developing public good technologies that will advance the state of geospatial AI. It is broader than agriculture. Agriculture is one of the core focus areas of Taylor Geospatial, but the remit is really the entire field of geospatial AI.
Daniel: How did you get involved in all this? How did you become an expert in agriculture and remote sensing?
Hannah: Really just by following what seemed interesting and where I could make an impact. I started in computer science because it felt like an incredible area where you can do so much with so little. By typing on my computer I can create things out of nothing that can impact people’s lives globally. That interest led to another, and that is where I found myself working with NASA Harvest, NASA Acres and Taylor Geospatial. Satellite data in general is such an untapped, underutilised resource for so many different application areas. But I have also really just followed the people — working with people who are fun to work with, who have similar energy and goals, and who really want to get things done rather than talk about it.
Daniel: Do you still have a sense of wonder when you look at satellite data?
Hannah: Oh, for sure. Sometimes I get lost. We’ll be looking at one thing, then I look over to the side and think, what is going on here? And 10 minutes later I’m asking myself what I was supposed to be doing.
Daniel: I’m glad there is someone else out there like that too. People often talk about it as a resource to be mined and exploited, but when I look at it I just think, this image was taken from outer space.
Hannah: Totally. And with the frequency of these observations, a lot of people don’t realise you can see not just where you are today, but where you were yesterday, a few days before, a few decades before. It’s amazing.
Why Field Boundaries Were The Missing Ingredient
Daniel: Could you introduce Fields of the World?
Hannah: Fields of the World started as a benchmark dataset with Taylor Geospatial a few years ago, back when it was called Taylor Geospatial Engine. Jen Marcus, the executive director at the time, wanted a first project that would exemplify their innovation bridge program — showing how you could move technology from a research setting into commercial settings faster, developing research technologies together with academic researchers and excellent geospatial engineers, and shrinking the gap between research and deployment. There are so many things you could pick to build public resources for geospatial AI, but in agriculture one of the key missing ingredients was field boundaries.
Hannah: It felt like a problem that isn’t fundamentally hard. It’s not like we were trying to move an intractable amount of mass from the Earth’s surface to Mars. It’s just that nobody had really done it, and nobody had gathered the data and the ecosystem needed to do it. If we focused on that, we could demonstrate the whole pipeline: create new data and model resources, get them into the hands of users quickly, and have a fast feedback cycle between the academics and the users. So we went after field boundaries as the solvable fruit that so many people needed. We had been hearing about it for years — if we only had field boundaries — but every effort was fragmented.
Daniel: You took the next question out of my mouth. If we only had field boundaries, what could we do? Why is a polygon delineating a field so fundamental?
Hannah: If you don’t have a polygon, you have pixels. What we had before field boundaries was lots of land cover maps and cropland maps, all at the pixel level. If you want insights at the field scale — what is the yield in this field, when might it be harvested, what are people growing, how many fields were created in the last couple of years because of deforestation — almost everything people want to know about agriculture comes down to field units. What we have instead are pixels from satellite data and the products we derive from them.
Why Drawing A Field Boundary Is Hard
Daniel: I’m going to open a can of worms here. What is the hard part about creating field polygons from pixels?
Hannah: People think it must be simple, until you show somebody a satellite image and say, okay, draw the boundary. Even in the United States, or other countries with large-scale agriculture where you might picture big rectangles in the Midwest, it isn’t easy. Where exactly is the boundary? If you’re drawing a line, is it in the middle of where it goes from road to grass? It is never actually that clear, especially in relatively coarse satellite pixels.
Hannah: Another challenge is that within a growing season what appears to be the boundary can change. Look at a satellite image on one date versus another and the shape of that field can look different depending on where it is in its management cycle. There are also a lot of parcels that might look like fields, and in places where agriculture isn’t mechanised the boundary between fields can be really hard to identify.
Hannah: There is an enormous range in the appearance of fields around the world, and even within countries. This is where I get lost in the imagery — I’ll be somewhere in South America looking at the tiniest fields I have ever seen right next to the biggest fields I have ever seen, right next to these crazy pie-shaped centre pivots. You often have to look across multiple spectral bands as well as over a long period of time to get as close as you can to the true field boundary. And even then you might not get it, because you might not be able to see it. Somebody might have a rope separating their field from the next one, so capturing what a farmer would consider the field boundary might not even be possible. Compared to segmentation tasks in AI like detecting a person, a car or a bicycle, which are pretty uniform globally, there is a huge amount of diversity.
Daniel: And to highlight it — Fields of the World is over the entire world?
Hannah: Yes, it is global.
What Is Actually In The Dataset
Daniel: Can you tell us about the datasets you are using? Is it a mixture of a lot of different things, or one set of satellite images?
Hannah: Fields of the World started as a benchmark dataset, which brought together annotated field boundaries that were ground truth in some way. Some countries, especially in Europe, publish datasets of farmer-reported field boundaries that are really precise, really high quality and span the entire country. Others came from efforts that were manually annotated on satellite imagery or in the field. We paired those with Sentinel-2 RGB and near-infrared images that overlap them, and created chips that go into machine learning models.
Hannah: We have Sentinel-2 images for two different dates — we call them window A and window B in the paper — roughly planting and harvest. Mainly we are trying to get contrast so you can see where the boundaries between fields actually are. That, together with the tools and the dataset splits, the training, validation and test definitions, and all the other things that go with a benchmark dataset, became the original Fields of the World dataset. Around it we have built a whole ecosystem for field boundary segmentation, including, most recently, a publicly released global field boundary layer. That is the first global dataset of field boundaries: the predictions of our model, trained on the benchmark dataset, which covered about 24 countries, and then run over the entire world.
Daniel: How big are window A and window B in terms of a time period? I’m guessing it isn’t a snapshot of one hour of one day.
Hannah: A couple of months. We got the windows we used from the USDA crop calendars they produce for everywhere in the world. So we get several-month windows and pick the highest quality image within them.
Daniel: Highest quality in terms of cloud cover, I guess?
Hannah: Lowest cloud cover, lowest no-data — all those filters to get the best looking image. Sometimes you don’t find an image in that window at all, so for some places you expand it half a month at a time until the algorithm finds a good fit.
Daniel: You had ground truth for around 24 countries. How well did that translate to vastly different geographies? Did Europe, with its precise polygons, translate nicely to the Philippines?
Hannah: Sometimes yes, sometimes no.
The Global Layer, Its Confidence Score, And Your Feedback
Hannah: What we did in the global data release was pair the global dataset with a global confidence layer. We trained another small, interpretable model that takes all of the prediction metadata from the model trained on those 24 countries — probability scores, what other datasets like land cover say in comparison, a few different covariates — and predicts how likely it is that a boundary in the global layer is correct, a true positive. You can tune that confidence in the Global Data Explorer from zero to 100 and filter out fields we are less confident about. It really isn’t a simple mapping. You can’t say all small fields are bad, or everywhere it wasn’t trained on is bad. It does well in a lot of places, and it does not do well in some others. But the confidence layer does a pretty good job of knowing when it is wrong.
Daniel: Where do people go to do that? What is the address?
Hannah: The easiest place to start is fieldsofthe.world. Scroll down and you can click Open Explorer App, and you will see the global dataset.
Daniel: Can I download data from there?
Hannah: You can. We tiled the world into a download grid, which makes it very fast to download straight from your browser. Toggle on the download grid in the explorer, click any grid cell, and you get all of the field boundaries in that area.
Daniel: Fantastic resource. Will you be doing this on a regular basis? Can people treat this as a foundational dataset?
Hannah: Yes. We plan continual releases. Right now we have layers for 2024 and 2025, and we plan to do 2026 as well, but we are also continuing to make improvements. In the global explorer, if you zoom in, there are options to give feedback. You can rate the fields in the viewing area and give more detailed feedback on what worked well and what didn’t, and we are using all of that to make improvements behind the scenes. Sometime this year I hope we will have a big update to the benchmark dataset itself as well as the global layer.
Daniel: How clean and useful is that feedback going to be? With crowdsourcing models it can be difficult. Are you going to train people in what a good field looks like?
Hannah: That’s a very good point. What everybody does in the viewer is zoom into a place they know really well, like their house, and then say it doesn’t work here. So the feedback can be quite specific — but that’s okay, we want to know that too. We tried to land somewhere in between: we didn’t want it so easy that you could click a bunch of things and spam garbage, but we didn’t want it so lengthy or difficult that you start and then give up. So if you select poor, acceptable or good, you then have to select why — there are check boxes for fields missing, fields over-merged, and so on. I thought of this from apps like Uber, where if you give a rating it prompts you to explain why. It gives people a bit more accountability for what they are submitting. So far we haven’t really seen people spamming; we have seen pretty useful feedback.
Daniel: I would have thought the most useful feedback would be, zoom in and correct the polygon for us, adjust the corners.
Hannah: That is what we are working on and trying to figure out. A lot of people want that, including on our team, and I have been the one blocking it for a while, saying hold on — it is really not clear what a field is, so it is not something just anyone can do. What we are doing is collecting interest from people about who wants to contribute. Eventually I think we will have some sort of correction tool, but we need to put a lot of thought into how we make sure we are not essentially creating slop if we use that feedback to train our models. We are trying to predict something that is real and meaningful to agricultural decision-makers.
Daniel: So you are going to curate the feedback, and the people giving it.
Hannah: Exactly. We are looking at the interest people share through the feedback forms to identify who is interested and has the skills, or the responsibilities in their own job. We are really looking for people who want to be partners in this and help shape the dataset in the future, rather than crowdsourcing blindly.
Open Through And Through
Daniel: I have a sneaking suspicion some of this stack might be cloud native. Is that correct?
Hannah: Everything is cloud native. From the beginning, that was part of the idea of the innovation bridge through Taylor Geospatial. While we academics worked on the dataset and the tools, we had engineers who Taylor Geospatial calls tech fellows working alongside us to make sure we were using best practices, the best cloud native tools available, the right standards, and appropriate licences that don’t restrict use. That has led to a dramatically more usable output.
Daniel: Usable from the end user’s perspective, or from the perspective of the people creating it? I imagine there is less infrastructure involved if it is cloud native from the start.
Hannah: Both. Things like GeoParquet and other standards, and working with various partners to make sure we were producing cloud-optimised GeoTIFF mosaics and downloads in a form that would be easily accessible. A lot of work went on behind the scenes not just in using those tools but in identifying the right ones — and some things had to be developed or extended along the way.
Daniel: How does Fields of the World sit alongside everything else out there? Google produces big datasets, there are national cadastral records, there are commercial vendors. What gap are you filling?
Hannah: The biggest difference between what we have done and what other efforts are doing is that it is open source through and through. Everything we do is in the open. Anyone can contribute, you can see everything, everything is free to use. We are not selling these field boundaries. It is meant to be a shared, public good technology that everyone can benefit from and also share the responsibility to contribute to. There are companies with products that sell field boundaries, or something downstream of them, maybe tailored to particular regions or use cases. I don’t necessarily think this open layer competes with that, because most companies are not trying to sell the boundaries, they are trying to sell insights or tools on top of them. It is a rising tide that lifts all these different boats.
Daniel: Have you seen companies pick it up and ingest it into what they are already doing?
Hannah: We have a great relationship with the AnthroKrishi team at Google, who did the agricultural land understanding tool, and I know they are using our resources, like the benchmark dataset, to advance their tools — even though theirs is not a public model or a public global dataset. Our benchmark dataset lets them work on improving their product without having to put what they are doing out in the open.
Daniel: But like you say, a rising tide.
Hannah: We are not trying to arbitrate who can use it and who cannot. That is why we have made everything CC BY. A few of the datasets inside the Fields of the World benchmark inherit licences from the original labelled datasets, and some of those don’t allow fully unrestricted use — they have share-alike or something similar. So there is a more restrictive version of the dataset and a completely open one. We have results in our GitHub repo showing the difference in performance if you use each, and there really isn’t a huge difference. The global dataset is made with the CC BY licensed model as well.
Daniel: This might be a naive question, but is this a model I could take and fine-tune for my own geographic area?
Hannah: For sure. The model weights are public and the code is public, so you have a lot of options. You can download the weights and use them as a starting point to fine-tune for your region. You could take the benchmark dataset, add your own data and retrain from scratch — it is not a giant LLM-scale model, so that is very feasible, and there are instructions in our GitHub repo. You can also change the inputs to the model using the inference app: there is a Custom tab where, without writing any code, you can pick the model weights, the parameters, which input dates to use, and the images that go in, rather than our pre-populated ones.
Daniel: Would there be a benefit to the overall dataset if people did that and contributed it back?
Hannah: What we have found is that some people use what we have done as a starting point and then see what changes they can make — not just adding local data, but changing the loss function, changing the weighting of countries, various things that improve performance in a particular area and may improve performance overall. That is the benefit of having the benchmark dataset alongside it: we have a shared basis of comparison when people make these advances, so we can see how a change improves performance on a common set of tasks.
Foundation Models Versus A Well-Designed U-Net
Daniel: How does your model compare to something like Clay? Clay is the foundation model I know best, and it sounds like you are tuned for a very specific purpose where Clay is more, find anything that looks like this thing.
Hannah: Fields of the World, and the models trained on it, is a fully supervised task and model. Fields of the World is a downstream task for models like Clay. You wouldn’t use the features extracted from it in a general purpose way. Geospatial foundation models are a very different setup: they are trained in a self-supervised way without any labelled data, agnostic of a particular task, on lots of different data sources and locations, to do a generic proxy task. Then that model is adapted to downstream tasks like Fields of the World.
Daniel: So that could be an input into something like Fields of the World.
Hannah: Yes, and we actually tried that. We have a paper we just presented at CVPR, the Computer Vision and Pattern Recognition conference, called PRU — a tortured acronym for a practical recipe for field boundary segmentation at scale. We originally called it the Great Model Bake Off, as a nod to the Great British Bake Off. In the first paper, when we published the benchmark dataset, we were focused on advancing the dataset and didn’t really care which models we used; we tested a few, but the biggest gains were going to come from the new data, not a new architecture. In the bake off paper we said, okay, let’s test all the models. We did a huge sweep of hundreds of different architectures, loss functions and parameters, including about 10 geospatial foundation models, Clay among them.
Hannah: What we found is that those models still do not compare to a really well-designed U-Net for segmentation on this task. Again, they are designed for generic utility. To get performance that even got close to the fully supervised models that excelled — the U-Net recipe we ended on — you would need to do so much exploration of different decoders to add on to those pre-trained encoders. Clay actually ended up doing the best. None of them did great, but out of all of them Clay did pretty well, we think because it has the smallest patch size. The others had blurry representations of the inputs, which led to segmentations that were not very crisp, and field boundaries can be very narrow.
Daniel: It isn’t surprising that was the finding, but it is nice to have data behind it.
Hannah: It is not surprising, but it is disappointing. The goal of geospatial foundation models is that we should be able to use them to jumpstart progress on many different tasks. For now, maybe given the size of the data we have — there are about 70,000 samples in Fields of the World, so it is not a low-data task — the custom, fully supervised models work best.
Hannah: Another big consideration is cost. In that paper we have a metric we call throughput: the square kilometres per second that can be processed through each model. That translates directly into cost when you are predicting a global map. Foundation models are for the most part an order of magnitude larger, and an order of magnitude lower in throughput, than the custom U-Nets we use in practice. That is orders of magnitude more cost to predict a global map, so even if you got a small performance gain it would not be worth it.
Daniel: Is there one dataset I could show up with tomorrow that would make a massive difference to your accuracy? Is there an input you are missing, or is it just more training data?
Hannah: If you took the confidence layer, sampled all of the lowest confidence areas, had experts annotate them and added that to the model, that would definitely improve it. We are going to do that — we’ll be working on it this summer, and it is a big update we will be making to the model. Right now the shortest path to improvement is adding more data where the model does poorly.
Daniel: More data in terms of labels and annotations — what about higher resolution satellite data?
Hannah: That would also help. Not everywhere, but in a lot of places we are getting past the limits of Sentinel-2’s 10 metre pixels. In a lot of smallholder agricultural areas you really can’t see the boundary, or you get stair-stepped artefacts on the boundaries from that resolution. We are in conversations with Planet and other providers about how to bring higher resolution data into the dataset and the ecosystem in a way that is as open source and accessible as possible. Obviously, once you are working with paid data providers that gets a lot more complicated, so we want to do everything we can with public data — but higher resolution data would undoubtedly help.
Daniel: Sounds like you might be a big supporter of Common Space.
Hannah: Oh yes. Rhiannon Price of Common Space is also part-time with Taylor Geospatial, so she is involved in this as well, and with the Fields of the World team.
Who Pays For A Foundational Dataset
Daniel: When I think about foundational datasets I think about something repeatable, open, understandable, and something that is going to be here next year and the year after. A lot of the time that comes down to funding. Who pays for this? How do we know it will be there next year?
Hannah: Taylor Geospatial has been paying for this — for the research behind it, for the compute to provide it. But as we build a fairly large open source community around it, our hope is that people start taking on bits of that responsibility, and that it stays afloat through this community of users and the different companies using it themselves and then giving back through support. A company might use it for a commercial product that doesn’t go back into the open, but if this is a useful resource they contribute back to it. I think that is what we are already seeing happen.
Daniel: How do you avoid ending up like QGIS? QGIS is used everywhere by everyone, the products created with it are paid products, and they now have an okay funding model — but I don’t think it’s amazing. I don’t think they are swimming in cash.
Hannah: Are you asking how we make sure this doesn’t become my hobby for eternity, and I’m 80 years old updating this repo by myself? Honestly, if that is what has to happen, it probably will. But the fact that we get a lot of help from our agent friends, our LLMs, definitely helps us sustain things with less of our own time. That helps enormously with sustainability, and it also lowers the bar for a lot of people to participate in open source projects. Even though we still have a review system, so it isn’t going to be slop on slop, it does allow more people to make changes, especially the bite-size, low-hanging-fruit contributions in a community project.
Hannah: In terms of a long-term sponsorship model, I don’t know. There are several people at Taylor Geospatial who are much smarter and more practised at this than me — Chris Holmes, for example, who did GeoServer and a billion other things in the open source community, is part of it. I trust in him. We all do what we do best on this team.
Daniel: It is important to have specialists. When I think about infrastructure I think about that side of it as well. It would be nice if it is going to be here tomorrow, next year, and the year after, so we can build on top of it.
Hannah: Totally. We have so many organisations invested in its success. NASA Harvest is a partner, Microsoft AI for Good is a partner. By having all of these different partners involved, we will keep it going somehow in the community.
Daniel: What does this look like in five years? What are the big improvements going to be? Millimetre accuracy over the entire world?
Hannah: Performance will be way better. When you zoom in on your backyard in the portal, you will be a lot happier, I hope. I hope that over the next several years we improve a lot of the major limitations in the dataset, especially for smallholder farmers, who account for such a huge share of global agricultural production. You will also see a lot more sensors. This will move from being an ad hoc case of predicting one kind of field boundary with one kind of data, to a much more flexible ecosystem, and to more flexible models that can predict more about agricultural fields, not just more accurately.
Daniel: I also think that if you can nail the human-in-the-loop piece, with those experts around the planet who are invested in this helping out, that would be incredible.
Hannah: For sure. There is a lot of expertise, and a lot of people who want to solve this problem.
Daniel: Thank you very much for the work you do, and for taking the time to talk with me today. I will link up everything you have mentioned in the show notes so people can find this resource and play around with it themselves.
Hannah: Perfect. And share your feedback with us.
