The curbside might not seem like the most obvious focus point in terms of mapping the urban environment but when you start to think of the curb as a highly regulated space and when you consider the number of arrivals and departures that that place on the curb in crowded urban cities you might just change your mind. The curbside is actually an interface between different vehicle traffic and pedestrians that has to enable a wide variety of use cases. Coord is helping organisations map the curbside, the assets on the curb and the locations of regulated spaces.
This episode is sponsored by HiveMapper
A platform that takes video and creates 3D mapping layers based on that data. The video can be from a variety of different sensors, does not need to be vertically looking down on the geography and each 3D output is georeferenced!
You are more than welcome to reach out to me on social media, I would love to hear from you!
In Conversation
From Online Advertising to the Curb
Daniel: Hi Jacob, welcome to the podcast. You are the CTO of a company called Coord and you’re doing something really interesting in the mapping space in the urban environment. But before we dive into that, can you tell me a little bit about your journey — how did you get involved in mapping?
Jacob: Thanks Daniel, it’s good to talk to you. I’m a computer scientist by training, I’ve always worked in software my whole career, but I started in online advertising, which is super interesting in very different ways to what we do at Coord, and in very different ways to mapping in general — dealing with very large datasets, things happening very quickly. But I’ve always been interested in transportation in particular, just in my life, and so when I got the opportunity to work on this really interesting problem that I’m excited to get into with you, I thought it was something I had to jump at, even though I had no real professional experience with mapping before. So I really had to teach myself when I started at Coord, and when we put the company together, which has been a lot of fun. My experience at Coord and my experience with mapping are one and the same.
Daniel: Sounds like a good way to get into it — the very hands-on approach that we often see with entrepreneurs. So can you tell me a little bit about the company? What is Coord and what problems are you solving?
Jacob: We’re a company that’s trying to help cities, as well as people who live and work in cities, understand the curb. That space at the very edge of the road — which is funny as a place, because it’s not really a place that exists. You’re always either on one side or the other, you can never be really right there. But for a place that doesn’t exist it’s very important, because cities love to regulate it. It’s very valuable space, because if you want to pick someone up or drop someone off, if you want to drop off a package, if you want to park your car, often this is the place that you use to do those things. So there are a lot of people contesting the space, a lot of people who want to use it for different purposes — also people use it for bike lanes, for bus lanes, all this kind of stuff.
Jacob: And what we found when we looked into this area is that cities often don’t even have a map of the way that their curb is regulated today. So we start at the very beginning, with collecting data and trying to model and understand how the curb gets used right now, and then go from there to build tools to help, first of all, people who use the curb understand what the rules are, to make sure that they can abide by them and not get parking tickets; and then also help cities figure out how they can change the way they use the curb to make the space more productive and more useful for everyone who’s there.
The Law Is What the Signs Say
Daniel: Firstly, this is really interesting — the idea of mapping something that doesn’t really exist, that doesn’t really have a place, you’re either on one side or the other side of it. The other thing is when you said that cities often don’t have a map of this area. So what have they been using, or how has it been working until now?
Jacob: This is one of these interesting things where usually if you want to know what the law is, you can get a book and you read in the book and it says, oh, I can’t walk outside with a champion on my shoulder on a Thursday, or whatever it happens to be. But on the curb, the law is what the signs say. So cities put out signs and it says, here’s a loading zone, here you can park for two hours, here you have to pay the meter. And whatever the signs say, that’s what the law is. If a police officer walks by, or a meter person walks by, then they’ll look at the signs and use those signs.
Jacob: And cities make the signs, but they don’t have a record of where they put them. If they want to find out how a street’s regulated in order to change it, they will send usually an intern out there with a clipboard to mark down where the signs are and how far they are from each other. So people can take advantage of this. There’s actually a story here in New York City from two days ago that some post office employees taped over the hours on a sign so they could set their own hours, because they were tired of getting tickets in the middle of the day. When you don’t have a record, this is the kind of thing that can happen. People put up fake signs, people take down signs, they fall down in a rainstorm — and that actually changes the law. So this is not exactly a great state of affairs for anyone.
Daniel: No — and especially when I think about the likes of autonomous vehicles. A lot of what we do now is driven by the idea that it needs to work for us and for the machines, and human readable signs is something that doesn’t work really well for machines either.
Jacob: It doesn’t work well for machines, and to tell you the truth it doesn’t always work well for humans. There are some great stories of people just getting very confused with parking signs, and it happens all the time. I kind of like it when a city is designed for people, but the problem is the curb isn’t designed for people right now. It’s designed for this mishmash of expediency and maximizing parking ticket revenue and all of these different things that aren’t really for anyone.
Mapping With Phones and Augmented Reality
Daniel: So we’ve got this space that isn’t a space, it’s used by a lot of different stakeholders, it’s highly regulated, and it’s an interface between vehicle traffic and pedestrian traffic. There’s a lot going on there and it’s not really well mapped at the moment, if at all. How are you guys mapping it?
Jacob: This is a fun thing that’s an interesting mix of high tech and low tech. We have an app that we call the Coord Surveyor app, and what you do is you install it on your phone and walk down the street and take pictures of the various regulatory devices and other interesting features of the curb that you see. Whether this is a parking sign, whether this is a fire hydrant, whether this is a curb cut like for a driveway or an alleyway, or a crosswalk — whatever are the things in that city, that’s what you capture. And we hire people to walk the streets and use this app to collect the data.
Jacob: It’s an interesting mix of high tech and low tech, because on the one hand we actually use augmented reality technology. If you’ve played an augmented reality game where you move your phone and stuff appears to be in the real world coming through your phone’s camera — to do that, your phone has to be able to track you very accurately around the real world, and it can be much more accurate than GPS over short distances, like over the space of a single block. So we use that to help get very accurate positions for all these curb assets. But on the other hand, people always ask, well, can’t you just have a car drive down the street and use machine learning to get everything? And we’re being a bit scrappier and a bit simpler, just having people go and take pictures of the things that are important. So it’s low tech in that respect, but it turns out to work very well.
Daniel: If you’re collecting that kind of data via machine learning from a car, you’re also seeing things from a vehicle’s perspective, and I’m imagining that great chunks of the sidewalk are obstructed from vehicles, so you’d be missing a lot of that. Is that not the way it is?
Jacob: You miss a ton, especially in some places in California in particular, and also in some European cities. We were just looking at some projects in the UK and in Ireland recently and we noticed there’s a lot of stuff that’s down at pavement level, on the edge of the sidewalk or on the edge of the roadway. And those things — even if they’re for vehicles — once somebody’s already using that space, the regulators don’t care if you can see it while you’re driving by, because you can’t use it anyway, there’s a car there, there’s a truck there. So these things can be very difficult to get good imagery of from a vehicle.
Jacob: And then the funny thing about AI and machine learning — people will tell you it’s a great way to save a lot of human effort and the machines are taking our jobs and so forth. But getting a good model, and especially for these things that vary so much jurisdiction to jurisdiction — not just country to country but even city to city, sometimes even neighborhood to neighborhood — teaching a computer the stuff that we as humans know intuitively, because after all these systems are designed for us as humans to understand, can end up taking as much or more effort than sending people out on the street.
Daniel: I love the fact that you’re using mobile phones to do this, I think there’s a great use of the technology. I love the fact that you’ve got that crowdsourcing approach to it — I can envision armies of people descending on New York City and mapping everything, in the same way we’ve seen with OpenStreetMap. But I can definitely see some problems around the ground truthing of that data. I can see people collecting signs and saying yes, that’s a sign — what about the transcriptions? And what about the fact that those parking signs work in pairs? We see one and then we see another one, and between those two there’s a relationship. How do you record things like that?
Jacob: That’s a great question. First of all there’s the question of just making sure we have good and accurate data, so we make sure that we always send at least two people down every single curb, and if we don’t get enough agreement from the two people who go down, we send a third person. You think about crowdsourcing and you think, well, you’d have hundreds of people to do this — but when we go into a city it’s usually four or five or six people out on the street for a week, and you can do a ton of space with just that size of crew. It’s really amazing how effective people are, and how much you can get done with people power. So that means that we can get really good data just by getting high coverage and high quality of coverage from these groups of people.
Jacob: But then, understanding — not just having a picture of the thing but actually understanding it — is really interesting. For transcribing, we’re still people powered. This is another interesting thing, because there are a ton of off-the-shelf optical character recognition machine learning models, and we tried throwing them at this problem and they don’t do great. This is something that we’re really interested to explore in the future, but right now we actually use humans for this as well. We send the pictures to Mechanical Turk and have people transcribe them, and people do a very good job. People do a great job with occlusions, which is something that’s very difficult — so if you’ve got a tree branch in front of part of the sign, maybe there’s graffiti over part of the sign, or someone put a sticker on it. These things get vandalized a ton, and people are very good at reading around that and understanding the meaning in the context of what can be a very ambiguous scene.
Why Machines Struggle With Parking Signs
Daniel: That is really interesting, because I would assume that street signs are made — not for machines obviously, they’re human readable, they were designed for us first — but I would have thought they would be so standardized that it would be easy to create a model. This is what a stop sign looks like, this is what a give way sign looks like.
Jacob: They’re really much less standardized than you’d hope. Traffic control signs — things that control the vehicle when it’s moving — are much more standardized. Here in the US there’s a thing called the MUTCD, the Manual of Uniform Traffic Control Devices, that has pictures of all the signs and it’s very, very detailed, down to the millimeter of how big that arrow has to be and how big the border has to be. And they do have standardized parking signs, but they only control a tiny percentage of the stuff that you see in cities, because every city has their own problems and their own special ways of dealing with this. Nobody at the national level, much less the international level, has really determined a great symbology that can encompass all of these different things — which means that every city ends up building their own.
Jacob: And these are also designed — if you look at stuff like how the text is presented, it’s often very broken up into zones. You have this top left zone, you have this top right zone, you have this big number in the corner, and it’s designed so that you can get the most important thing right away. But then, when we’re doing the job that we’re doing of trying to understand the rules on the curb in total, we have to also get the least important thing, which can often be very difficult to see. You can have this huge difference in font size and huge difference in prominence between these different features, and that really confuses off-the-shelf OCR algorithms.
Daniel: I just want to take a step back. We’re collecting the data, we’re using crowdsourcing, and we’re doing it on mobile phones. One of the really important things here is the location of things — where are these objects relative to the other objects, because the ultimate goal is to map this. How are you solving for the urban canyon effect when we think about GPS and location?
Jacob: That’s a great question, and that’s again why we use this augmented reality technology. What we do is we have these people stand at the corner, and we know exactly where the corner is. This is interesting too, because another thing that cities don’t have is just physically a map of the edge of their pavement for every street — that’s often something that’s hard for cities to get. But we build that map, and so we know when you’re standing at a particular corner, corner of 10th Street and Third Avenue or whatever it happens to be. You can say, well, here I am at the corner, and you mark that position, which we know exactly where it is. And then your camera and your accelerometer and your gyroscope take over, and we can get a very good relative position from that spot. So even in an urban canyon we don’t have to use GPS, we just use your phone sensors to understand how you’re moving through space, and doing that we can get a more accurate position for you even in urban canyons.
Daniel: That’s really interesting. I was expecting, when you say we get this accurate positioning, that it would be relative, and then when I get the data out of the system again I would somehow have to georeference that against the real world. But that’s not what’s happening here.
Jacob: Well, it kind of is, but we’re using people power for that too. We just have you georeference by standing on the appropriate corner.
Daniel: So I’m georeferencing myself, or the phone, at the start at one position, and then from that you can build a map relative to that position. Which is really a very traditional way of doing things — you see these survey markers embedded in the pavement sometimes. A civil engineer friend of mine pointed one out to me, I hadn’t noticed one before, but now I notice them all the time. When someone’s surveying a plot of land, they go from that known position. So it’s really a very old-fashioned way of doing things: you start from a known position and you go relative to that.
Jacob: But it works. I mean, that’s why they’ve been doing it so long.
Who Uses This Data
Daniel: Can you talk a little bit about who is using this data and what they’re using it for?
Jacob: We’ve really got two main groups of users. On the one hand we have the cities, who often want to have this map of their curbs and to help understand how their curbs are regulated. A lot of cities are doing things like demand responsive pricing, where the price to park on a given curb goes up or down depending on demand. Cities are also doing a lot of pilots of loading zones, or ride hail pickup and drop off zones, and you can use this data to set up these pilots much better and much faster than ever before. So that’s one big group of users — we sell to the city department of transportation or to other city departments so they can use this data.
Jacob: The other group are fleets. One of our customers is the ride hail company Lyft, who are big here in the US. They use our data to help plan better pickup and drop off locations for their users. We also have other fleet companies who use it to make sure that when their cars are parked on the curb — maybe you have one of those free-floating car share companies, for instance — they really don’t want them to get towed, because that can be a major expense for them. So they can use this data to make sure that their cars are parked by their users in places where they don’t get towed or they don’t get large fines.
Daniel: Having an accurate map like this, and being able to push it out via an API and deliver it to these end users in a digital fashion, would definitely take the guesswork out of it for them and make a much more secure business model. And no one would be standing there going, well, that sign was there yesterday but it’s not there today.
Measuring Demand and Counting Pedestrians
Daniel: You talked a little bit about charging models — different cities charging for parking based on demand. How are they doing that without some sort of measure of the traffic volume, or how often these parking spaces are taken?
Jacob: That’s often one of the hardest parts, they need to understand what the demand is. The easiest way to get this data, if you have paid parking already, is you look at when people pay the meter and you look at these meter transaction rates, and that can work pretty well. But it’s never perfect, and the reason why — there are two you can probably guess. One is not everyone feeds the meter, especially there are some people who have special placards or permits that let them not, and some people, if they’re there for only a short amount of time, decide they’re going to risk it. And the other thing is some people pay too much and they leave before their time is up, and then the space is available but you don’t know it.
Jacob: So there are various different ways of ground truthing that sort of data. Sometimes it really is people walking around and doing a survey — and we’ve done that kind of surveying too, where you’re not looking for the assets on the curb, you’re looking for just how many cars are there, and are they still there if you come by an hour later. So getting that kind of data is something else that people do manually. Sometimes people embed sensors in the pavement so that they can sense this information — as cars drive, you have a little magnetometer in there, the magnetometer spikes and you get a reading, and so now you know what your true occupancy is of all your parking spaces. Which can work, but it’s a big upfront expenditure, and we think that that’s rarely the most cost effective way to get that data.
Daniel: Is this data that you work with yourselves? Do you do this kind of analysis, or do you hand over the map that you’ve made of the curbside and the assets on the curbside, and this kind of analysis is done by a third party — in this case perhaps a city?
Jacob: We don’t want to tell cities the answer, because we know that they have a lot of constraints that we don’t fully internalize. It’s their curb and they should be the ones managing it. But we do handle this data. In our analytics app, which is the thing that we give cities — it’s a desktop or laptop application that they go into and they can really understand and do analysis on their curbs — that does include, when it’s available and we can get it from anywhere the cities have it, or we can collect it as well, this curb occupancy information, as well as turnovers, so that’s how long cars are staying at the curb. So we handle that information as well.
Daniel: I’m thinking that this curbside data and asset collection in this form is going to be important well into the future. But how will things change, do you think? Are there any big changes on the horizon in terms of data collection that would help you, or in terms of the use case for the data that you’re collecting today?
Jacob: Connected vehicles, especially on this occupancy side, really have the potential to change the whole game. If you have a fleet of vehicles and they’re driving around the city all the time, you can use where they go and where they stop and where they unload and where they park as a great way to get this sort of data in a sample sense. And sometimes that’s exactly what you want — there are cities that demand that ride hail companies, for instance, turn over all of their trip data to the city authorities, and then they can use that to do a lot of great analysis and to make their streets a lot better. So there are obvious concerns there around privacy — trip data can be very identifying if the positioning is fine-grained, and especially if different trips are linked together from the same people. So you have to do some work to make sure that you don’t have these terrible privacy impacts from getting this kind of data. But if you can do it, then that’s really what we think is the most scalable way to get a handle on these problems.
Daniel: I could see this kind of trip data — obviously if you can anonymize it in some way, so we’re not knowing that this was Dave driving his car on the 6th of April and this was the exact route that he took — if we could do that, you could start to see what people are actually using these spaces for. And this would be another way of ground truthing, because instead of assuming that okay, we’ve zoned this for parking, people are only parking there, you might see a whole bunch of other activities happening in these spaces. I think that would be a really interesting dataset to look at.
Jacob: That’s something that we really want to get. And the great thing for us — a lot of times people are looking at travel across the city, so they really need to know, even if it’s not “this is Dave”, where are they leaving and where are they going to on the same trip. But what we need to know for the curb is just the arrivals and the departures, and they can be completely separate. We don’t have to tie them together, which means that a lot of the de-anonymizing effects of having this trip data go away. So we think that there’s really the potential to be very privacy sensitive in the way that we handle this data, at least from the perspective of the curb, and to really understand how people are using this.
Jacob: Now of course, in some cases most of the users of the curb don’t have their own vehicles, or at least don’t have their own motorized vehicles. You have bicycles, pedestrians, people taking transit. So we really want to make sure that we capture these uses as well, and that’s its own kettle of fish — even getting accurate pedestrian counts can be very difficult. But that’s something that we absolutely are going to have to start confronting as well.
Daniel: How are people solving this problem at the moment? How are people getting these pedestrian counts?
Jacob: It can be extremely challenging. I don’t think there’s a perfect way. There are some people who are using cell phone location data — so they go to the cell phone company and they pay to see how many people are pinging their towers. Or they have you install an app that’s a free app but actually it sends all your location data to a location data provider, and you can use this to get some amount of pedestrian counts. But again, ground truthing is very difficult.
Jacob: Here in New York City it’s actually really funny — the gold standard ground truth pedestrian count data is from, I think, the 70s. They actually did aerial flyovers. They had a plane fly over the middle of Manhattan one nice cloudless day with a camera pointing down, and then they had graduate students with clickers count every dot — every dot that was on the sidewalk was a pedestrian. And that’s how New York City decided how wide the sidewalk should be in midtown, to this day. So that’s another one of these problems that’s really unsolved, and that’s not one that I think we’re going to be confronting just yet. But it’s just something that’s really interesting, how much harder that is. For vehicle counts you have these tubes — I’m sure you’ve seen them, you run the tube over the pavement and cars drive over the tubes — and you can use them for bicycles as well. But pedestrians, you can’t do that.
Daniel: You’ve built your own app, so you have experience in this space, and you mentioned that some apps out there, you install them and then in the background they’re sending your location data away somewhere else to be used for a purpose that you’ve probably not signed up for. Can you see this going away in the future, in light of the current privacy concerns around spatial data?
Jacob: I’m not sure that’s the question for me to answer. I told you I worked in online advertising and I’m very happy to escape from that. One of the things I love about Coord is that we really don’t deal with any personally identifiable information at all. All of these things that we have that are out there in the world that we take pictures of are public infrastructure that in fact the people who own them want everyone to know about. And even if we are counting cars out there, we make sure that nothing personally identifiable leaves the device. But in terms of what the right balance is between privacy and getting the sort of valuable data out to planners and to other people to help better understand the world — I just think that that’s a balance that as a society we have to figure out, and I’m honestly not sure where it’s going to come to.
A Feedback Loop for Cities
Daniel: I’ve just got one final question before I let you go, and that is: when you look out into the future, what is the one thing that you’re most excited about in terms of Coord as a company and what you’re doing with geospatial?
Jacob: Really the thing that I’m excited about is, for the first time, building a feedback loop around street usage between the city and the users of the street that can update quickly. What that means is that cities can make policies that benefit their citizens and their residents much more easily than ever before. One of the huge problems around managing transportation in cities was just that things take so long. It takes a long time to do an experiment, to put stuff out, because it involves often pouring concrete, or at very least anchoring new signs into the ground, doing all these kinds of things. And then even once you do that, you have to have this whole education campaign so people really understand what it is that you did and make sure to abide by your new rules. So transportation can often be a very slow moving space from a public sector perspective.
Jacob: I think that by using this new technology and bringing all this information into the digital world, we have the opportunity to give cities these amazing new tools for benefiting the public and for helping to arbitrate between the different users and the different interests around the curb in particular, and the street in general. And I don’t even know what they’re going to do with this — I don’t know what the resolutions are going to be, I think it’s going to be different in a lot of different cities. But I think that no matter what, cities are going to be able to build their policies better and to start doing more, faster, and more inventive things to manage this huge part of their transportation infrastructure.
Daniel: That’s a really great observation — that when we experiment with cities it takes a long time. You mentioned we have to pour concrete, put up signs, change the infrastructure; there’s a huge amount of inertia we have to overcome to get anything done there. But at the same time we have this very dynamic thing that we’re seeking to regulate. Transport is changing so fast — how we transport ourselves, the different actors involved, and where we do it. So scooters, for example, we’re moving away from the road onto the footpath, and we’ve got all these things being mixed together. So we’ve got a really fast pace, highly fluid thing here in transportation, and then we have the structural inertia of the physical city that’s difficult to change. That’s a difficult challenge to be presented with.
Jacob: Very well said. But you know, we love difficult challenges. I always tell my team, if your job weren’t hard then they wouldn’t be paying you the big bucks for it. So, feel the same way for companies.
Daniel: I really want to thank you for taking the time to do this interview with me, I’ve really enjoyed it. Before I let you go, maybe you could tell the listeners where they could go to learn more about you and your company.
Jacob: Sure, you can visit us on the web at coord.co — learn all about what we’re doing. We’re happy to have a chat if you’re interested in working with our data.
Daniel: Jacob, thanks again, much appreciated.
