Voice Assistants and Spatial Applications
In the modern world, voice technology is becoming more and more commonplace. In 2019, 20% of Google searches were voice queries and in general, a hefty share of searched questions are users looking for information on things in their local area. To complement this, over one-third of Americans now own a smart speaker. Understanding the current landscape, it is unsurprising that some innovators are looking to give the people what they want, a smart speaker compatible application that will answer their spatial questions. Smart speakers, such as Amazon’s Alexa, and Google Home, are interactive speakers that allow users to control their smart homes. This largely involves tasks such as turning lights on or off, playing music, or telling a silly joke on demand. These platforms also offer app stores with custom libraries of functions for users to take advantage of. Amazon’s applications are called skills, where Google’s are called actions. Independent companies and developers can contribute to these app stores to add even more targeted functionality. Although there are already a plethora of skills and actions on the market, smart voice assistance is still a very under explored space. Spatial queries in particular leave a lot to be desired, but hold a great amount of potential. As smart devices store a user profile of home address, phone number, email, and a number of other preferences, the figurative distance to be traveled to start answering people’s spatial questions is shorter than one might think. The trick lies in translating how humans ask questions, to how computers understand questions, then encouraging humans to continue to take advantage of this emerging technology as new capabilities are released. A significant part of this is porting web page based applications to voice based interactions. Where a web map may be the driving force behind answering a user’s question, that simply cannot be communicated in the same way strictly through words. This requires rethinking how people interact with spatial information at its core.How Do Voice Skills and Actions Work?
As everyone here may know, humans and computers speak different languages. Computer scientists have spent a long time navigating how to better translate between the two, and now we benefit from an extensive catalog of options. The people over at Q11 in particular are focused on adding geographical questions to this growing dictionary to allow you to ask “Alexa, when will my garbage be picked up?” or “Hey Google, what is going on downtown tonight?” and get back a quick and useful result. Accomplishing this task is simple in theory, but takes time to implement correctly. The general workflow involves first converting the user’s question to a query, pushing it to the appropriate municipality’s API and backend, then returning an understandable response to the user. Converting user questions into actionable queries has been the mission of smart speakers from the beginning. The process, called natural language processing (NLP) involves converting the spoken word into text, then extracting information from that text to build a query or command. The phrases people use for requests are called ‘utterances’. Lists of synonyms are created for the words used in utterances. These lists are expanded in order to help train the program for the many language variations that the application will likely encounter from the public. While applications like Q11 will respond to specific requests, for example “Alexa, what construction projects are there within 2 miles of me?”, it is also engineered to create queries from more general questions like “Hey Google, what construction projects are there near my house?”. Once the utterance has become a query, it gets fed into the relevant locality’s API. As there is no set standard for how GIS services are administered, stored, and accessed, it is necessary for this API setup to be done manually using Qwhery.cloud by the municipality’s GIS or IT department. This allows them to take advantage of the potential features of the Q11 application, as long as they have a feature service available that matches the user’s needs. Considering the discontinuity between various governments’s backend GIS systems, it is unlikely that we will see Amazon or Google themselves swoop in with their own custom spatial skills and actions. It is simply more efficient for each organization to integrate their own APIs, REST endpoints, and services. When the localities themselves are involved in this step, it also allows them to better regionally adapt their queries and outputs. Where “nearby” may mean 10 minutes in a small town, it may mean an hour in a larger city.The Future of Spatial Voice Querying
Voice assistants and artificial intelligence in general have come a long way very quickly, but of course, there is still ground to cover. We have already seen next gen features incorporated, like Q11 notifying users when the garbage truck should be expected, but there are both technical, and social challenges in the way of future uses of geospatial voice apps. A broad goal that GIS departments have in implementing tools like voice and web applications is to reduce calls they get to 311 centers, and to generally produce better informed citizens. This mission began with geospatial web apps, and the issues began with promoting awareness of these resources to the target audience. Geospatial voice tasks and actions face the same obscurity problem. If people are not aware they can ask Alexa what the zoning limitations of their property are, they probably will not think to. By the time they have researched the answer to their question, they may not have the need to ask it again in the future. This conundrum is why developers want to facilitate reusable content – questions that need to be asked over and over again. This produces returning customers who may be more willing to invest the time to explore additional capabilities. A huge market for potential use of spatially-aware voice apps is commercial applications. A user might be able to ask “Alexa, what is there to do downtown tonight?” and get a number of events to attend. These can be sourced from the municipality’s Economic Development office, or even go down the path of sponsored messages. Further opportunities in the private sector lie in integrations with things like the Yelp API, where a user could ask for recommendations of where to find the best pizza in their city or neighborhood. Once the word gets out on the possibilities behind spatial voice queries, there will be no looking back.In Conversation
From Municipal GIS to a Voice Startup
Daniel: Matt, welcome to the podcast. Can you introduce yourself?
Matt: I’m Matt Patrician, CEO and co-founder of Qwhery. I’ve been in the geospatial industry for almost 20 years, most of that in the public sector working as a GIS specialist, GIS web specialist, and supervisor. I’ve worked closely with large municipality teams building solutions to help them answer questions using geospatial technology, and I’ve spent a lot of time building up open-data programmes.
Daniel: How did that lead to building a voice assistant?
Matt: I originally saw the concept through Alexa-enabled devices in Johns Creek, Georgia — they’d put together a product letting citizens ask a defined set of questions through their Alexa device. Brilliant idea — using technology that’s already in people’s homes to trick them into using geospatial without them knowing it, because web maps can be tricky to learn or aren’t always the most adoptable channel. The opportunity I saw was scaling that into a platform many municipalities could integrate their GIS and open data with.
What Alexa Can Do for a Municipality
Daniel: What does a typical query sound like?
Matt: Things like “Alexa, ask Q11, when will my garbage be picked up?” or “Alexa, ask Q11, why wasn’t my garbage picked up?” You can pull in geospatial awareness: “Alexa, ask Q11, what’s being built nearby?” or “Alexa, ask Q11, what’s being built within 5 kilometres of my home?”
The Q11 Architecture: Cloud, Skills, and Actions
Daniel: So you’ve built logic that sits inside Alexa and Google Assistant?
Matt: Two sides. On one we have the Qwhery Cloud — our platform that lets municipalities integrate their GIS systems through web services and APIs. On the other we have voice applications. Amazon calls them skills, Google calls them actions. We’ve built both under the name Q11 — a play on 311, the call centre number residents would ring to ask questions of their municipality. Q11 is available in each platform’s marketplace, and a user engages with it on whatever device they’re comfortable with. It connects to the Qwhery Cloud, which connects to each municipality’s business systems and returns information to the user.
Daniel: Is Q11 one skill that does many things, or a collection?
Matt: One app that does many things. A municipality picks which capabilities to enable — waste collection, council members and wards, events calendar — and connects their REST endpoints to those. When a user interacts with Q11 we use their device location to figure out which municipality to connect to. So it’s one app with broad capabilities, configured per city.
Teaching the Assistant What “Nearby” Means
Daniel: How do you insert geospatial logic — “nearby,” “near me,” “around me” — into queries against a database?
Matt: Google and Amazon use natural language processing — NLP — to turn spoken words into text and structured data we can pass to APIs. We take a phrase like “what’s being built nearby” — an utterance — and define synonyms to train the model on variations. When someone says “nearby” we set a parameter in the Qwhery Cloud (say, 2.5 kilometres) and pass that to the ArcGIS feature service API along with the construction-projects layer. The API returns the projects within 2.5 km of the requester’s current location.
Daniel: Are there regional differences in how Alexa or Google interpret “near” or “far”? In Denmark anything more than 90 minutes is far; in Australia it’s close.
Matt: Not built into the platforms inherently — you have to bake it into your skill. Where I am near Toronto we have wards and a single member of council per ward; in some US customers we have districts with multiple council members per district. In the Qwhery Cloud, the municipal administrator has options to select that match their region — those become parameters in the query and the spoken response.
Notifications and Writing Back to the Database
Daniel: Can these devices push notifications too?
Matt: On our roadmap. The first use case was reminding residents when the garbage truck is on their road — even calling out “this week is recycling week” or “this week is organic waste week.” We’re really excited to tap into those notification APIs on both platforms.
Daniel: Can users write back to the database — report a pothole, a broken streetlight?
Matt: Yes — we’re developing that today for a customer. We hook into ArcGIS REST endpoints with around 50 different deficiency types. Anything from “there’s a funny smell when I turn on my taps” to “how do I turn off my water?” — captured via voice and stored in their database. A lot of the form fields can be pre-filled from the user’s Google or Amazon profile — location, phone number, email. For free-form input we store the text blob; for pick lists you have to keep options short, because listening to more than three or four choices gets tedious.
Why Google and Amazon Won’t Just Do This
Daniel: Do you worry Google or Amazon will just build this themselves?
Matt: It crosses my mind. But looking at municipal open-data efforts, there still isn’t a standard for zoning data, council ward data, service requests — nothing like GTFS for transit feeds. So I don’t worry too much about Amazon or Google consuming all that openly shared information, because there’s no standard to consume against. The Qwhery Cloud is the integration point — municipalities connect their REST endpoint and map their data spec on the fly. We’re not connected to ArcGIS by necessity — anything with a GeoJSON endpoint works, we read and write XML, we know GTFS. We started with ArcGIS because Qwhery is an emerging Esri partner.
Adoption Is Harder Than Engineering
Daniel: What’s been the biggest challenge?
Matt: We’re at the very beginning of voice as an interface — really exciting times ahead, especially as we move into screenless devices and concepts like the metaverse where you may not have hands free. Cities have spent years building awareness for their web maps — “who’s my councillor,” “what ward am I in,” “what’s the zoning for my property.” Now everyone has a smart speaker, or one is sitting on a shelf, or there’s voice on their phone. People are more comfortable speaking than navigating a buried web map. The challenge is stickiness — you ask “what’s the zoning for my property” once, not every day. So our biggest goal beyond making the assistant useful is helping municipalities lower call-centre volumes and shorten wait times, especially during a pandemic when many service desks moved entirely to phone.
Daniel: If you could build the killer app, what would it be?
Matt: “What’s happening around me?” After two years of pandemic restrictions, people are ready to head back out — “what can I do downtown?” “What’s happening nearby?” That’s an interesting concept and lots of people would use it. Municipalities and economic development offices already have that information — their job is to shine a light on what’s happening. We could also tap into open APIs like Yelp for “where can I get a five-star slice of pizza nearby?”
When Voice Isn’t the Right Answer — and Advice for Side Hustlers
Daniel: When is voice not the right answer?
Matt: When you’re trying to replicate a web front-end experience. Early last year we tried to convert a complex, large menu system to voice and halfway through realised it just wasn’t working. We had to re-engineer the entire interaction. Big complicated menu systems are hard to model and even harder for end users to navigate by ear. Web maps rely on visuals — thumbnails, imagery, graphs — that voice can’t substitute for.
Daniel: Are you working on Qwhery full-time?
Matt: Evenings and weekends right now — a passion project. Looking for new ways of doing things has always been my sport, so instead of watching my favourite team I work on Qwhery. The hope is to grow it into a full-time venture.
Daniel: Any advice for others starting something on the side?
Matt: If you have the passion and tenacity to keep going, you’ll know when it’s time to go all in — usually there’s a breakthrough contract that demands more time than evenings and weekends allow. Start by writing down your idea, building your business plan, and thinking about your network. Find a mentor — someone who’s done it. In late 2019 I called up an industry contact who’d started a successful geospatial business and asked for lunch. A lot of what they told me wasn’t flashy. Building a business is not easy. It’s lonely. Nobody cares about your success except you. You’ll feel rejection often. You need to push through if you believe in your idea.
Daniel: Where can people reach you?
Matt: LinkedIn or Twitter — DM me. Visit qwhery.com (Q-W-H-E-R-Y dot com). Or just say “Alexa, open Q11.”
