Accelerate SF Conference

October in San Francisco

The sun is shining bright, bringing the hint of warmth to my head and ears. I'm cruising at 20 mph down the Embarcadero along the San Francisco Bay. The wind is blowing against my arms and legs and keeping me cool. It's like natural air conditioning. This feeling of sunshine and comfort, in the middle of a techno-historic city, surrounded by technologists, dreamers, and doers...this is why I moved here in May 2012.

Setting Up My Portable Workstation

My e-cargo-bike makes it easy work. In my panniers I have a backpack packed with my portable coding workstations. A Lenovo laptop (running Ubuntu), a portable OLED monitor, a ZSA Moonlander split, ortho-linear, fully programmable ergonomic keyboard pre-programmed with Colemak and all my shortcuts, and a pair of Focal Elegia headphones. I'm prepared for 24 hours of programming, collaborating, and making something awesome.

I've also managed to stuff three poster boards showing how my company Aerbits solves the illegal dumping reporting issue for cities like SF. My mission is to get in front of more decision makers and to build relationships, so I can see the problems from their perspectives.

Inspiring Prompts

After chaining up my e-cargo-bike in the Fort Mason parking lot, and stashing my portable bluetooth speaker in under a jacket in the pannier, I head up a concrete ramp and through glass doors, above which hang the marquis for Founders Inc. The third floor of the Founders Inc. workspace is filled with chairs all facing a small stage. TVs are showing Founders Inc., and a sign shows the Accelerate SF imagery of the Golden Gate Bridge in some sort of generative AI crafted image. It's an idyllic vision for the future of San Francisco.

The organizers talk about the recent advances in AI, and mention that Mayor Breed uses ChatGPT to help her understand and ask questions about the 700-page City Charter. The head organizer mentions that there are three Supervisors alongside the Mayor that are "abundance minded" and want to see the city recover from the pandemic by using all the available tools.

City Leaders and Their Wish Lists

Three SF Supervisors are introduced, Dorsey, Stephanie, and Engardio. Each of them gives a short speech about their vision for the city.

Supervisor Matt Dorsey

Supervisor Dorsey talks about the need for more housing, and the need to make it easier to build housing. He recalls back to his experience working in tech alongside several prominent technologists, and how he wants to see the city use technology to help solve the city's problems. He mentions how this is a time like no other, and that this technology has the potential to deliver on the promise of a better future. He mentions that the city went through a process to make more data publicly available, and now we have the opportunity to make that data comprehensible and actionable.

Supervisor Catherine Stefani

Supervisor Stefani talks about her nephew who understands all this technology, and how she wants to make sure that the city is a place that uses these new technologies to help people. She looks to the attendees to make this technology simple and available to everyone.

Supervisor Joel Engardio

Supervisor Engardio talks about the need for more transparency in government, and how he wants to see more data and more open data.

It's clear that these three Supervisors are keen on adopting new technology that will help solve San Francisco's pains. What's not clear is how we can get a majority of Supervisors to clear the way for an AI powered future. I am eager to find out how this unfolds over the next several months, and years.

Political Will

I've never been to a hackathon with this much political will behind a single technology. It's inspiring to see the city leadership so engaged and so willing to work with the technologists in the room. I'm excited to see what we can do in the next 24 hours. The room is filled with optimism and a buzz around the potential for technology to help the city recover from the pandemic, but also to make unprecedented changes to the way we all access public services.

Let the Hacking Begin

I'm surrounded by so many new faces. We've all been chatting in Discord for days in preparation for this moment. Armin is a fellow engineer, early in his career, looking for a mentor and a team to join. I'm happy to have him on my team. We had prepared an ambitious plan to make street sweeping schedules, routes, FAQs, and also reminders available to user via SMS, using Generative AI to handle the conversational aspects, and a RAG (Retrieval Augmented Generator) to lookup valuable context for use questions and answers. However, there is a wrinkle.

A Wrinkle

Days prior I started setting up a new Long Code SMS phone number... and both Twilio and AWS are telling me it's going to take two weeks to get it approved! It wasn't going to work. So Armin and I are brainstorming new ideas. I start building a progressive web app to serve up the street sweeping schedules. I'm using Next.js, React, and Tailwind CSS. I hear that three other teams are working on the same or similar problem, and two of them already have SMS numbers they can use. Without SMS, we can't deliver on the promise of making this technology available to everyone, and neither Armin nor I think our new scope is very compelling. So, we agree to change course. It's not the best start, but we're still optimistic.

A New Direction

We need something compelling and different. I have a unique dataset, which was curated by an AI pipeline I designed for aerbits.ai. It's a dataset of 120,000+ images of the streets of San Francisco. Each image has been labeled with the type of trash in the image, a precise geotag, and a timestamp. I've been using this dataset to train a model to detect illegal dumping. From my conversation with the San Francisco Public Works Department and the SF 311 Technology and Call Center Departments, I've heard numerous times how problematic the sheer volume of resident reports has been. The city receives several million resident reports every year. This has caused the city's call center to grow to 250+ agents, while residents still complain about issues with the feedback and even the service they receive.

What if we could do two things:

• Provide residents with photographic proof of service delivered
• Provied 311 agents with a tool to help them quickly and accurately triage, validate, and close resident reports

The Solution

Aerbits is built to capture a high resolution image of every street in the city every day, and to use AI to detect illegal dumping....but we also can detect anything that can be seen from the air. We could provide the 311 agents a tool, using our dataset, to look up a high-resolution (sub 1cm / pixel) aerial image of any location in the city and to use that image to validate and close resident reports. We could also provide residents with a link to that image, so they can see the service delivered. This would be a win-win for everyone.

The Execution

It was a late start indeed. After spending time chatting with other teams, last minute brainstorming, and finally landing on an idea, we had just 12 hours left to execute. On top of that, I have dad duties for 3 of those hours. It was going to be tight.

The Search View

I started by adding a new page into aerbits.ai, with a simple search interface, and an image grid, where a user could see the results of their search. The search form included a text input for the address, a number field for a search radius, and a date picker for the date range.

The Map View

I then added a new page to show the map view. This page would show a view of the location with all the images from the search results. Agents could use this view to validate and close resident reports, and capture a screenshot of the image to send to the resident.

The Full Demo

It's pretty simple, but it works. Think about this, in two days, we're able to build this new feature on top of aerbits.ai's existing stack. This is with zero AI, just geolocation and a large high resolution dataset of aerial images of San Francisco.... but we can take this a step further, we can use AI to detect the status of a reported issue, automatically closing the ticket, or raising the issue to the attention of managers. This would save the 311 agents literally millions of minutes each year. At just $20/hr, that's on the order of $10M in savings every year.

Conclusion

We were able to use the existing dataset, and the existing AI models to deliver a new feature that could help the city of San Francisco. We were able to do this because we had a unique dataset, and a unique AI pipeline. Although we didn't win the hackathon, we did get to present our idea to the judges, and I had a chance to speak with the Mayor and the city Attorney.

This was an amazing and fun event. Not only did I get a chance to build something cool, and potentially impactful. I also got to spend time with so many amazing people.

Thank You to My Team

Special thanks to my teammates for the event, Armin, Steven, and to the organizers of the event, and to the Mayor, the City.