Researchers at the University of Toronto’s Faculty of Applied Science & Engineering have created a way to use Google Maps Street View images to assess existing structures. With the aid of machine learning,  the images available online can be used to generate data to help determine the age, height and size of existing structures.

The study was published in the Journal of Industrial Ecology.

“This is the first paper we know of where people took a picture that shows you the front of the building and then predicts things that you can’t see in the picture,” says Professor Shoshanna Saxe (CivMin), who led the research team through the Centre for the Sustainable Built Environment. 

“My motivations were very focused on embodied carbon research use, but this will be useful for lots of different people. I’ve talked to researchers who are looking at understanding water usage for future planning, or resilience assessments.”

As Street View is prevalent nearly everywhere, the new method offers a cost-effective way to generate significant building data.

“We spent maybe $1,000 on photos to get data that would otherwise cost millions of dollars to obtain,” says Saxe.

“Nobody has millions of dollars to spend on just building dimensions, so this is the difference between being able to work on these problems and not. Having methods that can let us understand neighbourhoods and buildings at scale is really useful,” she says.

The team trained the AI to estimate building attributes based on external images of the structure, achieving 70% accuracy for age prediction and 80% accuracy for area prediction.

“Being able to assess the exteriors allows a sort of educated guess at the interiors and the kinds of uses the occupants put on local infrastructure,” says co-author Alex Olson (MIE MASc 2T0), a senior AI researcher at the Centre for Analytics and Artificial Intelligence Engineering (CARTE) at U of T.

“It gives a strong estimate of the resources used in building, maintaining and operating the buildings.”  

Saxe expands, “This is information you can’t get from traditional methods of just looking at maps or plans — you need to see structures. One of the distinctions is we’re predicting what the internal square footage of the building is. And, although obviously that tracks with the size of the outside of the building, it’s actually harder to predict. And you also can’t see how old the building is from the outside.

“If you have experience, you can walk around and say, that building looks about this old to me, this building looks about that old to me and so on. But there’s all kinds of things about it that make it hard, including renovations. The front can be different from the back. And is the frontage brick, glass or is it concrete? Knowing the age of the building is important, as it tells you what materials were used and what embodied carbon there is. And, also, how it performs.”

The ability to see beyond the facades of buildings with AI could help planners better understand the resource needs of cities and prioritize future infrastructure in areas of greatest need. 

“You want to understand where there’s underused resources or infrastructure in your city,” says Olson.

“It sounds like we should already have the data, but we really don’t. With this, while it doesn’t model the future, it does quite accurately describe what the current situation is and allows us to use the data for planning our resource uses and what we want to do in the future.”