Professor Exposes Drivers of San Francisco’s Housing Crisis

With its majestic hills, mystic fog and magical afternoon light, San Francisco is one of the most photogenic and expensive cities in the world, with the median home price hitting $1.4 million in 2023.

It’s a city that Moira O’Neill, an urban planning and local government law scholar at the University of Virginia, knows well, even though she was born and raised on the other side of the bay, in Oakland. These two California cities have been in the media for years because of their rising housing costs and the debate around the relationship between housing supply, gentrification and displacement. But the cities are at the center of O’Neill’s research about what drives regional and neighborhood-level inequality.

These topics aren’t abstract for O’Neill. She grew up poor, in a city much less affluent than its neighbors in the region, and went to public schools that were de facto segregated. For that reason, understanding how law can remedy or perpetuate inequality has long been a personal mission for O’Neill — an effort that culminated last week in the heralded release of a California-funded investigative research report that prompted the state to require the city to overhaul a zoning and permitting process that has stymied housing construction.

As O’Neill told The New York Times after her report was released, San Francisco has progressive zoning laws on paper, but its actual practices — which her report uncovered — have resulted in a city that excludes middle- and lower-income workers.

“It’s a progressive city, but there’s this contradiction,” she said in the Times article. “It’s really, really important to highlight not just for California, but for the country, how it’s possible to use procedural rules to be exclusive and block the ability to house people.”

Over the years, pundits and politicians have thrown out various explanations for the astronomical home prices in San Francisco, from generally blaming liberal policies to more specific issues such as the booming tech economy, the cost of labor and building materials, and the expense of addressing environmental regulations.

That last claim was a tipping point for O’Neill, whose scholarship harnesses empirical and data-driven evidence to support state and local government efforts to address both climate change and inequality. Now holding a joint faculty appointment in UVA’s School of Architecture and School of Law, O’Neill still serves as an associate research scientist at the University of California, Berkeley, where she also previously taught at the College of Environmental Design and at the law school.

When she began working as a researcher at Berkeley in 2012, O’Neill took a close look at how other fields — like urban planning and public health — understood and evaluated law. That’s when she began designing a project to use more detailed data and interdisciplinary methods to tease out the details of how state and local law shape housing development patterns in very different cities.

In 2016, she started talking about her work to a colleague at Berkeley Law, Eric Biber, who taught environmental law. “I told him about my concerns and said, ‘Here’s why this matters to me,’” O’Neill said. “‘Because urban economists are saying that it’s environmental law in California that’s blocking housing, and from working in this space, I don’t think it’s that simple and I don’t think we have the right data to fix anything.’”

Biber agreed, but he didn’t need convincing to join her efforts. “He had his own reasons for wanting to launch this work,” O’Neill said.

Working with a law student, they launched an initial study in San Francisco. “So I had partners — we still had no money, but we had a team,” O’Neill laughed.

By 2017, a program officer at the Silicon Valley Community Foundation took notice of their work and provided O’Neill and Biber their first grant to fund expansion of the work into four neighboring cities in the region. O’Neill and her colleagues released their first working paper in February 2018, sharing initial findings from the dataset that would become known as CALES, short for Comprehensive Assessment of Land Use Entitlements Study.

Within months, the paper’s findings would show up in the analysis supporting legislative reform to close loopholes in existing state housing law. “Now we had a proof of concept on what precise, detailed data could do for legislative reform.”

Other funding then showed up, including money from the California Air Resources Board.

Over the next several years, O’Neill and her colleagues at Berkeley continued to collaborate on papers looking at the CALES data from different angles — but always with consistent methodology. In one instance, the CALES data showed that local governments were choosing to write their local law to trigger the state environmental law that other scholars said blocked housing construction. In another, the CALES data showed that state climate policy — often blamed for displacement — could not overcome the challenge of local exclusionary land use regulation.

Earlier this year O’Neill was able to use the data to explore the effects of recent state legislation that requires cities that aren’t meeting their housing production targets to approve “qualifying” affordable developments. Qualifying developments are those where a specified amount of the units are priced below market rate, where the proposed project will be built on parcels that are not environmentally sensitive and where they meet other special labor criteria.

“If they conform to all the city and state rules, you must say yes,” O’Neill said. “For those developments, you can no longer apply a discretionary local review process.”

The point of removing a locale’s discretion for otherwise compliant projects is to prevent well-organized — and well-heeled — neighbors from blocking affordable or mixed-income development by raising a ruckus at zoning and planning hearings, she said.

The work that led to last week’s eventful release began last fall, when California’s Department of Housing and Community Development launched an investigation into San Francisco’s land use policy and practices. The agency provided O’Neill a grant to expand CALES in San Francisco and analyze the data to identify barriers to housing approval and affordable housing construction. O’Neill was also tasked with checking to see whether San Francisco’s processes were consistent with California’s housing law, which requires cities to zone and plan to provide their fair share of affordable housing to the region. That law keeps home rule intact, but it asks cities to equitably spread both the burdens and the benefits of housing construction.

“The state’s newest changes to this area of housing law do more to task cities with revising local laws that have essentially walled off neighborhoods from multifamily housing, in order to relieve the burden on the neighborhoods that have carried the load for so long,” O’Neill said. “This is one of the ways cities create opportunities for social and economic mobility — by making sure that poor people aren’t concentrated in one part of the city.”

The report released last week showed how San Francisco has maneuvered around California’s housing law — which applies to its local zoning and planning law — through its local law, including a provision in its business and tax code. That provision requires discretionary review of all permits of any type.

“There is no proposal to do anything that is not subject to discretionary review,” O’Neill said. “If you want to do anything, even build a deck — there’s always the element of notification, and neighbors and ‘interested parties’ can just request a hearing on it.”

O’Neill calls it “process to an extreme.”

O’Neill and a team of researchers — which included Tim Dodson, a third-year UVA Law student — also looked at every detail of how San Francisco’s process unfolds in practice. Some of what she found came through the CALES data set. Other insights came in through interviews because informal conditions for approval were rarely referenced in hearing transcripts and documents.

“What developers, planners and housing advocates in the city shared is that the process allowed the city to impose conditions for approval that are not codified, that are not enumerated in writing, that are not predictable and that come up in between hearings,” O’Neill said. “That’s what people are reporting and it’s very hard to track whether the city is imposing conditions that are not allowed under state laws, because it’s happening outside of the formal public notice-and-hearing context.”

The research resulted in a 94-page academic report with statistical analyses, mapping and comparative charts. The biggest takeaway, O’Neill said, is that the housing approval process in San Francisco makes it hard to create any kind of new multifamily housing, affordable or otherwise.

“In other words, San Francisco is exclusionary — despite having inclusionary laws on paper, such as an inclusionary housing program, tenant protections, and a lot of land zoned for dense residential use compared to other cities,” she said.

This arduous process has meant it takes developers two years, on average, to get the initial green light for developments that already comply with all of San Francisco’s local law, or “code compliant” development. San Francisco then self-reports to the state that it takes another 605 days, on average, to get building permits for specific items like plumbing and electrical hookups. The unpredictable timeframes have driven some developers out of the market and deterred others from entering, she said.

The state is now mandating 18 specific actions the city must take, including eliminating the right of any individual to object to projects that comply with city rules and speeding up building permits once a project is approved. The state’s report also provides another 10 recommended actions.

According to the state report, if the city doesn’t make the required changes within the specified timeframes, California could withhold state funding and revoke local control over development in San Francisco.

While the San Francisco project had personal relevance for O’Neill, she said she hopes to be part of building a more equitable and sustainable future for Charlottesville, Albemarle and the South generally — a place where her husband, Malo Hutson, the dean of UVA’s School of Architecture, has family ties.

“I believe in research that helps local and state governments implement policy to tackle climate change and inequality — that’s everything I do in my research,” she said. “I love teaching land use law and state and local government law, because of how relevant it is to our daily lives. It may not be the area of law you think about first when you come to law school, but the fact is, your day-to-day experience is deeply impacted by state and local laws that affect your choice of schools, how you get to work and your ability to pay for your housing.”