Managing Boston effectively using data has been a central focus for Mayor Marty Walsh since he took office in January of 2014.
In Boston, when an ambulance arrives at an address in under six minutes, when a resident’s call to 311 is answered in less than 30 seconds, or when a restaurant inspector finds a health code violation before anyone gets sick, there is one common thread — Boston’s Citywide Analytics Team. The team is on a mission to improve the way the City of Boston delivers services using the power of data. Now in its third year, the team has grown significantly, worked with nearly every department, and launched dozens of successful projects to improve life in the city.
Managing Boston effectively using data has been a central focus for Mayor Marty Walsh since he took office in January of 2014. He hired Jascha Franklin-Hodge as Chief Information Officer (CIO) in June of 2014 and formally announced the creation of the Citywide Analytics Team in his 2015 State of the City address. According to Matt Mayrl, former Deputy CIO and Director of the Citywide Analytics Team, the concept for the team “grew out of the mayor’s commitment to both get better data for him to make decisions, and the realization that he needed to start equipping the rest of the organization with data-driven, actionable information in order to realize the full range of organizational change he envisioned.” The team’s origins and growth hold lessons for other cities seeking to start their own analytics efforts.
THE ORIGINS OF THE CITYWIDE ANALYTICS TEAM
Mayor Walsh began moving the city toward his data-driven vision with the appointment of Franklin-Hodge as CIO, joined by Mayrl, previously the chief of staff of Boston’s Department of Public Works. Franklin-Hodge and Mayrl were tasked with building out the team to develop the city’s data capacity, which they undertook with thoughtful consideration. Franklin-Hodge wanted to assemble a team that had “the skills to both do the technical heavy lifting and the organizational heavy lifting – building relationships, understanding need, and understanding where data could be an effective lever for empowerment and transformation.” Because they were starting a new effort for a city with over 16,000 employees across 42 departments, they knew their team needed to be able to work with departments at very different levels in their data use and understanding. They hired business analysts, visualization experts, and data scientists in addition to bringing in the city’s existing open data and performance management efforts. That January, the team — at the time only six people — moved physically to the same office.
The analytics team built on several notable efforts of the prior administration under Mayor Tom Menino, which included the pioneering Citizens Connect app (now BOS:311), the Mayor’s Office of New Urban Mechanics (which still operates as the mayor’s civic innovation group), and the Boston About Results (BAR) performance system. Chris Dwelley, Director of Performance Management, said the impetus for the new structure “was a realization that we were not going to be able to accomplish what we wanted to with a two- to- four person, fragmented team. We needed to leverage existing resources while developing our strategy and building out the team.” The Citywide Analytics Team united different people and projects working to improve performance and leverage data and enabled them to streamline their efforts toward a common vision for a data-driven Boston.
Many new hires in the first year were chosen because of experience working with customers, broadly defined, which the team’s leaders viewed as the necessary experience for working with departmental clients of the analytics team. The Citywide Analytics Team was initially structured with project managers in specific verticals that the team was addressing, in addition to a group of business analysts working on any project that needed them and three purely technical staff members. The mix of positions has evolved over time based on the needs of the team.
The Citywide Analytics Team is based in the Department of Innovation and Technology (DoIT), a location chosen because of the resources available in the department as well as the existing relationship DoIT had with much of the city’s data. Franklin-Hodge noted that the broader IT access lowers technical barriers to getting data and allows the team to surface cross-departmental possibilities. He called the DoIT location “a natural fit” for analytics because the department already had access to many of the systems producing data and could also guide broader technology procurement and use throughout the enterprise.
The analytics team has a strong focus on serving departments collaboratively and helping them solve problems. They choose projects based on requests from departments in addition to ideas nominated by the team. Mayrl said the initial projects that the team took on in 2015 were selected with a few goals in mind — “something that has a clearly visible public impact to begin the process of communicating how data is changing the city; something with a financial impact to demonstrate some financial returns; and something that demonstrates the data team’s interest and willingness to work with all levels of the agency.” The team sought to start with individual projects for departments and then build up to more strategic discussions about how they could help departments achieve their missions. From the beginning, the team has placed a strong emphasis on sustained, in-person engagement with department contacts, which has enabled the successes that they have achieved to date. They also put a high value on working with all levels of the department, from the executive level to front-line staff. Franklin-Hodge said that “often, technologists are bringing an engineering mindset to problems, which is that if we build a better mousetrap, then people will use it. In reality, it’s always messier than that – technology alone rarely moves the needle on big problems. But technology coupled with a nuanced understanding of how it will be used, relationships that allow people to approach change with a sense of trust and partnership – that is the fundamental difference between success and failure.” This customer-centric mentality has infused the team’s work with departments.
FROM DASHBOARDS TO CITYSCORE
One of the first things the Citywide Analytics Team took on at the mayor’s request was the creation of a dashboard for Boston’s performance. They pulled together an initial version within a week, and have been refining the Mayor’s Dashboard ever since. The dashboard displays daily data about city performance in areas ranging from potholes filled to homicides, with the metrics displayed based on Walsh’s current priorities. Once the mayor began asking questions based on the data on the dashboard, other department and cabinet heads requested similar dashboards to equip them to respond. The subsequent cabinet and departmental dashboards that the Citywide Analytics Team created are tied to the strategic goals of each, such as equity and customer experience. Some departments also use weekly team or individual reports to track their work. Franklin-Hodge said, “the way we are approaching our work is to ask the question, what tools can we put in the hands of managers and employees that will fundamentally help them transform the effectiveness of their teams and deliver better quality of life outcomes for the people of Boston?” He views dashboards as a good way to start a conversation around data — “they can both be interesting and can create controversy – what are you measuring, why are you measuring it that way? Those discussions in the right environment can be valuable ways to engage people around issues of data literacy and priorities.”
As they developed dashboards and thought through the best data-driven ways to manage performance, the Citywide Analytics Team took on a new task — developing a high-level way to measure the city’s progress. They launched CityScore last spring, a performance tool that rolls up standardized scores on key metrics from across city departments into an overall easy-to-understand dashboard that the mayor, department heads, city employees, and the public can all view.
CityScore is distinguished from the city’s prior efforts in two important ways: it is updated daily, enabling quick interventions, and it standardizes the scoring of all metrics to make them easy to understand. Dwelley noted, “A lot of our early performance efforts were more focused on longer-term trends. With daily updates, we can get ahead of situations and prevent them from snowballing into something larger.” For its scoring, CityScore uses 22 city metrics, each of which is based on performance divided by target for that metric, a method which enables them all to be viewed on the same scale. A score of one means the city has met the target, below one means it’s not meeting its target, and above one means the city is exceeding the target. Metrics without targets are scored using standard deviation and a three-year historical average to monitor overall increase or decrease in the metrics.
A beta internal version went live in January 2016 and a public version launched in March 2016. In August, Boston announced an open-source toolkit for CityScore, which several cities are using to generate their own versions.
The CityScore dashboard sits prominently in the office of the mayor as well as his chief of staff, Dan Koh. Koh was a driving force behind the CityScore concept, which he said “has already had a profound effect on the way we do business.” In one example, the mayor noticed that EMS response times were lagging in the CityScore. Upon investigation, he learned that Boston had not upgraded its ambulance fleet to account for a growing population. Now, ten new ambulances are in the budget for the next fiscal year. This discussion also led to a partnership between the Citywide Analytics Team and Emergency Medical Services to conduct in-depth analysis of 911 dispatch data to identify trends, hotspots, and opportunities for targeted improvements. This kind of concrete action is exactly the intention of CityScore. Said Koh, “At the end of the day, this is meaningless unless residents see a difference in how city services are delivered to them.”
CityScore sits on top of all of the city’s other data efforts and will guide future priorities and projects. According to Dwelley, “CityScore ties it all together – if we are being successful in all of our projects, we will be moving the needle on CityScore.” Although departments will naturally work to improve performance on the metrics included in CityScore, Koh said that the culture still emphasizes creativity in problem-solving, but through quantitative as well as qualitative lenses. Departments are working on creative multivariable ways of measuring their performance, which will improve CityScore over time. In addition to sparking these new conversations about performance, CityScore is also leading to skill-building in departments; the Citywide Analytics Team is working to train department staff in Tableau and other data analysis and visualization software to be able to answer questions and explore data beyond CityScore.
“At the end of the day, this is meaningless unless residents see a difference in how city services are delivered to them.”
The launch of CityScore brought new attention to the metrics included in the dashboard and sparked a larger conversation about performance measurement. As a result, the city is currently embarking on a more comprehensive change to its performance management system. Chris Dwelley is leading this effort, which began with a comprehensive review of all 3500 measures tracked in the city. A process of extensive analysis and review with departments has narrowed these measures down to 350. Dwelley said, “the idea is to have fewer measures but have them be much more reflective of the administration’s and the department’s priorities.” He pointed to the utility and participation benefits of fewer, more targeted metrics. Streamlining and unifying the metrics used will ensure that the mayor, cabinet chiefs, department heads, employees, and residents are all looking at the same priority measurements. Said Franklin-Hodge, “By choosing to hold ourselves accountable from the mayor on down, we are creating the correct managerial incentives to make the changes that improve quality of life.”
BUILDING INTELLIGENCE[1]
As part of their ongoing work with departments, members of the Citywide Analytics Team began spending time with dispatchers in the Boston Fire Department (BFD) in 2015 to understand their operational challenges and identify ways that data could be leveraged as a tool for the BFD. They worked together to define the business need for a new hazard data tool that would enable firefighters responding to an emergency call about a building to assess the situation and plan a response. The result: the Building Intelligence System, a web app that integrates seven city data sources to provide a streamlined view of individual buildings in Boston. Firefighters and dispatchers can now easily view permitting, inspection, code violation, and hazard data, allowing them to make better decisions and improve firefighter safety.
Franklin-Hodge noted that two keys to the project’s success were partnership with the department – to ensure the team was solving a core operational problem in a useful way – and the citywide vantage point that enabled the team to integrate data across departments. He described Building Intelligence as an example of a “glue layer” that “sits across multiple departmental IT systems and allows them to be linked together.”
The tool was built in-house in about six months using Google Maps and the ArcGIS REST API. Because it uses the ubiquitous Google Maps, the interface requires little training for users to understand it. Building Intelligence integrates seven sources of data – permits, inspectional violations, assessing building characteristics, building plans, hydrant and firebox locations, and computer-aided dispatch (CAD) data, which includes asbestos, biohazards, and other information collected by BFD. In the process of integrating this data, the team discovered data-quality issues with some sources that lacked Street Address Management (SAM) identifiers, a system developed by the city’s GIS team to ensure uniform address data throughout the city. They were able to correct this while preparing the data sources for Building Intelligence, leading to an increase in data quality and uniformity across departments.
The beta desktop version of Building Intelligence launched in 2016. When someone calls 911 about a fire, dispatchers enter the address into the web app, which generates a map of hydrant and fire box locations with a side panel showing hazard information like building date, land use type, and floors. The dispatcher relays the relevant information to firefighters by radio, but future phases will make this tool directly accessible to firefighters in the field on tablet and mobile devices.
Firefighters in Boston can now respond to emergencies with far more information, helping them to effectively and safely battle fires in the city. In the future, the Citywide Analytics Team hopes to expand the tool to other departments that need full access to building-level information for front-line employees, including building inspectors and police officers. The successful development and implementation of Building Intelligence demonstrates the value of the Citywide Analytics Team’s approach of engaging departments in data projects to ensure that the resulting tools will be used throughout the enterprise and change core daily operations.
THE FUTURE OF THE TEAM
In May of 2016, Mayor Walsh appointed Andrew Therriault as the city’s first Chief Data Officer (CDO), a role with responsibilities including leading the Citywide Analytics Team. With his appointment, the city’s separate GIS and data integration teams became part of the Citywide Analytics Team, which Therriault reorganized into verticals including performance management, open data, GIS, data science, and project development. The team is currently working to hire a group of Analytics Project Managers who will specialize in specific areas like public safety, education, and housing to develop closer relationships with the relevant departments and programs. These specialists will bring in the expertise of the performance management team and data scientists on specific projects and work with departments to improve their own internal data management and analysis capabilities. Therriault is emphasizing full-team meetings, collaborative projects, and cross-training as the team expands to develop a unified data team for Boston.
Therriault hopes to build on the team’s prior successes with more predictive work, particularly in preventable risk. He said, “Whether it’s someone getting sick at a restaurant, or dropping out of school, or having a drug overdose, if we can identify problems before they blow up it is much easier to handle them.” The team is already working with the Inspectional Services Department to prioritize restaurant inspections based on predictive modeling, which has increased the rate of inspectors finding critical violations by 20-25% over previous years. Another current priority area is street safety, part of Boston’s Vision Zero commitment to reducing crashes. A Vision Zero map allows residents to add text descriptions to any point in the city with traffic safety concerns. With data science volunteer group Data for Democracy, the Citywide Analytics Team is working on a way to automatically read these unstructured text reports and combine them with other data about road conditions, traffic, and past accidents to build a model to predict crashes on city streets. The team also recently launched a new open data hub, Analyze Boston, part of the city’s Open Data to Open Knowledge project with the Boston Public Library. The Analyze Boston site was designed to not only provide a repository for Boston’s open datasets, but to also facilitate collaborations between the analytics team, other city departments, and members of the community.
As the team’s successes build, demand for its services has started to exceed capacity. To promote the continued growth of analytics in the city, the team is working to more deeply embed data skills throughout the organization and manage the evolution and sustainability of existing projects. Even as the team has grown and the scope of Boston’s data work has expanded, they haven’t lost sight of how their work contributes to the core mission of the city. Franklin-Hodge said, “What I’m consistently struck by is that when you actually get to talking with people who come in every day to do a job — to issue building permits, to fight fires, to pick up trash off the street, to manage a team of office workers — all of these people bring with them a sense of mission to the work that they do. When you can structure conversations about data around helping them deliver that mission more effectively – delivering cleaner streets, delivering a safer city, delivering a more efficient and friendlier face for businesses that are trying to invest in the town that we all call home — that’s the basis for implementing data-driven change.”