SCS Tool Improves Community Buy-In for Policy Decisions

May 1, 2025

Community policies, such as those for zoning or parking, are typically made at the top, which can leave community members feeling frustrated or left out of the process. Researchers at Carnegie Mellon University's School of Computer Science (SCS) developed the web-based tool PolicyCraft to help community members use concrete cases to collaboratively craft policy. The research team found that policies created with PolicyCraft received stronger support and greater consensus.

"We know people use concrete examples or cases to reason about policies, like zoning regulations. But there is a lack of systematic processes for doing this collaboratively," said Tzu-Sheng Kuo, a doctoral student in CMU's Human-Computer Interaction Institute (HCII).

The SCS research team included Kuo; Jane Hsieh, a doctoral student in the Software and Societal Systems Department; Haiyi Zhu, the Daniel P. Siewiorek Associate Professor in the HCII; and Kenneth Holstein, an HCII assistant professor. The team also collaborated with Quan Ze Chen and Amy Zhang from the University of Washington.

A community, organization or board can use PolicyCraft to collaboratively develop a policy or guideline for what should or shouldn't be allowed.

Even when people share values, they'll have different feelings about policies. Scenarios are often used in policy discussions to make abstract topics more manageable and help people get to the root of disagreements.

"We already know these concrete cases are important to help people reason through complex ideas," Kuo said. "But we needed to create a systematic process for reasoning through cases collaboratively. That's why we proposed PolicyCraft's workflow. Overall, we found that groups that use PolicyCraft develop policies that receive stronger community consensus."

The tool's workflow has three crucial steps: propose, critique and revise.

The process works as follows. First, someone proposes a policy. For example: Residents are allowed to build additional dwelling units (ADUs) on their properties. Then, community members access PolicyCraft and critique the proposed policy, pointing out ambiguities or potential issues. These critiques take the form of scenarios that could arise under the proposed policy. In this example, one scenario would be that a resident builds multiple ADUs to rent on their property. PolicyCraft users can then label the scenario as allowed under the current proposed policy, and then vote and share their reasoning about why the scenario should be allowed or disallowed, regardless of what the current policy says. For example, multiple ADUs may be allowed on one property under a current policy, but someone might object to it because of the strain it places on parking resources.

As people use concrete scenarios to propose and critique a policy, any issues with the policy become clear and it can be revised with the community's feedback. For the ADU policy example, an acceptable revision may be that only one ADU is allowed per property.

To see how people use PolicyCraft and understand its impact, researchers asked students to use the tool to develop policies around which uses of generative AI should be allowed in class. The team found that policies created with PolicyCraft received stronger support and greater consensus.

"PolicyCraft allows people to ground their discussion based on concrete cases, so they can understand why people agree or disagree with each other. We believe this ultimately leads to people developing a policy that they better support collectively and that they have a greater consensus on," Kuo said.

Researchers are examining how PolicyCraft can be used in different situations with nonprofits or companies in other countries. This work was presented at the Association for Computing Machinery's Conference on Human Factors in Computing Systems (CHI 2025) this week in Yokohama, Japan.

For More Information
Aaron Aupperlee | 412-268-9068 | aaupperlee@cmu.edu