Comment Letters
Coalition Comment Letter on NIST’s TEVV-Athlon Framework for Evaluating AI Systems
The comment letter, submitted October 6th, responds to the National Institute of Standards and Technology’s (NIST) Request for Input on the initial public draft of the TEVV-Athlon Framework for Evaluating AI Systems (NIST AI 200-2). The framework sets out a general process for the test, evaluation, verification, and validation (TEVV) of AI systems. The letter was signed by the National Fair Housing Alliance, the Center for Democracy and Technology (CDT), the Center for Organizing, the Interfaith Center on Corporate Responsibility, the Sikh American Legal Defense and Education Fund (SALDEF), the Center for Media and Digital Governance at the Open Markets Institute, the National Urban League, and Hispanic Tech & Telecommunications Partnerships (HTTP).
The letter commends the framework’s structure. Its recommendations focus on making the framework work for evaluations that uncover risks no one has identified yet, as well as those that confirm known ones:
1. Broaden the framework’s definition of TEVV beyond checking whether a system meets its design specifications, so it fully supports risk-driven evaluation.
2. Distinguish evaluations of known risks from discovery-oriented evaluations. Show iteration and refinement explicitly through a more complex worked example and through exploratory methods such as qualitative, log-based, and goal-free analysis, which are especially relevant for agentic AI.
3. Add a “Define & Construct” toolbox, at the same level of detail as the existing methods appendices, to guide organizations in defining what they are measuring before they measure it.
The letter also urges NIST to involve affected stakeholders in interpreting findings and setting acceptance criteria, not only in scoping the problem. It asks NIST to clarify whether multi-agent systems are covered, address multilingual and multicultural contexts, and develop domain-specific starter packs for sectors such as housing and lending to provide meaningful support for entities seeking to evaluate within consequential domains.
Coalition Comment Letter to the FTC on Surveillance Pricing
In a comment filed September 25th, the National Fair Housing Alliance (NFHA) and The Leadership Conference on Civil and Human Rights led a 27-organization coalition in urging the Federal Trade Commission (FTC) to ban surveillance pricing outright, responding to the agency’s proposed policy on personalized pricing.
The letter makes the following recommendations to the FTC.
1) Ban surveillance pricing outright using its existing authority under Section 5 of the FTC Act.
2) Short of a ban, require companies to explain why a price was set, including the data used, and give consumers a way to challenge it.
3) Require disclosure when data is collected as well as when that information informs a personalized price.
4) Make clear that enrolling in a loyalty program is not consent to surveillance pricing.
5) Measure personalized “discounts” against what other consumers pay for the same product at the same time, rather than against the seller’s list price.
Coalition Comment Letter on GSA’s Draft AI Procurement Clause: “Basic Safeguarding of Artificial Intelligence Systems”
The comment letter, submitted April 3rd, is in response to the General Services Administration’s (GSA) draft AI procurement clause titled “Basic Safeguarding of Artificial Intelligence Systems.“The updated GSA AI Clause would essentially extend some demands from the Pentagon to a wider array of government contracts, including the “any lawful government purpose” requirement and to require neutral AI tools that are free from ideology such as DEI.
As such the coalition letter makes the following recommendations: On permitted uses, the current “any lawful government purpose” standard is too broad and should be replaced with explicitly enumerated prohibited use cases and protected safety guardrails that cannot be overridden. On bias and civil rights, the so-called “Woke AI” provision should be dropped in its current form and replaced with a concrete civil rights compliance standard. On data protection, existing segregation and training prohibitions should be kept and strengthened, with new protections added for demographic data and mandatory bias auditing requirements. Finally, on surveillance, GSA should assess the risk of government AI configurations spilling over into private sector use, require contractors to certify their systems will not be repurposed for surveillance, and create public notice requirements for any AI monitoring.
Coalition Comment Letter on NIST’s Draft Guidance for AI Evaluations
The comment letter, submitted on March 31st, addressed to the Center for AI Standards and Innovation is in response to NIST’s draft guidance on AI evaluations. 18 organizations signed on, including NCLC, Center for AI and Digital Policy (CAIDP), Center for Democracy and Technology (CDT), and the Electronic Privacy and Information Center (EPIC).
The recommendations in the letter center on the following priorities:
1. Incorporate civil rights principles, including antidiscrimination measures such as disparate treatment and disparate impact testing as explicit requirements for AI evaluation frameworks.
2. Encourage and incentivize the development of domain-specific benchmarks, with particular attention to housing, lending, criminal justice, child welfare, education, and employment as priority domains for benchmark development.