Humane Intelligence and NIST partner to launch a competition aimed at uncovering flaws in AI productivity software
A new government-supported initiative is actively seeking participants to help evaluate AI-enabled office productivity software. The goal is to stress-test generative AI tools, a crucial method for identifying potential biases, inaccuracies, and privacy risks, such as the leaking of sensitive information in AI-generated responses.
Humane Intelligence, a nonprofit focused on AI ethics, has partnered with the National Institute of Standards and Technology to launch a competition aimed at uncovering flaws in AI office productivity tools. The first round of the competition will take place virtually, and is open to anyone in the U.S. interested in red-teaming AI models. Participants will be tasked with running through a series of test scenarios to identify as many “violative outcomes” as possible. Successful participants will be invited to an in-person red-teaming event in late October, held at the Conference on Applied Machine Learning in Information Security in Arlington, Virginia.
In an effort to enhance the testing process, Humane Intelligence is also reaching out to generative AI model operators, encouraging them to donate access to their models for evaluation. Qualifying products must be designed for workplace productivity, such as coding and process automation, and operators must be open to having their models tested for a range of potential issues.
This competition is one of several challenges that Humane Intelligence is organizing in collaboration with government partners and non-governmental organizations in the coming weeks. By broadening the scope of AI red-teaming, Humane Intelligence aims to engage people from diverse backgrounds, not just those with coding experience.
For those interested in taking part in the initial round of the competition, the deadline to sign up is September 9.
