John Schulman said AI alignment should become public science, and Mark Chen discussed the self-accelerating progress of AI research and development.
BIBIBI
AT A GLANCE
John Schulman advocates for making AI alignment a public science, and increasing public participation in shaping AI values.
Article
OpenAI Co-founder John Schulman: AI Alignment Should Become Public Science, Not Be Dominated by a Few Tech Companies
Breaking AI News Flash: John Schulman, co-founder of Thinking Machines and OpenAI, stated on the A16Z podcast that AI alignment should not be the exclusive research of a handful of tech companies, but rather develop into a global public science, enabling more people to participate in shaping AI values and avoiding over-reliance on the judgment of a few companies.
As recursive self-improvement (RSI) gradually makes progress, AI has begun assisting in model development. OpenAI Chief Research Officer Mark Chen revealed that early signs of an RSI acceleration effect have emerged in the AI development process, with current progress at about 25% to 50%, but AI still struggles to independently determine valuable research directions and has not yet achieved independent development of next-generation models.
Mark Chen also warned that as models become more autonomous, prolonged operation could lead to unintended secondary objectives if their goals conflict with human intentions. However, more powerful AI may also help researchers monitor model behavior and detect and fix alignment issues.
John Schulman emphasized that AI alignment is not only a technical issue but also concerns who decides the values that AI should follow, and that users and society should have more space to participate in and shape AI values.
Reports quote John Schulman's views on the A16Z podcast: AI alignment should not be dominated by a few tech companies, but become a global public science.
02
John Schulman emphasized that AI alignment is not just a technical issue, but also involves who decides the values that AI should follow.
03
According to Mark Chen, the report states that early signs of a recursive self-improvement acceleration effect have appeared in the AI research and development process, with progress at around 25% to 50%.
04
Mark Chen said that AI still has difficulty making independent decisions about valuable research directions, and self-developed next-generation models have not yet been achieved.
05
Mark Chen warns that as model autonomy increases, if the goals conflict with human intentions, prolonged operation might produce unexpected secondary goals. More powerful AI is also expected to help monitor model behavior and fix alignment issues.
AI-assisted interpretation
The following is analysis, separate from reported facts. Verify important claims independently.
Here, AI alignment focuses on whether AI’s goals and behaviors align with human intentions, and who has the authority to determine the values it should follow. Schulman advocates for greater involvement of users and society, rather than relying solely on a few companies. The report also outlines progress in the development of AI-augmented models, but makes it clear that this does not yet mean AI can independently develop next-generation models.
Why it matters to readers
This discussion also involves the decision-making power over AI development capabilities and values: who participates in setting goals, and how the models are monitored after autonomy is enhanced.
A distinction should be made between "AI-assisted R&D" and "AI-driven R&D," and a distinction should also be made between the advocacy for public participation in AI governance and the systems that have already been implemented.
Risks and unknowns
The material is a media retelling; the original podcast content was not provided, so the full context of the remarks cannot be verified.
The measurement target, calculation criteria, and basis for “progress of approximately 25% to 50%” are not specified; therefore, it cannot be interpreted as meaning that the independent R&D capability has been completed by a corresponding proportion.
How the public can participate in shaping AI values—no specific implementation plans are provided in the article.
Related Developments
Loading event timeline…
The unexpected secondary target is a risk warning, and AI helping to fix alignment issues is a possibility statement, not a verified result.
Related concepts
AI alignment
This term is not in the glossary yet. Browse related concepts in the glossary.