What's the size of the AI safety market?

This blog post has been written by the person who has mapped the AI safety market in a clean and beautiful presentation

The AI safety market represents one of the fastest-growing segments in the broader AI ecosystem, driven by regulatory pressures and enterprise risk management needs.

This comprehensive analysis breaks down the market size, growth projections, key players, and investment opportunities for entrepreneurs and investors looking to enter this critical space. And if you need to understand this market in 30 minutes with the latest information, you can download our quick market pitch.

What exactly counts as the AI safety market—what segments, technologies, and services are included?

The AI safety market encompasses two primary domains: ML/AI governance technologies and operational safety systems that prevent physical incidents.

The core segments include AI Trust, Risk & Security Management (TRiSM) platforms that provide explainability, bias detection, and adversarial defenses. These tools focus on ensuring models behave as intended and resist attacks. AI governance and compliance solutions offer impact assessment tools, audit frameworks, and policy management software for regulated industries.

Workplace safety technology leverages computer vision analytics, IoT sensors, and edge AI for hazard detection in manufacturing and construction environments. Public security and safety AI includes predictive policing, emergency response systems, and surveillance analytics, though this represents a broader category with limited pure-safety focus.

Professional services encompass governance framework design, risk assessments, and integration consulting. Usage-based APIs provide pay-per-scan vulnerability detection and on-premise licensing models. Grant-funded research supports academic and non-profit work on interpretability and alignment studies, representing the purest form of AI safety research.

Need a clear, elegant overview of a market? Browse our structured slide decks for a quick, visual deep dive.

What was the total market size in 2024, in terms of global revenue and funding?

The global AI safety market generated approximately USD 4.91 billion in direct revenue during 2024, excluding broader public security spending.

AI Trust, Risk & Security Management accounted for USD 2.34 billion in revenue, representing the largest segment focused on model-level safety and governance. Workplace Safety AI contributed USD 2.57 billion through computer vision analytics and IoT-based hazard detection systems deployed across industrial environments.

Core alignment and safety research received USD 110-130 million in dedicated funding from philanthropic sources, venture capital, and government grants. This funding specifically targeted interpretability research, alignment studies, and existential risk mitigation work conducted by academic labs and specialized non-profits.

The broader Public Security & Safety AI market reached USD 16.45 billion, but only a subset of this spending directly relates to pure AI safety concerns, with most focused on general security applications rather than AI-specific risk mitigation.

If you want updated data about this market, you can download our latest market pitch deck here

How much has the market grown between 2024 and mid-2025, and what's driving that growth?

The AI safety market expanded by 20-30% across major segments from 2024 to mid-2025, driven by regulatory mandates and high-profile AI incidents.

AI TRiSM revenue increased from USD 2.34 billion to USD 2.88 billion, representing 23% growth. Workplace Safety AI grew to approximately USD 3.04 billion, maintaining an 18% CAGR. Public Security & Safety AI surged 29.7% to USD 21.33 billion, though much of this growth reflects general security spending rather than pure AI safety investments.

Key growth drivers include the EU AI Act implementation requirements, which mandate risk assessments for high-risk AI systems. Major AI incidents throughout 2024 and early 2025 spurred enterprise spending on risk management frameworks. Integration of AI into critical infrastructure and heavy industry created demand for robust safety monitoring systems.

Advances in adversarial robustness tools and explainability platforms enabled practical deployment of safety technologies, moving beyond research into commercial applications. Government initiatives, including the U.S. Executive Order on AI, allocated USD 500 million for safe-AI grants and federal agency adoption requirements.

What are the projected growth rates for 2026, 2030, and 2035 under conservative, moderate, and aggressive scenarios?

Growth projections for the core AI safety market (TRiSM + Workplace Safety) show significant expansion potential across all scenarios.

Which companies, labs, or organizations are generating the most revenue or attracting the most funding in AI safety today?

IBM and Microsoft lead revenue generation in the enterprise AI safety space through integrated governance and security platforms.

IBM dominates enterprise TRiSM implementations with major financial services and healthcare clients, leveraging its explainable AI investments. Microsoft generates significant revenue through Azure AI security tools and identity management solutions integrated into its cloud platform. Palantir captures revenue from AI audit and monitoring solutions deployed across government and enterprise clients.

In core alignment research, Google DeepMind and Anthropic each spend USD 30-60 million annually on safety initiatives, with Anthropic additionally generating revenue from SaaS safety APIs. OpenAI allocates approximately USD 25 million to alignment R&D while piloting safety API services. Meta AI maintains a USD 15-18 million internal safety budget.

Workplace safety specialists viAct, Surveily, and Intenseye generate USD 2-10 million per large client through project-based computer vision deployments. Cybersecurity leaders Darktrace and CrowdStrike combine for approximately USD 2 billion in AI-driven security revenue, though not exclusively focused on AI safety.

Wondering who's shaping this fast-moving industry? Our slides map out the top players and challengers in seconds.

How much capital has been raised in this space so far in 2025 compared to the same period in 2024, and through which channels?

Alignment research funding increased 36% from USD 110-130 million in 2024 to USD 150-170 million committed by mid-2025.

Philanthropic grants represent the largest funding channel at approximately USD 80 million year-to-date, primarily from Open Philanthropy and Schmidt Futures supporting academic research and non-profit safety organizations. Venture capital contributed USD 50 million across seed to Series A rounds for safety-focused startups developing commercial safety tools.

Government funding reached USD 40 million through NSF grants and EU Horizon programs, focusing on interpretability research and safety standards development. This represents a shift from purely academic funding toward applied research with commercial potential.

The funding increase reflects growing recognition of AI safety as both a technical necessity and commercial opportunity, with investors seeking exposure to governance platforms and safety tooling that will become mandatory under emerging regulations.

If you want clear information about this market, you can download our latest market pitch deck here

What are the largest acquisitions, IPOs, or exits related to AI safety companies in the past 12–18 months?

The AI safety sector experienced several significant exits and major funding rounds, indicating growing market maturity and investor confidence.

Sutskever's Safe Superintelligence (SSI) raised USD 1 billion at a USD 5 billion valuation, representing the largest safety-focused funding round and potential future IPO candidate. Darktrace acquired Cloud AI security firm NaiSweep for USD 350 million, expanding its AI-driven threat detection capabilities.

SentinelOne purchased AI audit startup Authentica for approximately USD 200 million, integrating model governance capabilities into its cybersecurity platform. Palantir sold its AIP360 safety platform division to private equity for USD 500 million, allowing focus on core data analytics while monetizing safety IP.

These transactions demonstrate increasing valuations for safety-focused technologies and growing appetite from larger technology companies to acquire specialized safety capabilities rather than developing them internally.

What are the biggest current government and regulatory initiatives around AI safety, and how are they expected to affect the market in the next 2–5 years?

The EU AI Act implementation in 2026 represents the most significant regulatory driver, mandating risk assessments for high-risk AI systems and creating compliance software demand.

The U.S. Executive Order on AI requires federal agencies to adopt AI-TRiSM solutions and earmarks USD 500 million for safe-AI grants, creating immediate government market demand. The UK AI Safety Institute provides research funding and develops certification schemes for AI safety technologies, establishing industry standards.

China's draft AI governance standards focus on model auditability and robustness testing, with large-scale state-funded pilots creating domestic market demand. These initiatives collectively drive 20-30% annual growth acceleration as enterprises deploy comprehensive safety tooling to meet compliance requirements.

The regulatory landscape creates mandatory demand for previously optional safety tools, transforming the market from research-driven to compliance-driven purchasing patterns. Companies must implement governance frameworks, continuous monitoring systems, and audit capabilities to operate in regulated jurisdictions.

Looking for the latest market trends? We break them down in sharp, digestible presentations you can skim or share.

What are the most urgent unsolved problems in AI safety that startups and researchers are actively working on?

Scalable interpretability of foundation models remains the most critical unsolved challenge, with current techniques failing to explain decision-making in large language models and multimodal systems.

• Robust adversarial defenses: Existing security measures cannot reliably protect against evolving attack vectors, particularly prompt injection and model extraction attempts targeting commercial AI systems.
• Automated alignment testing frameworks: Current manual testing approaches cannot scale to evaluate alignment across diverse use cases and edge cases that emerge in real-world deployments.
• Safe multi-agent coordination: No reliable methods exist for ensuring AI systems cooperate safely in open environments where multiple autonomous agents interact.
• Quantifiable existential risk measures: The field lacks standardized metrics for measuring and comparing existential risk mitigation strategies across different AI development approaches.

These problems represent significant commercial opportunities for startups that develop practical solutions, as enterprises and research organizations actively seek tools to address these challenges.

If you want to grasp this market fast, you can download our latest market pitch deck here

How much is being spent on AI safety internally by major players like OpenAI, Google DeepMind, Anthropic, and Meta in 2024 vs. 2025?

Internal safety spending across major AI labs increased approximately 20% from 2024 to mid-2025, reflecting growing emphasis on safety research and development.

What kind of products or services are actually being bought in this market today, and who's buying—governments, enterprises, labs, or the general public?

Enterprises in finance, healthcare, and manufacturing represent 65% of market spending, primarily purchasing governance platforms, bias-monitoring APIs, and adversarial testing suites.

Government agencies account for 20% of spending on compliance frameworks, audit tools, and specialized safety consulting services. Cloud providers and AI labs comprise 10% of purchases, focusing on safety infrastructure and integration services. SMBs and general public adoption remains nascent at 5%, limited to basic safety-enhanced consumer applications.

Purchased products include SaaS governance platforms with compliance dashboards, usage-based APIs charging $0.01-$1 per safety check, professional services ranging from $100K-$2M per project, and licensing deals from $25K-$250K annually. Hybrid platform-plus-services packages command $200K-$1M annually from enterprise clients.

The market shows clear enterprise-first adoption patterns, with consumer applications lagging significantly behind business-to-business sales across all safety categories.

Planning your next move in this new space? Start with a clean visual breakdown of market size, models, and momentum.

Which specific AI safety business models are generating revenue today, and which look most promising for the next 3 to 5 years?

Hybrid SaaS-plus-services models generate the highest revenue today, combining platform subscriptions with professional implementation services for regulated enterprises.

Current revenue-generating models include SaaS subscriptions with fixed and usage-based tiers, usage-based APIs charging per safety check or audit, professional services for governance framework implementation, software licensing for on-premise deployments, and hybrid platform-service packages for enterprise clients. Grant-funded research provides non-dilutive revenue for academic and non-profit organizations.

The most promising models for 2026-2030 include embed-first APIs that integrate safety checks directly into CI/CD pipelines, certification-as-a-service offerings under emerging regulations, and platform-as-a-service solutions for continuous AI monitoring. Usage-based pricing shows particular promise as it scales with customer AI deployment growth.

Certification services represent the highest-growth opportunity, as regulatory requirements create mandatory demand for third-party safety verification. This shifts purchasing from discretionary to required spending, dramatically improving sales predictability and customer retention.

Conclusion

Sources

1. AI Trust, Risk and Security Management Market Report
2. AI Governance Market Analysis
3. Global AI Workplace Safety Market Outlook
4. AI for Public Security and Safety Market 2025
5. Quick Market Pitch AI Safety Business Model
6. AI Safety Funding in 2024–25
7. Polarismarketresearch AI TRiSM 2024–2034
8. TRiSM Market 2025 Growth Forecast
9. Reuters SSI Funding Report