US Government Says China's Best AI Models Lag Behind. Experts Aren't So Sure

The US National Institute of Standards and Technology (NIST) evaluated DeepSeek V4 Pro and concluded that Chinese AI models lag behind US counterparts, but the methodology has drawn significant criticism. Experts question the use of private benchmarks and a cost-comparison filter that conveniently excluded all US models except GPT-5.4 mini, suggesting the evaluation may be politically motivated rather than scientifically rigorous.

The NIST CAISI evaluation represents a notable instance of potential methodological bias in AI capability assessment. By employing private benchmarks inaccessible to independent verification and applying a cost-comparison filter that systematically excluded competitive US models from the analysis, the evaluation raises fundamental questions about scientific objectivity in geopolitical technology assessment. This approach mirrors broader patterns where government agencies shape narratives around US technological superiority during intensifying US-China competition.

The controversy reflects deeper structural tensions in AI evaluation. Public benchmarks like MMLU, HumanEval, and others would provide transparent, reproducible results, yet NIST opted for proprietary metrics. The selective inclusion criteria—keeping only GPT-5.4 mini for comparison—appears designed to produce predetermined conclusions rather than genuine capability assessment. This methodology matters because governments, investors, and companies rely on authoritative evaluations to make strategic decisions about resource allocation and technology adoption.

The market implications are substantial. If Chinese AI models are incorrectly characterized as inferior through flawed analysis, Western companies may overestimate their competitive advantages, potentially slowing innovation and investment. Conversely, if the evaluation was intentionally skewed, it damages the credibility of US scientific institutions and fuels legitimate skepticism about official technology assessments. Investors in AI infrastructure, compute resources, and related technology stocks face increased uncertainty about which models truly represent the technological frontier.

Moving forward, independent researchers and technology firms should conduct their own rigorous evaluations using publicly available benchmarks. The AI industry requires transparent methodology and reproducible results to function effectively in an increasingly polarized geopolitical environment.

• →NIST's evaluation used private benchmarks and restricted comparison criteria that excluded most US AI models, raising objectivity concerns
• →The methodology appears designed to produce predetermined conclusions rather than conduct genuine capability assessment
• →Critics highlight that public benchmarks would provide transparent, reproducible results unavailable in this evaluation
• →Flawed evaluations may mislead investors and companies about competitive positioning in the AI technology landscape
• →Independent, transparent assessments using standardized benchmarks are essential for credible technology comparison