Which AI Model Configuration Fits My News Pipeline? 71 Put To The Test

🔍 Read the full analysis: Which AI Model Configuration Fits My News Pipeline? 71 Put To The Test on ThorstenMeyerAI.com

Buying for a business?Offer from Amazon

Get business pricing on office and shipping supplies

• Business-only prices and quantity discounts
• Tax-exempt purchasing
• Multiple users, one account, clear invoices

Create a free Business account →Free to register

As an affiliate, we earn on qualifying purchases.Hide

Thorsten Meyer says a test of 71 AI model configurations for news writing found GPT-6 Astra produced the highest-scoring articles, while much cheaper GPT-6 Luna was the model he chose for production. The results, published October 13 on ThorstenMeyerAI.com, also prompted him to remove two backup models after judges flagged invented quotes or invalid output.

Meyer tested models from September 8 to October 13 using the same prompt as his production pipeline, with two real news sources: a 6,340-character product filing and a 619-character political news item. Each configuration generated English and German versions. Two AI judges, GPT-6.1 Sol and Claude Opus 5.5, assessed source fidelity, fabrication, clarity about uncertainty, news value, structure and polish; German articles were also rated for language quality.

In the initial test, GPT-6 Astra scored 4.00 out of 5 in both languages and received a 5.00 source-fidelity score. In a follow-up run of 12 articles, it averaged 3.93, with a reported 95% bootstrap interval of 3.83–4.02. GPT-6 Luna averaged 3.67 in that run, with source fidelity of 4.71 and valid output for every article. Meyer reported costs of about $0.12 per Astra article and about $0.0012 per Luna article; the reported generation times were roughly 50 and 17 seconds, respectively.

The test also compared six reasoning-effort settings for Luna. Meyer said the scores did not show a reliable improvement as effort increased: the maximum setting cost about 1.43 cents per article and took 211 seconds, versus 0.12 cents and 17 seconds at the production setting, “none.” He also reported that GPT-4.1 nano scored 1.62 in English and 1.68 in German, with source fidelity of 1.00, while locally run gpt-oss-120b produced no valid articles in four attempts.

Cost and Accuracy in Production

The findings illustrate why a model that wins a writing benchmark may not be the practical choice for a high-volume news pipeline. In Meyer’s results, Astra’s higher scores came with a reported price about 100 times Luna’s. Luna’s lower cost and consistent validity led him to use it, despite its lower average score. That trade-off matters to publishers weighing article quality against the recurring cost and speed of automated production.

The fallback results point to a separate editorial risk: a system may rely on backup models when its preferred model is unavailable, but those alternatives still need to meet the same standards. Meyer said judges identified fabricated quotations in one fallback’s output, while another failed the pipeline’s validity check. A failover chain can preserve output availability, but it can also pass along material that needs to be rejected or reviewed.

The test is a report about one publisher’s workflow, sources and evaluation method, not a universal ranking of AI writing systems. Its practical message is that publishers can measure whether their own models preserve attribution, avoid unsupported detail and return usable formatting, rather than relying only on general-purpose leaderboards.

How Meyer Ran the Test

Meyer’s benchmark focused on a specific task: turning a supplied news source into an article using his production prompt. He included API and subscription models, free-tier options accessed through OpenRouter and open-weight models running on his own hardware. The test covered both English and German editions, intended for global readers and for audiences in Germany, Austria and Switzerland.

The 619-character source was meant to reflect the short inputs that Meyer said account for much of his news work. He also ran a robustness test in which the strongest configurations produced ten additional articles, mostly from short sources. Meyer cautioned that differences below about 0.3 points in the two-article test could be noise, and said he considered the larger follow-up more reliable. The comparison therefore reflects a defined set of prompts and sources, not every news format or model use case.

Meyer said he removed GPT-4.1 nano and gpt-oss-120b from his news route on October 7. The source material identifies GPT-6 Luna as the production choice after that change, although it does not specify the full current failover configuration.

“For a high-volume task, the right model is the cheapest one that clears your quality bar reliably, not the one at the top of the chart.”

— Thorsten Meyer, in the October 13 report

Limits of the Model Comparison

The results come from Meyer’s own pipeline, a limited set of source material and an evaluation by two AI judges. The report does not establish how the same configurations would perform across other publishers’ prompts, a wider range of beats, longer investigations or different editorial standards. The judges’ scores are also evaluations, not independent proof that every article is accurate.

The reported intervals and robustness results offer more information than the initial two-article comparison, but the test remains bounded. Meyer said differences under about 0.3 points in the smaller run should be treated as noise. It is also unclear from the supplied report what checks, if any, human editors apply to generated articles before publication, or what the current backup sequence is after the two weaker models were removed.

Monitoring the Production Route

Meyer’s report says GPT-6 Luna is the production model after the October 7 removal of the two fallback systems. It does not announce a new testing date or name a replacement backup model. For now, the next practical step described by the results is ongoing monitoring: checking that Luna continues to produce valid articles, preserve source details and meet the publisher’s quality threshold as models and production needs change.

Any future ranking will be most useful if it retains the features of this test that connect it to actual use: real sources, the production prompt, both language editions, validity checks and follow-up examples. Meyer’s findings show that the cost and performance balance can change by task, so a new model’s general benchmark standing alone would not establish whether it belongs in this news pipeline.

Key Questions

Which model scored highest in the test?

GPT-6 Astra scored 4.00 out of 5 in both English and German in the initial test and averaged 3.93 across 12 articles in the robustness run, according to Meyer.

Why did Meyer choose GPT-6 Luna for production?

Meyer reported that Luna scored 3.67 in the robustness run, produced valid output for every article and cost about $0.0012 per article, compared with about $0.12 for Astra. He chose Luna as the lower-cost option that met his quality bar.

Did higher reasoning effort improve the articles?

Meyer said the differences across Luna’s six reasoning-effort settings were within the test’s noise. The maximum setting cost about 1.43 cents per article and took 211 seconds, compared with 0.12 cents and 17 seconds at the no-effort setting.

What happened to the fallback models?

Meyer said he removed GPT-4.1 nano and gpt-oss-120b from the news route on October 7. In his tests, judges flagged invented quotes in GPT-4.1 nano output, while gpt-oss-120b produced no valid articles in four attempts.

Do these results establish the best model for every newsroom?

No. The report tests Meyer’s prompt, sources, languages and pipeline. Its rankings do not establish how the models would perform under other newsroom workflows or editorial requirements.

Source: ThorstenMeyerAI.com