Two new research papers published this week illuminate a sharpening double-edged dynamic in artificial intelligence: large language models are now capable of generating PowerShell malware that closely mimics real-world attacks, while separately, a novel detection framework shows promising results in identifying AI-generated text even when adversaries actively try to obscure it.
Researchers from Federico II University of Naples have released an experimental framework demonstrating that open-weight large language models, when fine-tuned or prompted appropriately, can produce PowerShell malware that bears a striking resemblance to genuine malicious code observed in the wild.
The study, authored by Luciano Pianese, Vittorio Orbinato, Pietro Liguori, and Roberto Natella, found that LLM-generated malware triggered operating system events nearly identical to those produced by real threats — recording a median Jaccard similarity index of 84.5%. In nearly half of all cases tested, the overlap was complete, meaning the AI-generated code was behaviourally indistinguishable from authentic malware.
The team developed a sandbox-based dynamic analysis environment specifically designed to evaluate AI-generated malicious scripts. They also released a manually curated dataset of real-world PowerShell malware, annotated in plain language, intended to support future training and benchmarking of language models in a security research context.
PowerShell is a widely used scripting language built into Windows, making it a frequent vehicle for cyberattacks including ransomware deployment, credential theft, and lateral movement across corporate networks. The findings raise urgent questions about how easily threat actors — including those with limited technical expertise — could weaponise commercially available or open-source AI tools.
"Generative AI has emerged as a significant cybersecurity threat, with several recent attack campaigns leveraging LLMs to generate code for malicious purposes," the authors wrote, noting that understanding these offensive capabilities is now essential for defenders.
On the defensive side of the equation, a separate team of researchers has introduced Triospect, a detection framework designed to identify AI-generated text even when it has been deliberately manipulated to evade existing detectors.
Authored by Guangsheng Bao and colleagues, Triospect approaches detection from three angles simultaneously: statistical patterns, core content or ideas, and stylistic expression. By combining these dimensions, the system proved significantly more resilient than current baseline detectors when tested against 17 distinct adversarial attack methods, across 12 domains and 17 source AI models.
On the Humanize-16K benchmark — a dataset of texts specifically altered to fool detectors — Triospect improved detection accuracy by 22.3 percentage points in AUROC (a standard measure of classifier performance) and by 13 percentage points in true positive rate. On the adversarial RAID benchmark, it achieved gains of 9.1% and 22% respectively.
The researchers have released both their code and data publicly on GitHub, inviting further scrutiny and development from the research community.
Taken together, the two studies underscore a rapidly escalating challenge for cybersecurity professionals: as AI becomes more capable of generating harmful content and code, detection tools must evolve in parallel — and the race between offence and defence shows no sign of slowing.
Analysis
Why This Matters
- The PowerShell malware findings suggest the barrier to entry for cyberattacks is falling sharply — a motivated but technically unsophisticated actor could potentially use publicly available LLMs to generate functional malware, widening the pool of potential threat actors significantly.
- Organisations relying on behavioural analysis to detect malware may find existing tools increasingly strained, as AI-generated threats can now replicate the OS-level signatures of known malicious code with high fidelity.
- The public release of both the malware dataset and the Triospect detection code raises dual-use concerns, though researchers argue open publication accelerates defensive research more than it aids attackers.
Background
The use of scripting languages like PowerShell in cyberattacks is well established. High-profile incidents including the SolarWinds breach and numerous ransomware campaigns have relied on PowerShell to execute payloads, move laterally through networks, and evade detection. PowerShell's deep integration into Windows administration makes it difficult to block outright without disrupting legitimate operations.
The emergence of large language models capable of generating functional code — beginning with tools like GitHub Copilot in 2021 and accelerating with the public release of ChatGPT in late 2022 — introduced new concerns about AI-assisted malware development. Early warnings from security researchers suggested LLMs could lower the skill threshold for writing malicious scripts, and anecdotal reports of threat actors experimenting with AI-generated code began surfacing in 2023 and 2024.
At the same time, the proliferation of AI-generated text has prompted a parallel arms race in detection. Early detectors based on perplexity and statistical token patterns proved vulnerable to paraphrasing attacks and dedicated "humanisation" tools, pushing researchers toward more sophisticated multi-dimensional approaches like Triospect.
Key Perspectives
Security researchers: The Naples team argues that studying LLM-generated malware in controlled conditions is essential for building better defences. Publishing datasets and frameworks, they contend, gives defenders a head start over attackers who are likely already experimenting with these tools.
AI detection researchers: The Triospect team sees multi-dimensional analysis — combining statistical, semantic, and stylistic signals — as the most promising path to robust AI-text detection, particularly as single-feature detectors continue to be defeated by adversarial manipulation.
Critics and sceptics: Some security professionals argue that publishing curated malware datasets and detailed generation frameworks, even in academic settings, provides a meaningful uplift to malicious actors. Others caution that benchmark performance does not always translate to real-world detection reliability, and that determined adversaries will continue to adapt.
What to Watch
- Whether AI safety policies at major LLM providers — including OpenAI, Meta, and Mistral — prove sufficient to prevent fine-tuning or prompting of their models for malware generation at scale.
- Regulatory responses: the EU AI Act and emerging US cybersecurity executive orders may introduce new obligations around dual-use AI capabilities, potentially affecting how open-weight models are released.
- The uptake of Triospect and similar multi-dimensional detectors by platform operators, academic publishers, and enterprise security vendors as adversarial AI-text attacks become more common.