German English Slovak Swedish
Materna IT Service Provid...  |  Insights  |  Blog  |  Using language models sec...
08.07.2025
Blog
Data & AI
Cyber Security

Using language models securely: How to protect AI from manipulation

Large language models such as ChatGPT and Gemini are impressive – but they are not invulnerable. Manipulation through misinformation or hidden instructions poses a real threat. That is why new security strategies are needed. Materna shows how it can be done.

Adobe Stock ©
Eduard Hübner
Experte für IT-Sicherheit

What if your AI solution were to provide systematically manipulated information? That is exactly what can happen – for example, through so-called ‘LLM grooming’ attacks. In such attacks, attackers flood the internet with targeted false information. Language models that access external data uncritically adopt this content. The result: incorrect answers, distorted narratives – and, in the worst-case scenario, misjudgements with real-world consequences.

Backdoor attack: indirect prompt injections

Another point of entry is what are known as ‘indirect prompt injections’. Attackers hide instructions within seemingly harmless content – such as a website or an email. If this information is processed by a language model, it can lead to unexpected behaviour: the AI no longer follows the user, but the attacker.

Traditional IT security falls short here. That is why Materna relies on a comprehensive security concept – from architecture right through to live operation: 

  • Least Privilege: Language models are only granted access to data that is absolutely essential.
  • Input filters: Incoming content is checked for suspicious patterns.
  • Monitoring: Unusual activity is automatically detected.
  • Distributed responsibility: Security mechanisms operate at multiple levels. 

Protective measures in practice

Some tried-and-tested strategies: 

  • Scanning before processing: Specialised models check inputs before they reach the main model.
  • Dual-LLM approaches: one model analyses, the other responds – without direct access to raw data.
  • Static and dynamic code analysis: This enables security vulnerabilities to be identified and rectified at an early stage. 

Stress testing for AI – using real-world attacks

Security testing does not end with go-live. Materna relies on regular penetration tests, for example using NVIDIA’s open-source tool garak. It simulates realistic attacks on AI systems – from simple jailbreaks to complex prompt injections. This is the only way to reliably identify and rectify vulnerabilities.

Conclusion: Security is not a state, but a process

AI can speed up processes and improve decision-making. However, without suitable protective measures, it quickly becomes a risk. Therefore, anyone wishing to use AI securely and responsibly must factor in security from the outset – technologically, strategically and continuously.

AI white paper available for download

Have you seen our AI white paper yet? You can download it here!

Eduard Hübner
Experte für IT-Sicherheit

Eduard Hübner ist ein Experte für IT-Sicherheit, der sich mit der Schnittstelle zwischen künstlicher Intelligenz und Cybersicherheit befasst. Durch seine Forschung trägt er zur Absicherung von KI-Systemen bei, wobei er stets ein angemessenes Gleichgewicht zwischen Sicherheit und Flexibilität dieser Systeme im Auge hat.

write a message

Related articles

Event
Frankfurt am Main
18.11.2026
BMC Helix Roadshow 2026

Meet Materna at the BMC Helix Roadshow 2026 on 18 November in Frankfurt and discover how modern service management, Agentic AI and compliance requirements work together successfully. Look forward to exclusive insights into the BMC Helix roadmap, the…

Read more
Event
Dortmund
05.11.2026
XCS Day (Xchange in Cyber Security)

At XCS Day, experts from the business and public sectors will once again come together to discuss current challenges and strategies relating to cyber security and regulation. Find out how organisations are strengthening their resilience to cyber…

Read more
Event
Dortmund
03.11.2026
FMS Information Day

On 3 November 2026, we will update you on the latest developments relating to the Federal Government’s FMS and encourage mutual exchange. The keen interest you show in every issue demonstrates that the FMS is, and will remain, a key element of…

Read more
Event
Wien
13.10.2026
Agentic AI Roundtable in Vienna

How does Agentic AI deliver real added value for IT and service organisations? At the Agentic AI Roundtable in Vienna, Materna, BMC Helix and other experts will discuss specific use cases, current challenges and next steps for businesses. Look…

Read more
Event
Berlin
13.10.2026 - 15.10.2026
Smart Country Convention (SCCON)

Come and meet us in Hall 27 at Stand 206 at the leading event for the digital state and public services. SCCON is a must-attend event for all stakeholders actively driving and shaping digital transformation. At our stand, you’ll find a use case…

Read more
Event
Amsterdam
06.10.2026 - 08.10.2026
Atlassian Team ’26 Europe

From 6 to 8 October 2026, Atlassian will be bringing the community together for ‘Team ’26 Europe’ in Amsterdam. Look forward to hands-on sessions covering AI, new products and features, live demos and certification opportunities. You’ll also have the…

Read more
Event
Berlin
01.10.2026
Open Insurance Dialog 4.0 (#OID4.0)

Open Insurance Dialog 4.0: FiDA, Identity and New Opportunities for Insurers

The insurance industry is facing a profound transformation: FiDA (Financial Data Access) and Open Insurance are creating new opportunities for data-driven services,…

Read more
Event
Online
30.09.2026
Webcast: ‘Understanding the mainframe, accelerating migration: How banks and insurers are harnessing hidden knowledge’

Mainframe and legacy systems contain decades of business logic – often inadequately documented and difficult to access. From 10.00 to 11.30, Materna and Nomain will demonstrate how AI-powered analyses can make business logic, dependencies and…

Read more
Event
Düsseldorf
30.09.2026
27th NRW ÖV Symposium

Artificial intelligence, digital sovereignty, the cloud, the data economy and cyber security form the framework for the digital transformation of public administration in North Rhine-Westphalia. This popular platform for information and dialogue on…

Read more
Event
Public Sector
Online
24.09.2026
Webcast: ‘From a jungle of registers to a digital ecosystem.’

How register modernisation, the ‘once-only’ principle and NOOTS are bringing about lasting change in public administration. Find out what challenges public authorities face, which strategies are proving effective, and how a future-proof register…

Read more

Materna Information & Communications SE

Digital matters.

We are here for you

Robert-Schuman-Straße 20 44263 Dortmund
+49 231 559 9 - 00
Contact us
Customer Service
© Materna 2026
Imprint
| Data Privacy
| Data protection settings