Working with generative AI such as ChatGPT has now become part of everyday life. A person submits a query, and the AI produces a response. The more precise the query, the better the result. In the process, the AI is constantly learning, whether through user feedback or through operator-led training of the Large Language Model (LLM) on which the AI is based.
For use in public authorities, such generative AI has three drawbacks: Firstly, it derives its knowledge through training from sources available on the internet – and in a very generalised manner. Secondly, it does not forget knowledge once it has been trained, which has particularly negative consequences when that knowledge quickly becomes outdated. Thirdly, the AI can only provide very general answers to a question if it does not have access to specific context – such as agency-specific working instructions or legal interpretations. The knowledge it has been trained on is sufficient for providing superficial answers to enquiries, but not for more detailed information or even decision-making support for case officers.
RAG: AI incorporates non-public data into its response
This is not the case with a RAG system. With this, the AI also accesses knowledge with which it was not trained, because the underlying data is not publicly accessible, or only to a limited extent. This may include laws, regulations, procedural guidelines, fee schedules or pre-formulated sentence modules – or even particularly sensitive, personal data. Furthermore, the AI does not ‘learn’ from this data, nor does it subsequently incorporate it into its model.
The solution offered by Materna and Elastic ensures that non-public data remains under the control of its owner, whilst still being used primarily to generate a specific response.
RAG thus combines publicly available or pre-trained knowledge with non-public data.
Benefits of RAG for public authorities and citizens
An example: ChatGPT receives the following query: ‘What is the property tax rate?’ The AI would then either engage in a dialogue and ask for more precise information (property tax rate for which town?) or it would generate a result at random. However, if the AI is instructed in advance to access only RAG data (current tables, predefined sentence modules, etc.) from the City of Dortmund when handling enquiries about the property tax assessment rate, it will always generate its response based on this data. This ensures that the information provided is accurate and legally sound. A system running on the City of Dortmund’s server would therefore always take the City of Dortmund’s assessment rate into account, even if the question does not mention the city’s name at all.
When using RAG systems, it is therefore essential to ensure that the underlying data is actually protected and is not made public or used for the general training of the Large Language Model (LLM). This is for data protection reasons, but also for procedural ones. This is because the quality of the RAG-generated responses depends heavily on how well the data is maintained. The datasets must always be up to date. The solution from Materna and Elastic can also be deployed on-premises, i.e. installed and operated entirely within the authority’s secure environment.
Generative AI with RAG could therefore revolutionise communication between public authorities or quasi-governmental bodies such as health insurance providers, as a chatbot on the homepage could already answer enquiries from citizens or customers very precisely and, depending on the data available, even in a personalised manner.
Faster responses, more time for advice
Communication with public authorities is often difficult: long response times, limited opening hours and complicated correspondence are just some of the hurdles. If a public authority automates its communication using RAG, citizens can submit enquiries regardless of opening hours and receive well-founded and legally compliant information immediately.
This benefits both sides: case officers can focus on complex cases and provide personalised advice, whilst applicants with standard enquiries receive their decisions more quickly.
If you would like to find out more about our RAG solutions, please send an email to [email protected]. We will then provide you with a recording of our webcast ‘It’s RAG-Time: Confidently integrating internal government knowledge into generative AI solutions’.
