In 2021, a leading government department responsible for supporting executive decision-making needed to better understand public sentiment around COVID-19 regulations and government initiatives. While they had implemented a feedback system on their information pages where citizens could rate content with thumbs up/down and provide comments, they lacked the capability to process and analyse this valuable data effectively.
The department's strategic priorities centred on collaboration, accountability, integrity, and innovation to boost the state's prosperity. A key focus was applying data insights to enhance policy development, service delivery, and performance measurement. However, without proper analysis of citizen feedback, they were missing crucial insights that could help align their policies with public sentiment and improve their communication during the pandemic.
The Solution
Our team developed a comprehensive data solution that combined expertise in Data Architecture, Data Engineering, Data Science and Business Intelligence. We created an automated API and data pipeline to ingest user feedback data in near real-time, leveraging the power of Google Cloud Platform technologies including:
- Google Cloud Functions
- Google Natural Language AI
- Google BigQuery
- Google Data Studio
- Google Apigee API Management
The solution incorporated advanced natural language processing to automatically generate numerical scores representing citizen sentiment. These insights were visualised through an intuitive dashboard that used graphics to symbolise positive, negative, and neutral feelings, making data interpretation accessible for all stakeholders.
Innovation in Action
A standout feature of this project was the implementation of large-scale sentiment analysis combined with topic modelling. The system could identify emerging hot topics in citizen feedback, enabling the department to respond rapidly by publishing targeted information addressing public concerns.
The data pipeline was optimised for real-time processing, ensuring that citizen sentiment analysis was continuously updated on the reporting dashboard. This immediate feedback loop allowed the department to gauge where to focus their information publishing efforts, resulting in a more responsive and citizen-centric approach to public communication.
Measurable Impact
The project delivered several key outcomes:
1. Real-time Data Processing: Successfully ingested all feedback data with zero loss in near real-time, enabling large-scale sentiment analysis.
2. Improved Content Strategy: The department could assess and modify their guidelines and policies based on quantifiable public sentiment.
3. Enhanced Citizen Experience: By identifying and responding to emerging topics of concern, the department could proactively address information gaps and clarify guidelines.
4. Digital Transformation: The solution represented a significant step forward in the department's digital journey, establishing a data-driven approach to citizen engagement.
Beyond Initial Scope
The implementation of topic modelling capabilities exceeded initial project expectations, providing the department with deeper insights into recurring themes in public feedback. This additional layer of analysis enhanced their ability to:
- Identify trending concerns
- Prioritise content development
- Allocate resources effectively
- Improve the overall quality of public information
Technical Evolution
Since the initial implementation, the technology landscape has drastically changed, with the introduction of cutting-edge AI capabilities. In today’s environment we would take a different technical approach to solving this customer’s challenge. We would leverage an advanced Large Language Model (LLM) and multimodal AI processing, enabling a more nuanced sentiment analysis that can understand context, emotional undertones, and cultural references. The architecture could also be enhanced with edge computing for privacy sensitive processing, a real-time vector database for efficient pattern matching and now all processing can occur within the one cloud platform. This modernised approach would decrease false positives, and processing costs through optimised LLM usage. The hybrid architecture would combine traditional sentiment analysis with modern AI capabilities.
This approach could also be applied across industries such as Retail and Banking for some of the following:
- Product Feedback Analysis
- Customer Service Enhancement
- Service Quality Monitoring
- Risk Management
Looking Forward
This project demonstrates how advanced data analytics can transform government communication and citizen engagement, particularly during crisis periods. By establishing a robust feedback loop between citizens and government, the solution has created a foundation for continuous improvement in public service delivery and communication effectiveness.
The success of this initiative has set a new standard for data-driven decision-making in government communications, showcasing how technology can bridge the gap between policy makers and the public they serve.