Automated content generation Agent for MediaWiki using AI and LangChain technology.
September 1, 2024
5 minutes read
SUMMARY
Our client is a consulting company specializing in MediaWiki. Their main role is to help organizations manage and structure content on the MediaWiki platform. The company’s clients range from small businesses to large corporations that use MediaWiki for organizing internal documentation, knowledge bases, and collaboration. The company offers support and consulting services in content creation, management, and updates on MediaWiki, tailoring the platform to meet the unique needs of different users.
The project task was to develop an AI agent capable of collecting and analyzing all the existing content belonging to a user (such as a company) on the MediaWiki platform. Based on this analysis, the agent would consider the specific style, formatting, structure, and layout of the company’s content. The main goal was to create a system that could automatically generate new articles or pages on a given topic, while maintaining consistency in style and structure.
Additionally, we implemented a Web UI interface that allows users to generate new content directly through the interface. This user-friendly interface enables clients to interact with the AI content generation tool without needing to access backend systems or perform manual coding. This makes the process of content generation more accessible and efficient for non-technical users, allowing them to create high-quality articles in line with their existing content.
TECH STACK
LangChain
GPT
Python
DELIVERY TIMELINE
1 week
Solution Architecture Design
2 Weeks
LangChain Agent Development & Customization
1 Week
REST API and UI Development
1 Week
Deployment & Testing
TECH CHALLENGE
Advanced RAG System: This project involved developing a full-fledged Retrieval-Augmented Generation (RAG) system where relevant information for content generation is retrieved from a large database using FAISS (Facebook AI Similarity Search). One of the core challenges was ensuring that the agent could maintain a message history, allowing the user to modify the generated content by making edits. Unlike typical question-answer systems, this agent comprises multiple LangChain submodules: message history storage, a question-answer agent, a RAG chain, and a conversation RAG chain. Each component had to be integrated seamlessly to support both efficient retrieval of relevant information and a dynamic editing process for generated content.
Handling MediaWiki Markup: MediaWiki utilizes a specific markup language that is critical for structuring the pages correctly. Ensuring the agent adhered to this markup was a significant challenge, as any deviation could result in improperly formatted pages or broken links. The system needed to not only generate relevant content but also strictly follow MediaWiki's markup rules. This required incorporating these markup constraints into the content generation process, so the output was well-formed, structurally sound, and ready to be used within the MediaWiki environment.
SOLUTION
Implementation via LangChain: We implemented the core functionality using LangChain, tailoring it to work as a modified Retrieval-Augmented Generation (RAG) system. The system was designed to allow the client to seamlessly connect their own MediaWiki instance. Once connected, the agent was able to gather all the content associated with the user (or organization) from their MediaWiki. We built a retriever database using FAISS (Facebook AI Similarity Search) to enable efficient retrieval of relevant information.The AI agent itself was equipped with several critical submodules. First, it included a history module that tracked past interactions, allowing users to make edits and refine generated content. This was crucial for maintaining flexibility and enabling iterative content creation, where users could make modifications based on previously generated outputs. This history module was part of a larger RAG agent, which handled both the retrieval and generation of content. On top of this RAG agent, we layered a Conversation Chain module, which ensured that the interaction felt like a continuous dialogue. This allowed the user to seamlessly flow between content retrieval, generation, and modification in an intuitive conversation-like manner.
REST-API Development: To integrate this functionality with other systems, we developed a REST-API in Python. This API allowed the client to programmatically interact with the content generation tool, submit topics for new content, retrieve generated content, and even refine or edit content directly through API requests. The API was designed with simplicity and scalability in mind, ensuring that it could be used in different environments and easily integrated into existing workflows or platforms.
User Interface (UI): In addition to the backend functionality, we developed a straightforward Web UI interface to enhance accessibility. This UI allowed users to interact with the content generation system without needing technical expertise. Through the interface, users could input the desired topics, review and edit generated articles, and ensure the output matched their expectations before finalizing it. The UI provided a user-friendly environment that empowered clients to engage directly with the AI system, making the process of content generation more intuitive and accessible for non-technical users.
.svg)
