A conversational AI shopping assistant for IKEA that combines natural language understanding, semantic search, and browser automation to create an intelligent, clickless cart management experience.
- Vague Query Detection: Automatically detects queries like "looking for chairs" without details
- Interactive Chips: Clickable options for chair type, price range, and colors
- Multi-Turn Conversations: Understands refinements like "less than $250" after showing results
- Context Awareness: Remembers previous searches and merges new constraints
- Natural Language: Say "add the ergonomic chair with armrests" instead of "add the first one"
- LLM-Powered Matching: Semantic understanding of product descriptions
- Automated Actions: Browser automation adds/removes items automatically
- Video Recordings: Every cart action recorded with split-screen display
- Cart Total Calculation: Ask "what's my cart total?" for detailed breakdown
- Dynamic Layout: Chat (60%) + Video Panel (40%) during cart operations
- Closeable Panel: ร button returns to full-width chat
- Smooth Transitions: 0.3s animations for professional feel
- Auto-Detection: Videos automatically extracted to side panel
- Semantic Search: Find products using natural language
- Hybrid Matching: Combines keyword and vector search
- Smart Filtering: Price range, colors, features extraction
- ChromaDB: Fast vector similarity search
- Context Awareness: Remembers previous searches and cart items
- Product Cards: Beautiful HTML product displays with images
- Formatted Responses: HTML formatting with emphasis and bullet points
- Session Management: Persistent state across interactions
- Python 3.11+
- Google API Key (for Gemini LLM)
- Chromium/Chrome (for Playwright)
- Clone the repository
git clone <your-repo-url>
cd ClickelssUI- Install dependencies
pip install -r requirements.txt- Install Playwright browsers
playwright install chromium- Set up environment variables
cp .env.example .env
# Edit .env and add your GOOGLE_API_KEY- Run the application
python web/app.py- Open your browser
http://127.0.0.1:5000
You: "looking for chairs"
Agent: [Shows interactive chips for type, price, color]
You: [Clicks "Office chair" and "Under $200"]
Agent: [Displays filtered results]
You: "office chairs with armrests"
Agent: [Shows 5 results]
You: "less than $250"
Agent: [Refines results to show only chairs under $250]
You: "what's my cart total?"
Agent: Your cart has:
โข FLINTAN Office chair - $139.00
Subtotal: $139.00
Tax (8%): $11.12
TOTAL: $150.12
You: "add the chair with lumbar support"
Agent: โ
Added MARKUS Office chair to cart!
[Split-screen shows video of action]
ClickelssUI/
โโโ agent/ # AI Agent Layer
โ โโโ ikea_agent.py # Main conversational agent
โ โโโ product_resolver.py # LLM product matching
โ โโโ rag_tool.py # RAG search interface
โ โโโ tools/cart_tools.py # Cart operation tools
โ
โโโ automation/ # Browser Automation
โ โโโ browser_manager.py # Playwright session management
โ โโโ ikea_cart.py # Cart automation + video recording
โ
โโโ scraper/ # Data Collection
โ โโโ ikea_scraper.py # Web scraper
โ โโโ rag_manager.py # ChromaDB manager
โ โโโ data_processor.py # Data cleaning
โ
โโโ web/ # Web Interface
โ โโโ app.py # Flask application
โ
โโโ data/ # Product Data
โ โโโ raw/ # Scraped JSON
โ โโโ chroma_db/ # Vector embeddings
โ
โโโ tests/ # Test Suite
โโโ test_product_selection_simple.py
โโโ test_remove_from_cart.py
- Python 3.11 - Main language
- Flask 3.1 - Web framework
- Playwright 1.47 - Browser automation
- Google Gemini 2.5 Flash - Primary LLM
- LangGraph 0.2 - Agent orchestration
- LangChain Core - Tool integration
- ChromaDB 0.5 - Vector database
- ONNX Embeddings - Default embedding model
- Semantic Search - Hybrid keyword + vector
# Required
GOOGLE_API_KEY=your_gemini_api_key_here
# Optional (for OpenAI embeddings)
OPENAI_API_KEY=your_openai_key_here- Port: 5000 (default)
- Debug Mode: Enabled in development
- Session Storage: File-based (
flask_session/) - Video Storage:
videos/(auto-created, cleared on restart) - Screenshot Storage:
screenshots/(auto-created)
| Metric | Performance |
|---|---|
| Product Search | < 1 second |
| LLM Processing | ~1-2 seconds |
| Cart Operations | ~5-8 seconds |
| Video Recording | Real-time |
| Multi-Turn Refinement | < 1 second |
- โ Clarification flow for vague queries with interactive chips
- โ Multi-turn conversation with refinement support
- โ Cart total calculation via LLM with detailed breakdown
- โ Split-screen UI with closeable video panel
- โ Improved follow-up formatting with HTML and product cards
- โ Natural language product matching
- โ Confidence-based decision making
- โ Automatic clarification for ambiguous queries
- โ Comprehensive test suite (5/5 passing)
- โ RAG-powered product search
- โ Browser automation with Playwright
- โ Video recording of cart actions
- โ Flask web interface
- Single Session: Only one user session supported at a time
- IKEA Website Changes: May break if IKEA updates their UI
- Browser State: Requires manual login to IKEA first (cookies persist)
- Video Storage: Videos cleared on server restart
- Multi-category support (beyond chairs)
- User preference learning
- Price tracking and alerts
- Image-based product search
- Multi-language support
- Voice interface
- Mobile app
- ARCHITECTURE.md - Detailed system architecture
- .env.example - Environment configuration template
- Fork the repository
- Create a feature branch
- Make your changes
- Add tests
- Submit a pull request
This project is for educational purposes. Please respect IKEA's terms of service and robots.txt.
- LangChain - Agent framework
- LangGraph - State machine orchestration
- Playwright - Browser automation
- ChromaDB - Vector database
- Google Gemini - LLM capabilities
For issues or questions:
- Check existing issues
- Review documentation
- Create a new issue with details
Built with โค๏ธ using LLMs, RAG, and Browser Automation
Version: 3.0.0
Last Updated: November 2025
Status: โ
Production Ready