Melissa AI

Melissa AI is a React Native mobile application that serves as your personal AI kitchen assistant. It leverages Google's Gemini AI / OpenRouter to generate creative recipes based on text, voice, or image inputs. The app features local recipe storage, ingredient substitution suggestions, and PDF cookbook generation.

Screenshots 📸

Architecture

Frontend (React Native + Expo)

Melissa/
├── app/                    # Expo Router screens
│   ├── index.tsx               # Home screen with recipe generation
│   ├── recipe.tsx              # Recipe display and editing screen
│   ├── CookbookGenerator.tsx   # PDF cookbook creation
│   └── _layout.tsx             # Navigation layout
├── components/             # Reusable UI components
│   ├── MicModal.tsx            # Voice recording interface
│   ├── IngredientModal.tsx     # Ingredient substitution UI
│   ├── SavedRecipesList.tsx    # Recipe management
│   └── ...
├── hooks/                  # Custom React hooks
│   ├── useRecipeGenerator.ts        # Recipe generation logic
│   ├── useIngredientSuggestions.ts  # Substitution logic
│   ├── useCookbookGenerator.ts      # PDF generation
│   └── ...
├── store/                  # Zustand state management
│   ├── userStore.ts            # User preferences
│   └── membershipStore.ts      # Premium features
└── types/                      # TypeScript definitions

Backend (Node.js + Express)

Melissa-Backend/
├── server.js               # Express server entry point
├── controllers/            # Request handlers
│   ├── recipeController.js      # Recipe generation logic
│   ├── ingredientController.js  # Substitution logic
│   └── cookbookController.js    # PDF generation
├── services/               # External service integrations
│   ├── geminiClient.js          # Google Gemini AI
│   ├── whisperClient.js         # Audio transcription
│   ├── whisperClient.py         # Python Whisper model
│   └── fallbackClient.js        # Backup AI service
├── routes/                 # API endpoints
│   ├── recipe.js                # /api/recipes/*
│   ├── ingredient.js            # /api/ingredient/*
│   ├── cookbook.js              # /api/cookbook/*
│   └── transcribe.js            # /api/transcribe
└── utils/                  # Helper functions
    └── formatPrompt.js          # AI prompt formatting

Tech Stack

Frontend

• React Native - Cross-platform mobile framework
• Expo - Development platform and build tools
• TypeScript - Type-safe JavaScript
• Expo Router - File-based navigation
• Zustand - State management
• AsyncStorage - Local data persistence
• Expo AV - Audio recording capabilities
• Expo Image Picker - Photo selection
• Expo Speech - Text-to-speech feedback

Backend

• Node.js - JavaScript runtime
• Express - Web application framework
• Google Gemini AI / OpenRouter - Recipe generation and NLP
• OpenAI Whisper - Speech-to-text transcription
• Multer - File upload handling
• PDFLaTeX - PDF cookbook generation
• Python - Whisper model integration

Getting Started

Prerequisites

• Node.js (v16 or higher)
• Python 3.8+ (for Whisper transcription)
• Expo CLI
• Android Studio or Xcode (for mobile development)
• PDFLaTeX (only for cookbook generation)

Installation

1. Clone the repository

2. Setup Frontend

cd Melissa
npm install

3. Setup Backend

cd Melissa-Backend
npm install

# Setup Python virtual environment for Whisper
python -m venv venv
# Windows
venv\Scripts\activate
# Mac/Linux
source venv/bin/activate

# Install Whisper
pip install openai-whisper

4. Configure Environment Variables

Create a .env file in Melissa-Backend/:

GEMINI_API_KEY=your_google_gemini_api_key
OPENROUTER_API_KEY=your_openrouter_api_key  # Optional: for fallback API
MAX_PHOTOS=2

5. Update Backend URL

In the frontend code, update the config.ts backend URL to match your local IP:

• Create a config.ts in Melissa dir
• Add "export const BACKEND_URL = 'local_IP_address';"

Running the Application

1. Start the Backend Server

cd Melissa-Backend
node server.js

2. Start the Frontend

cd Melissa
npx expo start

3. Run on Device

• Scan the QR code with Expo Go app (iOS/Android)
• Or press a for Android emulator
• Or press i for iOS simulator

How It Works

Recipe Generation Flow

1. User Input: Users provide input through text, voice, or images
2. Request Processing: Frontend packages the input with metadata (servings, difficulty, cookware)
3. AI Processing: Backend sends formatted prompts to Gemini AI / OpenRouter fallback
4. Response Parsing: AI response is parsed into structured recipe format
5. Display: Recipe is displayed with ingredients and instructions
6. Local Storage: Users can save recipes locally for offline access

Cookbook Generation

1. User selects saved recipes
2. Backend formats recipes into LaTeX document
3. PDFLaTeX compiles the document
4. PDF is returned as base64
5. User can view, share, or print the cookbook

API Endpoints

Recipe Generation

• POST /api/recipes/generate-recipe - Generate new recipe
• POST /api/recipes/adjust-recipe - Modify existing recipe

Ingredient Management

• GET /api/ingredient/suggest/:ingredient - Get substitution suggestions
• POST /api/ingredient/suggest/swap-amount - Calculate substitution amounts

Transcription

• POST /api/transcribe - Convert audio to text

Cookbook

• POST /api/cookbook/generate - Create PDF cookbook