# Agora-Conversational-AI-Agent-Creator

**Repository Path**: mirrors_AgoraIO-Community/Agora-Conversational-AI-Agent-Creator

## Basic Information

- **Project Name**: Agora-Conversational-AI-Agent-Creator
- **Description**: No description available
- **Primary Language**: Unknown
- **License**: Not specified
- **Default Branch**: main
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2025-11-15
- **Last Updated**: 2026-03-08

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# Conversational AI

Voice-first AI conversations with multiple characters and virtual avatars. Built with Next.js 14 and powered by Agora's Conversational AI platform.

## Features

- **Voice Activity Monitor**: Real-time animated bars showing audio capture with color gradients
- **Multiple AI Characters**: Choose from Santa, Lawyer, Doctor, Quiz Master, Wellness Coach, Puzzle Master, and AI Assistant
- **Custom Character Creation**: Create your own AI characters with custom avatars, voices, and personalities
- **AI-Powered Character Generation**: Generate professional character details using OpenAI GPT-4
- **20+ Voice Options**: Choose from 20 different ElevenLabs voices with audio previews
- **Virtual Avatars**: Lifelike HeyGen avatars with synchronized lip movement
- **Dual Modes**: Virtual Avatar (video + audio) or Voice Only (audio-only) conversations
- **Responsive Design**: Optimized for both desktop and mobile devices
- **Real-time AI**: Powered by Agora's Conversational AI with OpenAI GPT-4, ElevenLabs TTS, and HeyGen avatars
- **Character Management**: Save, delete, and manage custom characters with session persistence

## Tech Stack

- **Framework**: Next.js 14 with App Router
- **Language**: TypeScript
- **Styling**: Tailwind CSS
- **UI Components**: shadcn/ui
- **Icons**: Lucide React
- **AI Integration**: Agora Conversational AI API
- **TTS**: ElevenLabs (20+ voices)
- **Avatars**: HeyGen
- **LLM**: OpenAI GPT-4 (for character generation)

## Prerequisites

Before running this application, you'll need:

1. **Agora Account**: Sign up at [Agora.io](https://agora.io) and create a project
2. **OpenAI API Key**: Get your API key from [OpenAI](https://platform.openai.com)
3. **ElevenLabs API Key**: Sign up at [ElevenLabs](https://elevenlabs.io)
4. **HeyGen API Key**: Sign up at [HeyGen](https://heygen.com)

## Environment Variables

Create a `.env.local` file in the root directory with the following variables:

```env
# Agora Configuration
AGORA_APP_ID=your_agora_app_id
AGORA_CUSTOMER_ID=your_agora_customer_id
AGORA_CUSTOMER_SECRET=your_agora_customer_secret
AGORA_APP_CERTIFICATE=your_agora_app_certificate

# API Keys
OPENAI_API_KEY=your_openai_api_key
ELEVENLABS_API_KEY=your_elevenlabs_api_key
HEYGEN_API_KEY=your_heygen_api_key
```

### Getting Agora Credentials

1. Go to [Agora Console](https://console.agora.io)
2. Create a new project or select an existing one
3. Navigate to "Project Management" → "Project Settings"
4. Copy the App ID, Customer ID, Customer Secret, and App Certificate

## Installation

1. Clone the repository:

```bash
git clone <repository-url>
cd ai4
```

2. Install dependencies:

```bash
npm install
```

3. Set up environment variables (see above)

4. Run the development server:

```bash
npm run dev
```

5. Open [http://localhost:3000](http://localhost:3000) in your browser

## Usage

### Built-in Characters

1. **Select AI Character**: Choose from Santa, Lawyer, Doctor, Quiz Master, Wellness Coach, Puzzle Master, or AI Assistant
2. **Choose Mode**: Toggle between Virtual Avatar (video + audio) or Voice Only
3. **Join Conversation**: Click the "Join Conversation" button to start

### Custom Characters

1. **Create Custom Character**: Click "Create Custom Character" button
2. **AI Generation**: Use the AI button to generate character details automatically
3. **Manual Setup**: Or manually configure:
   - **Character Name**: Enter a name for your character
   - **Description**: Brief description (max 80 characters)
   - **Avatar**: Choose from 30+ available avatars
   - **Voice**: Select from 20+ ElevenLabs voices with audio previews
   - **System Message**: Define character behavior and personality
   - **Greeting Message**: Initial message when conversation starts
4. **Save Character**: Click "Create Character" to save
5. **Use Character**: Select your custom character from the dropdown
6. **Manage Characters**: Hover over custom characters to delete them

### Voice Activity

- Watch the animated bars to see your audio being captured
- The AI will respond based on the selected character
- Real-time voice activity monitoring with color gradients

## Project Structure

```
src/
├── app/
│   ├── api/
│   │   ├── setup-conversation/
│   │   │   └── route.ts          # Token and channel generation
│   │   ├── join-conversation/
│   │   │   └── route.ts          # AI agent creation
│   │   ├── leave-agent/
│   │   │   └── route.ts          # Agent cleanup
│   │   └── generate-personality/
│   │       └── route.ts          # AI character generation
│   ├── conversation/
│   │   └── page.tsx              # Active conversation interface
│   ├── globals.css               # Global styles
│   ├── layout.tsx                # Root layout
│   └── page.tsx                  # Character selection interface
├── components/
│   ├── ui/                       # shadcn/ui components
│   ├── CustomPersonalityModal.tsx # Custom character creation modal
│   └── VoiceSelector.tsx         # Voice selection component
├── hooks/
│   └── useAgoraCall.ts           # Agora RTC hook
└── lib/
    ├── personalities.tsx         # Character definitions and helpers
    └── utils.ts                  # Utility functions
```

## API Endpoints

### POST /api/setup-conversation

Generates tokens and channel for initial conversation setup.

**Request Body:**

```json
{
  "personality": "santa|lawyer|doctor|quizmaster|wellnessCoach|puzzleMaster|assistant|custom_1234567890",
  "isAvatarMode": true|false
}
```

**Response:**

```json
{
  "success": true,
  "channelName": "conversation_1234567890_abc123",
  "uid": 1000,
  "userToken": "agora_token_here",
  "agentToken": "agora_token_here",
  "avatarToken": "agora_token_here",
  "personality": {
    "id": "santa"
  },
  "isAvatarMode": true
}
```

### POST /api/join-conversation

Creates the AI agent for an existing conversation.

**Request Body:**

```json
{
  "personality": "santa|lawyer|doctor|quizmaster|wellnessCoach|puzzleMaster|assistant|custom_1234567890",
  "isAvatarMode": true|false,
  "existingChannel": "conversation_1234567890_abc123",
  "existingUid": 1000,
  "existingUserToken": "agora_token_here",
  "existingAgentToken": "agora_token_here",
  "existingAvatarToken": "agora_token_here",
  "customPersonalityData": {
    "name": "IT Professional",
    "description": "Expert IT professional...",
    "systemMessage": "You are an IT professional...",
    "greetingMessage": "Hello! I'm your IT...",
    "voiceId": "21m00Tcm4TlvDq8ikWAM",
    "avatarId": "Bryan_IT_Sitting_public"
  }
}
```

### POST /api/generate-personality

Generates character details using OpenAI GPT-4.

**Request Body:**

```json
{
  "personalityType": "creative and engaging"
}
```

**Response:**

```json
{
  "success": true,
  "personality": {
    "name": "IT Professional",
    "description": "Expert IT professional helping with technical support",
    "systemMessage": "You are an IT professional...",
    "greetingMessage": "Hello! I'm your IT..."
  }
}
```

### POST /api/leave-agent

Stops the AI agent when leaving the conversation.

**Request Body:**

```json
{
  "agentId": "agent_id_from_join_response"
}
```

## Characters

### Built-in Characters

- **Santa**: Jolly and festive holiday companion
- **Lawyer**: Professional legal advisor and consultant
- **Doctor**: Caring medical professional and health advisor
- **Quiz Master**: Lively quizmaster hosting a 20-question general knowledge quiz
- **Wellness Coach**: Positive and encouraging wellness coach
- **Puzzle Master**: Clever and engaging logic puzzle challenger
- **AI Assistant**: Helpful and knowledgeable AI companion

### Custom Characters

Users can create unlimited custom characters with:

- **Professional Focus**: IT Professional, Language Tutor, Fitness Coach, etc.
- **Custom Avatars**: Choose from 30+ HeyGen avatars
- **Voice Selection**: 20+ ElevenLabs voices with audio previews
- **AI Generation**: Automatic character detail generation
- **Session Persistence**: Characters saved in browser session

### Character Configuration

Each character has:

- **System Message**: Defines character behavior and personality
- **Greeting Message**: Initial message when joining
- **Voice ID**: ElevenLabs voice for TTS
- **Avatar ID**: HeyGen avatar for video mode
- **Icon**: Lucide React icon for UI
- **Description**: Brief character description (max 80 characters)

## Voice Options

### Available Voices (20+)

- **Rachel**: Young American female
- **Drew**: Middle-aged American male
- **Clyde**: Middle-aged American male
- **Paul**: Middle-aged American male
- **Aria**: Middle-aged American female
- **Domi**: Young American female
- **Dave**: Young British male
- **Roger**: Middle-aged American male
- **Fin**: Old Irish male
- **Sarah**: Young American female
- **Antoni**: Young American male (well-rounded)
- **Laura**: Young American female (sassy)
- **Thomas**: Young American male (meditative)
- **Charlie**: Young Australian male (energetic)
- **George**: Middle-aged British male (mature)
- **Emily**: Young American female (calm)
- **Elli**: Young American female (emotional)
- **Callum**: Middle-aged male (gravelly)
- **Patrick**: Middle-aged American male (shouty)
- **River**: Middle-aged neutral (calm)

### Voice Features

- **Audio Previews**: Click to hear voice samples
- **Character Labels**: Age, gender, accent information
- **Professional Descriptions**: Use case and style information
- **Multi-language Support**: Some voices support multiple languages

## Customization

### Creating Custom Characters

1. Click "Create Custom Character" button
2. Use AI generation or manually configure:
   - Character name and description
   - Avatar selection from 30+ options
   - Voice selection with audio previews
   - System message for behavior
   - Greeting message for initial interaction
3. Save and use your custom character

### AI Character Generation

The app uses OpenAI GPT-4 to generate:

- **Professional Character Names**: IT Professional, Language Tutor, etc.
- **Descriptions**: Concise character descriptions (max 80 characters)
- **System Messages**: Detailed behavior and personality definitions
- **Greeting Messages**: Engaging conversation starters

### Modifying Built-in Characters

Update the character definitions in `src/lib/personalities.tsx`:

- **Voice IDs**: ElevenLabs voice identifiers
- **Avatar IDs**: HeyGen avatar identifiers
- **System Messages**: Character and behavior definitions
- **Greeting Messages**: Initial conversation starters

## Data Storage

### Local Storage

Custom characters are stored in browser local storage:

- **Persistence**: Characters persist across browser sessions and restarts
- **Cross-page Access**: Available across all pages
- **Manual Cleanup**: Only cleared when user manually clears browser data
- **JSON Format**: Structured data for easy management

### Character Data Structure

```json
{
  "id": "custom_1703123456789",
  "name": "IT Professional",
  "description": "Expert IT professional helping with technical support",
  "systemMessage": "You are an IT professional...",
  "greetingMessage": "Hello! I'm your IT...",
  "voiceId": "21m00Tcm4TlvDq8ikWAM",
  "avatarId": "Bryan_IT_Sitting_public",
  "isCustom": true
}
```

## Deployment

### Vercel (Recommended)

1. Push your code to GitHub
2. Connect your repository to Vercel
3. Add environment variables in Vercel dashboard
4. Deploy

The app is configured with:

- `eslint.ignoreDuringBuilds: true`
- `typescript.ignoreBuildErrors: true`
- `output: 'standalone'`

### Other Platforms

The app can be deployed to any platform that supports Next.js:

- Netlify
- Railway
- DigitalOcean App Platform
- AWS Amplify

## Troubleshooting

### Common Issues

1. **"Missing Agora configuration"**: Ensure all environment variables are set
2. **"Failed to join conversation"**: Check your Agora credentials and API keys
3. **"Custom character data is required"**: Ensure custom characters have all required fields
4. **Avatar not showing**: Verify your HeyGen API key and avatar ID
5. **No audio**: Check ElevenLabs API key and voice ID
6. **Voice activity bars not moving**: Check microphone permissions
7. **AI generation not working**: Verify OpenAI API key and quota

### Debug Mode

Enable debug logging by checking browser developer tools or server logs.

## Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Test thoroughly
5. Submit a pull request

## License

This project is licensed under the MIT License.

## Support

For issues related to:

- **Agora Services**: Contact [Agora Support](https://agora.io/support)
- **OpenAI**: Check [OpenAI Documentation](https://platform.openai.com/docs)
- **ElevenLabs**: Visit [ElevenLabs Support](https://elevenlabs.io/support)
- **HeyGen**: Contact [HeyGen Support](https://heygen.com/support)