Articles / DeepSeek ‘Fat Fish’ Transformed into Interactive Live2D AI Companion

DeepSeek ‘Fat Fish’ Transformed into Interactive Live2D AI Companion

7 11 月, 2026 • 3 min read • DeepSeekLive2D

DeepSeek ‘Fat Fish’ Transformed into Interactive Live2D AI Companion

A groundbreaking DIY project merges AI, Live2D animation, and real-time multimodal interaction — all in a desktop-ready virtual roommate.


🌟 Viral Sensation Meets Virtual Reality

DeepSeek’s beloved “Fat Fish” mascot has taken the internet by storm — sparking memes, fan art, and now, an innovative technical implementation: a fully interactive, voice-enabled, screen-aware Live2D AI companion.

This isn’t just another chatbot. It’s a responsive, animated persona that:
– ✅ Talks back using custom TTS with character-consistent voice cloning
– ✅ Reacts to mouse touches (ears, head, hands) with unique animations
– ✅ Watches your screen in real time and delivers witty, context-aware commentary
– ✅ Switches seamlessly between text and voice input/output
– ✅ Runs locally or via lightweight web deployment

Fat Fish Live2D Demo


🛠️ Technical Architecture Overview

The project builds upon the open-source AIRI framework — an extensible AI virtual partner platform — and enhances it with three core layers:

Layer Technology Key Capability
Presentation Live2D Cubism + Seethrough auto-decomposer PSD-based rigging, physics-driven motion, real-time lip-sync
Cognition DeepSeek API (via Qoder CN integration) Role-aware LLM responses, dynamic personality tuning via system prompts
Perception & Interaction Whisper ASR + GPT-SoVITS TTS + Screen capture + Vision model hooks Full duplex voice chat, screen-aware commentary, gesture-triggered reactions

Live2D Model Preview


🧩 Step-by-Step Implementation Highlights

1. Character Creation Pipeline

  • Generated Q-style whale maid concept art using Kimi K3
  • Automated layer decomposition with Seethrough, followed by AI-assisted repair of critical parts (eyes, mouth, ears)
  • Mesh binding & motion physics configured for expressive idle behaviors

Layer Decomposition Tool

2. AI Integration Stack

  • Integrated DeepSeek API via secure key injection in AIRI’s backend config
  • Fine-tuned system prompt for “Fat Fish” persona: playful, observant, gently sarcastic, privacy-conscious
  • Enabled automatic screen analysis — without reading sensitive content — to trigger contextual quips (e.g., “Back at DeepSeek’s house again?”)

Screen-Aware Interaction

3. Voice & Presence Enhancement

  • Trained voice clone using GPT-SoVITS, sourced from Bilibili fan videos (5–10 sec clean samples)
  • Synchronized mouth animation to audio amplitude in real time
  • Configured Whisper ASR with auto-submit mode → enabling true “video-call-like” conversation flow

Touch Interaction Demo


📦 Open Source & Accessibility

  • ✅ Full source code published on GitHub (link in original article comments)
  • ✅ Zero-code setup supported via Qoder CN — a Chinese-language AI coding assistant
  • ✅ Lightweight local deployment: runs smoothly on mid-tier laptops (no GPU required)
  • ✅ Modular design: easily adaptable to GPT, Claude, Qwen, or other LLM backends

💡 Pro Tip: The entire pipeline was built using ~1000 Qoder CN credits — optimized for cost and speed during their Double 11 promotion.

Qoder CN Workflow


🚀 What’s Next?

This project exemplifies the democratization of embodied AI — where creativity, open tools, and accessible frameworks converge. Future extensions could include:
– Multi-character switching (e.g., “GPT Maid”, “Claude Scholar”)
– AR overlay via WebXR
– Real-time emotion inference from webcam feed
– Plugin ecosystem for third-party actions (calendar, music, notifications)

Whether you’re a developer, designer, or AI enthusiast — this is your invitation to build the next generation of living interfaces.

Final Deployment Preview

Project origin: JackCui | Source: AITNT AI News Platform