Qwen2.5-Omni Multimodal AI
Qwen2.5-Omni is a cutting-edge multimodal AI model capable of processing text, images, audio, and video. See what makes this AI model special!
Platform: Replicate
Multimodal AIVoice SynthesisReal-time InteractionVideo Understanding
2.8k runs
L40S
License Check Required🚀Function Overview
A multimodal model that processes text, images, audio, and video inputs to generate text responses and natural speech, supporting real-time streaming interactions for conversational applications.
Key Features
- End-to-end processing of text, images, audio, and video inputs
- Real-time streaming text and speech generation
- TMRoPE architecture for synchronized video/audio timing
- Robust performance across ASR, translation, and visual reasoning benchmarks
Use Cases
- •Real-time conversational AI with multimodal inputs
- •Video analysis and description
- •Multilingual speech recognition and synthesis
- •Audio-video content interaction systems
⚙️Input Parameters
prompt
stringText prompt for the model
image
stringOptional image input
audio
stringOptional audio input
video
stringOptional video input
system_prompt
stringSystem prompt for the model
use_audio_in_video
booleanWhether to use audio in video
voice_type
stringVoice type for audio output
generate_audio
booleanWhether to generate audio output
💡Usage Examples
Example 1
Input Parameters
{
"video": "https://replicate.delivery/pbxt/MmJqxKbRSknHd9fwtTEbywWqDDdhgsx5tNYLIDnFqJ9j5ObC/draw.mp4",
"voice_type": "Chelsie",
"system_prompt": "You are Qwen, a virtual human developed by the Qwen Team, Alibaba Group, capable of perceiving auditory and visual inputs, as well as generating text and speech.",
"generate_audio": true,
"use_audio_in_video": true
}Output Results
{
"text": "Oh, that's a really cool drawing! It looks like a guitar. You've got the body and the neck drawn in a simple yet effective way. The lines are clean and the shape is well-defined. What made you choose to draw a guitar?",
"voice": "https://replicate.delivery/xezq/xpYIpzUZvRIIONe0qe1CfAcq53wfz46ecayvhcPCRY3mcI7jC/output.wav"
}
Quick Actions
Technical Specifications
- Hardware Type
- L40S
- Run Count
- 2.8k
- Commercial Use
- Unknown/Restricted
- Platform
- Replicate