Tradução de apoio para consulta. Código e identificadores técnicos preservados. Consultar referência original ↗

Áudio e vídeo, compreensão de arquivos.

Texto + imagem/input de áudio

Começando.

? Pré-requisitos ?

[Pegue uma chave API] (/api-reference/preparation/api-keyE o define como uma variável de ambiente./api-reference/preparation/export-api-key-env).
Qwen-Omni Só suporta chamadas compatíveis com OpenAI./api-reference/preparation/install-sdk). O OpenAI Python SDK requer versão 1.52.0+. O SDK Node.js requer a versão 4.68.0+.

Este exemplo envia um texto para o Qwen-Omni API e retorna uma resposta de streaming que inclui texto e áudio.

import os
import base64
import soundfile as sf
import numpy as np
from openai import OpenAI

# 1. Initialize the client
client = OpenAI(
  api_key=os.getenv("DASHSCOPE_API_KEY"),  # Make sure the environment variable is configured
  base_url="https://maas.qwencloudapi.com/compatible-mode/v1",
)

# 2. Initiate the request
try:
  completion = client.chat.completions.create(
    model="qwen3.5-omni-plus",
    messages=[{"role": "user", "content": "Who are you?"}],
    modalities=["text", "audio"],  # Specify text and audio output
    audio={"voice": "Tina", "format": "wav"},
    stream=True,  # Must be set to True
    stream_options={"include_usage": True},
  )

  # 3. Process the streaming response and decode the audio
  print("Model response:")
  audio_base64_string = ""
  for chunk in completion:
    # Process the text part
    if chunk.choices and chunk.choices[0].delta.content:
      print(chunk.choices[0].delta.content, end="")

    # Collect the audio part
    if chunk.choices and hasattr(chunk.choices[0].delta, "audio") and chunk.choices[0].delta.audio:
      audio_base64_string += chunk.choices[0].delta.audio.get("data", "")

  # 4. Save the audio file
  if audio_base64_string:
    wav_bytes = base64.b64decode(audio_base64_string)
    audio_np = np.frombuffer(wav_bytes, dtype=np.int16)
    sf.write("audio_assistant.wav", audio_np, samplerate=24000)
    print("\nAudio file saved to: audio_assistant.wav")

except Exception as e:
  print(f"Request failed: {e}")

// Preparations before running:
// Universal for Windows/Mac/Linux:
// 1. Make sure Node.js is installed (version >= 14 is recommended)
// 2. Run the following command to install the necessary dependencies:
//    npm install openai wav

import OpenAI from "openai";
import { createWriteStream } from 'node:fs';
import { Writer } from 'wav';

// Define an audio conversion function: convert a Base64 string and save it as a standard WAV audio file
async function convertAudio(audioString, audioPath) {
  try {
    // Decode the Base64 string into a Buffer
    const wavBuffer = Buffer.from(audioString, 'base64');
    // Create a WAV file write stream
    const writer = new Writer({
      sampleRate: 24000,  // Sample rate
      channels: 1,        // Single channel
      bitDepth: 16        // 16-bit depth
    });
    // Create an output file stream and establish a pipeline connection
    const outputStream = createWriteStream(audioPath);
    writer.pipe(outputStream);

    // Write PCM data and end writing
    writer.write(wavBuffer);
    writer.end();

    // Use a Promise to wait for the file to be written
    await new Promise((resolve, reject) => {
      outputStream.on('finish', resolve);
      outputStream.on('error', reject);
    });

    // Add extra wait time to ensure audio integrity
    await new Promise(resolve => setTimeout(resolve, 800));

    console.log(`\nAudio file successfully saved as ${audioPath}`);
  } catch (error) {
    console.error('An error occurred during processing:', error);
  }
}

//  1. Initialize the client
const openai = new OpenAI(
  {
    apiKey: process.env.DASHSCOPE_API_KEY,
    baseURL: "https://maas.qwencloudapi.com/compatible-mode/v1"
  }
);
// 2. Initiate the request
const completion = await openai.chat.completions.create({
  model: "qwen3.5-omni-plus",
  messages: [
    {
      "role": "user",
      "content": "Who are you?"
    }],
  stream: true,
  stream_options: {
    include_usage: true
  },
  modalities: ["text", "audio"],
  audio: { voice: "Tina", format: "wav" }
});

let audioString = "";
console.log("Model response:")

// 3. Process the streaming response and decode the audio
for await (const chunk of completion) {
  if (Array.isArray(chunk.choices) && chunk.choices.length > 0) {
    // Process the text content
    if (chunk.choices[0].delta.content) {
      process.stdout.write(chunk.choices[0].delta.content);
    }
    // Process the audio content
    if (chunk.choices[0].delta.audio) {
      if (chunk.choices[0].delta.audio["data"]) {
        audioString += chunk.choices[0].delta.audio["data"];
      }
    }
  }
}
// 4. Save the audio file
convertAudio(audioString, "audio_assistant.wav");

curl -X POST https://maas.qwencloudapi.com/compatible-mode/v1/chat/completions \
-H "Authorization: Bearer $DASHSCOPE_API_KEY" \
-H "Content-Type: application/json" \
-d '{
  "model": "qwen3.5-omni-plus",
  "messages": [
    {
      "role": "user",
      "content": "Who are you?"
    }
  ],
  "stream":true,
  "stream_options":{
    "include_usage":true
  },
  "modalities":["text","audio"],
  "audio":{"voice":"Tina","format":"wav"}
}'

Resposta

Depois que você correr o Python Ou Node.js código, a resposta do texto é devolvida e um arquivo de áudio chamado audio_assistant.wav é salvo no mesmo diretório que seu arquivo de código.

Model response:
I am a large language model developed by Alibaba Cloud. My name is Qwen. How can I help you?

Correndo o HTTP O código retorna texto e dados de áudio codificados pela Base64 diretamente no audio Campo.

data: {"choices":[{"delta":{"content":"I"},"finish_reason":null,"index":0,"logprobs":null}],"object":"chat.completion.chunk","usage":null,"created":1757647879,"system_fingerprint":null,"model":"qwen3.5-omni-plus","id":"chatcmpl-a68eca3b-c67e-4666-a72f-73c0b4919860"}
data: {"choices":[{"delta":{"content":" am"},"finish_reason":null,"index":0,"logprobs":null}],"object":"chat.completion.chunk","usage":null,"created":1757647879,"system_fingerprint":null,"model":"qwen3.5-omni-plus","id":"chatcmpl-a68eca3b-c67e-4666-a72f-73c0b4919860"}
......
data: {"choices":[{"delta":{"audio":{"data":"/v8AAAAAAAAAAAAAAA...","expires_at":1757647879,"id":"audio_a68eca3b-c67e-4666-a72f-73c0b4919860"}},"finish_reason":null,"index":0,"logprobs":null}],"object":"chat.completion.chunk","usage":null,"created":1757647879,"system_fingerprint":null,"model":"qwen3.5-omni-plus","id":"chatcmpl-a68eca3b-c67e-4666-a72f-73c0b4919860"}
data: {"choices":[{"finish_reason":"stop","delta":{"content":""},"index":0,"logprobs":null}],"object":"chat.completion.chunk","usage":null,"created":1764763585,"system_fingerprint":null,"model":"qwen3.5-omni-plus","id":"chatcmpl-e8c82e9e-073e-4289-a786-a20eb444ac9c"}
data: {"choices":[],"object":"chat.completion.chunk","usage":{"prompt_tokens":207,"completion_tokens":103,"total_tokens":310,"completion_tokens_details":{"audio_tokens":83,"text_tokens":20},"prompt_tokens_details":{"text_tokens":207}},"created":1757940330,"system_fingerprint":null,"model":"qwen3.5-omni-plus","id":"chatcmpl-9cdd5a26-f9e9-4eff-9dcc-93a878165afc"}

Linguagens apoiadas

Qwen3.5-Omni supported languages

** Idiomas de entrada (74 línguas):** Chinês, Inglês, Alemão, Francês, Italiano, Tcheco, Indonésio, Tailandês, Coreano, Polonês, Japonês, Vietnamita, Finlandês, Português, Espanhol, Holandês, Russo, Malaio, Catalão, Sueco, Turco, Ucraniano, Romeno, Eslovaco, Dinamarquês, Islandês, Norueguês (Bokmal), Macedônio, Grego, Húngaro, Galego, Filipino, Croata, Bósnia, Esloveno, Búlgaro, Cazaque, Bielorrusso, Letão, Estoniano, Azerbaijão, Uyghur, Swahili, Hindi, Esperanto, Quirgyz, Tajik, Cebuano, Afrikaans, Árabe, Lituano, Javanese, Bengali, Persa, Hebraica, Punjabi, Gujarati, Mongol, Asturiana, Kannada, Marathi, Interlingua, Malayalam, Maltesa, Norueguesa Nynorsk, Telugu, Urdu, Georgiana, Basco, Tamil, Odia, Sérvia, Maori

** Dialetos de entrada (39 dialetos):** Dialeto do nordeste do mandarim, dialeto de Guizhou, cantonês, dialeto de Henan, cantonês de Hong Kong, Xangai, dialeto de Shaanxi, dialeto de Tianjin, dialeto de Taiwan, Hokkien, dialeto de Yunnan, dialeto de Anhui, dialeto de Fujian, dialeto de Gansu, dialeto de Guangdong, dialeto de Hubei, dialeto de Hunan, dialeto de Jiangxi, dialeto de Shandong, dialeto de Shanxi, dialeto de Sichuan, dialeto de Guangxi, dialeto de Hainan, dialeto de Chongqing, dialeto de Changsha, dialeto de Hangzhou, dialeto de Hefei, dialeto de Yinchuan, dialeto de Zhengzhou, dialeto de Shenyang, dialeto de Wenzhou, Wuhan Dialeto, dialeto de Kunming, dialeto de Taiyuan, dialeto de Nanchang, dialeto de Jinan, dialeto de Lanzhou, dialeto de Nanjing, Hakka, Min Sulista

** Linguagens de áudio de saída (29 idiomas):** Chinês, Inglês, Alemão, Italiano, Português, Espanhol, Japonês, Coreano, Francês, Russo, Tailandês, Indonésio, Árabe, Vietnamita, Turco, Finlandês, Polonês, Hindi, Holandês, Tcheco, Urdu, Tagalo, Sueco, Dinamarquês, Hebraico, Islandês, Malaio, Norueguês, Persa

Dialeto de Sichuan, dialeto de Pequim, dialeto de Tianjin, dialeto de Nanjing, dialeto de Shaanxi, cantonês, Min do Sul

Qwen3-Omni-Flash supported languages

** Idiomas de entrada e saída (11 idiomas):** Chinês, Inglês, Alemão, Francês, Italiano, Tailandês, Coreano, Japonês, Russo, Espanhol, Português

Dialetos de entrada/saída (8 dialetos): Dialeto de Sichuan, Xangai, Cantonês, Min do Sul, Dialeto de Shaanxi, Dialeto de Nanjing, Dialeto de Tianjin, Dialeto de Pequim

Para nomes de modelos, contexto, preços e versões instantâneas, veja [lista de modelos] (/developer-guides/getting-started/model-selection). Para limites de solicitação simultânea, veja Limites de taxa.

Desempenho do modelo

Análise de conteúdo de áudio e vídeo

<table style={{width: "100%", borderCollapse: "collapse", tableLayout: "fixed"}}> <tr> <th style={{textAlign: "left", width: "50%", paddingRight: "12px", paddingBottom: "8px"}}> Entrada</th> <th style={{textAlign: "left", paddingLeft: "12px", paddingBottom: "8px"}}> Saída</th> </tr>

  Gere uma descrição detalhada e cronometrada deste vídeo.
</td>

<td style={{verticalAlign: "top", paddingLeft: "12px"}}>
   Uma rua da cidade ensopada por chuva enche a tela larga. A fotografia de longa exposição cria faixas de luzes vermelhas e azuis no pavimento molhado. Um homem solitário com um casaco escuro e camisa clara com uma gravata caminha em direção à câmera ao longo da calçada direita.

  A câmera inclina-se para baixo.  Dentro de um raio de lanterna, um lençol amarelo e envelhecido está numa poça rasa, bordas carbonizadas e cantos enrolados. O homem ajoelha-se, estendendo os dedos com luvas para o documento, ondulando ao se aproximar da mão.

   Apenas as datas parciais são mostradas.
</td>

</tr> </table>

Compreensão multidialética

  Descreva as características vocais, estado emocional e letras completas para cantores em diferentes dialetos chineses.
</td>

<td style={{verticalAlign: "top", paddingLeft: "12px"}}>
  Este áudio apresenta apresentações de rap de cantores de várias regiões da China. **Hangzhou dialeto (Wu)** -- A voz do cantor masculino é suave e suave, com a suavidade e calor distintivos do dialeto Wu.  Sua entonação sobe e cai suavemente, e seu ritmo é leve e rápido, criando um efeito de contar histórias relaxado.  Ele descreve o estilo de vida lento de Hangzhou e mudanças urbanas, expressando afeto por sua cidade natal.

  * Apenas resultados parciais são mostrados.*
</td>

</tr> </table>

Geração de legendas líricas

  Transcreva as letras da música e forneça datas para cada linha.
</td>

<td style={{verticalAlign: "top", paddingLeft: "12px"}}>
  O fio de gato passa pela luz do luar nas árvores.

  * Apenas resultados parciais são mostrados.*
</td>

</tr> </table>

Programação de áudio e vídeo

<td style={{verticalAlign: "top", paddingLeft: "12px"}}>
  <video src="https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/en-US/20260330/fyybth/%E9%9F%B3%E8%A7%86%E9%A2%91%E7%BC%96%E7%A8%8B-02.mp4" controls style={{display: "block", width: "100%"}} />
</td>

</tr> </table>

Uso

Saída de transmissão

Todos os pedidos para Qwen-Omni Deve definir stream=True.

Configuração do modelo

Configure parâmetros, avisos e comprimentos de áudio-vídeo baseado no seu caso de uso para equilibrar custo, velocidade e qualidade.

Audio-video understanding

Ver a tabela estruturada nos quadros de referência desta página ↓

Recommended structured prompt for audio-video understanding

  ````text
  Provide a detailed description of the video.
  It should explicitly include three sections:
  1. A structured chronological storyline of **every noticeable audio and visual details**
  2. A structured list of all visible text. For each text element, include start timestamp, end timestamp, the exact text content, the appearance characteristics. If no text appears, explicitly state so.
  3. A structured speech-to-text transcription, include speaker（Corresponding to the character or voice‑over in Section 1, including their accent and tone）, exact spoken content, start timestamp, end timestamp, and speaking state (prosody, emotion, and style). If no speech appears, explicitly state so.
  Aside from these three required sections, you are free to organize any additional content in any way you find helpful. This additional content can include global information about the entire video or localized information about specific moments. You may choose the topic of this extra content freely.
  Output Format:
  ```
  ##  História
  <xx:xx.xxx> - <xx:xx.xxx>
  <an unstructured long paragraph in natural language describing what happened during this period, blending both audio and video details.>
  <xx:xx.xxx> - <xx:xx.xxx>
  <an unstructured long paragraph in natural language describing what happened during this period, blending both audio and video details.>
  <xx:xx.xxx> - <xx:xx.xxx>
  <an unstructured long paragraph in natural language describing what happened during this period, blending both audio and video details.>
  ...
  ##  Texto visível
  <xx:xx.xxx> - <xx:xx.xxx>
  "<element>": <appearance>
  "<element>": <appearance>
  <xx:xx.xxx> - <xx:xx.xxx>
  "<element>": <appearance>
  "<element>": <appearance>
  "<element>": <appearance>
  <xx:xx.xxx> - <xx:xx.xxx>
  "<element>": <appearance>
  ...
  ## Palestrantes e Transcrição
  Perfis de falante:
  <speaker> - <profile>
  <speaker> - <profile>
  <speaker> - <profile>
  ...
  <xx:xx.xxx> - <xx:xx.xxx>
  - Sim. <speaker>
  Estado: <description>
   Conteúdo: "<content>"
  <xx:xx.xxx> - <xx:xx.xxx>
  - Sim. <speaker>
  Estado: <description>
   Conteúdo: "<content>"
  <xx:xx.xxx> - <xx:xx.xxx>
  - Sim. <speaker>
  Estado: <description>
   Conteúdo: "<content>"
  ...
  ## <another section>
  <paragraphs>
  ## <another section>
  <paragraphs>
  ...
  ```
  ````

Para descrições de vídeos longos, segmente-os primeiro.

Audio understanding

Equilibrar custo e qualidade controlando o comprimento do áudio e a complexidade.

Ver a tabela estruturada nos quadros de referência desta página ↓

Recommended structured prompt for audio understanding

  ````text
  Provide a detailed description of the audio.

  It should explicitly include two sections:

  1. A structured chronological storyline of **every noticeable audio details**
  2. A structured speech-to-text transcription, include speaker（Corresponding to the character or voice‑over in Section 1, including their accent and tone）, exact spoken content, start timestamp, end timestamp, and speaking state (prosody, emotion, and style). If no speech appears, explicitly state so.

  Aside from these two required components, you are free to organize any additional content in any way you find helpful. This additional content can include global information about the entire audio or localized information about specific moments. You may choose the topic of this extra content freely.

  Output Format:

  ```
  ##  História

  <xx:xx.xxx> - <xx:xx.xxx>
  <an unstructured long paragraph in natural language describing what happened during this period, blending both audio details.>

  <xx:xx.xxx> - <xx:xx.xxx>
  <an unstructured long paragraph in natural language describing what happened during this period, blending both audio details.>

  <xx:xx.xxx> - <xx:xx.xxx>
  <an unstructured long paragraph in natural language describing what happened during this period, blending both audio details.>

  ...

  ...

  ## Palestrantes e Transcrição

  Perfis de falante:
  <speaker> - <profile>
  <speaker> - <profile>
  <speaker> - <profile>
  ...

  <xx:xx.xxx> - <xx:xx.xxx>
  - Sim. <speaker>
  Estado: <description>
   Conteúdo: "<content>"

  <xx:xx.xxx> - <xx:xx.xxx>
  - Sim. <speaker>
  Estado: <description>
   Conteúdo: "<content>"

  <xx:xx.xxx> - <xx:xx.xxx>
  - Sim. <speaker>
  Estado: <description>
   Conteúdo: "<content>"

  ...

  ## <another section>

  <paragraphs>

  ## <another section>

  <paragraphs>

  ...
  ```
  ````

Para descrições de áudio longo, segmente-o primeiro.

Modo de pensar

Para ativar/desativar, saída de streaming, e thinking_budget Veja, veja./developer-guides/text-generation/thinking).

Qwen3-Omni-Flash é um modelo de pensamento híbrido (enable_thinking predefinições para false).

No modo de pensar, definir modalities: ["text"] - A saída de áudio não é suportada quando o pensamento é ativado.

Pesquisa na Web

O Qwen3.5-Omni Séries suportam busca na web para recuperar informações em tempo real e executar raciocínio. Activar a pesquisa na Web usando o enable_search parâmetro e conjunto search_strategy para agent.

import os
from openai import OpenAI

client = OpenAI(
  api_key=os.getenv("DASHSCOPE_API_KEY"),
  base_url="https://maas.qwencloudapi.com/compatible-mode/v1",
)

try:
  completion = client.chat.completions.create(
    model="qwen3.5-omni-plus",
    messages=[{
      "role": "user",
      "content": "Please look up today's date and day of the week, and tell me what major holidays fall on this date."
    }],
    stream=True,
    stream_options={"include_usage": True},
    extra_body={
      "enable_search": True,
      "search_options": {
        "search_strategy": "agent"
      }
    }
  )

  print("Model response (includes real-time information):")
  for chunk in completion:
    if chunk.choices and chunk.choices[0].delta.content:
      print(chunk.choices[0].delta.content, end="")
  print()

except Exception as e:
  print(f"Request failed: {e}")

import OpenAI from "openai";

const openai = new OpenAI({
  apiKey: process.env.DASHSCOPE_API_KEY,
  baseURL: "https://maas.qwencloudapi.com/compatible-mode/v1"
});

const completion = await openai.chat.completions.create({
  model: "qwen3.5-omni-plus",
  messages: [{
    "role": "user",
    "content": "Please look up today's date and day of the week, and tell me what major holidays fall on this date."
  }],
  stream: true,
  stream_options: {
    include_usage: true
  },
  extra_body: {
    enable_search: true,
    search_options: {
      search_strategy: "agent"
    }
  }
});

console.log("Model response (includes real-time information):");

for await (const chunk of completion) {
  if (Array.isArray(chunk.choices) && chunk.choices.length > 0) {
    if (chunk.choices[0].delta.content) {
      process.stdout.write(chunk.choices[0].delta.content);
    }
  }
}
console.log();

curl -X POST https://maas.qwencloudapi.com/compatible-mode/v1/chat/completions \
-H "Authorization: Bearer $DASHSCOPE_API_KEY" \
-H "Content-Type: application/json" \
-d '{
  "model": "qwen3.5-omni-plus",
  "messages": [
    {
      "role": "user",
      "content": "Please look up today'\''s date and day of the week, and tell me what major holidays fall on this date."
    }
  ],
  "stream": true,
  "stream_options": {
    "include_usage": true
  },
  "enable_search": true,
  "search_options": {
    "search_strategy": "agent"
  }
}'

A busca na web é suportada apenas no **Qwen3.5-Omni Série **. O search_strategy O parâmetro só aceita agent.
Veja [Pricing] (/developer-guides/getting-started/pricing) para informações de cobrança relacionadas com o agent Estratégia.

Entrada multimodal

Vídeo e texto de entrada

Você pode inserir vídeo como [uma lista de imagens] (#image-list-format) ou como [um arquivo de vídeo] (#video-file-format-can-understand-audio-in-the-video). Se você inserir um arquivo de vídeo, o modelo também pode entender o áudio no vídeo.

O seguinte código de exemplo usa um URL de vídeo da internet como exemplo. Para inserir um vídeo local, veja [Input Base64-codificado arquivos locais] (#input-base64-encoded-local-files). A saída de transmissão é necessária para todas as chamadas.

Formato de arquivo de vídeo (pode entender áudio no vídeo)

Número de arquivos:
Qwen3.5-Omni até 512 arquivos usando URLs públicas, até 250 arquivos usando codificação Base64.
Qwen3-Omni-Flash Só é permitido um arquivo.
Tamanho do arquivo:
Qwen3.5-Omni Até 2 GB, até 1 hora de duração.
Qwen3-Omni-Flash Até 256 MB, até 150 segundos de duração.
Formatos de arquivos: MP4, AVI, MKV, MOV, FLV, WMV, etc.
Informações visuais e de áudio em arquivos de vídeo são cobradas separadamente.

import os
from openai import OpenAI

client = OpenAI(
  api_key=os.getenv("DASHSCOPE_API_KEY"),
  base_url="https://maas.qwencloudapi.com/compatible-mode/v1",
)

completion = client.chat.completions.create(
  model="qwen3.5-omni-plus",
  messages=[
    {
      "role": "user",
      "content": [
        {
          "type": "video_url",
          "video_url": {
            "url": "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20241115/cqqkru/1.mp4"
          },
        },
        {"type": "text", "text": "What is the video about?"},
      ],
    },
  ],
  modalities=["text", "audio"],
  audio={"voice": "Tina", "format": "wav"},
  stream=True,
  stream_options={"include_usage": True},
)

for chunk in completion:
  if chunk.choices:
    print(chunk.choices[0].delta)
  else:
    print(chunk.usage)

import OpenAI from "openai";

const openai = new OpenAI(
  {
    apiKey: process.env.DASHSCOPE_API_KEY,
    baseURL: "https://maas.qwencloudapi.com/compatible-mode/v1"
  }
);
const completion = await openai.chat.completions.create({
  model: "qwen3.5-omni-plus",
  messages: [
    {
      "role": "user",
      "content": [{
        "type": "video_url",
        "video_url": { "url": "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20241115/cqqkru/1.mp4" },
      },
      { "type": "text", "text": "What is the video about?" }]
    }],
  stream: true,
  stream_options: {
    include_usage: true
  },
  modalities: ["text", "audio"],
  audio: { voice: "Tina", format: "wav" }
});

for await (const chunk of completion) {
  if (Array.isArray(chunk.choices) && chunk.choices.length > 0) {
    console.log(chunk.choices[0].delta);
  } else {
    console.log(chunk.usage);
  }
}

curl -X POST https://maas.qwencloudapi.com/compatible-mode/v1/chat/completions \
-H "Authorization: Bearer $DASHSCOPE_API_KEY" \
-H "Content-Type: application/json" \
-d '{
  "model": "qwen3.5-omni-plus",
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "type": "video_url",
          "video_url": {
            "url": "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20241115/cqqkru/1.mp4"
          }
        },
        {
          "type": "text",
          "text": "What is the video about"
        }
      ]
    }
  ],
  "stream":true,
  "stream_options": {
    "include_usage": true
  },
  "modalities":["text","audio"],
  "audio":{"voice":"Tina","format":"wav"}
}'

Formato da lista de imagens

Número de imagens

Qwen3.5-OmniMínimo de 2 imagens, máximo de 2048 imagens.

Qwen3-Omni-FlashMínimo de 2 imagens, máximo de 128 imagens.

import os
from openai import OpenAI

client = OpenAI(
  api_key=os.getenv("DASHSCOPE_API_KEY"),
  base_url="https://maas.qwencloudapi.com/compatible-mode/v1",
)

completion = client.chat.completions.create(
  model="qwen3.5-omni-plus",
  messages=[
    {
      "role": "user",
      "content": [
        {
          "type": "video",
          "video": [
            "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20241108/xzsgiz/football1.jpg",
            "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20241108/tdescd/football2.jpg",
            "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20241108/zefdja/football3.jpg",
            "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20241108/aedbqh/football4.jpg",
          ],
        },
        {"type": "text", "text": "Describe the process shown in this video"},
      ],
    }
  ],
  modalities=["text", "audio"],
  audio={"voice": "Tina", "format": "wav"},
  stream=True,
  stream_options={"include_usage": True},
)

for chunk in completion:
  if chunk.choices:
    print(chunk.choices[0].delta)
  else:
    print(chunk.usage)

import OpenAI from "openai";

const openai = new OpenAI(
  {
    apiKey: process.env.DASHSCOPE_API_KEY,
    baseURL: "https://maas.qwencloudapi.com/compatible-mode/v1"
  }
);
const completion = await openai.chat.completions.create({
  model: "qwen3.5-omni-plus",
  messages: [{
    role: "user",
    content: [
      {
        type: "video",
        video: [
          "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20241108/xzsgiz/football1.jpg",
          "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20241108/tdescd/football2.jpg",
          "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20241108/zefdja/football3.jpg",
          "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20241108/aedbqh/football4.jpg"
        ]
      },
      {
        type: "text",
        text: "Describe the process shown in this video"
      }
    ]
  }],
  stream: true,
  stream_options: {
    include_usage: true
  },
  modalities: ["text", "audio"],
  audio: { voice: "Tina", format: "wav" }
});

for await (const chunk of completion) {
  if (Array.isArray(chunk.choices) && chunk.choices.length > 0) {
    console.log(chunk.choices[0].delta);
  } else {
    console.log(chunk.usage);
  }
}

curl -X POST https://maas.qwencloudapi.com/compatible-mode/v1/chat/completions \
-H "Authorization: Bearer $DASHSCOPE_API_KEY" \
-H "Content-Type: application/json" \
-d '{
  "model": "qwen3.5-omni-plus",
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "type": "video",
          "video": [
            "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20241108/xzsgiz/football1.jpg",
            "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20241108/tdescd/football2.jpg",
            "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20241108/zefdja/football3.jpg",
            "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20241108/aedbqh/football4.jpg"
          ]
        },
        {
          "type": "text",
          "text": "Describe the process shown in this video"
        }
      ]
    }
  ],
  "stream": true,
  "stream_options": {
    "include_usage": true
  },
  "modalities": ["text", "audio"],
  "audio": {
    "voice": "Tina",
    "format": "wav"
  }
}'

Áudio e texto

Número de arquivos:
Qwen3.5-Omni até 2048 arquivos usando URLs públicas, até 250 arquivos usando codificação Base64.
Qwen3-Omni-Flash Só é permitido um arquivo.
Tamanho do arquivo:
Qwen3.5-Omni Até 2 GB, até 3 horas de duração.
Qwen3-Omni-Flash Até 100 MB, até 20 minutos de duração.
** Formatos de arquivos:** Suporta formatos principais, como AMR, WAV, 3GP, 3GPP, AAC e MP3.

Para inserir um arquivo de áudio local, veja [Input Base64-codificado arquivos locais] (#input-base64-encoded-local-files). A saída de transmissão é necessária para todas as chamadas.

import os
from openai import OpenAI

client = OpenAI(
  api_key=os.getenv("DASHSCOPE_API_KEY"),
  base_url="https://maas.qwencloudapi.com/compatible-mode/v1",
)

completion = client.chat.completions.create(
  model="qwen3.5-omni-plus",
  messages=[
    {
      "role": "user",
      "content": [
        {
          "type": "input_audio",
          "input_audio": {
            "data": "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20250211/tixcef/cherry.wav",
            "format": "wav",
          },
        },
        {"type": "text", "text": "What is this audio about"},
      ],
    },
  ],
  modalities=["text", "audio"],
  audio={"voice": "Tina", "format": "wav"},
  stream=True,
  stream_options={"include_usage": True},
)

for chunk in completion:
  if chunk.choices:
    print(chunk.choices[0].delta)
  else:
    print(chunk.usage)

import OpenAI from "openai";

const openai = new OpenAI(
  {
    apiKey: process.env.DASHSCOPE_API_KEY,
    baseURL: "https://maas.qwencloudapi.com/compatible-mode/v1"
  }
);
const completion = await openai.chat.completions.create({
  model: "qwen3.5-omni-plus",
  messages: [
    {
      "role": "user",
      "content": [{
        "type": "input_audio",
        "input_audio": { "data": "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20250211/tixcef/cherry.wav", "format": "wav" },
      },
      { "type": "text", "text": "What is this audio about" }]
    }],
  stream: true,
  stream_options: {
    include_usage: true
  },
  modalities: ["text", "audio"],
  audio: { voice: "Tina", format: "wav" }
});

for await (const chunk of completion) {
  if (Array.isArray(chunk.choices) && chunk.choices.length > 0) {
    console.log(chunk.choices[0].delta);
  } else {
    console.log(chunk.usage);
  }
}

curl -X POST https://maas.qwencloudapi.com/compatible-mode/v1/chat/completions \
-H "Authorization: Bearer $DASHSCOPE_API_KEY" \
-H "Content-Type: application/json" \
-d '{
  "model": "qwen3.5-omni-plus",
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "type": "input_audio",
          "input_audio": {
            "data": "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20250211/tixcef/cherry.wav",
            "format": "wav"
          }
        },
        {
          "type": "text",
          "text": "What is this audio about"
        }
      ]
    }
  ],
  "stream":true,
  "stream_options":{
    "include_usage":true
  },
  "modalities":["text","audio"],
  "audio":{"voice":"Tina","format":"wav"}
}'

Imagem e texto

Qwen-Omni Modelos suportam múltiplas entradas de imagem. Os requisitos para as imagens de entrada são os seguintes:

Número de imagens:
Quando passou como uma URL pública: até 2048** imagens por pedido.
Quando passado como Base64-codificado cordas: até 250 imagens por pedido.

Além desses bonés por pedido, os símbolos totais de todas as imagens e todos os textos devem ser **menos que o comprimento máximo de entrada do modelo.

** Tamanho da imagem:**
- Qwen3.5 Cada arquivo de imagem deve ter 20 MB ou menos.
- Qwen3-Omni-Flash Cada arquivo de imagem deve ter 10 MB ou menos.
A largura e altura da imagem devem ser maiores que 10 pixels. A proporção não deve exceder 200:1 ou 1:200.
Para tipos de imagens suportadas, veja [Compreensão visual e vídeo] (/developer-guides/multimodal/vision).

O seguinte código de exemplo usa uma URL de imagem da internet como exemplo. Para inserir uma imagem local, veja [Input Base64-codificado arquivo local] (#input-base64-encoded-local-files). A saída de transmissão é necessária para todas as chamadas.

import os
from openai import OpenAI

client = OpenAI(
  api_key=os.getenv("DASHSCOPE_API_KEY"),
  base_url="https://maas.qwencloudapi.com/compatible-mode/v1",
)

completion = client.chat.completions.create(
  model="qwen3.5-omni-plus",
  messages=[
    {
      "role": "user",
      "content": [
        {
          "type": "image_url",
          "image_url": {
            "url": "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20241022/emyrja/dog_and_girl.jpeg"
          },
        },
        {"type": "text", "text": "What scene is depicted in the image?"},
      ],
    },
  ],
  # Set the output data modality. Two are currently supported: ["text","audio"] and ["text"].
  modalities=["text", "audio"],
  audio={"voice": "Tina", "format": "wav"},
  # stream must be set to True to avoid errors.
  stream=True,
  stream_options={
    "include_usage": True
  }
)

for chunk in completion:
  if chunk.choices:
    print(chunk.choices[0].delta)
  else:
    print(chunk.usage)

import OpenAI from "openai";

const openai = new OpenAI(
  {
    apiKey: process.env.DASHSCOPE_API_KEY,
    baseURL: "https://maas.qwencloudapi.com/compatible-mode/v1"
  }
);
const completion = await openai.chat.completions.create({
  model: "qwen3.5-omni-plus",
  messages: [
    {
      "role": "user",
      "content": [{
        "type": "image_url",
        "image_url": { "url": "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20241022/emyrja/dog_and_girl.jpeg" },
      },
      { "type": "text", "text": "What scene is depicted in the image?" }]
    }],
  stream: true,
  stream_options: {
    include_usage: true
  },
  modalities: ["text", "audio"],
  audio: { voice: "Tina", format: "wav" }
});

for await (const chunk of completion) {
  if (Array.isArray(chunk.choices) && chunk.choices.length > 0) {
    console.log(chunk.choices[0].delta);
  } else {
    console.log(chunk.usage);
  }
}

curl -X POST https://maas.qwencloudapi.com/compatible-mode/v1/chat/completions \
-H "Authorization: Bearer $DASHSCOPE_API_KEY" \
-H "Content-Type: application/json" \
-d '{
    "model": "qwen3.5-omni-plus",
    "messages": [
    {
      "role": "user",
      "content": [
        {
          "type": "image_url",
          "image_url": {
            "url": "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20241022/emyrja/dog_and_girl.jpeg"
          }
        },
        {
          "type": "text",
          "text": "What scene is depicted in the image?"
        }
      ]
    }
  ],
    "stream":true,
    "stream_options":{
        "include_usage":true
    },
    "modalities":["text","audio"],
    "audio":{"voice":"Tina","format":"wav"}
}'

Conversa de várias voltas

Quando você usa o recurso de conversação multi-turnos de Qwen-Omni Modelos, notem o seguinte:

Mensagens assistentes no array de mensagens suportam apenas dados de texto.
Mensagem do usuário: uma mensagem do usuário pode conter texto e dados de apenas uma outra modalidade. Em uma conversa multi-turno, você pode usar diferentes modalidades em mensagens de usuário separadas.

import os
from openai import OpenAI

client = OpenAI(
  api_key=os.getenv("DASHSCOPE_API_KEY"),
  base_url="https://maas.qwencloudapi.com/compatible-mode/v1",
)

completion = client.chat.completions.create(
  model="qwen3.5-omni-plus",
  messages=[
    {
      "role": "user",
      "content": [
        {
          "type": "input_audio",
          "input_audio": {
            "data": "https://dashscope.oss-cn-beijing.aliyuncs.com/audios/welcome.mp3",
            "format": "mp3",
          },
        },
        {"type": "text", "text": "What is this audio about"},
      ],
    },
    {
      "role": "assistant",
      "content": [{"type": "text", "text": "This audio says: Welcome to QwenCloud"}],
    },
    {
      "role": "user",
      "content": [{"type": "text", "text": "Can you tell me about this company?"}],
    },
  ],
  # Set the output data modality. Two are currently supported: ["text","audio"] and ["text"].
  modalities=["text"],
  # stream must be set to True to avoid errors.
  stream=True,
  stream_options={"include_usage": True},
)

for chunk in completion:
  if chunk.choices:
    print(chunk.choices[0].delta)
  else:
    print(chunk.usage)

import OpenAI from "openai";

const openai = new OpenAI(
  {
    apiKey: process.env.DASHSCOPE_API_KEY,
    baseURL: "https://maas.qwencloudapi.com/compatible-mode/v1"
  }
);
const completion = await openai.chat.completions.create({
  model: "qwen3.5-omni-plus",
  messages: [
    {
      "role": "user",
      "content": [
        {
          "type": "input_audio",
          "input_audio": {
            "data": "https://dashscope.oss-cn-beijing.aliyuncs.com/audios/welcome.mp3",
            "format": "mp3",
          },
        },
        { "type": "text", "text": "What is this audio about" },
      ],
    },
    {
      "role": "assistant",
      "content": [{ "type": "text", "text": "This audio says: Welcome to QwenCloud" }],
    },
    {
      "role": "user",
      "content": [{ "type": "text", "text": "Can you tell me about this company?" }]
    }],
  stream: true,
  stream_options: {
    include_usage: true
  },
  modalities: ["text"]
});

for await (const chunk of completion) {
  if (Array.isArray(chunk.choices) && chunk.choices.length > 0) {
    console.log(chunk.choices[0].delta);
  } else {
    console.log(chunk.usage);
  }
}

curl -X POST https://maas.qwencloudapi.com/compatible-mode/v1/chat/completions \
-H "Authorization: Bearer $DASHSCOPE_API_KEY" \
-H "Content-Type: application/json" \
-d '{
  "model": "qwen3.5-omni-plus",
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "type": "input_audio",
          "input_audio": {
            "data": "https://dashscope.oss-cn-beijing.aliyuncs.com/audios/welcome.mp3"
          }
        },
        {
          "type": "text",
          "text": "What is this audio about"
        }
      ]
    },
    {
      "role": "assistant",
      "content": [
        {
          "type": "text",
          "text": "This audio says: Welcome to QwenCloud"
        }
      ]
    },
    {
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": "Can you tell me about this company?"
        }
      ]
    }
  ],
  "stream": true,
  "stream_options": {
    "include_usage": true
  },
  "modalities": ["text"]
}'

Parse Base64-codificado saída de dados de áudio

A saída de áudio de Qwen-Omni Modelos são dados codificados da Base64 entregues em um fluxo. Você pode usar uma variável de cordas para acumular os dados da Base64 de cada fragmento quando chegar. Depois que o fluxo estiver completo, decodifique a string final para criar o arquivo de áudio. Alternativamente, decodificar e reproduzir cada fragmento em tempo real como é recebido.

# Installation instructions for pyaudio:
# APPLE Mac OS X
#   brew install portaudio
#   pip install pyaudio
# Debian/Ubuntu
#   sudo apt-get install python-pyaudio python3-pyaudio
#   or
#   pip install pyaudio
# CentOS
#   sudo yum install -y portaudio portaudio-devel && pip install pyaudio
# Microsoft Windows
#   python -m pip install pyaudio

import os
from openai import OpenAI
import base64
import numpy as np
import soundfile as sf

client = OpenAI(
  api_key=os.getenv("DASHSCOPE_API_KEY"),
  base_url="https://maas.qwencloudapi.com/compatible-mode/v1",
)

completion = client.chat.completions.create(
  model="qwen3.5-omni-plus",
  messages=[{"role": "user", "content": "Who are you"}],
  # Set the output data modality. Two are currently supported: ["text","audio"] and ["text"].
  modalities=["text", "audio"],
  audio={"voice": "Tina", "format": "wav"},
  # stream must be set to True to avoid errors.
  stream=True,
  stream_options={"include_usage": True},
)

# Method 1: Decode after the generation is complete
audio_string = ""
for chunk in completion:
  if chunk.choices:
    if hasattr(chunk.choices[0].delta, "audio"):
      try:
        audio_string += chunk.choices[0].delta.audio["data"]
      except Exception as e:
        print(chunk.choices[0].delta.content)
  else:
    print(chunk.usage)

wav_bytes = base64.b64decode(audio_string)
audio_np = np.frombuffer(wav_bytes, dtype=np.int16)
sf.write("audio_assistant_py.wav", audio_np, samplerate=24000)

# Method 2: Decode while generating (comment out the code for Method 1 to use Method 2)
# # Initialize PyAudio
# import pyaudio
# import time
# p = pyaudio.PyAudio()
# # Create an audio stream
# stream = p.open(format=pyaudio.paInt16,
#                 channels=1,
#                 rate=24000,
#                 output=True)

# for chunk in completion:
#     if chunk.choices:
#         if hasattr(chunk.choices[0].delta, "audio"):
#             try:
#                 audio_string = chunk.choices[0].delta.audio["data"]
#                 wav_bytes = base64.b64decode(audio_string)
#                 audio_np = np.frombuffer(wav_bytes, dtype=np.int16)
#                 # Play the audio data directly
#                 stream.write(audio_np.tobytes())
#             except Exception as e:
#                 print(chunk.choices[0].delta.content)

# time.sleep(0.8)
# # Clean up resources
# stream.stop_stream()
# stream.close()
# p.terminate()

// Preparations before running:
// Universal for Windows/Mac/Linux:
// 1. Make sure Node.js is installed (version >= 14 is recommended)
// 2. Run the following command to install the necessary dependencies:
//    npm install openai wav
//
// To use the real-time playback feature (Method 2), you also need:
// Windows:
//    npm install speaker
// Mac:
//    brew install portaudio
//    npm install speaker
// Linux (Ubuntu/Debian):
//    sudo apt-get install libasound2-dev
//    npm install speaker

import OpenAI from "openai";

const openai = new OpenAI(
  {
    apiKey: process.env.DASHSCOPE_API_KEY,
    baseURL: "https://maas.qwencloudapi.com/compatible-mode/v1"
  }
);
const completion = await openai.chat.completions.create({
  model: "qwen3.5-omni-plus",
  messages: [
    {
      "role": "user",
      "content": "Who are you?"
    }],
  stream: true,
  stream_options: {
    include_usage: true
  },
  modalities: ["text", "audio"],
  audio: { voice: "Tina", format: "wav" }
});

// Method 1: Decode after the generation is complete
// Requires installation: npm install wav
import { createWriteStream } from 'node:fs';  // node:fs is a built-in Node.js module, no installation required
import { Writer } from 'wav';

async function convertAudio(audioString, audioPath) {
  try {
    // Decode the Base64 string into a Buffer
    const wavBuffer = Buffer.from(audioString, 'base64');
    // Create a WAV file write stream
    const writer = new Writer({
      sampleRate: 24000,  // Sample rate
      channels: 1,        // Single channel
      bitDepth: 16        // 16-bit depth
    });
    // Create an output file stream and establish a pipeline connection
    const outputStream = createWriteStream(audioPath);
    writer.pipe(outputStream);

    // Write PCM data and end writing
    writer.write(wavBuffer);
    writer.end();

    // Use a Promise to wait for the file to be written
    await new Promise((resolve, reject) => {
      outputStream.on('finish', resolve);
      outputStream.on('error', reject);
    });

    // Add extra wait time to ensure audio integrity
    await new Promise(resolve => setTimeout(resolve, 800));

    console.log(`Audio file successfully saved as ${audioPath}`);
  } catch (error) {
    console.error('An error occurred during processing:', error);
  }
}

let audioString = "";
for await (const chunk of completion) {
  if (Array.isArray(chunk.choices) && chunk.choices.length > 0) {
    if (chunk.choices[0].delta.audio) {
      if (chunk.choices[0].delta.audio["data"]) {
        audioString += chunk.choices[0].delta.audio["data"];
      }
    }
  } else {
    console.log(chunk.usage);
  }
}
// Execute the conversion
convertAudio(audioString, "audio_assistant_mjs.wav");

// Method 2: Generate and play in real time
// You must first install the necessary components according to the instructions for your system above.
// import Speaker from 'speaker'; // Import the audio playback library

// // Create a speaker instance (configuration matches WAV file parameters)
// const speaker = new Speaker({
//     sampleRate: 24000,  // Sample rate
//     channels: 1,        // Number of sound channels
//     bitDepth: 16,       // Bit depth
//     signed: true        // Signed PCM
// });
// for await (const chunk of completion) {
//     if (Array.isArray(chunk.choices) && chunk.choices.length > 0) {
//         if (chunk.choices[0].delta.audio) {
//             if (chunk.choices[0].delta.audio["data"]) {
//                 const pcmBuffer = Buffer.from(chunk.choices[0].delta.audio.data, 'base64');
//                 // Write directly to the speaker for playback
//                 speaker.write(pcmBuffer);
//             }
//         }
//     } else {
//         console.log(chunk.usage);
//     }
// }
// speaker.on('finish', () => console.log('Playback complete'));
// speaker.end(); // Call based on the actual end of the API stream

Arquivos locais codificados pela base de entrada 64

Images

Este exemplo usa o arquivo localmente salvo [eagle.png] (https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/en-US/20250415/zgfeuf/eagle.png).



  ```python Python
  import os
  from openai import OpenAI
  import base64

  client = OpenAI(
    api_key=os.getenv("DASHSCOPE_API_KEY"),
    base_url="https://maas.qwencloudapi.com/compatible-mode/v1",
  )

  #  Base64 encoding format
  def encode_image(image_path):
    with open(image_path, "rb") as image_file:
      return base64.b64encode(image_file.read()).decode("utf-8")

  base64_image = encode_image("eagle.png")

  completion = client.chat.completions.create(
    model="qwen3.5-omni-plus",
    messages=[
      {
        "role": "user",
        "content": [
          {
            "type": "image_url",
            "image_url": {"url": f"data:image/png;base64,{base64_image}"},
          },
          {"type": "text", "text": "What scene is depicted in the image?"},
        ],
      },
    ],
    # Set the output data modality. Two are currently supported: ["text","audio"] and ["text"].
    modalities=["text", "audio"],
    audio={"voice": "Tina", "format": "wav"},
    # stream must be set to True to avoid errors.
    stream=True,
    stream_options={"include_usage": True},
  )

  for chunk in completion:
    if chunk.choices:
      print(chunk.choices[0].delta)
    else:
      print(chunk.usage)
  ```

  ```javascript Node.js
  import OpenAI from "openai";
  import { readFileSync } from 'fs';

  const openai = new OpenAI(
    {
      apiKey: process.env.DASHSCOPE_API_KEY,
      baseURL: "https://maas.qwencloudapi.com/compatible-mode/v1"
    }
  );

  const encodeImage = (imagePath) => {
    const imageFile = readFileSync(imagePath);
    return imageFile.toString('base64');
  };
  const base64Image = encodeImage("eagle.png")

  const completion = await openai.chat.completions.create({
    model: "qwen3.5-omni-plus",
    messages: [
      {
        "role": "user",
        "content": [{
          "type": "image_url",
          "image_url": { "url": `data:image/png;base64,${base64Image}` },
        },
        { "type": "text", "text": "What scene is depicted in the image?" }]
      }],
    stream: true,
    stream_options: {
      include_usage: true
    },
    modalities: ["text", "audio"],
    audio: { voice: "Tina", format: "wav" }
  });

  for await (const chunk of completion) {
    if (Array.isArray(chunk.choices) && chunk.choices.length > 0) {
      console.log(chunk.choices[0].delta);
    } else {
      console.log(chunk.usage);
    }
  }
  ```

Audio

Este exemplo usa o arquivo localmente salvo [welcome.mp3] (https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/en-US/20250430/vicmsy/welcome.mp3).



  ```python Python
  import os
  from openai import OpenAI
  import base64
  import numpy as np
  import soundfile as sf
  import requests

  client = OpenAI(
    api_key=os.getenv("DASHSCOPE_API_KEY"),
    base_url="https://maas.qwencloudapi.com/compatible-mode/v1",
  )

  def encode_audio(audio_path):
    with open(audio_path, "rb") as audio_file:
      return base64.b64encode(audio_file.read()).decode("utf-8")

  base64_audio = encode_audio("welcome.mp3")

  completion = client.chat.completions.create(
    model="qwen3.5-omni-plus",
    messages=[
      {
        "role": "user",
        "content": [
          {
            "type": "input_audio",
            "input_audio": {
              "data": f"data:;base64,{base64_audio}",
              "format": "mp3",
            },
          },
          {"type": "text", "text": "What is this audio about"},
        ],
      },
    ],
    # Set the output data modality. Two are currently supported: ["text","audio"] and ["text"].
    modalities=["text", "audio"],
    audio={"voice": "Tina", "format": "wav"},
    # stream must be set to True to avoid errors.
    stream=True,
    stream_options={"include_usage": True},
  )

  for chunk in completion:
    if chunk.choices:
      print(chunk.choices[0].delta)
    else:
      print(chunk.usage)
  ```

  ```javascript Node.js
  import OpenAI from "openai";
  import { readFileSync } from 'fs';

  const openai = new OpenAI(
    {
      apiKey: process.env.DASHSCOPE_API_KEY,
      baseURL: "https://maas.qwencloudapi.com/compatible-mode/v1"
    }
  );

  const encodeAudio = (audioPath) => {
    const audioFile = readFileSync(audioPath);
    return audioFile.toString('base64');
  };
  const base64Audio = encodeAudio("welcome.mp3")

  const completion = await openai.chat.completions.create({
    model: "qwen3.5-omni-plus",
    messages: [
      {
        "role": "user",
        "content": [{
          "type": "input_audio",
          "input_audio": { "data": `data:;base64,${base64Audio}`, "format": "mp3" },
        },
        { "type": "text", "text": "What is this audio about?" }]
      }],
    stream: true,
    stream_options: {
      include_usage: true
    },
    modalities: ["text", "audio"],
    audio: { voice: "Tina", format: "wav" }
  });

  for await (const chunk of completion) {
    if (Array.isArray(chunk.choices) && chunk.choices.length > 0) {
      console.log(chunk.choices[0].delta);
    } else {
      console.log(chunk.usage);
    }
  }
  ```

Video file

Este exemplo usa o arquivo local [spring\_mountain.mp4] (https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/en-US/20250430/ngisno/spring_mountain.mp4).



  ```python Python
  import os
  from openai import OpenAI
  import base64
  import numpy as np
  import soundfile as sf

  client = OpenAI(
    api_key=os.getenv("DASHSCOPE_API_KEY"),
    base_url="https://maas.qwencloudapi.com/compatible-mode/v1",
  )

  #  Base64 encoding format
  def encode_video(video_path):
    with open(video_path, "rb") as video_file:
      return base64.b64encode(video_file.read()).decode("utf-8")

  base64_video = encode_video("spring_mountain.mp4")

  completion = client.chat.completions.create(
    model="qwen3.5-omni-plus",
    messages=[
      {
        "role": "user",
        "content": [
          {
            "type": "video_url",
            "video_url": {"url": f"data:;base64,{base64_video}"},
          },
          {"type": "text", "text": "What is she singing?"},
        ],
      },
    ],
    # Set the output data modality. Supported modalities are ["text", "audio"] and ["text"].
    modalities=["text", "audio"],
    audio={"voice": "Tina", "format": "wav"},
    # stream must be set to True. Otherwise, an error occurs.
    stream=True,
    stream_options={"include_usage": True},
  )

  for chunk in completion:
    if chunk.choices:
      print(chunk.choices[0].delta)
    else:
      print(chunk.usage)
  ```

  ```javascript Node.js
  import OpenAI from "openai";
  import { readFileSync } from 'fs';

  const openai = new OpenAI(
    {
      apiKey: process.env.DASHSCOPE_API_KEY,
      baseURL: "https://maas.qwencloudapi.com/compatible-mode/v1"
    }
  );

  const encodeVideo = (videoPath) => {
    const videoFile = readFileSync(videoPath);
    return videoFile.toString('base64');
  };
  const base64Video = encodeVideo("spring_mountain.mp4")

  const completion = await openai.chat.completions.create({
    model: "qwen3.5-omni-plus",
    messages: [
      {
        "role": "user",
        "content": [{
          "type": "video_url",
          "video_url": { "url": `data:;base64,${base64Video}` },
        },
        { "type": "text", "text": "What is she singing?" }]
      }],
    stream: true,
    stream_options: {
      include_usage: true
    },
    modalities: ["text", "audio"],
    audio: { voice: "Tina", format: "wav" }
  });

  for await (const chunk of completion) {
    if (Array.isArray(chunk.choices) && chunk.choices.length > 0) {
      console.log(chunk.choices[0].delta);
    } else {
      console.log(chunk.usage);
    }
  }
  ```

Image list (as video)

Este exemplo usa os arquivos locais [football1.jpg] (https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/en-US/20250415/spqrrx/football1.jpg), [football2.jpg] (https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/en-US/20250415/vtnhyr/football2.jpg), [football3.jpg] (https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/en-US/20250430/enmhbp/football3.jpg), e [football4.jpg] (https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/en-US/20250430/ognmyc/football4.jpg).



  ```python Python
  import os
  from openai import OpenAI
  import base64
  import numpy as np
  import soundfile as sf

  client = OpenAI(
    api_key=os.getenv("DASHSCOPE_API_KEY"),
    base_url="https://maas.qwencloudapi.com/compatible-mode/v1",
  )

  #  Base64 encoding format
  def encode_image(image_path):
    with open(image_path, "rb") as image_file:
      return base64.b64encode(image_file.read()).decode("utf-8")

  base64_image_1 = encode_image("football1.jpg")
  base64_image_2 = encode_image("football2.jpg")
  base64_image_3 = encode_image("football3.jpg")
  base64_image_4 = encode_image("football4.jpg")

  completion = client.chat.completions.create(
    model="qwen3.5-omni-plus",
    messages=[
      {
        "role": "user",
        "content": [
          {
            "type": "video",
            "video": [
              f"data:image/jpeg;base64,{base64_image_1}",
              f"data:image/jpeg;base64,{base64_image_2}",
              f"data:image/jpeg;base64,{base64_image_3}",
              f"data:image/jpeg;base64,{base64_image_4}",
            ],
          },
          {"type": "text", "text": "Describe the procedure in this video."},
        ],
      }
    ],
    # Set the output data modality. Supported modalities are ["text", "audio"] and ["text"].
    modalities=["text", "audio"],
    audio={"voice": "Tina", "format": "wav"},
    # stream must be set to True. Otherwise, an error occurs.
    stream=True,
    stream_options={"include_usage": True},
  )

  for chunk in completion:
    if chunk.choices:
      print(chunk.choices[0].delta)
    else:
      print(chunk.usage)
  ```

  ```javascript Node.js
  import OpenAI from "openai";
  import { readFileSync } from 'fs';

  const openai = new OpenAI(
      {
      apiKey: process.env.DASHSCOPE_API_KEY,
      baseURL: "https://maas.qwencloudapi.com/compatible-mode/v1"
      }
  );

  const encodeImage = (imagePath) => {
      const imageFile = readFileSync(imagePath);
      return imageFile.toString('base64');
    };
  const base64Image1 = encodeImage("football1.jpg")
  const base64Image2 = encodeImage("football2.jpg")
  const base64Image3 = encodeImage("football3.jpg")
  const base64Image4 = encodeImage("football4.jpg")

  const completion = await openai.chat.completions.create({
      model: "qwen3.5-omni-plus",
      messages: [{
      role: "user",
      content: [
      {
      type: "video",
      video: [
      `data:image/jpeg;base64,${base64Image1}`,
      `data:image/jpeg;base64,${base64Image2}`,
      `data:image/jpeg;base64,${base64Image3}`,
      `data:image/jpeg;base64,${base64Image4}`
      ]
      },
      {
      type: "text",
      text: "Describe the procedure in this video."
      }
      ]
      }],
      stream: true,
      stream_options: {
      include_usage: true
      },
      modalities: ["text", "audio"],
      audio: { voice: "Tina", format: "wav" }
  });

  for await (const chunk of completion) {
      if (Array.isArray(chunk.choices) && chunk.choices.length > 0) {
      console.log(chunk.choices[0].delta);
      } else {
      console.log(chunk.usage);
      }
  }
  ```

Referência da API

Para os parâmetros de entrada e saída de Qwen-Omni, veja [Completação de bate-papo API] (/api-reference/chat/openai-chat).

Limites de faturamento e taxa

Regras de cobrança

Qwen-Omni é faturado com base no número de fichas para diferentes modalidades, como áudio, imagem e vídeo. Veja [Pricing] (/developer-guides/getting-started/pricing) para detalhes de preços.

Regras para converter áudio, imagens e vídeos em fichas

?Audio ?

Qwen3.5-Omni Série: entrada áudio tokens totais = duração do áudio (em segundos) x 7; Saída de áudio tokens totais = Duração de áudio (em segundos) x 12,5
Qwen3-Omni-FlashTotal de fichas para entrada e saída de áudio = duração de áudio (em segundos) x 12,5

Se a duração do áudio é menor que 1 segundo, é calculado como 1 segundo.

Imagens

Qwen3.5-Omni series e Qwen3-Omni-Flash1 token por 32 x 32 Pixels.

Qwen3.5-Omni série requer um mínimo de 24 fichas por imagem, enquanto outros modelos requerem um mínimo de 4 fichas. O máximo padrão é de 1280 fichas. Qwen3.5-Omni série suporta o vl_high_resolution_images parâmetro para aumentar o limite de resolução para 16384 tokens (não suportado por Qwen3-Omni-Flash). Use o seguinte código para estimar o total de fichas para uma imagem:

import math
from PIL import Image  # pip install Pillow

# ============ Model parameter configuration (modify as needed) ============

# Image factor: 32 for Qwen3.5-Omni series and Qwen3-Omni-Flash
IMAGE_FACTOR = 32

# Token minimum: 24 for Qwen3.5-Omni series; 4 for Qwen3-Omni-Flash
MIN_TOKENS = 24

# High-resolution mode (only supported by Qwen3.5-Omni series, not by Qwen3-Omni-Flash)
# True  → Token maximum 16384
# False → Token maximum 1280 (default)
VL_HIGH_RESOLUTION_IMAGES = False

# ============ Pixel range (auto-calculated from above) ============

MIN_PIXELS = MIN_TOKENS * IMAGE_FACTOR * IMAGE_FACTOR
MAX_PIXELS = (16384 if VL_HIGH_RESOLUTION_IMAGES else 1280) * IMAGE_FACTOR * IMAGE_FACTOR

def smart_resize(height, width, factor=IMAGE_FACTOR,
                 min_pixels=MIN_PIXELS, max_pixels=MAX_PIXELS):
  """Align image dimensions to factor multiples and scale to [min_pixels, max_pixels] range."""
  h_bar = max(factor, round(height / factor) * factor)
  w_bar = max(factor, round(width / factor) * factor)

  if h_bar * w_bar > max_pixels:
    beta = math.sqrt((height * width) / max_pixels)
    h_bar = math.floor(height / beta / factor) * factor
    w_bar = math.floor(width / beta / factor) * factor
  elif h_bar * w_bar < min_pixels:
    beta = math.sqrt(min_pixels / (height * width))
    h_bar = math.ceil(height * beta / factor) * factor
    w_bar = math.ceil(width * beta / factor) * factor

  return h_bar, w_bar

def token_calculate(image_path=''):
  if len(image_path) > 0:
    image = Image.open(image_path)
    height = image.height
    width = image.width
    print(f"Original size: {width}x{height}")
    resized_h, resized_w = smart_resize(height, width)
    token = int(resized_h * resized_w / (IMAGE_FACTOR * IMAGE_FACTOR)) + 2
    print(f"Resized: {resized_w}x{resized_h}, Tokens: {token}")
    return token
  else:
    raise ValueError("Image path cannot be empty. Provide a valid image file path")

if __name__ == "__main__":
  token = token_calculate(image_path="xxx/test.jpg")

Vídeo

Arquivos de vídeo geram dois tipos de fichas: video_tokens (visual) e audio_tokens (áudio).

video_tokens

O procedimento de cálculo é complexo. Para mais informações, veja o seguinte código:

# Before use, install: pip install opencv-python
import math
import os
import logging
import cv2

# Fixed parameters
FRAME_FACTOR = 2

# For Qwen3.5-Omni and Qwen3-Omni-Flash, IMAGE_FACTOR is 32
IMAGE_FACTOR = 32

# Aspect ratio of video frames
MAX_RATIO = 200

# Lower limit for video frame pixels. For Qwen3.5-Omni and Qwen3-Omni-Flash: 128 * 32 * 32
VIDEO_MIN_PIXELS = 128 * 32 * 32

# Upper limit for video frame pixels. For Qwen3.5-Omni and Qwen3-Omni-Flash: 768 * 32 * 32
VIDEO_MAX_PIXELS = 768 * 32 * 32

FPS = 2
# Minimum number of extracted frames
FPS_MIN_FRAMES = 4

# Maximum number of extracted frames
# Maximum number of extracted frames for Qwen3.5-Omni and Qwen3-Omni-Flash: 128
FPS_MAX_FRAMES = 128

# Maximum pixel value for video input. For Qwen3.5-Omni and Qwen3-Omni-Flash: 16384 * 32 * 32
VIDEO_TOTAL_PIXELS = 16384 * 32 * 32

def round_by_factor(number, factor):
  return round(number / factor) * factor

def ceil_by_factor(number, factor):
  return math.ceil(number / factor) * factor

def floor_by_factor(number, factor):
  return math.floor(number / factor) * factor

def get_video(video_path):
  cap = cv2.VideoCapture(video_path)
  frame_width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
  frame_height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
  total_frames = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))
  video_fps = cap.get(cv2.CAP_PROP_FPS)
  cap.release()
  return frame_height, frame_width, total_frames, video_fps

def smart_nframes(total_frames, video_fps):
  min_frames = ceil_by_factor(FPS_MIN_FRAMES, FRAME_FACTOR)
  max_frames = floor_by_factor(min(FPS_MAX_FRAMES, total_frames), FRAME_FACTOR)
  duration = total_frames / video_fps if video_fps != 0 else 0
  if duration - int(duration) > (1 / FPS):
    total_frames = math.ceil(duration * video_fps)
  else:
    total_frames = math.ceil(int(duration) * video_fps)
  nframes = total_frames / video_fps * FPS
  nframes = int(min(min(max(nframes, min_frames), max_frames), total_frames))
  if not (FRAME_FACTOR <= nframes <= total_frames):
    raise ValueError(f"nframes should in interval [{FRAME_FACTOR}, {total_frames}], but got {nframes}.")
  return nframes

def smart_resize(height, width, nframes, factor=IMAGE_FACTOR):
  min_pixels = VIDEO_MIN_PIXELS
  total_pixels = VIDEO_TOTAL_PIXELS
  max_pixels = max(min(VIDEO_MAX_PIXELS, total_pixels / nframes * FRAME_FACTOR), int(min_pixels * 1.05))
  if max(height, width) / min(height, width) > MAX_RATIO:
    raise ValueError(f"absolute aspect ratio must be smaller than {MAX_RATIO}, got {max(height, width) / min(height, width)}")
  h_bar = max(factor, round_by_factor(height, factor))
  w_bar = max(factor, round_by_factor(width, factor))
  if h_bar * w_bar > max_pixels:
    beta = math.sqrt((height * width) / max_pixels)
    h_bar = floor_by_factor(height / beta, factor)
    w_bar = floor_by_factor(width / beta, factor)
  elif h_bar * w_bar < min_pixels:
    beta = math.sqrt(min_pixels / (height * width))
    h_bar = ceil_by_factor(height * beta, factor)
    w_bar = ceil_by_factor(width * beta, factor)
  return h_bar, w_bar

def video_token_calculate(video_path):
  height, width, total_frames, video_fps = get_video(video_path)
  nframes = smart_nframes(total_frames, video_fps)
  resized_height, resized_width = smart_resize(height, width, nframes)
  video_token = int(math.ceil(nframes / FPS) * resized_height / 32 * resized_width / 32)
  video_token += 2  # Visual marks
  return video_token

if __name__ == "__main__":
  video_path = "spring_mountain.mp4"  # Your video path
  video_token = video_token_calculate(video_path)
  print("video_tokens:", video_token)

audio_tokens
- Qwen3.5-Omni Série: entrada áudio tokens totais = duração do áudio (em segundos) x 7; Saída de áudio tokens totais = Duração de áudio (em segundos) x 12,5
- Qwen3-Omni-FlashTotal de fichas para entrada e saída de áudio = duração de áudio (em segundos) x 12,5
- Se a duração do áudio é menor que 1 segundo, é calculado como 1 segundo.

Cota livre

Para mais informações sobre como reclamar, consultar e usar sua quota livre, veja [Contingência livre para novos usuários] (/resources/free-quota).

? Limites de velocidade ?

Para regras de limite de taxa de modelo e FAQ, veja [Limites de taxa] (/developer-guides/administration/rate-limits).

Códigos de erro.

Se uma ligação falhar, veja [Mensagens de erro] (/api-reference/preparation/error-messages).

Lista de voz

Para as vozes, cada modelo omnimodal suporta, incluindo amostras e voice valores dos parâmetros, veja [lista de voz Omni-modal] (/developer-guides/speech/omni-voice-list).

Quadros de referência

Tabelas restauradas a partir da referência original; a ordem das colunas e os valores técnicos foram preservados.

Quadro 1

Use o caso.	Comprimento de áudio recomendado.	Prompt recomendado
Revisão rápida, baixo custo.	≤ 60 minutos	Simples prompt em 50 palavras
Extração de conteúdo (segmento de áudio longo)	≤ 60 minutos	Simples prompt em 50 palavras
Análise padrão (audio tagging)	≤ 2 minutos	Use o prompt estruturado abaixo.
Análise de fino tamanho (multiple speakers/cenas complexas)	≤ 1 minuto	Use o prompt estruturado abaixo.