The resources (image, video, voice) generated by our API are valid for 7 days. Please save the relevant resources as soon as possible to prevent expiration.

To experience our live avatar streaming feature in action, explore our demo built on the Agora streaming service: AKool Streaming Avatar React Demo.

​
Upload Streaming Avatar

POST https://openapi.akool.com/api/open/v3/avatar/create

Request Headers

ParameterTypeRequiredDescription
AuthorizationStringYesBearer token for API authentication. Obtain from GetToken endpoint.

Body Attributes

ParameterTypeValueDescription
urlStringAvatar resource link. It is recommended that the video be about one minute long, and the avatar in the video content should rotate at a small angle and be clear.
avatar_idStringavatar unique ID，Can only contain /^a-zA-Z0-9/.
typeString2Avatar type 2 represents stream avatar, When type is 2, you need to wait until status is 3 before you can use it, You can get the current status in real time through the interface GET https://openapi.akool.com/api/open/v3/avatar/detail.
url_fromNumber1，2url source, 1 means akool and other links, 2 means other third-party links (currently only supports YouTube / TikTok / X / Google Drive)

Response Attributes

ParameterTypeValueDescription
codeint1000Interface returns business status code(1000:success)
msgStringOKInterface returns status information
dataObject{ avatar_id: "xx", url: "", status: 1 }avatar_id: Used by creating live avatar interface. url: You can preview the avatar via the link. status: 1-queueing, 2-processing, 3-success, 4-failed

Example

Request

curl --location 'https://openapi.akool.com/api/open/v3/avatar/create' \
--header 'authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6IjY0ZDk4NWM1NTcxNzI5ZDNlMjk5OTQ3NyIsInVpZCI6Mzc4MzM3LCJlbWFpbCI6Imh1Y2hlbkBha29vbC5jb20iLCJjcmVkZW50aWFsSWQiOiI2NjE1MGZmM2Q5MWRmYjc4OWYyNjFmNjEiLCJmaXJzdE5hbWUiOiJjaGVuIiwiZnJvbSI6InRvTyIsInR5cGUiOiJ1c2VyIiwiaWF0IjoxNzEyNzE0ODI4LCJleHAiOjIwMjM3NTQ4Mjh9.e050LbczNhUx-Gprqb1NSYhBCKKH2xMqln3cMnAABmE' \
--header 'Content-Type: application/json' \
--data '{
    "url": "https://drz0f01yeq1cx.cloudfront.net/1721197444322-leijun000.mp4",
    "avatar_id": "HHdEKhn7k7vVBlR5FSi0e",
    "type": 2,
    "url_from": 1
}'

Response

{
  "code": 1000,
  "msg": "ok",
  "data": {
    "_id": "655ffeada6976ea317087193",
    "disabled": false,
    "uid": 1,
    "type": 2,
    "from": 2,
    "status": 1,
    "sort": 12,
    "create_time": 1700788730000,
    "name": "Yasmin in White shirt",
    "avatar_id": "Yasmin_in_White_shirt_20231121",
    "url": "https://drz0f01yeq1cx.cloudfront.net/1700786304161-b574407f-f926-4b3e-bba7-dc77d1742e60-8169.png",
    "modify_url": "https://drz0f01yeq1cx.cloudfront.net/1700786304161-b574407f-f926-4b3e-bba7-dc77d1742e60-8169.png",
    "gender": "female",
    "thumbnailUrl": "https://drz0f01yeq1cx.cloudfront.net/avatar/thumbnail/1700786304161-b574407f-f926-4b3e-bba7-dc77d1742e60-8169.png",
    "crop_arr": []
  }
}

​
Get Streaming Avatar List

GET https://openapi.akool.com/api/open/v4/liveAvatar/avatar/list

Request Headers

ParameterTypeRequiredDescription
AuthorizationStringYesBearer token for API authentication. Obtain from GetToken endpoint.

Query Parameters

ParameterTypeRequiredDefaultDescription
pageNumberNo1Page number for pagination
sizeNumberNo100Number of items per page (max 100)

Response Properties

PropertyTypeDescription
codeIntegerResponse status code (1000 indicates success)
msgStringResponse status message
dataObjectContainer for response data
data.countNumberTotal number of streaming avatars
data.resultArrayList of streaming avatar objects

Streaming Avatar Object Properties

PropertyTypeDescription
avatar_idStringUnique identifier for the streaming avatar
voice_idStringAssociated voice model identifier
nameStringDisplay name of the avatar
urlStringURL to access the streaming avatar
thumbnailUrlStringURL for the avatar’s preview thumbnail
genderStringAvatar’s gender designation
availableBooleanIndicates if the avatar is currently available for use
typeNumberAvatar type identifier (2 for streaming avatars)
fromNumberSource identifier for the avatar, 2 for official and 3 for user created

Example

Request

curl --location 'https://openapi.akool.com/api/open/v4/liveAvatar/avatar/list?page=1&size=100' \
--header 'Authorization: Bearer {{Authorization}}'

Response

{
  "code": 1000,
  "msg": "OK",
  "data": {
    "count": 1,
    "result": [
      {
        "_id": "67498e55cd053926b2e75cf2",
        "uid": 1,
        "type": 2,
        "from": 2,
        "avatar_id": "dvp_Tristan_cloth2_1080P",
        "voice_id": "iP95p4xoKVk53GoZ742B",
        "name": "tristan_dvp",
        "url": "https://static.website-files.org/assets/avatar/avatar/streaming_avatar/tristan_10s_silence.mp4",
        "thumbnailUrl": "https://static.website-files.org/assets/avatar/avatar/thumbnail/1716457024728-tristan_cloth2_20240522.webp",
        "gender": "male",
        "available": true
      }
    ]
  }
}

​
Create session

POST https://openapi.akool.com/api/open/v4/liveAvatar/session/create

Request Headers

ParameterValueDescription
AuthorizationBearer {token}Your API Key used for request authorization. getToken

Body Attributes

ParameterTypeValueDescription
avatar_idStringDigital human model in real-time avatar, The current system provides only one option: “dvp_Tristan_cloth2_1080P”. If you want to use a custom uploaded video, you need to call the https://openapi.akool.com/api/open/v3/avatar/create interface to create a template. This process takes some time to process. You can check the processing status through the interface https://openapi.akool.com/api/open/v3/avatar/detail. When status=3, you can use the avatar_id field to pass it in.
durationNumberThe duration of the session, in seconds. The maximum value is 3600 seconds.
ParameterTypeValueDescription
codeint1000Interface returns business status code(1000:success)
msgStringInterface returns status information
dataObject{ "_id": "", "status": 1, "credentials": {} }_id: Interface returns data, status: the status of session: [1:queueing, 2:processing, 3:completed, 4:failed], credentials: the join information for the Agora SDK

Example

Body

{
  "_id": "6698c9d69cf7b0d61d1b6420",
  "status": 1,
  "stream_type": "agora",
  "credentials": {
    "agora_uid": 100000, // The user ID for the Agora SDK.
    "agora_app_id": "", // The App ID used for the Agora SDK.
    "agora_channel": "", // The specified channel name for the Agora SDK.
    "agora_token": "", // The authentication token for the Agora SDK, currently the valid time is 5 minutes.
  }
}

Request

curl --location 'https://openapi.akool.com/api/open/v4/liveAvatar/session/create' \
--header 'authorization: Bearer token' \
--header 'content-type: application/json' \
--data '{
    "avatar_id": "dvp_Tristan_cloth2_1080P"
}'

Response

{
    "code": 1000,
    "msg": "OK",
    "data": {
        "_id": "6698c9d69cf7b0d61d1b6420",
        "uid": 100010,
        "type": 1,
        "status": 1,
        "stream_type": "agora",
        "credentials": {
          "agora_uid": 100000, // The user ID for the Agora SDK.
          "agora_app_id": "", // The App ID used for the Agora SDK.
          "agora_channel": "", // The specified channel name for the Agora SDK.
          "agora_token": "", // The authentication token for the Agora SDK, currently the valid time is 5 minutes.
        }
    }
}

​
Get Session Info Result

GET https://openapi.akool.com/api/open/v4/liveAvatar/session/detail?id=6698c9d69cf7b0d61d1b6420

Request Headers

ParameterValueDescription
AuthorizationBearer {token}Your API Key used for request authorization.getToken

Query Attributes

ParameterTypeValueDescription
idStringNULLvideo db id: You can get it based on the _id field returned by Create Session .

Response Attributes

ParameterTypeValueDescription
codeint1000Interface returns business status code(1000:success)
msgStringOKInterface returns status information
dataObject{ status:1, _id:"", credentials:{} }status: the status of live avatar:(1:queueing, 2:processing, 3:completed, 4:failed) credentials: the url of live avatar, _id: Interface returns data

Example

Request

curl --location 'http://openapi.akool.com/api/open/v4/liveAvatar/session/detail?id=6698c9d69cf7b0d61d1b6420' \
--header 'Authorization: Bearer token'

Response

{
  "code": 1000,
  "msg": "OK",
  "data": {
    "_id": "6698c9d69cf7b0d61d1b6420",
    "uid": 100010,
    "type": 1,
    "status": 3,
    "stream_type": "agora",
    "credentials": {
        "agora_uid": 100000, // The user ID for the Agora SDK.
        "agora_app_id": "", // The App ID used for the Agora SDK.
        "agora_channel": "", // The specified channel name for the Agora SDK.
    }
  }
}

​
Close Session

POST https://openapi.akool.com/api/open/v4/liveAvatar/session/close

Request Headers

ParameterValueDescription
AuthorizationBearer {token}Your API Key used for request authorization. getToken

Body Attributes

ParameterTypeValueDescription
idStringNULLsession id: You can get it based on the _id field returned by Create session .

Response Attributes

ParameterTypeValueDescription
codeint1000Interface returns business status code(1000:success)
msgStringOKInterface returns status information

Example

Request

curl --location 'http://openapi.akool.com/api/open/v4/liveAvatar/session/close' \
--header 'Authorization: Bearer token'

Response

{
  "code": 1000,
  "msg": "OK"
}

​
Get Session List

GET https://openapi.akool.com/api/open/v4/liveAvatar/session/list?page=1&size=10&status=1

Request Headers

ParameterValueDescription
AuthorizationBearer {token}Your API Key used for request authorization. getToken

Query Attributes

ParameterTypeValueDescription
pageNumber1Current number of pages, Default is 1.
sizeNumber10Current number of returns per page, Default is 100.
statusNumberNULLsession status: (1:queueing, 2:processing, 3:completed, 4:failed).

Response Attributes

ParameterTypeValueDescription
codeint1000Interface returns business status code(1000:success)
msgStringOKInterface returns status information
dataArray{count: 1, data: [{ credentials: {} }] }task_id: task id of session. url: the url of live avatar.

Example

Request

curl --location 'http://openapi.akool.com/api/open/v4/liveAvatar/session/list?page=1&size=10&status=1' \
--header 'Authorization: Bearer token'

Response

{
  "code": 1000,
  "msg": "OK",
  "data": {
    "count": 18,
    "result": [
      {
        "_id": "666d3006247f07725af0f884",
        "uid": 100010,
        "type": 1,
        "status": 1,
        "stream_type": "agora",
        "credentials": {
          "agora_uid": 100000, // The user ID for the Agora SDK.
          "agora_app_id": "", // The App ID used for the Agora SDK.
          "agora_channel": "" // The specified channel name for the Agora SDK.
        },
      }
    ]
  }
}

​
Live Avatar Stream Message

IAgoraRTCClient.on(event: "stream-message", listener: (uid: UID, pld: Uint8Array) => void)

IAgoraRTCClient.sendStreamMessage(msg: Uint8Array | string, flag: boolean): Promise<void>;

Send Data

Chat Type Parameters

ParameterTypeRequiredValueDescription
vNumberYes2Version of the message
typeStringYeschatMessage type for chat interactions
midStringYesUnique message identifier for conversation tracking
idxNumberYesSequential index of the message, start from 0
finBooleanYesIndicates if this is the final part of the message
pldObjectYesContainer for message payload
pld.textStringYesText content to send to avatar (e.g. “Hello”)

Command Type Parameters

ParameterTypeRequiredValueDescription
vNumberYes2Protocol version number
typeStringYescommandSpecifies this is a system command message
midStringYesUnique ID to track and correlate command messages
pldObjectYesContains the command details and parameters
pld.cmdStringYesCommand action to execute. Valid values: “set-params” (update avatar settings), “interrupt” (stop current avatar response)
pld.dataObjectNoParameters for the command (required for “set-params”)
pld.data.vidStringNoVoice ID to change avatar’s voice. Only used with “set-params”. Get valid IDs from Voice List API
pld.data.vurlStringNoCustom voice model URL. Only used with “set-params”. Get valid URLs from Voice List API
pld.data.langStringNoLanguage code for avatar responses (e.g. “en”, “es”). Only used with “set-params”. Get valid codes from Language List API
pld.data.modeNumberNoAvatar interaction style. Only used with “set-params”. “1” = Retelling (avatar repeats content), “2” = Dialogue (avatar engages in conversation)
pld.data.bgurlStringNoURL of background image/video for avatar scene. Only used with “set-params”

JSON Example

{
    "v": 2,
    "type": "chat",
    "mid": "msg-1723629433573",
    "idx": 0,
    "fin": true,
    "pld": {
      "text": "Hello"
    },
}

Receive Data

Chat Type Parameters

ParameterTypeValueDescription
vNumber2Version of the message
typeStringchatMessage type for chat interactions
midStringUnique message identifier for tracking conversation flow
idxNumberSequential index of the message part
finBooleanIndicates if this is the final part of the response
pldObjectContainer for message payload
pld.fromString”bot” or “user”Source of the message - “bot” for avatar responses, “user” for speech recognition input
pld.textStringText content of the message

Command Type Parameters

ParameterTypeValueDescription
vNumber2Version of the message
typeStringcommandMessage type for system commands
midStringUnique identifier for tracking related messages in a conversation
pldObjectContainer for command payload
pld.cmdString”set-params”, “interrupt”Command to execute: “set-params” to update avatar settings, “interrupt” to stop current response
pld.codeNumber1000Response code from the server, 1000 indicates success
pld.msgStringResponse message from the server

JSON Example

{
    "v": 2,
    "type": "chat",
    "mid": "msg-1723629433573",
    "idx": 0,
    "fin": true,
    "pld": {
      "from": "bot",
      "text": "Hello!  How can I assist you today? "
    }
}

Typescript Example

const client: IAgoraRTCClient = AgoraRTC.createClient({
  mode: 'rtc',
  codec: 'vp8',
});

client.join(agora_app_id, agora_channel, agora_token, agora_uid);

client.on('stream-message', (message: Uint8Array | string) => {
  console.log('received: %s', message);
});

​
Integrating Your Own LLM Service

Before dispatching a message to the WebSocket, consider executing an HTTP request to your LLM service.

const client: IAgoraRTCClient = AgoraRTC.createClient({
  mode: 'rtc',
  codec: 'vp8',
});

client.join(agora_app_id, agora_channel, agora_token, agora_uid);

client.on('stream-message', (message: Uint8Array | string) => {
  console.log('received: %s', message);
});

let inputMessage = 'hello';

try {
  const response = await fetch('https://your-backend-host/api/llm/answer', {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      question: inputMessage,
    }),
  });

  if (response.ok) {
    const result = await response.json();
    inputMessage = result.answer;
  } else {
    console.error("Failed to fetch from backend", response.statusText);
  }
} catch (error) {
  console.error("Error during fetch operation", error);
}

const message = {
  v: 2,
  type: "chat",
  mid: "msg-1723629433573",
  idx: 0,
  fin: true,
  pld: {
      text: inputMessage,
  },
};
client.sendStreamMessage(JSON.stringify(message), false);

​
Response Code Description

Please note that if the value of the response code is not equal to 1000, the request is failed or wrong

ParameterValueDescription
code1000Success
code1003Parameter error or Parameter can not beempty
code1008The content you get does not exist
code1009Youdo not have permission to operate
code1101Invalid authorization or Therequest token has expired
code1102Authorization cannot be empty
code1200The account has been banned
code1201create audio error, pleasetry again later
code1202The same video cannot be translated lipSync inthe same language more than 1 times
code1203video should be with audio
code1204Your video duration is exceed 60s!
code1205Create videoerror, please try again later
code1207The video you are using exceeds thesize limit allowed by the system by 300M
code1209Please upload a videoin another encoding format
code1210The video you are using exceeds thevalue allowed by the system by 60fp
code1211Create lipsync error, pleasetry again later