collect issues

[seven1240]

I found a few issues on collecting.

steps:

• make an outbound SIP call, the other side automatically answer and play audio
• collect

Problems:

• it returns no_match when speechTimeout = 10 which is less than the real audio
• and then a Missing the instance error
• some collect.updated lines shows duplicated message with a SPACE separated, which matches the log in the signalwire web console, e.g.
  • B collect.updated 好欢迎致的 好欢迎致电
  • B collect.updated 好欢迎致电烟台小樱桃网络科技有限公司烟 好欢迎致电烟台小樱桃网络科技有限公司烟台
• automatic_punctuation doesn't work
• continuous doesn't work

the js:

require('dotenv').config()
const { Voice } = require("@signalwire/realtime-api")
require("util").inspect.defaultOptions.depth = null;

const client = new Voice.Client({
  project: process.env.SIGNALWIRE_PROJECT_ID,
  token: process.env.SIGNALWIRE_TOKEN,
  contexts: ["home"],
});

const langA = "en-US";
const langB = "zh-CN";
const queueA = [];
const queueB = [];

async function setCollectCallbacks(call, leg) {
  call.on("collect.started", (collect) => {
    console.log(leg, "collect.started", 'collecting');
  });
  call.on("collect.startOfInput", (collect) => {
    console.log(leg, "collect.startOfInput Input collection started.");
  });
  call.on("collect.updated", (collect) => {
    console.log(leg, "collect.updated", collect.speech);
  });
  call.on("collect.ended", (collect) => {
    console.log(leg, "collect.ended", collect.speech);
  });
  call.on("collect.failed", (collect) => {
    console.log(leg, "collect.failed", collect.reason);
  });
}

async function startCollect(call, leg, lang) {
  console.log(leg, "collect starting");
  const collect = await call.collect({
    speech: {
      endSilenceTimeout: 2,
      speechTimeout: 10,
      language: lang,
      automatic_punctuation: true,
      // hints: ["sales", "support", "representative"],
    },
    continuous: true,
    partialResults: true,
    sendStartOfInput: true,
  });
  console.log(leg, "collect started");
  return collect;
}

async function collectLoop(collect, call, leg, lang) {
  // await collect.startInputTimers();
  // console.log("collect", collect)
  console.log(leg, 'waiting for collect result')
  const {type, result} = await collect.ended();
  console.log(type, result)

  // await collect.stop()
  // console.log(leg, 'collect stopped')
  // const newCollect = await startCollect(call, leg, lang)
  // await collectLoop(newCollect, call, leg, lang, queueA, queueB)
}

console.log("Call Handler Started");

(async() => {
  let peer
  try {
    peer = await client.dialSip({
      from: "sip:[email protected]",
      to: "sip:[email protected]:20003;transport=tcp",
    });
    console.log("B", "Call connected!");

    await setCollectCallbacks(peer, 'B')
    const collect = await startCollect(peer, 'B', 'zh-CN')
    await collectLoop(collect, peer, 'B', 'zh-CN', queueB, queueA)
    console.log('waiting call to end ...')
    await peer.waitFor('ended')
  } catch (error) {
    console.error("Error connecting call:", error);
  }
  console.log("hangup peer");
  await peer.hangup();
})();

console.log("ready");

full log:

(base) ➜  js node call.js
Call Handler Started
ready
B Call connected!
B collect starting
B collect.started collecting
B collect started
B waiting for collect result
B collect.startOfInput Input collection started.
B collect.updated 好
B collect.updated 花
B collect.updated 欢
B collect.updated 欢迎
B collect.updated 欢迎这
B collect.updated 好欢迎致的 好欢迎致电
B collect.updated 好欢迎致电压
B collect.updated 好欢迎致电烟
B collect.updated 好欢迎致电烟台
B collect.updated 好欢迎致电烟台小
B collect.updated 好欢迎致电烟台小一
B collect.updated 好欢迎致电烟台小英
B collect.updated 好欢迎致电烟台小樱桃
B collect.updated 好欢迎致电烟台小樱桃网
B collect.updated 好欢迎致电烟台小樱桃网络
B collect.updated 好欢迎致电烟台小樱桃网络可
B collect.updated 好欢迎致电烟台小樱桃网络科
B collect.updated 好欢迎致电烟台小樱桃网络科技
B collect.updated 好欢迎致电烟台小樱桃网络科技有
B collect.updated 好欢迎致电烟台小樱桃网络科技有限
B collect.updated 好欢迎致电烟台小樱桃网络科技有限公
B collect.updated 好欢迎致电烟台小樱桃网络科技有限公司
B collect.updated 好欢迎致电烟台小樱桃网络科技有限公司
B collect.updated 好欢迎致电烟台小樱桃网络科技有限公司烟 好欢迎致电烟台小樱桃网络科技有限公司烟台
B collect.updated 好欢迎致电烟台小樱桃网络科技有限公司烟台小
B collect.updated 好欢迎致电烟台小樱桃网络科技有限公司烟台小一
B collect.updated 好欢迎致电烟台小樱桃网络科技有限公司烟台小英
B collect.updated 好欢迎致电烟台小樱桃网络科技有限公司烟台小樱桃
B collect.updated 好欢迎致电烟台小樱桃网络科技有限公司烟台小樱桃是
B collect.updated 好欢迎致电烟台小樱桃网络科技有限公司烟台小樱桃是高
B collect.updated 好欢迎致电烟台小樱桃网络科技有限公司烟台小樱桃市高新
B collect.updated 好欢迎致电烟台小樱桃网络科技有限公司烟台小樱桃市高新技
B collect.updated 好欢迎致电烟台小樱桃网络科技有限公司烟台小樱桃市高新技术
B collect.updated 好欢迎致电烟台小樱桃网络科技有限公司烟台小樱桃市高新技术企
B collect.updated 好欢迎致电烟台小樱桃网络科技有限公司烟台小樱桃市高新技术企业
B collect.updated 好欢迎致电烟台小樱桃网络科技有限公司烟台小樱桃市高新技术企业做
B collect.updated 好欢迎致电烟台小樱桃网络科技有限公司烟台小樱桃市高新技术企业做了
B collect.updated 好欢迎致电烟台小樱桃网络科技有限公司烟台小樱桃市高新技术企业坐落在
B collect.updated 好欢迎致电烟台小樱桃网络科技有限公司烟台小樱桃市高新技术企业坐落在美
B collect.updated 好欢迎致电烟台小樱桃网络科技有限公司烟台小樱桃市高新技术企业坐落在美丽
B collect.updated 好欢迎致电烟台小樱桃网络科技有限公司烟台小樱桃市高新技术企业坐落在美丽的
B collect.updated 好欢迎致电烟台小樱桃网络科技有限公司烟台小樱桃市高新技术企业坐落在美丽的蝴
B collect.updated 好欢迎致电烟台小樱桃网络科技有限公司烟台小樱桃市高新技术企业坐落在美丽的花
B collect.updated 好欢迎致电烟台小樱桃网络科技有限公司烟台小樱桃市高新技术企业坐落在美丽的黄
B collect.updated 好欢迎致电烟台小樱桃网络科技有限公司烟台小樱桃市高新技术企业坐落在美丽的黄海
B collect.updated 好欢迎致电烟台小樱桃网络科技有限公司烟台小樱桃市高新技术企业坐落在美丽的黄海之
B collect.updated 好欢迎致电烟台小樱桃网络科技有限公司烟台小樱桃市高新技术企业坐落在美丽的黄海之滨
B collect.updated 好欢迎致电烟台小樱桃网络科技有限公司烟台小樱桃市高新技术企业坐落在美丽的黄海之滨
B collect.failed no_match
no_match { type: 'no_match' }
waiting call to end ...
Error: Missing the instance
    at voiceCallCollectWorker (/Users/seven/work/call-translator/js/node_modules/@signalwire/realtime-api/dist/index.node.js:5120:11)
    at voiceCallCollectWorker.next (<anonymous>)
    at next (/Users/seven/work/call-translator/js/node_modules/@redux-saga/core/dist/redux-saga-core.dev.cjs.js:1161:27)
    at proc (/Users/seven/work/call-translator/js/node_modules/@redux-saga/core/dist/redux-saga-core.dev.cjs.js:1112:3)
    at /Users/seven/work/call-translator/js/node_modules/@redux-saga/core/dist/redux-saga-core.dev.cjs.js:589:17
    at immediately (/Users/seven/work/call-translator/js/node_modules/@redux-saga/core/dist/redux-saga-core.dev.cjs.js:60:12)
    at runForkEffect (/Users/seven/work/call-translator/js/node_modules/@redux-saga/core/dist/redux-saga-core.dev.cjs.js:588:3)
    at runEffect (/Users/seven/work/call-translator/js/node_modules/@redux-saga/core/dist/redux-saga-core.dev.cjs.js:1208:7)
    at digestEffect (/Users/seven/work/call-translator/js/node_modules/@redux-saga/core/dist/redux-saga-core.dev.cjs.js:1275:5)
    at next (/Users/seven/work/call-translator/js/node_modules/@redux-saga/core/dist/redux-saga-core.dev.cjs.js:1165:9)
The above error occurred in task voiceCallCollectWorker
    created by worker
    created by voiceCallingWroker
Tasks cancelled due to error:
worker
voiceCallingWroker

When chang the speechTimeout to 50, it stuck on collect.updated and no ended or failed callback called, and the await collectio.ended() stuck forever. the last two line of log:

B collect.updated 好欢迎致电烟台小樱桃网络科技有限公司烟台小樱桃市高新技术企业坐落在美丽的黄海之滨让互联互通更顺畅样实时互动更高效小樱桃将竭诚为您服务
B collect.updated 好欢迎致电烟台小樱桃网络科技有限公司烟台小樱桃市高新技术企业坐落在美丽的黄海之滨让互联互通更顺畅样实时互动更高效小樱桃将竭诚为您服务

[seven1240]

two call ids mathes the above two cases:

c6d519d9-9ebc-4762-abbe-711cf7dcdff7
e04b1576-c209-464f-bbef-da47fec2289b

[seven1240]

package.json

{
  "dependencies": {
    "@signalwire/realtime-api": "^3.10.3",
    "dotenv": "^16.3.1"
  }
}