Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions examples/smart-cockpit/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,10 +97,10 @@ A preflight validates the Realtime configuration and all four ports before any c
- Users can create and run persistent cockpit-specific workflows by voice. The
backend Agent loads these workflows and composes existing MCP tools; they are
not dynamic MCP plugins, A2A Agent Card entries, or globally installed Agent Skills.
- The foreground Agent owns realtime conversation and directly calls weather,
vehicle-state, window, sunroof, headlight, and climate tools through standard
MCP. Explicit single vehicle-control commands execute inline without a redundant
second confirmation.
vehicle-state, window, sunroof, headlight, climate, and short in-route
navigation tools through standard MCP. Explicit single vehicle-control,
route-view, voice, and current-route preference commands execute inline
without a redundant second confirmation.
- Other cockpit work goes through the fixed `spawn_thinking` bridge. The example
backend attaches over A2A, and Qwen3.8-Flash discovers the complete backend MCP
surface for vehicle control, navigation, music, flash-buy, and custom workflows,
Expand Down
2 changes: 1 addition & 1 deletion examples/smart-cockpit/README_ZH.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ npm run example:smart-cockpit
- 用户可以通过语音创建和运行持久化的座舱自定义技能;技能是按座舱隔离的用户工作流,
由后台 Agent 加载后编排现有 MCP 工具。它不是动态 MCP 插件、A2A Agent Card 或全局 Agent Skill。
- 前台 Agent 负责实时聊天,通过标准 MCP 直接调用天气、车况以及车窗、天窗、
大灯和空调工具;用户明确说出的单次车辆控制指令直接执行,不再增加重复确认。
大灯、空调和短导航工具;用户明确说出的单次车辆控制、导航视图、导航播报和当前路线偏好指令直接执行,不再增加重复确认。
- 其他座舱任务通过固定的 `spawn_thinking` 桥梁提交给后台。示例后台通过 A2A
接入 Gateway,Qwen3.8-Flash 会发现并调用独立的后台 MCP 工具面,完成车控、
导航、音乐、闪购和自定义技能任务,包括有序的多途经点导航。
Expand Down
5 changes: 5 additions & 0 deletions examples/smart-cockpit/agent/executor.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,11 @@ export const COCKPIT_AGENT_PROMPT = `你是智能座舱的后台 Agent,负责
- 导航请求可以包含多个有序途经点。将中间地点放入 waypoints,最后一个地点作为 destination。
- 用户明确说“导航到”“带我去”“去某地”或“开始导航”时直接调用 navigation_start,成功后不要再次询问是否开始。
- 只有用户明确说“查路线”“怎么走”“多远”“多久”或“先看看路线”时才调用 navigation_route_query。
- 用户在已有导航中说“中途去一下”“顺路去”“加个途经点”时调用 navigation_add_waypoint;说“不去这个途经点了”“取消途经点”时调用 navigation_remove_waypoint。
- 用户在已有导航中说“目的地改成”“换个地方”时调用 navigation_change_destination;只说“换成不走高速”“改成少收费”“避开拥堵”时调用 navigation_set_route_strategy。
- 用户只是找地点或周边 POI、没有要求导航时,调用 navigation_search_place。
- “回家”“去公司”等常用地点导航优先调用 navigation_to_favorite;设置家/公司/学校地址时调用 navigation_set_favorite。
- 导航静音、详细播报、简洁播报调用 navigation_set_voice;查看全程、跟车视角、北向上调用 navigation_set_view。
- 闪购中,只有“看看”“搜一下”“有哪些”等浏览意图使用 search;“帮我点”“来一份”“就这个”“加入购物车”使用 add_to_cart,不得退回再次搜索。
- 闪购加购后必须先返回订单预览;只有用户在后续指令中明确确认后,才调用 confirm_order。
- 用户明确要求创建自定义技能时,调用 custom_skill_create 保存名称、简介和可执行步骤;未得到创建意图时不要擅自保存。
Expand Down
133 changes: 133 additions & 0 deletions examples/smart-cockpit/client/src/App.css
Original file line number Diff line number Diff line change
Expand Up @@ -833,6 +833,13 @@ button {
font-weight: 700;
}

.nav-voice-badge {
padding-left: 8px;
border-left: 1px solid rgba(95, 124, 134, 0.18);
color: #5c6a72;
font-weight: 700;
}

.nav-ai-dot {
width: 8px;
height: 8px;
Expand Down Expand Up @@ -1118,6 +1125,126 @@ button {
transform: scale(1.08);
}

.destination-search-panel {
position: absolute;
top: 22px;
left: 24px;
z-index: 4;
width: min(360px, calc(100% - 48px));
display: grid;
grid-template-columns: 1fr;
gap: 0;
border-radius: 8px;
background: rgba(255, 255, 255, 0.82);
border: 1px solid rgba(255, 255, 255, 0.9);
box-shadow: 0 12px 28px rgba(80, 111, 123, 0.14);
backdrop-filter: blur(14px);
overflow: hidden;
}

.destination-search-field {
min-height: 52px;
display: flex;
align-items: center;
gap: 10px;
padding: 0 18px;
background: rgba(255, 255, 255, 0.72);
border: 0;
border-bottom: 1px solid rgba(95, 124, 134, 0.12);
color: #7a8991;
font-size: 14px;
font-weight: 800;
text-align: left;
transition: background 140ms ease, color 140ms ease;
}

.destination-search-field:hover {
background: rgba(255, 255, 255, 0.94);
color: #40515a;
}

.destination-search-field .icon {
width: 19px;
height: 19px;
flex-shrink: 0;
}

.destination-shortcuts {
display: grid;
grid-template-columns: 1fr;
}

.destination-shortcut {
min-height: 58px;
display: grid;
grid-template-columns: 34px minmax(0, 1fr) 56px;
align-items: center;
gap: 10px;
padding: 9px 12px 9px 14px;
border-bottom: 1px solid rgba(95, 124, 134, 0.1);
}

.destination-shortcut:last-child {
border-bottom: 0;
}

.destination-shortcut-icon {
width: 34px;
height: 34px;
display: grid;
place-items: center;
border-radius: 50%;
background: rgba(95, 124, 134, 0.12);
color: #40515a;
font-size: 13px;
font-weight: 900;
}

.destination-shortcut.is-configured .destination-shortcut-icon {
background: rgba(53, 214, 194, 0.14);
color: #168b81;
}

.destination-shortcut-text {
min-width: 0;
display: flex;
flex-direction: column;
gap: 3px;
}

.destination-shortcut-text strong {
color: #26333a;
font-size: 14px;
line-height: 1.1;
}

.destination-shortcut-text span {
max-width: 100%;
color: #6b7980;
font-size: 12px;
line-height: 1.25;
white-space: nowrap;
overflow: hidden;
text-overflow: ellipsis;
}

.destination-shortcut-action {
height: 32px;
border-radius: 8px;
border: 1px solid rgba(53, 214, 194, 0.34);
background: rgba(53, 214, 194, 0.14);
color: #168b81;
font-size: 12px;
font-weight: 800;
transition: background 140ms ease, border-color 140ms ease, color 140ms ease;
}

.destination-shortcut-action:hover {
background: #35d6c2;
border-color: #35d6c2;
color: #fff;
}

.map-overview-btn {
flex-shrink: 0;
height: 40px;
Expand Down Expand Up @@ -2519,6 +2646,7 @@ button {
gap: 4px;
}

.chat-clear,
.chat-close {
width: 28px;
height: 28px;
Expand All @@ -2529,6 +2657,11 @@ button {
transition: background 150ms ease, color 150ms ease;
}

.chat-clear:hover {
background: rgba(95, 124, 134, 0.1);
color: #2e3c44;
}

.chat-close:hover {
background: rgba(215, 68, 103, 0.1);
color: #d74467;
Expand Down
39 changes: 36 additions & 3 deletions examples/smart-cockpit/client/src/App.jsx
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,12 @@ function getClientId() {
return id
}

function createClientId() {
const id = crypto.randomUUID()
localStorage.setItem('clientId', id)
return id
}

function getStoredChoice(key, fallback, validValues) {
const value = localStorage.getItem(key)
return validValues.includes(value) ? value : fallback
Expand All @@ -86,13 +92,14 @@ function parseHash() {
}

export default function App() {
const clientId = useMemo(() => getClientId(), [])
const [clientId, setClientId] = useState(() => getClientId())
const cockpitId = import.meta.env.VITE_COCKPIT_ID || 'default'
const {
state: cockpitState,
progress: cockpitProgress,
activity: cockpitActivity,
execute: executeCockpitCommand,
reset: resetCockpitState,
} = useCockpitState(cockpitId)
const {
skills: customSkills,
Expand Down Expand Up @@ -140,6 +147,14 @@ export default function App() {
runCockpitCommand('music_play', { query: PLAYLIST[index]?.title })
}, [runCockpitCommand])

const navigateToFavorite = useCallback((favoriteType) => {
runCockpitCommand('navigation_to_favorite', { favoriteType })
}, [runCockpitCommand])

const openDestinationInput = useCallback(() => {
setShowChat(true)
}, [])

useEffect(() => {
localStorage.setItem(PERSONA_STORAGE_KEY, selectedPersona)
}, [selectedPersona])
Expand Down Expand Up @@ -357,6 +372,15 @@ export default function App() {
sendInput([{ type: 'text', text }])
), [sendInput])

const handleClearDebug = useCallback(() => {
voiceAssistantMessageIdRef.current = null
setChatMessages([])
setClientId(createClientId())
resetCockpitState().catch(error => {
console.warn('Cockpit reset failed', error)
})
}, [resetCockpitState])

useEffect(() => {
const targetScreen = cockpitScreenForProgress(visualProgress, {
navigationActive: navState.status === 'navigating',
Expand Down Expand Up @@ -402,7 +426,16 @@ export default function App() {
onToggleVoiceMute={toggleVoiceMute}
/>
{screen === 'main' && (
<MapPanel navState={navState} navProgress={visualProgress} mapActions={mapActions} routeStrategy={routeStrategy} onStrategyChange={setRouteStrategy} />
<MapPanel
navState={navState}
navProgress={visualProgress}
mapActions={mapActions}
routeStrategy={routeStrategy}
onStrategyChange={setRouteStrategy}
onFavoriteNavigate={navigateToFavorite}
onFavoriteSetup={openDestinationInput}
onSearchDestination={openDestinationInput}
/>
)}
{screen === 'music' && (
<MusicPanel musicState={musicState} onPlay={musicPlay} onPause={musicPause} onNext={musicNext} onPrev={musicPrev} onSelectTrack={musicSelectTrack} />
Expand All @@ -423,7 +456,7 @@ export default function App() {
/>
)}
</div>
{showChat && <ChatPanel onClose={toggleChat} messages={chatMessages} onMessagesChange={setChatMessages} onSendMessage={handleTextMessage} voiceActive={!voiceMuted} />}
{showChat && <ChatPanel onClose={toggleChat} onClear={handleClearDebug} messages={chatMessages} onMessagesChange={setChatMessages} onSendMessage={handleTextMessage} voiceActive={!voiceMuted} />}
</div>

<Dock screen={screen} onNavigateHome={navigateHome} onOpenSettings={() => openSettings('persona')} onToggleChat={toggleChat} carState={carState} musicState={musicState} onTogglePlay={musicState.playing ? musicPause : musicPlay} onOpenMusic={openMusic} onOpenFlashBuy={openFlashBuy} />
Expand Down
23 changes: 23 additions & 0 deletions examples/smart-cockpit/client/src/components/ChatPanel.jsx
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,15 @@ const TOOL_TAGS = {
navigation_start: { label: '开始导航', cls: 'tag-nav' },
navigation_route_query: { label: '路线查询', cls: 'tag-nav' },
navigation_stop: { label: '停止导航', cls: 'tag-nav' },
navigation_add_waypoint: { label: '增加途经点', cls: 'tag-nav' },
navigation_remove_waypoint: { label: '删除途经点', cls: 'tag-nav' },
navigation_change_destination: { label: '变更目的地', cls: 'tag-nav' },
navigation_set_route_strategy: { label: '路线偏好', cls: 'tag-nav' },
navigation_search_place: { label: '地点搜索', cls: 'tag-nav' },
navigation_to_favorite: { label: '常用地点导航', cls: 'tag-nav' },
navigation_set_favorite: { label: '设置常用地点', cls: 'tag-nav' },
navigation_set_voice: { label: '导航播报', cls: 'tag-nav' },
navigation_set_view: { label: '导航视图', cls: 'tag-nav' },
maps_geo: { label: '地理编码', cls: 'tag-nav' },
maps_text_search: { label: '地点搜索', cls: 'tag-nav' },
maps_search_detail: { label: '地点详情', cls: 'tag-nav' },
Expand Down Expand Up @@ -50,6 +59,7 @@ function getDefaultPosition(panel) {

export default function ChatPanel({
onClose,
onClear,
messages,
onMessagesChange,
onSendMessage,
Expand Down Expand Up @@ -136,6 +146,19 @@ export default function ChatPanel({
<div className="chat-header" onMouseDown={handleDragStart}>
<span className="chat-title">Qwen Audio Agent Smart Cockpit · 调试</span>
<div className="chat-header-actions">
<button
className="chat-clear"
onClick={onClear}
onMouseDown={event => event.stopPropagation()}
aria-label="清空"
title="清空"
>
<svg className="icon icon-sm" viewBox="0 0 24 24" aria-hidden="true">
<path d="M4 7h16" stroke="currentColor" strokeWidth="2" strokeLinecap="round" />
<path d="M10 11v6M14 11v6" stroke="currentColor" strokeWidth="2" strokeLinecap="round" />
<path d="M6 7l1 13h10l1-13M9 7V4h6v3" stroke="currentColor" strokeWidth="2" strokeLinejoin="round" />
</svg>
</button>
<button className="chat-close" onClick={onClose} aria-label="关闭">
<svg className="icon icon-sm" viewBox="0 0 24 24" aria-hidden="true">
<path d="M18 6 6 18M6 6l12 12" stroke="currentColor" strokeWidth="2" strokeLinecap="round" />
Expand Down
Loading