virt-screen 集成示例

面向用户的产品说明见 WET 扩展屏。本文档面向开发者：提炼 virt-screen/ 中的 WetRTC 连接代码，便于理解端到端架构。

virt-screen 是 WetRTC 的生产级参考实现：Electron 主机多屏桌面捕获 + NestJS Socket.IO 信令 + 手机浏览器接收。

源码位置

模块	路径
发送端 composable	`virt-screen/src/renderer/composables/useMultiScreenShare.ts`
接收端页面	`virt-screen/src/renderer/views/ReceiverView.vue`
WetRTC Vue 封装	`virt-screen/src/renderer/composables/useWetRTC.ts`
Socket.IO 信令	`virt-screen/src/renderer/composables/useSignalChannel.ts`
信令服务端	`virt-screen/src/kernel/gateway/signaling.gateway.ts`
低延迟常量	`virt-screen/src/shared/low-latency-config.ts`

架构概览

mermaid

sequenceDiagram
  participant Host as Electron 主机
  participant Kernel as NestJS Kernel :3450
  participant Mobile as 手机浏览器

  Host->>Kernel: join-room screen-{displayId}
  Host->>Host: captureDesktop + WetRTC sendonly
  Host->>Kernel: connect (passive, 等 offer)

  Mobile->>Kernel: join-room screen-{displayId}
  Mobile->>Kernel: signal offer
  Kernel->>Host: signal offer
  Host->>Kernel: signal answer
  Kernel->>Mobile: signal answer
  Note over Host,Mobile: ICE / SRTP 直连或经 STUN
  Mobile->>Mobile: video 播放 + playoutDelayHint

每块屏幕一个房间：roomId = screen-${displayId}
每房间一条 WebRTC 连接：不做 simulcast 多 viewer（viewer 离开后会 disconnect + connect 等待下一个）
当前仅视频：audio: false，未推音频轨

双端 WetRTC 角色（必读）

两端 initiator / polite 必须成对，否则 SDP 冲突或无人发 offer：

/**
 * virt-screen 双端 WetRTC 角色对照（集成 WetRTC 时最容易配反的组合）
 */
export const VIRT_SCREEN_WETRTC_ROLES = {
  electronHost: {
    file: 'virt-screen/src/renderer/composables/useMultiScreenShare.ts',
    direction: 'sendonly' as const,
    initiator: false,
    polite: true,
    roomId: 'screen-${displayId}',
    deviceId: 'host-${displayId}',
    captures: 'Electron getUserMedia + chromeMediaSourceId',
    waitsFor: '移动端 createOffer',
  },
  mobileReceiver: {
    file: 'virt-screen/src/renderer/views/ReceiverView.vue',
    direction: 'recvonly' as const,
    initiator: true,
    polite: false,
    roomId: 'query.roomId（扫码带入）',
    deviceId: 'receiver-xxx（localStorage 持久化）',
    captures: '无（只播放远端 video track）',
    waitsFor: '无，连接后立即发 offer',
  },
}

端	direction	initiator	polite	谁发 offer
Electron 主机	`sendonly`	`false`	`true`	手机
手机浏览器	`recvonly`	`true`	`false`	手机

1. 信令：Socket.IO → SignalChannel

Kernel 将 signal 事件转发到同 room 其他客户端（见 signaling.gateway.ts 的 handleSignal）。

/**
 * 来源：virt-screen/src/renderer/composables/useSignalChannel.ts
 * 将 WetRTC 的 SignalChannel 接到 NestJS + Socket.IO 信令服务（kernel :3450）。
 */
import { io, type Socket } from 'socket.io-client'
import type { SignalChannel, SignalMessage } from '@wetspace/wetrtc'

export interface JoinRoomPayload {
  roomId: string
  deviceId: string
  name?: string
  platform?: string
}

export interface SignalChannelIdentity {
  deviceId: string
  name?: string
  platform?: string
}

const KERNEL_ORIGIN = 'https://localhost:3450'

export function createSignalChannel(
  roomId: string,
  getIdentity: () => SignalChannelIdentity,
) {
  const socket: Socket = io(KERNEL_ORIGIN, {
    transports: ['websocket', 'polling'],
    path: '/socket.io/',
  })

  const emitJoinRoom = () => {
    const { deviceId, name, platform } = getIdentity()
    socket.emit('join-room', { roomId, deviceId, name, platform } satisfies JoinRoomPayload)
  }

  socket.on('connect', emitJoinRoom)

  const channel: SignalChannel = {
    async send(data: SignalMessage) {
      const { deviceId } = getIdentity()
      socket.emit('signal', { roomId, ...data, from: deviceId })
    },
    onMessage(handler) {
      const listener = (msg: SignalMessage & { roomId: string }) => {
        if (msg.roomId === roomId) handler(msg)
      }
      socket.on('signal', listener)
      return () => socket.off('signal', listener)
    },
  }

  return {
    channel,
    onPeerJoined(handler: (payload: { deviceId?: string }) => void) {
      socket.on('peer-joined', handler)
      return () => socket.off('peer-joined', handler)
    },
    onPeerLeft(handler: (payload: { deviceId?: string }) => void) {
      socket.on('peer-left', handler)
      return () => socket.off('peer-left', handler)
    },
    close() {
      if (socket.connected) socket.emit('leave-room', { roomId })
      socket.disconnect()
    },
  }
}

WetRTC 只关心 SignalChannel.send / onMessage；join-room、peer-joined、peer-left 由 virt-screen 扩展处理。

2. 低延迟配置

/**
 * 来源：virt-screen/src/shared/low-latency-config.ts
 * Electron 发送端共用的 WetRTC 视频编码与 codec 偏好。
 */
import type { VideoEncodingOptions } from '@wetspace/wetrtc'

export const CAPTURE_MAX_WIDTH = 1920
export const CAPTURE_MAX_HEIGHT = 1080
export const CAPTURE_MAX_FRAME_RATE = 75

export const SCREEN_VIDEO_ENCODING: VideoEncodingOptions = {
  contentHint: 'motion',
  maxFrameRate: CAPTURE_MAX_FRAME_RATE,
  maxBitrate: 4_000_000,
  degradationPreference: 'maintain-framerate',
}

export const RECEIVER_PLAYOUT_DELAY_HINT = 0
export const PREFERRED_VIDEO_CODEC = 'h264' as const

传入 WetRTC 构造函数：

videoEncoding: SCREEN_VIDEO_ENCODING,
preferredVideoCodec: PREFERRED_VIDEO_CODEC,

3. Electron 发送端

桌面捕获

/**
 * 来源：virt-screen/src/renderer/utils/capture-desktop.ts
 * Electron 桌面捕获：getUserMedia + chromeMediaSourceId（非 getDisplayMedia）。
 */
import {
  CAPTURE_MAX_FRAME_RATE,
  CAPTURE_MAX_HEIGHT,
  CAPTURE_MAX_WIDTH,
} from './low-latency-config'

/** sourceId 来自主进程 desktopCapturer.getSources() */
export async function captureDesktopSource(sourceId: string): Promise<MediaStream> {
  return navigator.mediaDevices.getUserMedia({
    audio: false,
    video: {
      mandatory: {
        chromeMediaSource: 'desktop',
        chromeMediaSourceId: sourceId,
        maxWidth: CAPTURE_MAX_WIDTH,
        maxHeight: CAPTURE_MAX_HEIGHT,
        maxFrameRate: CAPTURE_MAX_FRAME_RATE,
      },
    },
  } as MediaStreamConstraints)
}

sourceId 来自主进程 desktopCapturer.getSources()，通过 IPC 传给渲染进程（见 virt-screen/src/main/screen-sources.ts）。

启动会话

/**
 * 来源：virt-screen/src/renderer/composables/useMultiScreenShare.ts（startSession 核心流程）
 *
 * Electron 主机 = sendonly + 被动协商（initiator: false, polite: true）
 * 房间 ID：screen-${displayId}，每块屏独立一条 WebRTC 连接。
 */
import { WetRTC } from '@wetspace/wetrtc'
import { createSignalChannel } from './signal-channel-socketio'
import { captureDesktopSource } from './capture-desktop-electron'
import { PREFERRED_VIDEO_CODEC, SCREEN_VIDEO_ENCODING } from './low-latency-config'

interface ScreenSource {
  display_id: string
  id: string
}

export async function startScreenShareSession(source: ScreenSource) {
  const roomId = `screen-${source.display_id}`

  const signalHandle = createSignalChannel(roomId, () => ({
    deviceId: `host-${source.display_id}`,
  }))

  const rtc = new WetRTC({
    signal: signalHandle.channel,
    direction: 'sendonly',
    initiator: false,   // 等移动端 recvonly 发起 offer
    polite: true,       // Perfect Negotiation 礼貌方
    videoEncoding: SCREEN_VIDEO_ENCODING,
    preferredVideoCodec: PREFERRED_VIDEO_CODEC,
  })

  rtc.on('statechange', (state, prev) => {
    console.log('[host]', roomId, prev, '→', state)
    // viewer 断开后：idle ← connected → 触发 handleViewerLeft 重新 connect 等待新 viewer
  })

  rtc.on('stats', (snap) => {
    console.log('[host] fps', snap.frameRate, 'codec', snap.codec, 'rtt', snap.roundTripTime)
  })

  signalHandle.onPeerLeft((payload) => {
    if (payload.deviceId?.startsWith('host-')) return
    // disconnect + connect：保持捕获，等待下一个手机扫码
  })

  const stream = await captureDesktopSource(source.id)
  for (const track of stream.getVideoTracks()) {
    rtc.addTrack(track, stream)
    track.addEventListener('ended', () => rtc.dispose(), { once: true })
  }

  await rtc.connect()
  return { rtc, signalHandle, stream }
}

Vue 集成要点（HomeView.vue）：

const activeDisplayId = computed(() => selectedSource.value?.display_id ?? null)
const { getSession, sessionRegistry } = useMultiScreenShare(
  computed(() => store.sources),
  activeDisplayId,  // 仅捕获当前选中的屏，降低 GPU 负载
)

ShareSession 暴露 state、localStream、stats，供 ScreenStreamPanel.vue 预览与 QR 码展示。

Viewer 离开后的行为

useMultiScreenShare 在 peer-left 或 statechange: connected → idle 时调用 handleViewerLeft：

rtc.disconnect() — 发 bye、重建 PeerConnection，不停止桌面捕获
rtc.connect() — 回到 signaling，等待下一个手机扫码

4. 手机接收端

/**
 * 来源：virt-screen/src/renderer/views/ReceiverView.vue + useWetRTC.ts
 *
 * 手机浏览器 = recvonly + 主动协商（initiator: true, polite: false）
 * URL：/app/#/receiver?roomId=screen-<displayId>
 */
import { WetRTC, applyReceiverPlayoutDelay } from '@wetspace/wetrtc'
import { createSignalChannel } from './signal-channel-socketio'
import { PREFERRED_VIDEO_CODEC, RECEIVER_PLAYOUT_DELAY_HINT } from './low-latency-config'

function getReceiverDeviceProfile() {
  const key = 'wet-receiver-device-id'
  let deviceId = localStorage.getItem(key)
  if (!deviceId) {
    deviceId = `receiver-${crypto.randomUUID().slice(0, 12)}`
    localStorage.setItem(key, deviceId)
  }
  return {
    deviceId,
    name: 'Mobile 设备',
    platform: 'Mobile',
  }
}

export async function connectMobileReceiver(roomId: string, videoEl: HTMLVideoElement) {
  const signalHandle = createSignalChannel(roomId, () => getReceiverDeviceProfile())

  const rtc = new WetRTC({
    signal: signalHandle.channel,
    direction: 'recvonly',
    initiator: true,    // 主动 createOffer
    polite: false,      // 与主机 polite:true 配对
    preferredVideoCodec: PREFERRED_VIDEO_CODEC,
  })

  rtc.on('statechange', (state) => {
    if (state === 'connected') {
      applyReceiverPlayoutDelay(rtc.peerConnection!, RECEIVER_PLAYOUT_DELAY_HINT)
    }
  })

  rtc.on('track', (ev) => {
    const stream = ev.streams[0]
    if (!stream) return
    videoEl.srcObject = stream
    videoEl.playsInline = true
    videoEl.muted = true
    void videoEl.play()
    applyReceiverPlayoutDelay(rtc.peerConnection!, RECEIVER_PLAYOUT_DELAY_HINT)
  })

  await rtc.connect()

  return {
    async leave() {
      await rtc.disconnect()
      rtc.dispose()
      signalHandle.close()
    },
  }
}

// 用法：roomId 与主机 QR 码一致，例如 screen-2779098405
// await connectMobileReceiver('screen-2779098405', document.querySelector('video')!)

路由：/receiver?roomId=screen-xxx（QR 码由 QrCodePanel.vue 生成，指向 Kernel HTTPS 地址）。

useWetRTC 封装了 state / remoteStream / connect / disconnect，并在 connected 与 track 时调用 applyReceiverPlayoutDelay。

5. 信令事件一览

事件	方向	用途
`join-room`	客户端 → 服务端	加入 Socket.IO room
`signal`	双向（经服务端转发）	WetRTC SDP / ICE
`peer-joined`	服务端 → 同 room	对端上线
`peer-left`	服务端 → 同 room	对端离线，主机触发重连等待
`leave-room`	客户端 → 服务端	页面关闭时清理
`devices-changed`	服务端广播	刷新设备列表 UI

6. 与通用示例的差异

主题	屏幕分享示例	virt-screen
信令	内存 / 演示用	Socket.IO + NestJS
捕获	`getDisplayMedia`	Electron `chromeMediaSourceId`
协商	单端 demo 常 sendonly + initiator	sendonly 被动 / recvonly 主动
多屏	单路	每 displayId 独立 room，按需捕获
低延迟	文档示例参数	`low-latency-config.ts` 集中配置

7. 本地调试

bash

# 仓库根目录
cd virt-screen
npm run dev          # Electron + Kernel https://localhost:3450

手机访问：https://<局域网IP>:3450/app/#/receiver?roomId=screen-<displayId>

相关文档：低延迟屏幕共享 · 信令指南 · WetRTC API

virt-screen 集成示例 ​

架构概览 ​

双端 WetRTC 角色（必读） ​

1. 信令：Socket.IO → SignalChannel ​

2. 低延迟配置 ​

3. Electron 发送端 ​

桌面捕获 ​

启动会话 ​

Viewer 离开后的行为 ​

4. 手机接收端 ​

5. 信令事件一览 ​

6. 与通用示例的差异 ​

7. 本地调试 ​