SignalR+AForge实现视频会话[WPF]

版权声明：本文为博主原创文章，未经博主允许不得转载。https://blog.csdn.net/lordwish/article/details/51785023

AForge是基于.NET的强大视频分析库，而SignalR是微软推出的实时通信技术，两者结合起来实现简单的视频会话。预期的效果是实现：在线终端刷新、会话请求、会话接受、会话拒绝、会话繁忙、会话结束。本示例采用USB摄像头。

1服务端及辅助类

1.1创建服务端

[ 使用WPF创建SignalR服务端]

1.2在线终端刷新

[ WPF+SignalR实现用户列表实时刷新]

1.3通信状态辅助类

public enum MessageState
{
VideoApply,//视频会话请求
VideoOpen,//视频会话开始
VideoRefuse,//视频会话拒绝
VideoBusy,//视频会话繁忙
VideoOver,//视频会话结束
Null
}

1.4服务端方法

在MyHub类中添加方法MessageConnect，其中参数messageid是由会话发起端生成的唯一Guid,用来标识此次会话，当会话接受端拒绝、繁忙或会话过程中任一方终止会话时，会话结束，移除messageid。因为会话状态消息在两个终端之间来回传输的，所以从消息的角度讲，终端发送消息时就是sender，接收消息时就是receiver了。

public Task MessageConnect(ClientModel sender,ClientModel receiver,string messageid,MessageState status)
{
return Clients.Client(receiver.ConnectionId).messageconnect(receiver, sender, messageid, status);
}

2客户端方法

使用NuGet添加AForge程序包，只安装AForge.Video和AForge.Controls就够了，图方便全装也可以。

2.1页面布局

摄像头图像显示使用AForge.Controls中的控件VideoSourcePlayer，所以要引用AForge.Controls命名空间。由于视频实际上是以图像帧的形式传输的，格式为bitmap，所以对方图像显示在PictureBox中，对System.Windows.Form的引用也是不能少的。

xmlns:aforge="clr-namespace:AForge.Controls;assembly=AForge.Controls"
xmlns:form="clr-namespace:System.Windows.Forms;assembly=System.Windows.Forms"

<WrapPanel>
<Grid Margin="10,10,5,10" VerticalAlignment="Center" HorizontalAlignment="Center">
<Grid.RowDefinitions>
<RowDefinition></RowDefinition>
<RowDefinition Height="50"></RowDefinition>
</Grid.RowDefinitions>
<WindowsFormsHost Width="320" Height="240" VerticalAlignment="Center" HorizontalAlignment="Center">
<form:PictureBox x:Name="imgCapture" Width="320" Height="240"></form:PictureBox>
</WindowsFormsHost>
<StackPanel Grid.Row="1" Orientation="Horizontal" VerticalAlignment="Center" HorizontalAlignment="Center">
<TextBlock Name="txtName"></TextBlock>
</StackPanel>
</Grid>
<Grid Margin="5,10,10,10" VerticalAlignment="Center" HorizontalAlignment="Center">
<Grid.RowDefinitions>
<RowDefinition></RowDefinition>
<RowDefinition Height="50"></RowDefinition>
</Grid.RowDefinitions>
<WindowsFormsHost Width="320" Height="240" VerticalAlignment="Center" HorizontalAlignment="Center">
<aforge:VideoSourcePlayer x:Name="sourcePlayer" Width="320" Height="240"></aforge:VideoSourcePlayer>
</WindowsFormsHost>
<StackPanel Grid.Row="1" VerticalAlignment="Center" Orientation="Horizontal" HorizontalAlignment="Center">
<Button Name="btnCapture" Width="90" Height="30" Click="btnCapture_Click">捕获</Button>
<Button Name="btnClose" Width="90" Height="30" Margin="10,0,0,0" Click="btnClose_Click">关闭</Button>
</StackPanel>
</Grid>
</WrapPanel>

2.2发送会话请求

选择在线终端列表上的任一用户，双击发起会话请求。

private void dgList_MouseDoubleClick(object sender, MouseButtonEventArgs e)
{
if (dgList.SelectedItem == null)
{
return;
}
client = (ClientModel)dgList.SelectedItem;
var result = MessageBox.Show("确定要进行视频会话？", "会话提醒", MessageBoxButton.OKCancel, MessageBoxImage.Question, MessageBoxResult.OK);
if (result == MessageBoxResult.OK && client != null)
{
string messageid = Guid.NewGuid().ToString();
MyHub.Invoke("MessageConnect", MyClient, client, messageid, MessageState.VideoApply);
}
}

2.3会话状态监听

接收到会话请求时，首先会判断本地是不是已经有正在进行的视频会话了，如果有就返回繁忙标识，没有就打开摄像头。这里对会话关闭状态是为了在一方终止会话关闭摄像头时，另一方也能收到关闭通知从而关闭摄像头。

private void ConnectListener()
{
try
{
MyHub.On<ClientModel, ClientModel, string, MessageState>("messageconnect", (sender, receiver, id, state) =>
{
switch (state)
{
case MessageState.VideoApply:
{
if (messageid == "")
{
var result = MessageBox.Show("是否接收" + sender.ClientName + "的会话请求？", "会话提醒", MessageBoxButton.YesNo, MessageBoxImage.Question, MessageBoxResult.Yes);
if (result == MessageBoxResult.Yes)
{
this.Dispatcher.Invoke(delegate
{
client = sender;
messageid = id;
txtName.Text = sender.ClientName;
OpenVideo();//打开摄像头
});
}
else
{
MyHub.Invoke("MessageConnect", receiver, sender, id, MessageState.VideoRefuse);
}
}
else
{
MyHub.Invoke("MessageConnect", receiver, sender, id, MessageState.VideoBusy);
}
}
break;
case MessageState.VideoOpen:
{
this.Dispatcher.Invoke(delegate
{
client = sender;
messageid = id;
txtName.Text = sender.ClientName;
OpenVideo();//打开摄像头
});
}
break;
case MessageState.VideoRefuse:
{
MessageBox.Show(sender.ClientName + "拒绝了您的视频会话请求！");
}
break;
case MessageState.VideoBusy:
{
MessageBox.Show(sender.ClientName + "正忙，请稍后发送请求！");
}
break;
case MessageState.VideoOver:
{
this.Dispatcher.Invoke(delegate
{
CloseVideo();//关闭摄像头
});
}
break;
}
});
}
catch (Exception)
{
throw;
}
}

2.4打开摄像头

在打开摄像头后，为控件绑定NewFrame事件开始发送图像帧，同时开启线程videoThread接收图像帧，并显示在PictureBox中。发送和接收其实都是用了最基本的Socket。

private Socket videoSocket;
private Thread videoThread;
private object bitmapTemp = new object();
private IPEndPoint receiver = null;
private byte[] buffTemp = new byte[2 * 1024 * 1024];
private void OpenVideo()
{
try
{
receiver = new IPEndPoint(IPAddress.Parse(client.ClientIP), VideoPort);
videoSocket = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
uint IOC_IN = 0x80000000;
uint IOC_VENDOR = 0x18000000;
uint SIO_UDP_CONNRESET = IOC_IN | IOC_VENDOR | 12;
videoSocket.IOControl((int)SIO_UDP_CONNRESET, new byte[] { Convert.ToByte(false) }, null);
videoSocket.Bind(new IPEndPoint(IPAddress.Parse(ClientIp), VideoPort)); FilterInfoCollection videoDevices = new FilterInfoCollection(FilterCategory.VideoInputDevice);
if (videoDevices.Count > 0)
{
sourcePlayer.VideoSource = new VideoCaptureDevice(videoDevices[0].MonikerString);
sourcePlayer.NewFrame += SourcePlayer_NewFrame;
sourcePlayer.Start();
videoThread = new Thread(ReceiveVideo);
videoThread.IsBackground = true;
videoThread.Start();
}
else
{
MessageBox.Show("未找到视频设备！");
} }
catch (Exception ex)
{
MessageBox.Show(ex.Message);
//throw;
}
}


2.5发送图像帧

private void SourcePlayer_NewFrame(object sender, ref Bitmap image)
{
lock (bitmapTemp)
{
System.Drawing.Image imageTemp = image.Clone(new System.Drawing.Rectangle(0, 0, image.Width, image.Height), image.PixelFormat);
MemoryStream stream = new MemoryStream();
imageTemp.Save(stream, System.Drawing.Imaging.ImageFormat.Jpeg);
stream.Position = 0;
byte[] buffImage = new byte[stream.Length];
stream.Read(buffImage, 0, buffImage.Length);
videoSocket.BeginSendTo(buffImage, 0, buffImage.Length, SocketFlags.None, receiver, SendData, videoSocket);
stream.Dispose();
stream = null;
}
}
private void SendData(IAsyncResult ar)
{
Socket socket = (Socket)ar.AsyncState;
socket.EndSendTo(ar);
}

2.6接收图像帧

private void ReceiveVideo()
{
while (true)
{
bool result = videoSocket.Poll(5000, SelectMode.SelectRead);
if (result)
{
videoSocket.BeginReceive(buffTemp, 0, buffTemp.Length, SocketFlags.None, ReceiveData, videoSocket);
}
}
}
private void ReceiveData(IAsyncResult ar)
{
Socket socket = (Socket)ar.AsyncState;
int count = socket.EndReceive(ar);
if (count > 0)
{
byte[] buff = new byte[count];
Buffer.BlockCopy(buffTemp, 0, buff, 0, count);
MemoryStream ms = new MemoryStream(buff);
Bitmap bitmap = (Bitmap)System.Drawing.Image.FromStream(ms);
imgCapture.SizeMode = System.Windows.Forms.PictureBoxSizeMode.StretchImage;
imgCapture.Image = bitmap;
}
}

2.7关闭摄像头

private void btnClose_Click(object sender, RoutedEventArgs e)
{
try
{
MyHub.Invoke("MessageConnect", MyClient, client, messageid, MessageState.VideoOver);
CloseVideo();
client = null;
messageid = ""; }
catch (Exception)
{ throw;
}
}
private void CloseVideo()
{
videoThread.Abort();
if (sourcePlayer.IsRunning)
{
sourcePlayer.SignalToStop();
sourcePlayer.WaitForStop();
}
imgCapture.Image = null;
sourcePlayer.NewFrame -= SourcePlayer_NewFrame;
}


后记

总体来说运行效果还是不错的。最开始想用SignalR做图像帧的发送和接收，发现没经过压缩的图像数据量太大，SignalR根本反应不过来，还是用底层的Socket快。而SignalR中的很多特性都是从底层封装好的，在实时通信上比Socket要方便很多。此外视频控件并不局限于AForge，类似WPFMediaTookit之类的也可以做，原理都是一样的。