C#, AForge.Net – Examples for average color and motion detection

As I was a student at the chair in Data and Signal processing of TUM (http://tum.de), I had a course of Computer Vision which discusses the image processing and his uses in the real application. Matlab is often used for calculating, evaluating the algorithms and displaying data on the chart. However, in .NET I would like to introduce another library which is also powerful for image processing. That’s AForge.NET (http://aforgenet.com).

AForge.NET is a C# framework designed for developers and researchers in the fields of Computer Vision and Artificial Intelligence – image processing, neural networks, genetic algorithms, machine learning, robotics, etc.

1. Introduction

The reason that I come to AForge.NET, is my laziness. I often see films/video on my computer from my bed. And I am so lazy that when I go to sleep, I must crawl from the bed to the computer to turn it off. Therefore I have an idea to write a small program using my webcam to monitor the brightness in the room when I turn off the light, the room turns black and my program will turn off the computer. That means I can turn off my computer just by turning off the light. Is it great isn’t it? ^_^. So let’s go to the homepage of AForge.Net and download its installer (http://aforge.googlecode.com/files/AForge.NET%20Framework-2.2.2.exe) to have full of examples, documentation, etc…

1. Create new MVVM Light WPF 4 Project (you can use any type of project you want. This is just the GUI).
2. Insert an image control, a combo box and a TextBlock as below

<Grid x:Name="LayoutRoot">
	<Grid.RowDefinitions>
		<RowDefinition Height="8*"></RowDefinition>
		<RowDefinition Height="1*"></RowDefinition>
		<RowDefinition Height="1*"></RowDefinition>
	</Grid.RowDefinitions>
	<Image x:Name="imgCamera" Grid.Row="0" Source="{Binding CameraFrame}"></Image>
	<ComboBox Grid.Row="1" Margin="5" ItemsSource="{Binding Cameras}" SelectedIndex="{Binding CameraIndex, Mode=TwoWay}">
		<i:Interaction.Triggers>
			<i:EventTrigger EventName="SelectionChanged">
				<cmd:EventToCommand Command="{Binding SelectionChangedCommand, Mode=OneWay}" PassEventArgsToCommand="True" />
			</i:EventTrigger>                
		</i:Interaction.Triggers>
	</ComboBox>
	<TextBlock x:Name="tbAverageColor" Text="{Binding AverageColor, StringFormat='Average Color: \{0\}'}" Grid.Row="2" HorizontalAlignment="Center" VerticalAlignment="Center">            
		<TextBlock.Background>
			<SolidColorBrush Color="{Binding AverageColor}"></SolidColorBrush>
		</TextBlock.Background>
	</TextBlock>
</Grid>

3. It looks maybe a little involved for someone who doesn’t work with WPF MVVM. But just skipping it if you can’t understand, the code does nothing special than showing the control, binding them to data context and handling event SelectionChanged of the combo box. So that’s all preparations for the GUI. Now go to MainViewModel() to build our logics When the program is started, I will load all available webcams into combo box so that I can select later which one I would like to use. This action can take a long time so it should run asynchronously and I won’t block the GUI during its executing.

public MainViewModel()
{
	if (IsInDesignMode)
	{
		// Code runs in Blend --> create design time data.
	}
	else
	{
		// Code runs "for real"
		Task.Factory.StartNew(() => Initialize());		
	}
}

AForgeUtil aforgeUtil = null;
public List<string> Cameras {...}

private void Initialize()
{
	aforgeUtil = new AForgeUtil();
	Cameras = aforgeUtil.GetCameras();
}

In this initialization step, I create a Task asynchronously to initialize my custom AForgeUtil class and call GetCameras() function to get the list of the camera connected to the computer

private FilterInfoCollection filterInfoCollection = new FilterInfoCollection(FilterCategory.VideoInputDevice);
public  List<string> GetCameras()
{            
	return filterInfoCollection.Cast<FilterInfo>().ToList().Select(x=>x.Name).ToList();
}

As you can see, with the help of FilterInfoCollection in AForge.NET, I can easily get the list of cameras. When I start the program, the combo box will be filled with all webcams connected to my computer. To make the code above work, of course, we need to add references to AForge.Video and AForge.Video.DirectShow.

4. In next step, I would like to handle the SelectionChanged event of the combo box to start the selected webcam and streaming what it records on image control

private RelayCommand<SelectionChangedEventArgs> selectionChangedCommand;
public ICommand SelectionChangedCommand
{
	get
	{
		if (selectionChangedCommand == null)
			selectionChangedCommand = new RelayCommand<SelectionChangedEventArgs>(e => HandleSelectionChangedCommand(e));
		return selectionChangedCommand;
	}
}

private void HandleSelectionChangedCommand(SelectionChangedEventArgs e)
{
	Task.Factory.StartNew(() => aforgeUtil.StartCamera(CameraIndex));
}

private  VideoCaptureDevice videoCaptureDevice = null;
public  void StartCamera(int CameraIndex)
{
	videoCaptureDevice = new VideoCaptureDevice(filterInfoCollection[CameraIndex].MonikerString);
	videoCaptureDevice.NewFrame += new AForge.Video.NewFrameEventHandler(videoCaptureDevice_NewFrame);
	videoCaptureDevice.Start();
}

void videoCaptureDevice_NewFrame(object sender, AForge.Video.NewFrameEventArgs eventArgs)
{
	Messenger.Default.Send(new NotificationMessage<Bitmap>((Bitmap)eventArgs.Frame.Clone(), MessengerMessages.NewFrame));
}

public MainViewModel()
{
	...
		Task.Factory.StartNew(() => Initialize());
		Messenger.Default.Register<NotificationMessage<Bitmap>>(this, (message) => UpdateCameraFrame(message));
}

private void UpdateCameraFrame(NotificationMessage<Bitmap> message)
{
	if (message.Notification == MessengerMessages.NewFrame)
	{
		CameraFrame = message.Content.ToWpfBitmap();		          
	}   
}

public static class ExtensionMethod
{
	public static BitmapSource ToWpfBitmap(this Bitmap bitmap)
	{
		using (MemoryStream memorystream = new MemoryStream())
		{
			bitmap.Save(memorystream, ImageFormat.Bmp);
			memorystream.Position = 0;
			BitmapImage result = new BitmapImage();
			result.BeginInit();
			// According to MSDN, "The default OnDemand cache option retains access to the stream until the image is needed."
			// Force the bitmap to load right now so we can dispose the stream.
			result.CacheOption = BitmapCacheOption.OnLoad;
			result.StreamSource = memorystream;
			result.EndInit();
			result.Freeze();
			return result;
		}
	}

}

It’s really long, long code for a simple action. The SelectionChanged event will be handled by a new Task calling to StartCamera() function of AForgeUtil class with the index of the selected camera in the list. In StartCamera(), I create an instance of VideoCaptureDevice of given camera, subscribe to its NewFrame event and turn the camera on. That means anytime when a new frame comes, function videoCaptureDevice_NewFrame(…) will be called and I send the current frame back to MainViewModel() to show it on image control. MainViewModel receives this message, converts the current frame from Bitmap to WPFBitmap and “streams” it on image control by setting CurrentFrame property. ToWpfBitmap() is an extension method for converting Bitmap to WPFBitmap without causing memory leaking.

The code looks maybe a little complicated but most of them relate to communicating within the program. The part of AForge.Net is just some line of codes. But before starting the program, we must be sure that VideoCaptureDevice must be disposed of correctly when the program is closed to avoid the memory leak.

public static void ClearMain()
{
	_main.CloseCamera();
	_main.Cleanup();
	_main = null;
}

public void CloseCamera()
{
	aforgeUtil.CloseCamera();
}

public void CloseCamera()
{
	if (videoCaptureDevice != null && videoCaptureDevice.IsRunning)
	{
		videoCaptureDevice.SignalToStop();
		videoCaptureDevice.WaitForStop();
		videoCaptureDevice = null;
	}
}

You can override the CleanUp in MainViewModel and can dispose of the object too. It’s better than changing code in ViewModelLocator like me. Then now start the program to see if it works

5. The capturing must look smoothly and RAM should be about 40MB. If not, then check your code again to see if there is any position causing memory leak with any profiler. I have now current frame on my image control and what I still have to do now is calculating its average color to decide if the light in the room is turned off.

private void UpdateCameraFrame(NotificationMessage<Bitmap> message)
{
	if (message.Notification == MessengerMessages.NewFrame)
	{
		CameraFrame = message.Content.ToWpfBitmap();
		aforgeUtil.GetAverageColor(message.Content);                
	}   
}

public void GetAverageColor(Bitmap bitmap)
{
	Color tempColor;
	long sumA = 0, sumR = 0, sumG = 0, sumB = 0;
	for (int i = 0; i < bitmap.Height; i++)
	{
		for (int j = 0; j < bitmap.Width; j++)
		{
			tempColor = bitmap.GetPixel(j, i);
			sumA += Convert.ToInt32(tempColor.A);
			sumR += Convert.ToInt32(tempColor.R);
			sumG += Convert.ToInt32(tempColor.G);
			sumB +=Convert.ToInt32(tempColor.B);
		}
	}

	System.Windows.Media.Color result = new System.Windows.Media.Color();
	result.A = (byte)(sumA / (bitmap.Height * bitmap.Width));
	result.R = (byte)(sumR / (bitmap.Height * bitmap.Width));
	result.G = (byte)(sumG / (bitmap.Height * bitmap.Width));
	result.B = (byte)(sumB / (bitmap.Height * bitmap.Width));
	Messenger.Default.Send(new NotificationMessage<System.Windows.Media.Color>(result, MessengerMessages.NewAverageColor));
}

bool alreadyStart = false;
const int BlackThreshhold = 47;
public MainViewModel()
{
	...
		Messenger.Default.Register<NotificationMessage<Bitmap>>(this, (message) => UpdateCameraFrame(message));
		Messenger.Default.Register<NotificationMessage<System.Windows.Media.Color>>(this, (message) =>
			{
				AverageColor = message.Content;
				if (AverageColor.R < BlackThreshhold && AverageColor.G < BlackThreshhold && AverageColor.B < BlackThreshhold && alreadyStart)
				{
					alreadyStart = false;
					Messenger.Default.Send(MessengerMessages.RunYourActionNow);
				}
				else if (AverageColor.R > BlackThreshhold && AverageColor.G > BlackThreshhold && AverageColor.B > BlackThreshhold)
				{
					alreadyStart = true;
				}
			});
	}
}

public MainWindow()
{
	InitializeComponent();
	Closing += (s, e) =>
		{                    
			ViewModelLocator.Cleanup();
		};

	Messenger.Default.Register<string>(this, (message) => HandleStringMessage(message));
}

private void HandleStringMessage(string message)
{
	if (message == MessengerMessages.RunYourActionNow)
	{
		MessageBox.Show("Run your action now");
	}
}

Again the code looks a little confusing but it is really simple. When current frame changed, we’ll calculate its average color through summing color values and dividing through its area. After that, send color back to MainViewModel to show it on TextBlock and check if the color is nearly black. If yes, then show the message box. Of course, you should replace the message box with your defined action, for example, turn off the computer and make it standby.

6. We can extend the example with motion detection to decide if there is any motion in the room. If not, turn off the light or something like that or raise an alarm.

IMotionDetector motionDetector = new TwoFramesDifferenceDetector();
BlobCountingObjectsProcessing motionProcessing = new BlobCountingObjectsProcessing();
MotionDetector detector = null;
const int MinObjectsSize = 30;
public AForgeUtil()
{
	motionProcessing.HighlightColor = System.Drawing.Color.Green;
	motionProcessing.MinObjectsHeight = MinObjectsSize;
	motionProcessing.MinObjectsWidth = MinObjectsSize;            
	detector = new MotionDetector(motionDetector, motionProcessing);
	
}

void videoCaptureDevice_NewFrame(object sender, AForge.Video.NewFrameEventArgs eventArgs)
{
	MotionDetection((Bitmap)eventArgs.Frame.Clone());
}

public void MotionDetection(Bitmap bitmap)
{
	if (detector.ProcessFrame(bitmap) > 0.02)
	{
		if (motionProcessing.ObjectsCount > 1)
		{
			Messenger.Default.Send(MessengerMessages.IntrusionAlarm);
		}                
	}           

	Messenger.Default.Send(new NotificationMessage<Bitmap>(bitmap, MessengerMessages.NewFrame));
}

There are still a lot of interesting functionalities of AForge.Net which can be not all discussed at this small blog. I’ll write more about this library when I have time to make research with it. As usual, in the end, you’ll find the source code

2. Source code

Source code: https://bitbucket.org/hintdesk/dotnet-aforge-examples-for-average-color-and-motion-detection/src

6 thoughts on “C#, AForge.Net – Examples for average color and motion detection”

  1. Hi,

    i like your article. I am using Aforge to determine motion detection much like you are doing. I was also picking up changes in light (such as the sun rising or/and falling). I do not want to pick that up as motion. I understand your code for getting the average color but will not this change also for motion detection as well as for lighting changes? How would I use you code to separate lighting changes from motion changes?

    Thank You!

  2. @Alice: I have now no idea how to detect light motion but I put this question on top of my “research” list. If I can figure out something helpful, I’ll reply you here.

  3. hi. i am doing project on sleep monitoring system in visual studio. I need to do project that can automatically record the video when there is movement in user (change in accelerometer reading). can u help with with the code please…

  4. Can you please renew the link to the source code “AForge Webcam Average Color“?

    If possible please.

    Thank you very much!

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.