3D Scan/Kinect Makerbot/Howard Community College/Spring2012/P1501msn
Project documentation focuses on how the team organized itself to tackle the unknown. This is a creative process. The details learned should be captured in tutorials. Project documentation should focus on how the team organized itself to step out into the unknown. Failure is common. The single most effective way to increase engineering respect is to document failures in a creative way that stimulates, inspires and helps others continue the project. Projects are never done. There are always loose ends. There are always decisions that might have been made more clearly, there are always other alternatives that might have been considered. None of this stuff appears in a tutorial. It does appear in the team documentation.
Electronic Sections Expected
[edit | edit source]Problem Statement
[edit | edit source]Have an efficient way of checking attendance in classrooms by using a webcam.
Team Members
[edit | edit source]Summary
[edit | edit source]Our goal is to have an automatic check in system in classroom. Using the Kinect's camera and IR technology. The Kinect can view people in 3-D Space (x,y,z) and can identify bodies in its scan region. We will mount the Kinect that has the best view of the door entrance and have it connected to a PC that will automatically run the software on the appropriate times.
Poster
[edit | edit source]Story
[edit | edit source]The first week team members worked to gather information and find software for Microsoft's Kinect and to look into existing facial recognition programs. This week included searches in open source packages and Microsoft Kinect Toolkit for the Kinect and looked into addition features that the Kinect can do such as 3D recognition, voice recognition, Skeleton Structure viewer, and Motion detector.
- The second week the team continued working on week 1 assignment, exploring around the numerous features the Microsoft Toolkit has. Noticing the Toolkit has a Skeleton Viewer that identifies a human structure, we then tried to take advantage of this feature and use it to recognize and identify people. In so doing, we had to first be able to count the number of students entering and exiting an area. We brainstormed ideas on how to implement the Skeleton Viewer feature and use it to count people. As shown in the picture below, we wanted it to count as soon as it identifies a person.
Our first idea was to search through software online that already has this feature. However, as reading through forums, other users tweaked the code of the Toolkit that is written in C++ language. No one in our group knows anything about C++ language, but some team members know a little bit about java, so we continued to explore the classes and methods and did not go anywhere. Our second idea was to make a decision matrix and let friends and classmates take a quick survey whether to use webcam or Kinect in our project the decision was close, so we split the project into two. One person finds software for a regular webcam and two members explore the Kinect.
- The third week the team started coding the Kinect with the use of Microsoft Visual C# and OpenCV. We created algorithms as guideline in what methods and classes to make in order to start programming.
Motion(){
//mehtod that when detects a skeleton structure implemented with Kinect's toolkit I had in Week 1's assignment would take a snapshop
int NumberStudents = 0;
if skeleton detected && skeleton moves left{
for loop (Number of face recognized) //loop until number of recognized identities been recorded in attendance chart
String Name= snapshot(); //goes to snapshot method and takes picture and returns name of students
Attendance(Name); //goes to Attendance method uses the parameter of "Name" to check attendance.
NumberStudents = NumberStudents + 1
}else if skeleton detected && skeleton moves right {
NumberStudents = NumberStudents -1
}else {
null
}
system.out.print("Number of student attended class was: " + NumberStudents); //displays the number of students attended this class.
}
public String snapshot(){
Sting NameOfStudent;
//takes picture and saves picture to hard drive.
NameOfStudent = //search and identify faces from picture database
return NameOfStudent;
}
public void Attendance(Name){
//goes through Microsoft Excel and search through the parameter "Name" and marks the person present.
}
We also explored Keylemon a facial recognition program for webcams and attempted implementing it with OpenCV to see its compatibility. We also looked into Google software Picasa and looked into how its facial recognition works.
- During the fourth and final week, members were brainstorming mounting areas in the classroom. Pictures shown below:
a mount that will have velcro at the bottom to hold the Kinect.
One member found motion detection software Yawcam that automatically take a picture when any motion is detected. We then figured out that this could be implemented with Picasa's facial recognition software. As motion is done by students it is detected by Yawcam, it takes a picture and saves it to a specific location that Picasa will then automatically scan the directory in which Yawcam saves its picture and tries identify people from Picasa's people database. This was our working project.
Decision List
[edit | edit source]Our decisions were mostly based upon testing on ideas and software found; and improving upon failed tests. Our first decision was to either use Microsoft Kinect or just a regular webcam. We listed the advantages and disadvantages of using each of the devices. This video showed the 3D face recognition the Kinect can do by hacking it. Moreover, the Microsoft SDK Toolkit from this link has a feature that detects a person's body in a form of a skeleton. So we figured we could use this to identify human beings and use the 3D facial recognition to identify who the person is. However, the main point is that with the use of a regular webcam, we can also do any of these hacks. We decided to make a decision matrix and let friends and classmates take a short survey, the decision table is shown below.
| Device to use | Quality scale 1-2 (1-best quality, 2-worst quality) Total | Difficulty to finish project Scale 1-5 (1-easiest, 5-hardest) Total | Time consuming scale 1-5 (1-least time, 5 most time) | Total Total (Lowest-Best Highest-Not the Best) |
|---|---|---|---|---|
| Webcam | 19 | 26 | 28 | 73 |
| Microsoft Kinect | 7 | 32 | 36 | 75 |
So we decided to split the project into two since the decision matrix's result came so close together and to put more challenge into the team. We had two people deal with the Kinect and one person deal with a regular webcam. We thought that the Kinect would have more hacking and coding to do, so we would have more people there.
Having worked with the Kinect there were a lot of Classes and Methods we have to look into to implement codes. We had some experience in C++, Java, and Visual Basic to code a program from scratch. We came across different programming software such as OPENCV, Visual Basic, and Visual Studios that will be compatible with the Kinect's toolkit packages. We made algorithms after algorithms that we thought would be used as a guidelines to program. After stressful week of programming and downloading software to be used, we eventually decided to shifted from the Kinect to a regular webcam because we found a software that enables us to identify people through the discussion made in our presentation in class using Picasa. From this we all worked together and focused on using the webcam and searched through various software such as KeyLemon and Kinemote to improve our project.
Material List
[edit | edit source]1 Webcam
1 Kinect
Computers
In the future, it would be best to have a better webcam device. A webcam that has better picture quality, in our project we used a VGA webcam supplied by the school that took very blurry pictures which results in a difficult time in recognizing identities.
Software List
[edit | edit source]Software used in our projects:
Lenovo's Verface - facial recognition software that locks and unlocks user profile on windows based PC.
Microsoft Kinect SDK Toolkit - Toolkit that has a Skeletonial Viewer feature that can be tweek to identify and count the number of people.
Visual C# - Programming software from Microsoft whos primary language is C# has a built in references to the Kinect API. Allows you to easily load Kinect's packages and use throughout the project.
OpenCV- Another programming software that has references that are compatible with a regular webcam to create series of programs with the use of a regular webcam
Picasa by Google - Photo management program that has facial recognition capabilities it identifies people's face on pictures and can sort pictures by recognized people in photos.
Yawcam - A freeware webcam software that has motion detector applications that takes picture when something crosses the defined area. Also has capabilities for security purposes and can automatically transfer pictures to another PC on a network
The only softwares used were Picase and Yawcam, the other softwares could be used as well, however it would require more experienced programmers to hack the Kinect and program a Graphical User Interface (GUI).
Time
[edit | edit source]Total Time Spent of project: 24 hrs 1 min
Tutorials
[edit | edit source]Use these programs together. Complete Picasa tutorial before moving on to Yawcam tutorial.
Next Steps
[edit | edit source]Our project has finally work. However this could always be improved such as using Microsoft Kinect instead of a regular webcam, where the Kinect has the ability to replicate a 3D model of people's face to have a better identity of the forms and facial features of people's face. Moreover, instead of using other company's software, coding and making your own program will be best because it shows ownership and skill. Due to lack of time and inexperience C# programmers, we wanted to have a working model, so we used Picasa and Yawcam's features. We began writing algorithms, but was not familiar with C#'s language, if Kinect was compatible with Java this project would be much easier.