edit face recognition with documentation and prediction

Author: alexattia

FaceRecognition/README.md

@@ -2,12 +2,11 @@
 
 Modern face recognition with deep learning and HOG algorithm. 
 
-1. Find faces in image 
-2. Affine Transformations
-3. Encoding Faces
-4. Make a prediction
-
-Using the pipeline described [in this post from Adam Geitgey](https://medium.com/@ageitgey/machine-learning-is-fun-part-4-modern-face-recognition-with-deep-learning-c3cffc121d78). 
+1. Find faces in image (HOG Algorithm) 
+2. Affine Transformations (Face alignment using an ensemble of regression
+trees) 
+3. Encoding Faces (FaceNet) 
+4. Make a prediction (Linear SVM) 
 
 We are using the [Histogram of Oriented Gradients](http://lear.inrialpes.fr/people/triggs/pubs/Dalal-cvpr05.pdf) (HOG) method. Instead of computing gradients for every pixel of the image (way too much detail). We compute the weighted vote orientation gradients of 16x16 pixels squares. Afterward, we have a simple representation (HOG image) that captures the basic structure of a face. 
 All we have to do is find the part of our image that looks the most similar to a known trained HOG pattern. 
@@ -34,5 +33,11 @@ The training process works by looking at 3 face images at a time:
 - Load a picture of a different person and generate for the two pictures the 128 measurements 
 Then we tweak the neural network slightly so that it makes sure the measurements for the same person are slightly closer while making sure the measurements for the two different persons are slightly further apart.
 Once the network has been trained, it can generate measurements for any face, even ones it has never seen before!
+```
+face_encoder = dlib.face_recognition_model_v1(face_recognition_model)
+face_encoding = np.array(face_encoder.compute_face_descriptor(image, pose_landmarks, 1))
+```
+
+Finally, we need a classifier (Linear SVM or other classifier) to find the person in our database of known people who has the closest measurements to our test image. We train the classifier with the measurements as input.
 
-Finally, we need a classifier (Linear SVM or other classifier) to find the person in our database of known people who has the closest measurements to our test image. We train the classifier with the measurements as input.
+Thanks to Adam Geitgey who wrote a great [post](https://medium.com/@ageitgey/machine-learning-is-fun-part-4-modern-face-recognition-with-deep-learning-c3cffc121d78) about this, I followed his pipeline.

FaceRecognition/detect_recognize.py

@@ -9,30 +9,42 @@
 import pickle 
 import os
 import sys
+import argparse
+import time
 
 from sklearn.svm import SVC
 from sklearn.preprocessing import LabelEncoder
 
-import face_recognition_models
-
 face_detector = dlib.get_frontal_face_detector()
-face_recognition_model = face_recognition_models.face_recognition_model_location()
-face_encoder = dlib.face_recognition_model_v1(face_recognition_model)
+face_encoder = dlib.face_recognition_model_v1('./model/dlib_face_recognition_resnet_model_v1.dat')
 face_pose_predictor = dlib.shape_predictor('./model/shape_predictor_68_face_landmarks.dat')
 
 def get_detected_faces(filename):
+ """
+ Detect faces in a picture using HOG
+ :param filename: picture filename
+ :return: picture numpy array, face detector object with detected faces
+ """
  image = cv2.imread(filename)
  image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
  return image, face_detector(image, 1)
 
 def get_face_encoding(image, detected_face):
+ """
+ Encode face into 128 measurements using a neural net
+ :param image: picture numpy array
+ :param detected_face: face detector object with one detected face
+ :return: measurement (128,) numpy array 
+ """
  pose_landmarks = face_pose_predictor(image, detected_face)
  face_encoding = face_encoder.compute_face_descriptor(image, pose_landmarks, 1)
  return np.array(face_encoding)
 
 def training(people):
  """
- We need to have only one person/face per picture
+ Training our classifier (Linear SVC). Saving model using pickle.
+ We need to have only one person/face per picture.
+ :param people: people to classify and recognize
  """
  # parsing labels and reprensations
  df = pd.DataFrame()
@@ -51,6 +63,7 @@ def training(people):
  y = le.fit_transform(df[128])
  print("Training for {} classes.".format(len(le.classes_)))
  X = df.drop(128, axis=1)
+ print("Training with {} pictures.".format(len(X)))
 
  # training
  clf = SVC(C=1, kernel='linear', probability=True)
@@ -62,21 +75,71 @@ def training(people):
  with open(fName, 'wb') as f:
  pickle.dump((le, clf), f)
 
-def predict(filename):
- with open("./classifier.pkl", 'rb') as f:
- (le, clf) = pickle.load(f)
+def predict(filename, le=None, clf=None, verbose=False):
+ """
+ Detect and recognize a face using a trained classifier.
+ :param filename: picture filename
+ :param le:
+ :paral clf:
+ :param verbose:
+ :return: picture with bounding boxes and prediction
+ """
+ if not le and not clf:
+ with open("./classifier.pkl", 'rb') as f:
+ (le, clf) = pickle.load(f)
  image, detected_faces = get_detected_faces(filename)
+ prediction = []
+ # Verbose for debugging
+ if verbose:
+ print('{} faces detected.'.format(len(detected_faces)))
  img = np.copy(image)
  font = cv2.FONT_HERSHEY_SIMPLEX
  for face_detect in detected_faces:
  # draw bounding boxes
  cv2.rectangle(img, (face_detect.left(), face_detect.top()), 
  (face_detect.right(), face_detect.bottom()), (255, 0, 0), 2)
+ start_time = time.time()
+ # predict each face
  p = clf.predict_proba(get_face_encoding(image, face_detect).reshape(1, 128))
- if np.max(p) > 0.8:
+ # throwing away prediction with low confidence
+ a = np.sort(p[0])[::-1]
+ if a[0]-a[1] > 0.5:
  y_pred = le.inverse_transform(np.argmax(p))
- cv2.putText(img, y_pred, (face_detect.left(), face_detect.top()-5), font, 0.3, (255, 0, 0))
- return img
+ prediction.append([y_pred, (face_detect.left(), face_detect.top()), 
+ (face_detect.right(), face_detect.bottom())])
+ else:
+ y_pred = 'unknown'
+ # Verbose for debugging
+ if verbose:
+ print('\n'.join(['%s : %.3f' % (k[0], k[1]) for k in list(zip(map(le.inverse_transform, 
+ np.argsort(p[0])), 
+ np.sort(p[0])))[::-1]]))
+ print('Prediction took {:.2f}s'.format(time.time()-start_time))
+ 
+ cv2.putText(img, y_pred, (face_detect.left(), face_detect.top()-5), font, np.max(img.shape[:2])/1800, (255, 0, 0))
+ return img, prediction
 
 if __name__ == '__main__':
- people = os.listdir('./data/')
+ parser = argparse.ArgumentParser()
+ parser.add_argument('mode', type=str, help='train or predict')
+ parser.add_argument('--training_data',
+ type=str,
+ help="Path to training data folder.",
+ default='./data/')
+ parser.add_argument('--testing_data',
+ type=str,
+ help="Path to test data folder.",
+ default='./test/')
+
+ args = parser.parse_args()
+ people = os.listdir(args.training_data)
+ print('{} people will be classified.'.format(len(people)))
+ if args.mode == 'train':
+ training(people)
+ elif args.mode == 'test':
+ with open("./classifier.pkl", 'rb') as f:
+ (le, clf) = pickle.load(f)
+ for i, f in enumerate(glob.glob(args.testing_data)):
+ img, _ = predict(f, le, clf)
+ cv2.imwrite(args.testing_data + 'test_{}.jpg'.format(i), img)
+

README.md

@@ -10,21 +10,27 @@ The goal of this challenge is to build a model that predicts the count of bike s
 
 -->[French Explanations PDF](https://github.com/alexattia/Data-Science-Projects/blob/master/KaggleBikeSharing/Kaggle_BikeSharing_Explanations_French.pdf)
 
-## [Kaggle Understanding the Amazon from Space](https://github.com/alexattia/Data-Science-Projects/tree/master/KaggleAmazon) 
-Use satellite data to track the human footprint in the Amazon rainforest. 
-Deep Learning model (using Keras) to label satellite images.
+## [Twitter Parsing](https://github.com/alexattia/Data-Science-Projects/tree/master/TwitterParsing)
 
-## [Predicting IMDB movie rating](https://github.com/alexattia/Data-Science-Projects/tree/master/KaggleMovieRating)
-Project inspired by Chuan Sun [work](https://www.kaggle.com/deepmatrix/imdb-5000-movie-dataset) 
-How can we tell the greatness of a movie ? 
-Scrapping and Machine Learning 
+I've recently discovered the Chris Albon Machine Learning flash cards and I want to download those flash cards but the official Twitter API has a limit rate of 2 weeks old tweets so I had to find a way to bypass this limitation : use Selenium and PhantomJS. 
+Purpose of this project : Check every 2 hours, if he posted new flash cards. In this case, download them and send me a summary email.
+
+## [Face Recognition](https://github.com/alexattia/Data-Science-Projects/tree/master/FaceRecognition)
+
+Modern face recognition with deep learning and HOG algorithm. Using dlib C++ library, I have a quick face recognition tool using few pictures (20 per person).
 
 ## [Playing with Soccer data](https://github.com/alexattia/Data-Science-Projects/tree/master/KaggleSoccer)
 
 As a soccer fan and a data passionate, I wanted to play and analyze with soccer data. 
 I don't know currently what's the aim of this project but I will parse data from diverse websites, for differents teams and differents players. 
 
-## [Twitter Parsing](https://github.com/alexattia/Data-Science-Projects/tree/master/TwitterParsing)
 
-I've recently discovered the Chris Albon Machine Learning flash cards and I want to download those flash cards but the official Twitter API has a limit rate of 2 weeks old tweets so I had to find a way to bypass this limitation : use Selenium and PhantomJS. 
-Purpose of this project : Check every 2 hours, if he posted new flash cards. In this case, download them and send me a summary email.
+
+## [Kaggle Understanding the Amazon from Space](https://github.com/alexattia/Data-Science-Projects/tree/master/KaggleAmazon) 
+Use satellite data to track the human footprint in the Amazon rainforest. 
+Deep Learning model (using Keras) to label satellite images.
+
+## [Predicting IMDB movie rating](https://github.com/alexattia/Data-Science-Projects/tree/master/KaggleMovieRating)
+Project inspired by Chuan Sun [work](https://www.kaggle.com/deepmatrix/imdb-5000-movie-dataset) 
+How can we tell the greatness of a movie ? 
+Scrapping and Machine Learning