Abstract The boosting interest in face recognition is mainly driven by application demands. For a facial recognition system to work well in practice, it should automatically detect whether a face is present in the acquired image, and recognize the face from a general viewpoint. In addition, a realtime system requires more efficient and automatic algorithms than the video and image based applications. This work addresses these problems and presents novel techniques for the automated and application of detecting and recognizing people. The main objectiveis to make the work practically, fast and robustly. An advanced visual tracking scheme is employed for combining the power of various detectors. The face alignment algorithm is implemented based on Active Appearance Models. Various features extraction techniques, such as 2D DCT, Lifting Wavelets and Gabor Wavelets are investigated for facial features extraction. The discriminant analysis, including linear subspace based methods and kernel based methods, are also widely discussed. A new Enhanced Nearest Feature Space classifier is proposed which shows greater ability in facial feature classification. Lots of experiments are performed on the FERET dataset and ORL dataset in order to evaluate the efficiency of different approaches. The work is mainly divided into four parts: Part I gives a brief introduction to the face recognition technology and the realtime face recognition scheme. Part II mainly discusses the fast facial region detection, alignment and tracking for the realtime purpose. Skin-region detector, AdaBoost face detector, Active Appearance Models and Lucas-Kanade tracker are built into a incremental focusof attention scheme. Part III reports the face recognition techniques and evaluation based on the still images. The different combinations of various methods are investigated. Part VI shows the current implementation of the software package, which includes the system requirements, I/O definition, etc.