To fulfill the increasing demands on authentication methods on the smart mobile and wearable devices with small form factors and constrained screen displays, we introduce a novel authentication mechanism, Body-Taps, which authenticates a device based on the Tap-Code gestures in the form of hand movements captured through the built-in motion sensors. The Body-Taps require a user to set a TapCode as an unlock code for the device by tapping the device on the set anchor points on his or her own body. The target device is authenticated based on two criterion: (1) the user's knowledge of the set Tap-Code, and (2) the BodyTap gestures measured through the smart device's built-in motion sensors (accelerometer and gyroscope). Our experiments show that the proposed Body-Taps system can achieve an average authentication accuracy over 99.5% on a dataset comprising of 230 Body-Tap samples from 23 subjects, using Random Forest (RF), Neural Network (NNet), and Linear Discriminant Analysis (LDA) classifiers. Our work yields a light-weight, low-cost, and easy-to-use secure authentication system that requires minimal efforts and offers satisfactory usability.