With the introduction of battery-powered wireless embedded smart cameras, it has now become viable to deploy large numbers of spatially-distributed cameras with more flexibility in terms of camera locations. However, many challenges remain to be addressed to build operational, battery-powered, wireless smart-camera networks. Battery life is limited, and video processing tasks, such as foreground detection and tracking, consume considerable amount of energy. Thus, it is essential to design and implement light-weight algorithms and methods to increase the energy efficiency of each camera node, and thus the overall life-time of the camera network. We present an adaptive method based on tracking that significantly decreases the energy consumption of the embedded camera. The microprocessor on the camera board is sent to an idle state depending on the amount of activity in the scene. The amount of time the camera remains in idle mode is adaptively changed based on the speeds of tracked objects. Instead of continuously capturing and processing every frame, the camera drops frames during idle mode while preserving the tracking performance and thus system reliability at the same time. We present experimental results showing the energy-efficiency of the proposed method, and the gain in battery life. The proposed methodology provides 25% to 37% savings in the energy consumption, and 45:83% to 65% increase in the battery life depending on the number of objects in the scene and their speeds.