As Android has become the most prevalent operating system in mobile devices, privacy concerns in the Android platform are increasing. A mechanism for efficient runtime enforcement of informationflow security policies in Android apps is desirable to confine privacy leakage. The prior works towards this problem require firmware modification (i.e., modding) and incur considerable runtime overhead. Besides, no effective mechanism is in place to distinguish malicious privacy leakage from those of legitimate uses. In this paper, we take a bytecode rewriting approach. Given an unknown Android app, we selectively insert instrumentation code into the app to keep track of private information and detect leakage at runtime. To distinguish legitimate and malicious leaks, we model the user's decisions with a context-aware policy enforcement mechanism. We have implemented a prototype called Capper and evaluated its efficacy on confining privacy-breaching apps. Our evaluation on 4723 real-world Android applications demonstrates that Capper can effectively track and mitigate privacy leaks. Moreover, after going through a series of optimizations, the instrumentation code only represents a small portion (4.48% on average) of the entire program. The runtime overhead introduced by Capper is also minimal, merely 1.5% for intensive data propagation.