Abstract: Non-adherence to medication is a complex behavioral issue that costs hundreds of billions of dollars annually in the United States alone. Existing solutions to improve medication adherence are limited in their effectiveness and require significant user involvement. To address this, a minimally invasive mobile health system called DoseMate is proposed, which can provide quantifiable adherence data and imposes minimal user burden. To classify a motion time-series that defines pill-taking, we adopt transfer-learning and data augmentation based techniques that uses captured pill-taking gestures along with other open datasets that represent negative labels of other wrist motions. The paper also provides a design methodology that generalizes to other systems and describes a first-of-its-kind, in-the-wild, unobtrusively obtained dataset that contains unrestricted pill-related motion data from a diverse set of users.