Tuesday 12pm, 23 October 2018

Lily hu

Design of Machine Learning Models with Domain Experts

Lily Hu

Research Scientist - Salesforce Research -- MetaMind


How can machine learning models learn from prior human knowledge? How do we effectively leverage domain knowledge to create machine learning solutions? Humans possess domain knowledge that enables us to make predictions. For example, we can learn from books, articles, and encyclopedia entries to recognize new concepts. A machine learning model is proposed that uses natural language descriptions to identify samples of novel image classes unavailable during training time, a task called zero-shot learning. The model consists of a learner module that makes an initial prediction and a correction module that is trained to anticipate and correct errors based on the learner's training data. Experiments demonstrate that this approach leads to state-of-the-art performance on fine-grained zero-shot classification on natural language class descriptions. Prior knowledge may also be gained from human experiences and concepts from science and engineering, for example, to monitor the operations of wind turbines or commercial buildings. For these cases, a process is developed to collect domain knowledge from human experts to design machine learning solutions. Techniques from machine learning and statistics also aids the design process by selecting important sensors and reducing installation and deployment costs. This increases renewable energy output and energy efficiency to help tackle the important problem of sustainability.


Lily Hu is a Research Scientist in deep learning at Salesforce Research (MetaMind). She is interested in how machine learning models can learn from prior human knowledge and how machine learning solutions can be designed for specific domains. This includes learning from knowledge in natural language descriptions, tasks in computer vision and zero-shot learning, methods of collecting domain knowledge from human experts, and deep learning for specialized domains such as medicine, energy systems, and IoT. Lily earned her PhD and MS from the University of California, Berkeley in Mechanical Engineering in 2016 and 2014 and completed her BASci in Engineering Science from the University of Toronto.