Ruth Urner

Associate Professor
Department of Electrical Engineering and Computer Science
Lassonde School of Engineering
York University
Toronto Canada

Email: ruth "at" eecs "dot" yorku "dot" ca
Office: LAS 3016
Phone: 416-736-2100 x70128

Short bio.

---

Research

My research develops mathematical tools and frameworks for analyzing the possibilities and limitations of automated learning, with a focus on semi-supervised, active and transfer learning. Currently I am particularly interested developing formal foundations for topics relating to societal impacts of machine learning, such as human interpretability and algorithmic fairness.

---

Teaching

International teaching activities/teaching at summer schools

• Hausdorff School on Algorithmic Data Analysis, Hausdorff Center for Mathematics, Bonn, Germany.
  Course on Statistical Learning Theory, May 2022.
• SMILES: Online Summer of Machine Learning at Skoltech 2020, Moskow, Russia/online.
  Course on Learning Theory, August 2020.
• Pre-Doc Summer School on Learning Systems, ETH Zürich, Switzerland.
  Course and Tutorial on Learning Theory, July 2017.
• Machine Learning Theory (co-taught with Ilya Tolstikhin).
  Full semester course at Tübingen University, Germany, Winter Semester 2016/17.
• Machine Learning Summer School (MLSS), Tübingen, Germany.
  Tutorial for Learning Theory, July 2015 and June 2017.

Teaching at York

I am making some of my teaching material and general course information available here. If you are a student currently enrolled in a specific version of one of these courses, there is an eclass site, which you have access to. Please consult the eclass site as it contains the updated course information and material that is relevant to the current offering.

• MATH 1090 Introduction to Logic for Computer Science.
  Offered: F18, F19, F22.
• EECS 2001 Introduction to the Theory of Computation.
  Offered: W21, W23.
• EECS 4404/5327 Introduction to Machine Learning and Pattern Recognition.
  Offered: W18, W19, SU19, F20, W22, F22.
• EECS 6127 Machine Learning Theory.
  Offered: F17 (as EECS 6002), W19, W20, F20, F21, F22.
• EECS 6002 Machine Learning and Society (reading course).
  I offered this course in the SU19 term. If you are a graduate student and interested in attending a reading course on this (or a similar) topic, reach out to me!

---

Activities

Keynote talks

• Keynote speaker at Women in Data Science Regensburg Conference 2021, Regensburg, Germany, April 2021: Deciphering fairness desiderata for machine learning.
• Keynote speaker at The Sixth International Conference on Machine Learning, Optimization, and Data Science (LOD), Sienna, Italy, July 2020: Promises and Challenges of Transfer Learning.
• Keynote speaker at Women in Data Science Zurich Conference 2019, Zurich, Switzerland, April 2019: Invitation to thinking about Machine Learning.

Other invited talks/tutorials/panel discussions

• Invited talk at Workshop on Pitfalls of limited data and computation for Trustworthy ML \@ ICLR, Kigali, Rwanda/Zoom, May 2023: How (not) to Model an Adversary.
• Tutorial at Dagstuhl Seminar, Beyond Adaptation: Understanding Distributional Changes, Schloss Dagstuhl/online, August 2020: An Overview on Domain Adaptation and Modelling of Dataset Shift (joint with Shai Ben-David).
• Panelist at IEEE Toronto ComSoc: Moving the dial, Women in STEM Panel, Toronto/Zoom, July 2020.
• Panelist at YorkU Inclusion Day 2020, York University, February 2020: Artificial Intelligence and Human Rights at YorkU: A Panel Discussion on Impacts and Opportunities.
• Vignette at Fields/VISTA Mathematics of Vision Workshop 2019, Fields Institute, Toronto, October 2019: Vignette 10: Machine Learning.
• Invited talk at Lorentz Center Workshop: Theoretical Foundations of Learning from Easy Data, Lorentz Center, Leiden, Netherlands, November 2016: Data niceness and the use of unlabeled examples.

Event (co-)organization

• Vector Institute Machine Learning Theory Workshop University of Waterloo, Waterloo, Ontario, November 2023.
• Simons Institute Cluster Beyond Prediction - Interpretable ML for Extracting Insights and Guiding Actions, Simons Institute, Berkeley, USA, zoom version Summer 2020, in-person meeting Summer 2022.
• Women in Machine Learning Theory meetings: WiML-T@COLT 2019, WiML-T@COLT 2020 (co-organized with Claire Vernade), and WiML-T@ALT2021 (co-organized with Tosca Lechner).
  
• Machine Learning Summer School (MLSS) 2017, MPI for Intelligent Systems, Tübingen, Germany, June 2017.
• Dagstuhl Seminar Foundations of Unsupervised Learning, Schloss Dagstuhl, Germany, September 2016.

Top level program committee member/area chair

• International Conference on Algorithmic Learning Theory (ALT): 2017, 2021, 2022, 2023.
• International Conference on Artificial Intelligence and Statistics (AISTATS): 2018, 2019.
• Conference On Learning Theory (COLT): 2016, 2017, 2018, 2021, 2022, 2023, 2024.
• Symposium on Foundations of Responsible Computing (FORC): 2022.
• International Conference on Learning Representations (ICLR): 2021.
• International Conference on Machine Learning (ICML): 2016, 2017, 2018, 2019, 2020, 2021, 2022.
• International Joint Conferences on Artificial Intelligence (IJCAI): 2019.
• Conference on Neural Information Processing Systems (NeurIPS): 2016, 2017, 2018, 2019, 2020, 2021, 2022, 2023 (SAC).

---

Publications

Refereed conference and journal publications

• Adversarially Robust Learning with Uncertain Perturbation Sets.
  Tosca Lechner, Vinayak Pathak, and Ruth Urner. NeurIPS 2023.
• Strategic Classification with Unknown User Manipulations.
  Tosca Lechner, Ruth Urner, and Shai Ben-David. ICML 2023.
• Precision Recall Cover: A Method For Assessing Generative Models.
  Fasil Cheema and Ruth Urner. AISTATS 2023.
• Adversarially Robust Learning with Tolerance.
  Hassan Ashtiani, Vinayak Pathak, and Ruth Urner. ALT 2023. Arxiv version.
• Robustness Should not be at Odds with Accuracy.
  Sadia Chowdhury and Ruth Urner. FORC 2022. Arxiv version.
• Learning Losses for Strategic Classification.
  Tosca Lechner and Ruth Urner. AAAI-22 Main Technical Track. Selected for oral presentation. Full paper with appendix available in the Arxiv version.
• Open Problem: Are all VC-classes CPAC learnable?
  Sushant Agarwal, Nivasini Ananthakrishnan, Shai Ben-David, Tosca Lechner, and Ruth Urner. Open Problems @ COLT 2021.
• Identifying Regions of Trusted Predictions.
  Nivasini Ananthakrishnan, Shai Ben-David, Tosca Lechner, and Ruth Urner. UAI 2021.
• Black-box Certification and Learning under Adversarial Perturbations.
  Hassan Ashtiani, Vinayak Pathak, and Ruth Urner. ICML 2020. Arxiv version.
• On Learnability wih Computable Learners.
  Sushant Agarwal, Nivasini Ananthakrishnan, Shai Ben-David, Tosca Lechner, and Ruth Urner. ALT 2020.
• When can unlabeled data improve the learning rate?
  Christina Göpfert, Shai Ben-David, Olivier Bousquet, Sylvain Gelly, Ilya O. Tolstikhin, and Ruth Urner. COLT 2019.
• Active Nearest-Neighbor Learning in Metric Spaces.
  Aryeh Kontorovich, Sivan Sabato, and Ruth Urner. JMLR, 18(195):1-38, 2018.
  Conference version appeared at NIPS 2016.
• Lifelong Learning with Weighted Majority Votes.
  Anastasia Pentina and Ruth Urner. NIPS 2016.
• On Version Space Compression.
  Shai Ben-David and Ruth Urner. ALT 2016.
• Active Nearest Neighbors in Changing Environments.
  Christopher Berlind and Ruth Urner. ICML 2015.
• Efficient Learning of Linear Separators under Bounded Noise.
  Pranjal Awasthi, Maria-Florina Balcan, Nika Haghtalab, and Ruth Urner. COLT 2015.
• Hierarchical Label Queries with Data-Dependent Partitions.
  Samory Kpotufe, Ruth Urner, and Shai Ben-David. COLT 2015.
• Learning Economic Parameters from Revealed Preferences.
  Maria-Florina Balcan, Amit Daniely, Ruta Mehta, Ruth Urner, Vijay V. Vazirani. WINE 2014. Arxiv version.
• The sample complexity of agnostic learning under deterministic labels.
  Shai Ben-David and Ruth Urner. COLT 2014.
• Domain Adaptation--Can Quantity Compensate for Quality?
  Shai Ben-David and Ruth Urner. Annals of Mathematics and Artificial Intelligence (AMAI), 70(3): 185-202 (2014).
  Conference version with Shai Shalev-Shwartz appeared at ISAIM 2012.
• Generative Multiple-Instance Learning Models For Quantitative Electromyography.
  Tameem Adel, Ruth Urner, Benn Smith, Daniel Stashuk and Dan Lizotte. UAI 2013. Arxiv version.
• PLAL: CLuster-based active learning.
  Ruth Urner, Sharon Wulff and Shai Ben-David. COLT 2013.
• Monochromatic Bi-Clustering.
  Sharon Wulff, Ruth Urner and Shai Ben-David. ICML 2013.
• On the Hardness of Domain Adaptation (and the Utility of Unlabeled Target Samples).
  Shai Ben-David and Ruth Urner. ALT 2012.
• Learning from Weak Teachers.
  Ruth Urner, Shai Ben-David, Ohad Shamir. AISTATS 2012.
• Unlabeled Data can Speed up Prediction Time.
  Ruth Urner, Shai Ben-David, Shai Shalev-Shwartz. ICML 2011.
• Naïve Security in a Wi-Fi World.
  Colleen Swanson, Ruth Urner, Edward Lank. IFIPTM 2010: 32-47.
• On a disparity between relative cliquewidth and relative NLC-width.
  Haiko Müller and Ruth Urner. Discrete Applied Mathematics 158(7): 828-840 (2010).

Workshop Contributions

• Tosca Lechner, Ruth Urner:
  Learning Losses for Strategic Classification.
  Workshop on Learning and Decision-Making with Strategic Feedback - Workshop @ NeurIPS 2021.
• Jeff Edmonds, Karan Singh, Ruth Urner:
  Implications of Modeled Beliefs for Algorithmic Fairness in Machine Learning.
  Workshop on Algorithmic Fairness through the Lens of Causality and Robustness - Workshop @ NeurIPS 2021.
• Sadia Chowdhuri, Ruth Urner:
  On the (Un-)Avoidability of Adversarial Examples.
  Workshop on Theoretic Foundation, Criticism, and Application Trend of Explainable AI - Workshop @ ICML 2021. arxiv
• Fahmida Brishty, Ruth Urner, and Gerd Grau:
  Machine Learning Based Data Driven Approach for Optimized Inkjet Printed Electronics.
  Poster, Materials Research Society (MRS) Fall Meeting, Boston, MA, 2019. Journal version in preparation.
• Shai Ben-David, Ruth Urner:
  On Weight Ratio Estimation for Covariate Shift.
  The Third Workshop on Transfer and Multi-Task Learning: Trends and New Perspectives - Workshop @ NIPS 2015. pdf
  Best paper award!
• Christopher Berlind, Ruth Urner:
  Active Nearest Neighbors in Changing Environments.
  The Second Workshop on Transfer and Multi-Task Learning: Theory meets Practice - Workshop @ NIPS 2014. pdf
• Shai Ben-David, Ruth Urner:
  The sample complexity of agnostic learning with deterministic labels.
  Machine Learning track of ISAIM 2014. pdf
• Ruth Urner, Shai Ben-David:
  Probabilistic Lipschitzness: A niceness assumption for deterministic labels.
  Learning Faster from Easy Data - Workshop @ NIPS 2013. pdf
• Shai Ben-David, Ruth Urner:
  Domain Adaptation as Learning with Auxiliary Information.
  New Directions in Transfer and Multi-Task: Learning Across Domains and Tasks - Workshop @ NIPS 2013. pdf
• Sharon Wulff, Ruth Urner and Shai Ben-David:
  Monochromatic Bi-Clustering.
  Discrete Optimization in Machine Learning (DISCML) - Workshop @ NIPS 2012.
• Shai Ben-David, Nathan Srebro, Ruth Urner:
  Universal Learning vs. No Free Lunch Results.
  Philosophy and Machine Learning - Workshop @ NIPS 2011. pdf
• Shai Ben-David, Ohad Shamir, Ruth Urner:
  Learning from Weak Teachers.
  Snowbird Learning Workshop 2011; WiML Workshop 2011.
• Shai Ben-David, Shai Shalev-Shwartz, Ruth Urner:
  Domain Adaptation using Nearest Neighbors.
  Snowbird Learning Workshop, Fort Lauderdale, Florida, 2011.
• Haiko Müller, Ruth Urner:
  On the relative clique-width of graphs.
  Third Workshop on Graph Classes, Optimization, and Width Parameters - Eugene, Oregon, 2007.

Other

• Maria-Florina Balcan, Ruth Urner:
  Active Learning - Modern Learning Theory.
  Encyclopedia of Algorithms 2016: 8-13.

Theses

• Ruth Urner:
  Learning with non-Standard Supervision.
  PhD thesis, prepared under supervision of Wayne Oldford and Shai Ben-David at the University of Waterloo, 2013. pdf
• Ruth Urner:
  Maximale stabile Graphen.
  German Diploma thesis, prepared under supervision of Nils Rosehr at the University of Würzburg;
  Work also presented at: WueST Workshop Gruppen, Geometrie, Topologie; Wuerzburg, Germany, 2006. pdf