Peng Billy Xu (徐鹏)
Machine Learning Researcher and Software Engineer
Google Cloud Conversational AI
Email : pxu4 [at] ualberta [dot] ca

Feel free to reach out if you find anything interesting on my website or seek collaborations on relevant topics!

About Me (CV)

I'm a ML software engineer at Google, working on developing and applying state-of-the-art NLP technology on the next generation dialogue systems. Previously, I'm a senior ML researcher in Borealis AI, working on ML/NLP research and applications.

I did my master in Department of Computing Science at University of Alberta. I worked at the Database Group where I was advised by Prof. Denilson Barbosa. I'm also an active contributor of DBpedia. I've participated in GSoC 2016 as a student (Certificate) and GSoC 2017 and 2018 as a student mentor (Certificate) for DBpedia.

I got my B.Eng. degree in the School of Information and Communication Engineering of Beijing University of Posts and Telecommunications(BUPT). I finished my B.S. graduation thesis in CS department in Tsinghua university in June 2015. I really appreciate my thesis advisor Prof. Jie Tang for his guidance and help in that year.

News

  • Janurary 10, 2022: Start my journey at Google!
  • June 14, 2021: Check out our public demo Turing on Text-to-SQL Semantic Parsing!
  • May 25, 2021: One demo paper on Text-to-SQL Semantic Parsing accepted at ACL 2021.
  • May 6, 2021: One paper on Transformer Optimization accepted at ACL 2021.
  • June 1, 2020: One paper on Controllable Text Generation accepted at ICML 2020.
  • Janurary 6, 2020: One paper on Language Modelling accepted at AISTATS 2020.
  • May 14, 2019: One long paper on Coherence Modelling accepted at ACL 2019.
  • Feburary 22, 2019: One short paper on Neural Relation Extraction accepted at NAACL 2019.
  • July 12, 2018: Successfully defend my master thesis which is the runner up for the Outstanding Master Thesis Award.
  • Feburary 14, 2018: One long oral paper on Fine-Grained Entity Type Classification accepted at NAACL 2018.

Selected Peer-Reviewed Publications

[Full Publication List]
Optimizing Deeper Transformers on Small Datasets
Peng Xu, Dhruv Kumar, Wei Yang, Wenjie Zi, Keyi Tang, Chenyang Huang, Jackie Chi Kit Cheung, Simon J.D. Prince, Yanshuai Cao
The 59th Annual Meeting of the Association for Computional Linguistics (ACL), 2021.
Long paper, Aug. 2 - 6, 2021, ACL.
[ Paper ] [ Slides ] [ Video ][ Code ]
Related Transformer Tutorials (with Simon Prince):
Transformers I: Introduction
Transformers II: Extensions
Transformers III: Training
On Variational Learning of Controllable Representations for Text without Supervision
Peng Xu, Jackie Chi Kit Cheung, Yanshuai Cao
Thirty-seventh International Conference on Machine Learning (ICML), 2020.
Jul. 12 - 18, 2020, PMLR.
[ Paper ] [ Slides ] [ Video ][ Code ]
Better Long-Range Dependency By Bootstrapping A Mutual Information Regularizer
Yanshuai Cao*, Peng Xu* (* indicates equal contribution)
The 23rd International Conference on Artificial Intelligence and Statistics (AISTATS), 2020.
Palermo, Sicily, Italy, Jun. 3 - 5, 2020, PMLR.
[ Paper ] [ Code ]
Related Blogpost:
Shortcut Learning Hypothesis of Modern Language Models
A Cross-Domain Transferable Neural Coherence Model
Peng Xu, Hamidreza Saghir, Jin Sung Kang, Teng Long, Avishek Joey Bose, Yanshuai Cao, Jackie Chi Kit Cheung
The 57th Annual Meeting of the Association for Computational Linguistics (ACL), 2019.
Long paper, Florence, Italy, Jul.28 - Aug.2, ACL.
[ Paper ] [ Poster ] [ Code ]
Connecting Language and Knowledge with Heterogeneous Representations for Neural Relation Extraction
Peng Xu, Denilson Barbosa
The 17th Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), 2019.
Short paper, Minneapolis, MN, U.S.A., Jun. 2-7, ACL.
[ Paper ] [ Poster ] [ Code ]
Neural Fine-Grained Entity Type Classification with Hierarchy-Aware Loss
Peng Xu, Denilson Barbosa
The 16th Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), 2018.
Long oral paper, New Orleans, LA, U.S.A., Jun. 1-6, ACL.
[ Paper ] [ Slides ] [ Video ] [ Code ]

Thesis

  • Master Thesis, Towards Neural Information Extraction without Manual Annotated Data
    [Thesis] [Presentation] (Runner up for the Outstanding Master Thesis Award)

Granted Patents

  • System and Method for Cross-domain Transferable Neural Coherence Model
    US Patent 11,270,072 [Google Patents]

Pending Patents

  • System and Method for Transferable Natural Language Interface
    US Patent App. 17/508,914 [Google Patents]
  • System and Method for Controllable Machine Text Generation Architecture
    US Patent App. 16/881,843 [Google Patents]
  • System and Method for Machine Learning with Long-range Dependency
    US Patent App. 16/809,267 [Google Patents]

Awards

  • “Third-class Scholarship for Undergraduates”, 2013/2014
  • “Second-class Scholarship for Undergraduates”, 2012/2013
  • “Third-class Scholarship for Undergraduates”, 2011/2012

    • Last Updated on 8th May, 2022