Advances in Data Science Seminar: Controllable headline generation

Time: 14:00 - 15:00

Venue: Online

Sorry, this event has now ended.

Once you have registered for this event, a link to the event page will be emailed to you before the seminar.

We will be video recording all of the talks in this year's Advances in Data Science seminar series. If you have registered for any of the seminars and oppose to us doing so, please notify us immediately.

Speaker: Naoaki Okazaki

Title: Controllable headline generation


Headline generation is a special type of abstractive summarisation that generates a very short summary (headline) from a news article. Headline generation is reaching a practical level in terms of quality of generated headlines, leveraging the recent advances in encoder- decoder models and the vast amount of training data. However, real applications of headline generation must satisfy various constraints about generated outputs, which requires controllable encoder-decoder models. In this talk, I will present a simple but useful extension of a sinusoidal positional encoding that forces Transformer models to preserve the length constraint. Besides, I will also present an approach for improving the truthfulness in headlines, pointing a flaw of the current task setting, evaluation measures, and training data.


Naoaki Okazaki is a professor in School of Computing, Tokyo Institute of Technology, Japan. Prior to this faculty position, he worked as a post-doctoral researcher at the University of Tokyo (in 2007-2011), and as an associate professor at Tohoku University (2011-2017). He is also a visiting research scholar of the Artificial Intelligence Research Center (AIRC), National Institute of Advanced Industrial Science and Technology (AIST). His research areas include Natural Language Processing (NLP), Artificial Intelligence (AI), and Machine Learning. He is also known as a developer for CRFSuite, SimString, libLBFGS, and NLP100 Exercise.