Uploaded January 2023 | Updated September 2026, 8 hours ago

---

Thanks to Growth Intelligence for supporting this video. Discover new ways for your business to use language models: growthintelligence.com

The age of AI is upon us, but can it help us save the planet? In this video, we'll take a look at an application of foundation language models (particularly transformers and Google's BERT) to the task of identifying sustainable businesses.

0:00 Intro
0:38 Machine Learning
1:46 Transformers
5:09 Foundation Models
6:19 BERT
7:49 Fine-Tuning for Sustainability
10:01 Bias
11:01 Outro

------------------

FURTHER READING

Transformers
arxiv.org/pdf/1706.03762.pdf
towardsdatascience.com/transformers-89034557de14
analyticsvidhya.com/blog/2018/03/essentials-of-deep-learning-sequence-to-sequence-modelling-with-attention-part-i
analyticsvidhya.com/blog/2022/02/sentiment-analysis-using-transformers
analyticsvidhya.com/blog/2019/06/understanding-transformers-nlp-state-of-the-art-models
jalammar.github.io/illustrated-transformer

Foundation Models
research.ibm.com/blog/what-are-foundation-models
deeplizard.com/learn/video/5T-iXNNiwIs

BERT
huggingface.co/blog/bert-101
analyticsvidhya.com/blog/2019/09/demystifying-bert-groundbreaking-nlp-framework
towardsdatascience.com/what-exactly-happens-when-we-fine-tune-bert-f5dc32885d76

------------------

MUSIC

Glimpsing Infinity
Asher Fulero (YouTube Audio Library)

Lightless Dawn
Kevin MacLeod (incompetech)

One Sly Move
Kevin MacLeod (incompetech)

Stasis Attempt
Max McFerrin (YouTube Audio Library)

The World’s Fair
Godmode (YouTube Audio Library)