Uploaded June 2022 | Updated September 2026, 2 weeks ago
#gpt4chan #4chan #ai
GPT-4chan was trained on over 3 years of posts from 4chan's "politically incorrect" (/pol/) board.
(and no, this is not GPT-4)
EXTRA VIDEO HERE: youtube.com/watch?v=dQw4w9WgXcQ
Website (try the model here): gpt-4chan.com
Model (no longer available): huggingface.co/ykilcher/gpt-4chan
Code: github.com/yk/gpt-4chan-public
Dataset: zenodo.org/record/3606810#.YpjGgexByDU
OUTLINE:
0:00 - Intro
0:30 - Disclaimers
1:20 - Elon, Twitter, and the Seychelles
4:10 - How I trained a language model on 4chan posts
6:30 - How good is this model?
8:55 - Building a 4chan bot
11:00 - Something strange is happening
13:20 - How the bot got unmasked
15:15 - Here we go again
18:00 - Final thoughts
ERRATA:
- I stated that the model is better on the automated parts of TruthfulQA than any other GPT out there, which is incorrect. There exist some small GPT-models with similar performance, I was mainly talking about the flagship models, such as GPT-3 and GPT-J.
Links:
Merch: ykilcher.com/merch
TabNine Code Completion (Referral): bit.ly/tabnine-yannick
YouTube: youtube.com/c/yannickilcher
Twitter: twitter.com/ykilcher
Discord: ykilcher.com/discord
BitChute: bitchute.com/channel/yannic-kilcher
LinkedIn: linkedin.com/in/ykilcher
BiliBili: space.bilibili.com/2017636191
If you want to support me, the best thing to do is to share out the content :)
If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):
SubscribeStar: subscribestar.com/yannickilcher
Patreon: patreon.com/yannickilcher
Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq
Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2
Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m
Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
#gpt4chan #4chan #ai
GPT-4chan was trained on over 3 years of posts from 4chan's "politically incorrect" (/pol/) board.
(and no, this is not GPT-4)
EXTRA VIDEO HERE: youtube.com/watch?v=dQw4w9WgXcQ
Website (try the model here): gpt-4chan.com
Model (no longer available): huggingface.co/ykilcher/gpt-4chan
Code: github.com/yk/gpt-4chan-public
Dataset: zenodo.org/record/3606810#.YpjGgexByDU
OUTLINE:
0:00 - Intro
0:30 - Disclaimers
1:20 - Elon, Twitter, and the Seychelles
4:10 - How I trained a language model on 4chan posts
6:30 - How good is this model?
8:55 - Building a 4chan bot
11:00 - Something strange is happening
13:20 - How the bot got unmasked
15:15 - Here we go again
18:00 - Final thoughts
ERRATA:
- I stated that the model is better on the automated parts of TruthfulQA than any other GPT out there, which is incorrect. There exist some small GPT-models with similar performance, I was mainly talking about the flagship models, such as GPT-3 and GPT-J.
Links:
Merch: ykilcher.com/merch
TabNine Code Completion (Referral): bit.ly/tabnine-yannick
YouTube: youtube.com/c/yannickilcher
Twitter: twitter.com/ykilcher
Discord: ykilcher.com/discord
BitChute: bitchute.com/channel/yannic-kilcher
LinkedIn: linkedin.com/in/ykilcher
BiliBili: space.bilibili.com/2017636191
If you want to support me, the best thing to do is to share out the content :)
If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):
SubscribeStar: subscribestar.com/yannickilcher
Patreon: patreon.com/yannickilcher
Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq
Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2
Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m
Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
![[ML News] DeepMind tackles Math | Microsoft does more with less | Timnit Gebru launches DAIR](https://i.ytimg.com/vi/f2OgP49J7Pg/mqdefault.jpg "[ML News] DeepMind tackles Math | Microsoft does more with less | Timnit Gebru launches DAIR
#mlnews #deepmind #ai
The most trusted model in News!
Get started with Weights & Biases here: https://wandb.me/yannic
(its free forever for personal use)
OUTLINE:
0:00 - Intro
0:15 - Sponsor: Weights & Biases
3:10 - DeepMind tackles fundamental math
6:45 - Microsoft focuses on scaling effectively and efficiently
10:15 - NeurIPS Anthology Visualization
13:30 - Timnit Gebru launches research institute independent from big tech
16:50 - SageMaker Canvas for no-code ML
17:50 - Help, Help!
21:40 - Cornelius Emde wins the 3090
21:55 - A retrospective on the NeurIPS 2021 ethics review process
References:
DeepMind tackles fundamental math
https://deepmind.com/blog/article/exploring-the-beauty-of-pure-mathematics-in-novel-ways?utm_source=pocket_mylist
https://www.nature.com/articles/s41586-021-04086-x?utm_source=pocket_mylist
Microsoft focuses on scaling effectively and efficiently
https://www.microsoft.com/en-us/research/blog/efficiently-and-effectively-scaling-up-language-model-pretraining-for-best-language-representation-model-on-glue-and-superglue/?OCID=msr_blog_TNLRV5_tw
NeurIPS Anthology Visualization
https://neuripsav.vizhub.ai/blog/
https://neuripsav.vizhub.ai/
Timnit Gebru launches research institute independent from big tech
https://www.washingtonpost.com/technology/2021/12/02/timnit-gebru-dair/
https://www.dair-institute.org/about
https://www.theguardian.com/commentisfree/2021/dec/06/google-silicon-valley-ai-timnit-gebru
SageMaker Canvas for no-code ML
https://aws.amazon.com/blogs/aws/announcing-amazon-sagemaker-canvas-a-visual-no-code-machine-learning-capability-for-business-analysts/
Help, Help!
https://macberth.netlify.app/
https://huggingface.co/emanjavacas/MacBERTh/tree/main
https://developer.nvidia.com/blog/nvidia-announces-tensorrt-8-2-and-integrations-with-pytorch-and-tensorflow/?ncid=so-twit-314589#cid=dl13_so-twit_en-us
https://opacus.ai/
https://twitter.com/naotokui_en/status/1466320722825920515
https://colab.research.google.com/drive/1H_g60Q_XELJ2VJu4GF7KY8111ce4VLwd?usp=sharing#scrollTo=JyNp3rwoWOQd
https://twitter.com/ThomasSimonini/status/1466437571303649301?utm_source=pocket_mylist
https://github.com/karpathy/arxiv-sanity-lite
https://arxiv-sanity-lite.com/
https://www.youtube.com/watch?v=01ENzpkjOCE
https://github.com/Felix-Petersen/algovision
https://github.com/rentruewang/koila?utm_source=pocket_mylist
https://github.com/YeWR/EfficientZero
Cornelius Emde wins the 3090
https://twitter.com/CorEmde/status/1466122212000374793
A retrospective on the NeurIPS 2021 ethics review process
https://blog.neurips.cc/2021/12/03/a-retrospective-on-the-neurips-2021-ethics-review-process/
Links:
TabNine Code Completion (Referral): http://bit.ly/tabnine-yannick
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: https://twitter.com/ykilcher
Discord: https://discord.gg/4H8xxDF
BitChute: https://www.bitchute.com/channel/yannic-kilcher
LinkedIn: https://www.linkedin.com/in/ykilcher
BiliBili: https://space.bilibili.com/2017636191
If you want to support me, the best thing to do is to share out the content :)
If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):
SubscribeStar: https://www.subscribestar.com/yannickilcher
Patreon: https://www.patreon.com/yannickilcher
Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq
Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2
Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m
Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n")
![[ML News] Uber: Deep Learning for ETA | MuZero Video Compression | Block-NeRF | EfficientNet-X](https://i.ytimg.com/vi/fEKZC9mta8w/mqdefault.jpg "[ML News] Uber: Deep Learning for ETA | MuZero Video Compression | Block-NeRF | EfficientNet-X
#mlnews #muzero #nerf
Your regularly irregular updates on everything new in the ML world!
Merch: http://store.ykilcher.com
OUTLINE:
0:00 - Intro
0:15 - Sponsor: Weights & Biases
2:15 - Uber switches from XGBoost to Deep Learning for ETA prediction
5:45 - MuZero advances video compression
10:10 - Learned Soft Prompts can steer large language models
12:45 - Block-NeRF captures entire city blocks
14:15 - Neural Architecture Search considers underlying hardware
16:50 - Mega-Blog on Self-Organizing Agents
18:40 - Know Your Data (for Tensorflow Datasets)
20:30 - Helpful Things
Sponsor: Weights & Biases
https://wandb.me/yannic
References:
https://docs.wandb.ai/guides/integrations/other/openai
https://colab.research.google.com/github/wandb/examples/blob/master/colabs/openai/Fine_tune_GPT_3_with_Weights_%26_Biases.ipynb#scrollTo=rJdQqrC8Ablo
https://wandb.ai/borisd13/GPT-3/reports/Fine-Tuning-Tips-and-Exploration-on-OpenAI-s-GPT-3 VmlldzoxNDYwODA2
Uber switches from XGBoost to Deep Learning for ETA prediction
https://eng.uber.com/deepeta-how-uber-predicts-arrival-times/?utm_source=pocket_mylist
MuZero advances video compression
https://deepmind.com/blog/article/MuZeros-first-step-from-research-into-the-real-world
https://storage.googleapis.com/deepmind-media/MuZero/MuZero%20with%20self-competition.pdf
Learned Soft Prompts can steer large language models
https://ai.googleblog.com/2022/02/guiding-frozen-language-models-with.html
https://aclanthology.org/2021.emnlp-main.243/
Block-NeRF captures entire city blocks
https://arxiv.org/abs/2202.05263
https://arxiv.org/pdf/2202.05263.pdf
https://waymo.com/intl/zh-cn/research/block-nerf/
Neural Architecture Search considers underlying hardware
https://ai.googleblog.com/2022/02/unlocking-full-potential-of-datacenter.html
https://openaccess.thecvf.com/content/CVPR2021/papers/Li_Searching_for_Fast_Model_Families_on_Datacenter_Accelerators_CVPR_2021_paper.pdf
Mega-Blog on Self-Organizing Agents
https://developmentalsystems.org/sensorimotor-lenia/
https://flowers.inria.fr/
Know Your Data (for Tensorflow Datasets)
https://knowyourdata-tfds.withgoogle.com/#dataset=pass&filters=kyd%2Fcloud_vision%2Fface_probability:9&tab=RELATIONS&item=train%5B89%25%3A91%25%5D_27143&expanded_groups=cloud_vision
https://knowyourdata.withgoogle.com/
Helpful Things
https://twitter.com/casualganpapers/status/1490318575873241091
https://www.reddit.com/r/MachineLearning/comments/snmtzn/r_phd_thesis_on_neural_differential_equations/
https://arxiv.org/abs/2202.02435
https://github.com/vicariousinc/PGMax
https://www.vicarious.com/posts/pgmax-factor-graphs-for-discrete-probabilistic-graphical-models-and-loopy-belief-propagation-in-jax/?utm_content=197542312&utm_medium=social&utm_source=twitter&hss_channel=tw-204185426
https://diambra.ai/tournaments
https://github.com/diambra/diambraArena
https://www.youtube.com/watch?v=dw72POyqcqk&t=271s
https://gitlab.com/deepcypher/python-fhez
https://python-fhez.readthedocs.io/en/latest/
https://joss.theoj.org/papers/10.21105/joss.04101?s=09&utm_source=pocket_mylist
https://github.com/PyTorchLightning/metrics
https://torchmetrics.readthedocs.io/en/latest/
https://twitter.com/alanyttian/status/1492027524909449221?utm_source=pocket_mylist
https://github.com/google/evojax
https://arxiv.org/abs/2202.05008
https://www.reddit.com/r/MachineLearning/comments/snod8f/n_gym_now_has_a_documentation_website/?utm_source=dlvr.it&utm_medium=twitter
https://www.gymlibrary.ml/pages/api/#initializing-environments
Links:
TabNine Code Completion (Referral): http://bit.ly/tabnine-yannick
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: https://twitter.com/ykilcher
Discord: https://discord.gg/4H8xxDF
BitChute: https://www.bitchute.com/channel/yannic-kilcher
LinkedIn: https://www.linkedin.com/in/ykilcher
BiliBili: https://space.bilibili.com/2017636191
If you want to support me, the best thing to do is to share out the content :)
If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):
SubscribeStar: https://www.subscribestar.com/yannickilcher
Patreon: https://www.patreon.com/yannickilcher
Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq
Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2
Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m
Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n")
