Uploaded December 2022 | Updated September 2026, 2 hours ago
Merry Christmas everyone! In this empirical study I investigate if Pianoteq 8 has what it takes to sing. I made this video as an entry for the modartt Pianoteq 8 2022 video contest.
I discovered the idea of a speaking piano through art works of the Austrian composer Peter Ablinger (https://ablinger.mur.at/), who realizes the effect on a real piano with automated keys. This video uses my (attempt at...) independent recreation of this effect. As I found out, it's one of those things that is conceptually straightforward, but the devil's in the details :)
Credits:
* Text animations generated with camala (CAption MArkup LAnguage) - a hobby project I very recently started to scratch my itch :)
* The audio to midi software was written by me in python 3 with the help of the numpy library. Although I tried other techniques as well, using the short-time fourier transform appeared to give the best results.
* The video features additional images and video fragments reused with permission:
- Piano vector image: toppng.com/show_download/174296/stylised-piano-by-at-cyberscooty-a-stylised-piano-on-piano-clip-art/large
- Pine tree vector image: freesvg.org/raseone-tree-2
- Trollface vector image: freesvg.org/troll-face
- Snowflakes: youtube.com/watch?v=rjacJY1DogU
- Video by KoolShooters: pexels.com/video/santa-claus-listening-to-music-using-a-headphone-6193878
- Video by cottonbro studio: pexels.com/video/a-cat-walking-over-the-piano-keyboard-6853337
- Video by cottonbro studio: pexels.com/video/snow-queen-in-the-forest-7354840
- Video by Artem Podrez: pexels.com/video/two-women-making-a-dance-video-6003997
- Video by Francesco Morrone: pexels.com/video/kid-running-in-the-field-holding-a-guitar-4444485
- Video by RODNAE Productions: pexels.com/video/man-dancing-beside-a-christmas-tree-6518074
* Tools used:
- Audio recording with audacity (audacityteam.org/) using an AT2020USB+ microphone (audio-technica.com/nl-nl/at2020usb)
- Piano sounds are generated in real time by sending midi information to Pianoteq 8 standard version (modartt.com/). No external effects or samples are used while Pianoteq is playing.
- Python 3 programming language (https://www.python3.org)
- Camala caption generator (github.com/shimpe/camala), using toml (toml.io/en/), SVG (inkscape.org/develop/about-svg/), inkscape (inkscape.org/), vectortween(github.com/shimpe/pyvectortween) and moviepy (zulko.github.io/moviepy/)
- Video editing with kdenlive (kdenlive.org/en/)
- Screen, audio and webcam recording are done with obs studio (obsproject.com/)
- Everything is made on a laptop running an up-to-date EndeavourOS GNU/Linux (endeavouros.com/) with the pipewire/wireplumber (pipewire.org/) sound system which made desktop recording with audio a breeze. In terms of user-friendliness, pipewire is the best the best thing to happen to linux audio in a long time.
Merry Christmas everyone! In this empirical study I investigate if Pianoteq 8 has what it takes to sing. I made this video as an entry for the modartt Pianoteq 8 2022 video contest.
I discovered the idea of a speaking piano through art works of the Austrian composer Peter Ablinger (https://ablinger.mur.at/), who realizes the effect on a real piano with automated keys. This video uses my (attempt at...) independent recreation of this effect. As I found out, it's one of those things that is conceptually straightforward, but the devil's in the details :)
Credits:
* Text animations generated with camala (CAption MArkup LAnguage) - a hobby project I very recently started to scratch my itch :)
* The audio to midi software was written by me in python 3 with the help of the numpy library. Although I tried other techniques as well, using the short-time fourier transform appeared to give the best results.
* The video features additional images and video fragments reused with permission:
- Piano vector image: toppng.com/show_download/174296/stylised-piano-by-at-cyberscooty-a-stylised-piano-on-piano-clip-art/large
- Pine tree vector image: freesvg.org/raseone-tree-2
- Trollface vector image: freesvg.org/troll-face
- Snowflakes: youtube.com/watch?v=rjacJY1DogU
- Video by KoolShooters: pexels.com/video/santa-claus-listening-to-music-using-a-headphone-6193878
- Video by cottonbro studio: pexels.com/video/a-cat-walking-over-the-piano-keyboard-6853337
- Video by cottonbro studio: pexels.com/video/snow-queen-in-the-forest-7354840
- Video by Artem Podrez: pexels.com/video/two-women-making-a-dance-video-6003997
- Video by Francesco Morrone: pexels.com/video/kid-running-in-the-field-holding-a-guitar-4444485
- Video by RODNAE Productions: pexels.com/video/man-dancing-beside-a-christmas-tree-6518074
* Tools used:
- Audio recording with audacity (audacityteam.org/) using an AT2020USB+ microphone (audio-technica.com/nl-nl/at2020usb)
- Piano sounds are generated in real time by sending midi information to Pianoteq 8 standard version (modartt.com/). No external effects or samples are used while Pianoteq is playing.
- Python 3 programming language (https://www.python3.org)
- Camala caption generator (github.com/shimpe/camala), using toml (toml.io/en/), SVG (inkscape.org/develop/about-svg/), inkscape (inkscape.org/), vectortween(github.com/shimpe/pyvectortween) and moviepy (zulko.github.io/moviepy/)
- Video editing with kdenlive (kdenlive.org/en/)
- Screen, audio and webcam recording are done with obs studio (obsproject.com/)
- Everything is made on a laptop running an up-to-date EndeavourOS GNU/Linux (endeavouros.com/) with the pipewire/wireplumber (pipewire.org/) sound system which made desktop recording with audio a breeze. In terms of user-friendliness, pipewire is the best the best thing to happen to linux audio in a long time.
