You give this AI program a vocal sound file and lyrics and it will convert the lyrics to phonemes and align them with the sound. Unfortunately works only for English.
I added one script to convert the result to Praat's TextGrid, so it can be imported to SynthV with the Real Voice Notes from TextGrid script.
The quality is not very good as if you could use it directly without any manual work, but it helps a lot. You need only align the markers better and correct the phonemes sometimes.
Like the video if you want me to make an installation tutorial.
How to install if you know what to do. This worked for me:
Automatic Lyrics Alignment for Real Voice SynthV scriptHPsynthV2023-07-21 | Hi, I've got something new for you.
You give this AI program a vocal sound file and lyrics and it will convert the lyrics to phonemes and align them with the sound. Unfortunately works only for English.
I added one script to convert the result to Praat's TextGrid, so it can be imported to SynthV with the Real Voice Notes from TextGrid script.
The quality is not very good as if you could use it directly without any manual work, but it helps a lot. You need only align the markers better and correct the phonemes sometimes.
Like the video if you want me to make an installation tutorial.
How to install if you know what to do. This worked for me:
Music used: Millenium by Republica (YT restricted the video, so I used only drums for the final mix)
#realvoice #synthvHigh Pitch Voice Conversion with Tsukuyomi-chan [tutorial]HPsynthV2024-02-14 | This tutorial is about my new program "One Octave Up Voice Converter"
and about troubles you can have converting synthetic voice with SVC.
Tsukuyomi-chan will tell you better than me (but don't tell her she has an "akusento" :)
00:00 Intro 01:09 Octave trick 01:51 Doing it with SpectraLayers 03:06 Free variant with One Octave Up Voice Converter 03:45 How to start a command line from file explorer in Windows 04:26 Praat pitch (yes, you need Praat too) 06:30 Mixing 07:07 Results
I hope it will help.【Yuu AI】音の鳴るほうへ (Oto no Naru Hou e) / SouthSoilHome 【SynthV + soVITS SVC cover】HPsynthV2024-02-06 | I am back after a while with a cover of a beautiful song composed by @SouthSoilHome (and kindly asking for a permission to have it here).
Now I finally have everything what I'v always wanted - high quality singing voice synthesizer and possibility to use any voice I want (by use of SVC).
------------- Remarks:
* stems were separated by Ultimate Vocal Remover 5.6 (great SW!) github.com/Anjok07/ultimatevocalremovergui (I recommend Karaoke 5 model for separation backing vocals, also can remove echo and reverb)
* Spectral Layers used for cleaning/separating stems
* Praat and RealVoice script were used to convert the main vocal (original AI Kiritan) to SynthV github.com/hataori-p/real-voice (script notesFromTextGrid)
* Synthesized with Kyomachi Seika AI, params: tension -0.2, breathiness -1, gender 0.25, tone shift -200
There is also a second way to install it on Github, when this goes wrong. It worked for me on Windows 10. I intentionally install it for CPU only. It is easier and it's not necessary to run it on GPU.
After install if it complains some module is missing then try this:
free relaxing music Tibetan Bowls composed by Liborio Conti no-copyright-music.com/relaxingSubscribe to Hataoris new Utau & vocal synthesis channelHPsynthV2022-04-20 | Starting a new channel for Utau and vocal synthesis and copying stuff.
Tuning & timing copied from 【Tohoku Kiritan AI】High Range Test【CeVIO AIカバー】 by @2cada youtube.com/watch?v=ZuC3EfZ1qjg (except that highest part where Kiritan goes metal)
original performed by 初音ミク Hatsune Miku music & lyrics by 木村わいP
#utau #annanyui #highrangetest #暗鳴ニュイAnna Nyui - 「Melancholic」 暗鳴ニュイ「メランコリック」 (Utau cover - new synth engine test)HPsynthV2022-04-03 | This is a first test of my own synthesis engine which works with Utau voice bank. It can also stretch consonants precisely - I need it for natural voice copying.
The song has been tuned completely in SynthV with Tsuina-chan and then synthesized by the engine.
It is based on a technology which Praat uses for time stretching & pitch shifting (TD PSOLA algorithm). It is actually very simple and fast and no FFT is needed for the synthesis. VB preprocessing is done by Praat script.
I am using Lua for experimenting. There is still a lot of work. And the voice banks have to be top quality (sampling and otoing) - only Anna's raw bank works without manual changes.
Voice database Anna Nyui 暗鳴ニュイ VCV Raw (only E4 monopitch) aoka45.wixsite.com/annanyui/vcv Postprocessing: EQ, compressor, reverb
Song: メランコリック Melancholic A real voice copy (timing and pitch) from [Melancholic Cover by ENE]
#synthv #utau #annanyui #voicecopySynthV Real Voice Series - in Studio Basic - using external Lua scriptsHPsynthV2022-03-11 | I have changed the essential scripts to work outside the SynthV Studio. You could now be able to use it with the free Studio Basic.
#synthv #script #realvoice #koharurikka #voicecopy #synthvstudiobasicSynthV Real Voice Series - Notes to TextGrid - even easier with a project file (BONUS at the end)HPsynthV2022-03-06 | Thanks for 300 subscribers!
This is a demonstration video of a new script "RV Notes to TextGrid" and updated "RV Notes from TextGrid" from Real Voice Series.
If you can get notes with lyrics from some project file, the sync work in Praat is way much easier.
#synthv #script #realvoice #koharurikka #voicecopySynthV AI All Stars 「Don Fiction ドンフィクション」 by Tokimeki♡Sendenbu とき宣 (SynthV cover, params tracking)HPsynthV2022-02-23 | A real voice copy (timing, pitch & other params) cover.
Tried to automatically track params - tension, tone shift, gender & loudness. It's quite complicated, can't make it with a simple script.
The principle is to compare the vowel spectrum of the note with synthetic vowels pre-generated with different parameters - about 60k combinations (30 pitches, 6 vowels, 7 tensions, 7 genders, 7 tone shifts). Used Praat's MFCC coefficients for comparing the spectra and determination of the loudness difference.
I wanted a song with about 5 singers singing separately to assign different AI voice banks to them and compare how will they resemble the originals.
If you know the song or the group tell me in the comments which one is the most similar.
Synthesizer V Studio Pro 1.5.0
Voice banks: Kyomachi Seika AI (ver. 102) Tsuina-Chan AI (ver. 103) Tsurumaki Maki AI (ver. 107) Koharu Rikka AI (ver. 116) Saki AI (version 118)
#synthv #ときめき宣伝部 #voicecopyKyomachi Seika AI 「Koi wa Sensou」 Love is War 京町セイカ AI 「恋は戦争」 (SynthV cover + SVP)HPsynthV2022-02-14 | A real voice copy (timing and pitch) of Fye I used the new RV Growl script (maybe too much)
#synthv #seika #loveiswarSynthV Real Voice Series - Growl / Scream Script featuring Kyomachi Seika 京町セイカHPsynthV2022-02-10 | Inspired by a growl in Rishie-P's latest video I tried to create a growling script for SynthV.
This is a demonstration video of the script. Tuning is from Rishie-P (only part for demo)
Scripts for SynthV Studio Pro (working only in Pro) For installation see Github github.com/hataori-p/real-voice (please update, new scripts & versions)
#synthv #voicecopy #praatprogramSynthV Real Voice Series - New Easier Method 1 - Labeling in PraatHPsynthV2022-01-17 | First of all - THANK YOU for 100 subscribers !!
This video is a new - I think more straightforward and perhaps easier way to do the voice copy from real voice to synth voice.
Although I use Japanese it should be language independent. You have to know the phonetics of your language used.
The rule of thumb is to mark starts of vocals and pauses and label the vocals by whatever you would write as lyrics in SynthV.
There is also an option to deal with consonants - wait for another video.
Scripts for SynthV Studio Pro (working only in Pro) For installation see Github github.com/hataori-p/real-voice (please update, new scripts & versions)
#synthv #voicecopy #praatprogramSaki AI - Kassai - Applause (Hirauchi Maya voice copy) サキAI 「喝采」(平内真矢のボイスコピー)SynthV coverHPsynthV2022-01-17 | A voice copy (timing and pitch) of Hirauchi Maya's voice to Saki's voice.
Another enka style song from another ASH (Actor's School Hiroshima) graduate - Hirauchi Maya (now aka Hima). 12 yo in the video, qualified in Singing King 2016 contest with Chiaki Naomi's song. Song range E3 - Db5 is quite low. I like her deep voice.
#synthv #sakiAI #平内真矢 #陽真 #enkaTsuina-Chan AI 「Anata no Utahime」 Your Diva ついなちゃん AI 「あなたの歌姫」 (SynthV cover + SVP)HPsynthV2022-01-14 | A real voice copy (timing and pitch) of Saya Scarlet's voice
#synthv #tsuinachan #あなたの歌姫Speech Test SynthV AI - Saki, Maki, Rikka, Tsuina + bonusHPsynthV2022-01-10 | Test of a real voice copy (timing and pitch) of a speech part (phone call) from Berryz Koubou song Maji Good Chance Summer performed by Shimakura Rika from Beyooooonds
Different SynthV AI voice banks + bonus
Who do you like the most?
No synthesizer can keep up with Rika sama, I had to omit some syllables
Synthesizer V Studio Pro 1.5.0 Voice databases: Koharu Rikka AI ver.116, Tsurumaki Maki AI (JPN) (version 107), Tsuina-Chan AI (version 103), Saki AI (version 118), GENBU (Lite) Animen JP (version 100)
Song: Maji Good Chance Summer マジ グッドチャンス サマー (speech part) performed by 島倉りか Shimakura Rika (member of BEYOOOOONDS ビヨーンズ) Original artist: Berryz工房 Berryz Koubou
#synthv #voicecopy #ついなちゃん #弦巻マキ #小春六花 #サキちゃんSaki AI - Arekara (Misora Hibari AI voice copy) サキAI 「あれから」(美空ひばり[AI歌唱]のボイスコピー)HPsynthV2022-01-06 | An AI voice copy (timing and pitch) of Misora Hibari's AI voice to Saki's voice
I could not resist adding a choir of all my singers
#synthv #sakiAI #美空ひばり #enkaTsuina-Chan AI - 「Iroha Uta」 ついなちゃん AI 「いろは唄」 (SynthV cover)HPsynthV2021-12-30 | A real voice copy (timing and pitch) from [いろは唄 cover by yuki]
Synthesizer V Studio Pro 1.5.0 Voice database Tsuina-Chan AI (version 103) ah-soft.com/synth-v/tsuina/index.html Postprocessing: EQ, compressor, reverb, delay, deesser
#synthv #tsuinachan #いろは唄SynthV Real Voice Series - chapter II - Real Voice Tuning - part 4HPsynthV2021-12-28 | This is a second part (chapter) of the promised demo how to do a voice copy in Synth V. (I'm sorry for so many parts - I can upload only 15 min videos)
#synthv #voicecopySynthV Real Voice Series - chapter II - Real Voice Tuning - part 3HPsynthV2021-12-28 | This is a second part (chapter) of the promised demo how to do a voice copy in Synth V. (I'm sorry for so many parts - I can upload only 15 min videos)
#synthv #voicecopySynthV Real Voice Series - chapter II - Real Voice Tuning - part 2HPsynthV2021-12-28 | This is a second part (chapter) of the promised demo how to do a voice copy in Synth V. (I'm sorry for so many parts - I can upload only 15 min videos)
#synthv #voicecopySynthV Real Voice Series - chapter II - Real Voice Tuning - part 1HPsynthV2021-12-28 | This is a second part (chapter) of the promised demo how to do a voice copy in Synth V. (I'm sorry for so many parts - I can upload only 15 min videos)
#synthv #voicecopySynthV Real Voice Series - chapter I - Project PreparationHPsynthV2021-12-28 | This is a first part of the promised demo how to do a voice copy in Synth V.
You can skip this chapter to the second one, which is more interesting and come back if you decide to try it. Skip to youtube.com/watch?v=kAF0oj_4RnE
#synthv #voicecopySaki AI - Look at the stars in the sky (Dambara Ruru voice copy) サキAI 「見上げてごらん夜の星を」(段原瑠々のボイスコピー)HPsynthV2021-12-22 | A real voice copy (timing and pitch) of Dambara Ruru (Juice=Juice) to Saki's voice.
I think that Saki is a little underrated and Ruru is a utahime 歌姫 (no wonder, she's from Actor's School Hiroshima).
There's no official picture of Saki, so I made very stylized "deepfake" video from original Ruru by AnimeGANv2 neural net.
(there are different translations to English around, this one is by Norio Shimizu, native Japanese with fluent English)
#synthv #sakiAI #段原瑠々Tsuina Chan AI + Tsurumaki Maki AI 「PoPiPo ぽっぴっぽー」 (Synth V Cover)HPsynthV2021-12-18 | This is not a real voice copy.
I used UST from Ken-chan, converted to musicxml and synthesized with Neutrino AI Kiritan. Then took lengths of phonemes from Neutrino .lab file and changed lengths of notes in synthV. Then used synthV's Auto Pitch Tuning, also remade the English part for Maki and tuned some parts manually.
I wanted to have phoneme lengths also tuned by AI. But it's too complicated.
Synthesizer V Studio Pro 1.4.1 Voice databases: Tsuina-Chan AI (version 100), Tsurumaki Maki AI (ENG) (version 102)
#synthv #ついなちゃん #弦巻マキTsuina Chan AI + Tsurumaki Maki AI 「Colorful x Melody カラフル×メロディ - short」 (Synth V Cover)HPsynthV2021-12-08 | A real voice copy from 朝霞しお&天秤ひなみ cover (only first 1:29)
Synthesizer V Studio Pro 1.4.0 Voice databases: Tsuina-Chan AI (version 100), Tsurumaki Maki AI (JPN) (version 105)
#synthv #ついなちゃん #弦巻マキKoharu Rikka AI - Mirai e (Kudou Yume voice & motion copy) 小春六花AI 「未来へ」(工藤由愛のボイスとモーションコピー)HPsynthV2021-12-02 | A real voice copy (timing and pitch) of Kudou Yume (Juice=Juice) to Rikka's voice. I tried different AI voices, only Rikka and Saki were able to sing it right. Chose Rikka for her voice color.
Animation of Rikka generated by neural network (RESRGAN) from one picture, motions transfered by another AI from Yume's orignal video. (poor quality)
#synthv #小春六花 #KoharuRikkaGENBU (Lite) - 「家庭の事情 Family Matters by ZORN」 (rap, SynthV partial cover)HPsynthV2021-11-29 | A real voice copy (timing and pitch)
#synthv #genbu #zorcTsuina-Chan AI - 「Melancholic」 ついなちゃん AI 「メランコリック」 (SynthV cover)HPsynthV2021-11-26 | A real voice copy (timing and pitch) from [Melancholic Cover by ENE]
Synthesizer V Studio Pro 1.4.0 Voice database Tsuina-Chan AI (version 100) ah-soft.com/synth-v/tsuina/index.html Postprocessing: EQ, compressor, reverb
#synthv #tsuinachan #melancholicTsurumaki Maki 【Ashita sekai ga owaru to shite / by Ni-na】 (SynthV cover) 弦巻マキ 【明日世界が終わるとして・ニーナ】HPsynthV2021-11-16 | A real voice copy cover (timing and pitch) of Ni-na's original song to Maki's voice. This is my most favorite song from Ni-na.
Synthesizer V Studio Pro 1.3.0 Voice databases: Tsurumaki Maki AI (JPN) (version 104)
Postprocessing: EQ, compressor, reverb
Song: 明日世界が終わるとして (Ashita sekai ga owaru to shite, As the world ends tomorrow) by ニーナ (Ni-na) youtu.be/3ASfSZ6B4mQ
Other social network summary site: https://lit.link/nachippoo
#synthv #弦巻マキ #ニーナSaki AI - 「Boku no Migi Te」「僕の右手」「나의 오른손」 / Paran Maum (SynthV cover)HPsynthV2021-11-12 | A real voice copy (timing and pitch) of Doona Bae to Saki AI.
Synthesizer V Studio Pro 1.3.0 Voice database Saki AI (version 110) Postprocessing: EQ, compressor
Song: 僕の右手 (Boku No Migi Te - My Right Hand) performed by Paran Maum (orig. Blue Hearts) from EP We Are Paranmaum (2005) youtu.be/gjaiDXCjtu4
#synthv #SakiAI #ParanMaumKoharu Rikka AI + Tsurumaki Maki AI 「GIRL ZONE - Rap Part」 by 雨ノ森 川海 (Synth V Cover)HPsynthV2021-11-09 | A real voice copy (timing and pitch) of 雨ノ森 川海's song to Rikka's and Maki's voices. I was interested in how the rapping will go.
Synthesizer V Studio Pro 1.3.0 Voice databases: Koharu Rikka AI ver.100 Japanese, Tsurumaki Maki AI (JPN) (version 104)
#synthv #小春六花 #弦巻マキKoharu Rikka AI - Vitamin ME / BEYOOOOONDS (cover) 小春六花AI 「ビタミンME」BEYOOOOONDS(カバー)HPsynthV2021-10-25 | A real voices copy (timing and pitch) of BEYOOOOONDS' song (part) to Rikka's voice. There is an expressive speech in the song - interesting for synthesis.
Synthesizer V Studio Pro 1.3.0 Voice database Koharu Rikka AI ver.100 Japanese Backing vocals: Tsurumaki Maki AI (JPN Lite) (version 101), Saki AI (Lite) (version 101) Postprocessing: EQ, compressor, reverb, CloneEnsembleV42 for the shouts
Song: ビタミンME Vitamin ME (first part) from ビヨーンズ BEYOOOOONDS channel youtu.be/YhJHUxQtnkM
#synthv #小春六花 #synthesizervKoharu Rikka AI - Hatsukoi Cider (Yofuu Runo voice copy) 小春六花AI 「初恋サイダー」(豫風瑠乃のボイスコピー)HPsynthV2021-10-22 | A real voice copy (timing and pitch) of Yofuu Runo (Tsubaki Factory) to Rikka's voice.
Synthesizer V Studio Pro 1.3.0 Voice database Koharu Rikka AI ver.100 Japanese Postprocessing: EQ, compressor, reverb
#synthv #小春六花 #synthesizervKoharu Rikka AI - Hikkosenai Kimochi (Oda Sakura voice copy) 小春六花AI「引越せない気持ち」(小田さくらのボイスコピー)HPsynthV2021-10-18 | I tried to copy a real voice (timing and pitch) of Oda Sakura from Morning Musume to Rikka's voice. This song also has many dynamics changes, so I tried to use some tension control.
Synthesizer V Studio Pro 1.3.0 Voice database Koharu Rikka AI ver.100 Japanese Postprocessing: EQ, compressor, reverb
Song: 引越せない気持ち Hikkosenai Kimochi from live Hello!Project Solo Fest 2020 youtu.be/a0T8Tc3zOW0
Artist: 小田さくら Oda Sakura, Morning Musume '20 (original artist 松浦亜弥 Matsuura Aya)