New TTS Model! (collecting feedback before merging)

[souzatharsis]

Hi, I think we have a break-through... The audio sample below uses what I believe is NotebookLM's TTS model!

Would love your feedback - I'll merge into main once I collect enough feedback:

Sample 1: open-notebook.ai website
(longterm param set to half the default)
https://soundcloud.com/manoel-tobias-here/open-notebook?in=manoel-tobias-here/sets/new-tts-model&si=90e10620ed10402d9cfb1a666c298c82

Sample 2: Google Handbook on Prompt Engineering
(longterm param set to default)
https://soundcloud.com/manoel-tobias-here/prompt-eng-handbook?in=manoel-tobias-here/sets/new-tts-model&si=6ea2fc3783cd445fa8362322c8aa6555

[MalikMAlna]

Overall, it definitely sounds like an improvement over the older TTS model. At its worst, it can get a bit tinny, shakey, robotic, or hollow in some places, but not more than the older TTS. At its best, it sounds more natural, varied, and emotive (albeit a little too emotive in some places as well).

That all being said, I like it.

[souzatharsis]

Sample 3: podcastfy README
(longterm param set to a low value)
https://soundcloud.com/manoel-tobias-here/podcastfy?si=a6391b5312a341fabf1905e00013dac2

[souzatharsis]

Sample 4: 5h+ Lex Fridman Podcast with Dario Amodei - Anthropic's CEO
(longterm param: default value)
https://soundcloud.com/manoel-tobias-here/lex?si=b461f4c809d24b0f974da3a29303282f

[souzatharsis]

Sample 5: Myths and Legends of Ancient Greece and Rome (1880)
Style = [formal, educational]
https://soundcloud.com/manoel-tobias-here/myths-and-legends-of-ancient-greece-and-rome-1880?si=6a777fda2f6c493baa6db52788b7cf19

[souzatharsis]

Sample 6: The Autobiography of Benjamin Franklin
Style = Engaging
https://soundcloud.com/manoel-tobias-here/the-autobiography-of-benjamin-franklin?si=782d08022146486c9b94ba7bfc9eaae3

[druce]

that's awesome! worked very nicely!
https://www.youtube.com/watch?v=Fl0xP1Io72k

have to dig in and see how you did it!

[souzatharsis]

Thank you so much for the feedback!

This has been released: v0.4.0.

• All sample audio in README have been updated to use the new TTS Model. Added some longform podcasts too.

• I've updated python notebook describing longform podcast + new Google TTS model work:

https://github.com/souzatharsis/podcastfy/blob/main/podcastfy.ipynb

[druce]

I think podcastfy.__version__ still 0.3.6 unless I did something wrong

overshot requested length which was 10000 chars, says it's calling openai tts, maybe I'm doing it wrong, config is below?

sounds pretty good !

Generating 9 parts
Generated part 1/9: Size 1825 characters.
Generated part 2/9: Size 1602 characters.
Generated part 3/9: Size 1939 characters.
Generated part 4/9: Size 1975 characters.
Generated part 5/9: Size 1542 characters.
Generated part 6/9: Size 1679 characters.
Generated part 7/9: Size 1818 characters.
Generated part 8/9: Size 295 characters.
2024-11-16 14:37:07,305 - podcastfy.content_generator - INFO - Content generated successfully
2024-11-16 14:37:07,307 - podcastfy.content_generator - INFO - Response content saved to ./data/transcripts/transcript_c44f34be5bfd496e9b3b5bffe39f7c29.txt
Generated part 9/9: Size 3442 characters.
Transcript saved to ./data/transcripts/transcript_c44f34be5bfd496e9b3b5bffe39f7c29.txt
2024-11-16 14:37:09,139 - httpx - INFO - HTTP Request: POST https://api.openai.com/v1/audio/speech "HTTP/1.1 200 OK"

audio_file = generate_podcast(text=summary_content,
                              conversation_config=debate_config,
#                               transcript_only=True,
                              longform=True
                      
#                               tts_model='elevenlabs'        
                             )
                             
debate_config = {
    "word_count": 10000,  # Longer content for in-depth discussions
    "conversation_style": ["conversational", "fast-paced", "informal", "engaging", "funny", "thoughtful", "analytical", "balanced"],
    "roles_person1": "main summarizer",
    "roles_person2": "questioner/clarifier",
    "dialogue_structure": ["Introduction", "Discuss News of the day", "Conclusion"],
    "podcast_name": "Skynet and Chill",
    "podcast_tagline": "Today's AI news, lovingly curated by man and machine",
    "output_language": "English",
    "engagement_techniques": ["questions", "analogies", "humor"],
    "creativity": 0.5  # Lower creativity for more factual content
}

[druce]

there is a bit of repetition in the qa transcript, I need to probably step through in the debugger and see if it is duplicated in the prompts that are sent ... search for : propublica wrote about

summary.txt
transcript_c44f34be5bfd496e9b3b5bffe39f7c29.txt
audio

[souzatharsis]

Thank you @druce !

Sounds like it's working, right?

However, in order to use the new TTS model you need to tts_model=geminimulti.
See documentation for how to setup your API KEY to use Google's TTS models:

https://github.com/souzatharsis/podcastfy/blob/main/usage/config.md#setting-up-google-tts-model

I am still setting openai as the default model since it does not require extra setup steps as in the Google TTS Models.

[druce]

that worked, sounds great! YouTube

[souzatharsis]

Oh and thanks for letting me know about podcastfy.__version__; I've updated the init file now and pushed a new release, so this should be fixed now.

[souzatharsis]

wow that's so much better! We should create a markdown doc with side-by-side comparison of the TTS models available - such a huge impact in output quality and engagement! <http://linkedin.com/in/tharsissouza>

…

On Sat, Nov 16, 2024 at 5:28 PM Druce Vertes ***@***.***> wrote: that worked, sounds great! YouTube <https://youtu.be/0LjESoSexXc> — Reply to this email directly, view it on GitHub <#179 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADTMY3OV42U2QE66CUAVR532A6TFTAVCNFSM6AAAAABR3VISAWVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTCMRXHE4DANA> . You are receiving this because you modified the open/close state.Message ID: ***@***.*** com>

[souzatharsis]

Here's a sample from the famous Newsletter AI News:

Source: https://buttondown.com/ainews/archive/ainews-stripe-lets-agents-spend-money-with/

Audio:
https://soundcloud.com/high-lander123/ai_news_11-15?si=acce7c7e1f094be4972b3917e407c0dd