hckrnws
Nightingale – open-source karaoke app that works with any song on your computer
by rzzzzru
I've been working on a karaoke app called Nightingale. You point it at your music folder and it turns your songs into karaoke - separates vocals from instrumentals, generates word-level synced lyrics, and lets you sing with highlighted lyrics and pitch scoring. Works with video files too.
Everything runs locally on your machine, nothing gets uploaded. No accounts, no subscriptions, no telemetry.
It ships as a single binary for Linux, macOS, and Windows. On first launch it sets up its own isolated Python environment and downloads the ML models it needs - no manual installation of dependencies required.
My two biggest drivers for the creation of this were:
The lack of karaoke coverage for niche, avant-garde, and local tracks.
Nostalgia for the good old cheesy karaoke backgrounds with flowing rivers, city panoramas, etc.
Stem separation using the UVR Karaoke model (preserves backing vocals) or Demucs
Automatic lyrics via WhisperX transcription, or fetched from LRCLIB when available
Pitch scoring with player profiles and scoreboards
Gamepad support and TV-friendly UI scaling for party setups
GPU acceleration on NVIDIA (CUDA) and Apple Silicon (CoreML/MPS)
Built with Rust and the Bevy engine
Feedback and contributions welcome.
Just tried it with B.E.D - Walk Away[0], unfortunately it lost track of the lyrics after 30 secs (Model is "large-v3"). Will play around a bit more, as it would be great to have a working karaoke generator.
Some quick feedback:
- Needs a way to skip for-/backwards during playback to validate the result
- Sentences seem to be recognized (first letter has uppercasing), but periods aren't added
- Needs an option to edit results from the track analysis
hey mate! thanks for your feedback.
indeed, I'm running to two problems on the analyzer side: 1. align model sliding off (especially w/ chorus/back vocals present) 2. transcript skipping parts of lyrics in lyrics-heavy tracks (I tried a lot of russian rap, lol)
happy for contributions as I'm not that experienced w/ machine learning side of the project, mostly it was emperical "tweak the parameters and look what is changed"
also model only affects the transcript job (I need to make it clearer in the UI). For the alignment, it's a single model provided by whisperx
Periods in song lyrics?
I'm guessing they mean punctuation in general?
lyrics do not have punctuation.
Amazing work! I am thrilled someone was motivated to approach this problem and develop a creative solution like this. There are very limited options for Karaoke, especially in the FOSS space. Most Karaoke apps are super limited and that's driven many Karaoke enjoyers I know to YouTube in search of the songs they want to sing. This solution would give them the power to do even more songs, even better than what's out there now!
Questions for you:
1. What CUDA capability level is necessary for Nvidia GPU accelleration to work?
3. Are there any plans to support iGPU/NPU accelleration on AMD and Intel? Asking because those chips are most common in the mini computers sold at low cost these days.
My family members who love Karaoke and will be happy to try this. Looking forward to it!
hi!
1. Maxwell+ should work well 3. I would need to explore, you can join the discord or the mailing list on the website!
cheers!
I studied signal processing in university and my career evolved to not use what I studied. Decades ago, giving an algorithm a sound file and isolating tracks was difficult.
How does your implementation accomplish this? Were you involved or did you use something off the shelf?
Edit: ah, using neural nets, demucs. I wonder if there is pure math approach that can compete?
Excited to try this out. How well does WhisperX deal with lyrics in say Mandarin or Cantonese? Does it output Hanzi?
I haven't tried Mandarin and Cantonese, but tried Japanese. back at that time, it performed poorly. however, I've tweaked a bunch of settings since then, so maybe it has changed. Hanzi is a supported font and can be output, but the transcript/alignment quality might not be the best
Struggled somewhat with Tjamuku Ngurra by the Tjintu Desert Band, absolutely nailed Mariah Carey's Ken Lee.
how's this different from ultrastar singer?
https://github.com/rakuri255/UltraSinger
has anyone compared the quality of the two?
Can you bypass the LRCLIB lookup and always use Whisper?
I just want to say how much I love that you used Dean Blunt in the example video
one of my favorite artists and this one is one of my favorite tracks in general. cheers!
This looks like awesome awesome fun! Will let you know how it runs. What a wonderful idea <3
Just tried No_4mat's 1992... unfortunately it didn't work :(
Just downloaded source and built this to play around with it. I was a bit surprised that the first thing it did when I ran it was to start downloading binaries from the internet. It went off to fetch FFMpeg from some remote server, but I already have FFMpeg installed. Then it tried and failed to install its own Python interpreter, which is another thing that's already present on the system.
How come this is trying to install its own vendored dependencies, including executable binaries, instead of checking for what's already installed? That approach can lead to both security and performance issues.
Edit: the Python download isn't failing, but rather the application itself is looking for the executable interpreter in `lib` rather than `bin` once the download completes. I built the release tarball in the git repo, and I'm pretty amazed that such a basic error could make it into release code.
Further edit: I tried using the build script in the tarball rather than just doing a `cargo build -r`, and it started trying to install Docker containers! Docker to build a desktop application! What is going on here?
> instead of checking for what's already installed
Plenty of software come with their own Python runtime. Even Blender uses its own Python runtime. I can name so many apps with embedded Python runtime: Blender, Houdini, Bitwig, Substance Painter, Krita, etc. Checking for what's already installed isn't the norm. In Krita's case, it uses installed Python to build it... and in the building process it builds another Python runtime for its own!
This app should have probably bundled the runtime instead of downloading a new one though.
> install its own vendored dependencies
> lead to both security and performance issues
npm install and pip -r theoretically have the same kind of security issue. How many projects on github run this kind of command during build process? My guess is in the order of millions.
All reasonable Linux distro will patch these stupid things to usw the system interpreter.
It's not how it works. You can just install whichever linux distro of your favorite, download blender or krita, and see it uses its own python by default yourself.
And thankfully it's not how it works. If it were it'd break plugin ecosystems of many apps completely.
gentoo (a lot of lines removed):
$ strace blender-4.4
..
openat(AT_FDCWD, "/usr/lib64/libpython3.13.so.1.0", O_RDONLY|O_CLOEXEC) = 3
...
openat(AT_FDCWD, "/usr/lib/python3.13", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 12
...
And even if the build scripts are downloading deps, having the application itself install dependencies to the user's home directory at runtime is unheard of.
It's probably pertinent to mention that the Python installation ecosystem is a hot mess, with multiple ways of installing Python (e.g. standard Python installer, multiple different packages managers on different OSes, Conda, and myriad package managers which can also install Python. And of course, these can all be in different locations, and may have different approaches to installing libraries.
Which is to say, I don't blame the author for wanting a single installation that his app can manage and rely on, even though I wish it was different.
The app is vibecoded. The author isn't making decisions about these tradeoffs and possibly wasn't aware of the implications of these decisions at all. The robot they used tried to fulfill its given prompts at the expense of everything else, which is why it's looking in bad directories and trying to install Docker environments in the build script.
I suspect that some of the author's comments in this thread are vibe-written, also. They are LLM-flavored and contrast strongly vs. their regular commenting.
> The app is vibecoded. The author isn't making decisions about these tradeoffs and possibly wasn't aware of the implications of these decisions at all. The
I agree, but to be fair this is how I would code it, too. I would have probably bundled the Python interpreter and only downloaded the FFmpeg binary (because of its license), but that's a relatively minor difference.
Do.. people not already have ffmpeg on their machines? In that case it should use the system package manager.
Not all people do. Also it may be a different version that doesn't support the command line arguments that you use, or it may not have been compiled with the flags that you want. It's just less headache to vendor it.
It's not open source if it has to download binaries from the internet to run.
I agree with the sentiment but that definition would make open source Windows programs not open source either.
Why? Apparently even the build scripts download stuff here…
>How come this is trying to install its own vendored dependencies, including executable binaries, instead of checking for what's already installed? That approach can lead to both security and performance issues.
Because the person who vibecoded this had no idea they should have been doing that.
It’s a desktop app for mostly non-technical users, so bundling the runtime is a deliberate tradeoff to reduce setup friction and machine-specific breakage.
That said, an optional “use system environment if available” mode could make sense for advanced users. A PR for that would be welcome, as long as it also handles the real complexity involved: platform differences, Python package compatibility, GPU backends, and missing system/compiler flags.
> It’s a desktop app for mostly non-technical users, so bundling the runtime is a deliberate tradeoff to reduce setup friction and machine-specific breakage.
That's not a very reasonable justification, considering that dynamic linking of dependencies has been industry standard in software designed for "non-technical users" for the past thirty years or so, and is basically a solved problem.
I can understand having a downloadable archive that already includes things like FFMpeg and Python for Windows users (with everything already included in appropriate locations, so no runtime downloads necessary).
But this is an especially bad practice for Linux, since most of the vendored dependencies are already installed by default on pretty much every Linux distro, and package managers are designed to sort out and install appropriate dependencies on behalf users, so that the "non-technical" among them aren't exposed to the massive risks of having application software retrieve and execute arbitrary binaries from the internet.
The only thing it somewhat makes sense for would be the AI models it's retrieving, but even that ought to be implemented via a separate download/update script and not just baked into the main application runtime without even prompting the user that it's about to download a huge dataset.
> A PR for that would be welcome, as long as it also handles the real complexity involved: platform differences, Python package compatibility, GPU backends, and missing system/compiler flags.
These are the sort of things that config scripts at build time are designed to handle. It's already using Cargo here, which should be able to handle all of this just fine, so it's very perplexing to see that it isn't being used for this purpose, and what should be build-time dependency resolution is instead being palmed off to the application itself at runtime. That is an extremely strange -- and potentially dangerous -- approach.
> dynamic linking of dependencies has been industry standard
So it has, and I've been hating the excess complexity it brings for most of that thirty years! I'm glad to see the recent swing back toward self-contained executables. Where this author went wrong was not in vendoring the app's exact dependencies, which is a good idea, but in trying to download them and install them separately on first launch, rather than including them in the app bundle, where they can remain isolated from the rest of the system.
> So it has, and I've been hating the excess complexity it brings for most of that thirty years!
It reduces complexity compared to the administrative and security mess of every application having its own version of every library, let alone its own version of external tools and interpreters.
> I'm glad to see the recent swing back toward self-contained executables.
I wish there was one. Static linking is a great solution for this. Instead, we're seeing dynamically linked libraries being bundled alongside of executables in a way that increases complexity vastly.
> Where this author went wrong was not in vendoring the app's exact dependencies, which is a good idea, but in trying to download them and install them separately on first launch, rather than including them in the app bundle, where they can remain isolated from the rest of the system.
Agreed. Dependency resolution at build time is normal. Dependency resolution in user mode at runtime is crazy.
I'd agree if they actually were bundled rather than trying to hotlink files off the internet on first run which is just as fragile if not more so than using the system ones.
Such a setting will cause more headaches for the developer since it explodes their support matrix. Supporting a single version of dependencies and shipping those is a much more scalable solution for delivering high quality software.
Quite the opposite, in fact: if the app gains popularity that people start looking to build and package it for their respective OSes or distribution channels, the author will be inundated with support requests on account of doing dependency management in such an unusual and risky way.
"Normal" users wouldn't even encounter anything here, as they'd just install prebuilt binary packages with all of the dependencies already sorted out. As things stand, the application trying to install its own dependencies at runtime is creating a whole new class of user-facing issues to generate escalations (such as the app's failure to locate the Python interpreter it itself had just installed).
>in such an unusual and risky way
There is more risk in the shenanigans people who package software for distros do. Kdenlive suffered from big damage to their reputation due to all of the crashes packagers added by using incorrect versions of dependencies.
>"Normal" users wouldn't even encounter anything here, as they'd just install prebuilt binary packages with all of the dependencies already sorted out.
That's the benefit of just shipping what the developer released instead of swapping out dependencies under developers' feet.
hey both!
thanks for your feedback and reports, I'd be happy if they are added as issues on github.
as said in the separate comment, I really wanted an app to be as "grandma-proof" as possible, therefore I really wanted to have one binary that does the magic for you. it's a karaoke app, not a tool that is aimed at engineers.
we can indeed look at the local packages before downloading an executable, it's just not done yet but might be added in the future.
I've built this project out of passion and it's 100% open-source and free, so please keep this in mind when criticizing.
> I really wanted an app to be as "grandma-proof" as possible, therefore I really wanted to have one binary that does the magic for you.
Probably the best way to do that is to design, build, and distribute it like any other normal desktop application, and not come up with idiosyncratic and experimental methods for invoking bog-standard libraries and language interpreters.
On Windows, just include the necessary binaries as part of the application distribution itself, in hardcoded paths, without any runtime download of executables from unclear sources.
On Linux, use system defaults resolved at build time through a normal config script -- any "grandma users" on Linux will end up installing from distro repos, AppImage, Flatpak, etc, all of which have their own methods for handling dependencies, and is definitely not something the application should be trying to do by itself post-install.
noted, mate, and thanks for the feedback, really!
I'm not experienced in building desktop apps per-se, so I went with the thing that looked reasonable to me. all your comments are valid tho. I'll take a look how can I resolve this in the future.
cheers!
While I can understand that, why not replace python deps with wasm versions ?
Simply because I was not aware of this option, that's it. Happy to consider it and also happy to receive contributions.
Dog ate your shift key?
Personal attacks directed at people who build OSS are not cool, even if you take issue with their supposed use of AI or vendoring of dependencies.
Vibe coders are NOT programmers, they are leeches, feeding from the blood of the open source.
I doubt that you know the OP's background and knowledge.
If someone on here would direct at me the insinuation that a flaw in my software was the result of me having "no idea" about what I am doing, we would not be having a civil discussion.
In my view personal attacks should be flagged, but I don't have that ability because my account does not have enough Karma.
Look at the commit history. Vibe coded in 10 days, with obvious AI generated commit messages. Look at the diffs.
If you can't tell AI slop from handwritten code, that's your problem. I won't censor myself because of your opinions.
There is no need for me to look at "obvious AI generated commit messages" or check whether the code is handwritten.
Personal attacks are still against the rules of this site, and that's why you, or in this case the commenter before you, should have censored themselves. This is not a matter of opinions.
Calling a duck a duck is not a personal insult, no matter how much the duck dislikes being called a duck.
It reminds me of the billionaires that like to constantly tell people they could have "studied physics" when it's so obvious they would fail out of any intro calculus class (let's also ignore that they could easily pay for the top tutors to "study physics") but just want to code as smart and not come across as a moron, you're going to see the same thing with vibe sloppers too.
Yeah yeah. This app is open source too.
That's even worse, they're polluting the open source space with AI slop. Anyone learning from open source will not learn anything useful from it.
Man, nobody is learning anything any more, those days are gone. Programming with actual code is a niche hobby or a hyperspecialized profession now, like demoscene coders.
Besides, even before LLMs, it's not like anyone ever said "you shouldn't have open sourced this, we can't learn from your code". We just didn't bother reading that code.
Telling off people who contribute is not OK.
AI slop is not a contribution.
[flagged]
Nice detective work in an attempt to... ridicule my programming skills, I suppose?
It's a useful tool and I built it myself, with my own ten fingers, using my brain. That's more than vibe coders will ever do.
Meanwhile, your blog says in big text "I don't care for the joy of programming", so I don't consider your opinions on software development anywhere near relevant.
Excellent, on that we can agree.
Good.
Thank you for being a human that defends their craft against the wanton destruction wrought by VC + SV. They are destroying the planet to solve problems that don't exist.
This is a really embarrassing post. You stalked the author's online presence, turned up a TCP bridge utility, not really relevant to anything, and tried to shame the author for writing it, all so you can pretend you won an argument on the Internet?
It wasn't embarrassing that the GP shat all over someone's project, it was embarrassing that I pointed out that he gatekept people's contributions without contributing much himself?
Most people who contribute on FOSS seriously will not use github for anything important.
> How come this is trying to install its own vendored dependencies
"Why does this new software do X?" is probably answered by "the vibe worked on my system"
Not trying to sound like a luddite (I don't "vibe code" but I use LLM assistance), but after learning that Claude Code is a React app, I'm not surprised by anything anymore.
This is unfortunately becoming more common.
Just yesterday, I went to try out some cool new AI thing that was here on the front page of HN. It's written in Python. Great, I thought, that means I can put it into a virtualenv and just rm the whole tree when I'm done and my system will be exactly in the same state it was previously.
But sadly... no... the first time I ran it, this Python program started downloading and installing Node/NPM, and all kinds of other stuff to my machine WITHOUT even asking for permission. Sorry app developers, but my machine and my home directory are my workplace. They are curated property, you are NOT allowed to just install whatever you wish.
I expect this kind of behavior from programs whose only supported installation method is a curlpipe. (And I do avoid those.) I do not expect it from programs that claim to be installable by pip, or ship their own binaries. These NEED to be called out as vulnerable to supply-chain attacks at worst and extremely disrepectful to users at best.
I would, for safety, run the whole thing under a different user.
This guy just discovered static linking.
May he enjoy an eternal debate about static and dynamic linking, strong types and duck types, tabs and spaces, vim and emacs, less and more, and Kickers and Rockport.
Static linking packs the dependencies into the executable. This is basically dynamic vendoring.
there's a difference between static linking and downloading your dependencies over the net
What dependencies, these days, are not downloaded over the net?
If someone put this app on CDs and gave them out at a supermarket, well, it worked for AOL 30 years back but not now.
You don't want to add untrusted binary download sources in your Linux system. Your distrib is already doing that. This is not strictly about "downloads from the internet" of course
> How come this is trying to install its own vendored dependencies, including executable binaries, instead of checking for what's already installed? That approach can lead to both security and performance issues.
I've been sympathetic to your viewpoint, and I can see why this kind of thing is becoming more common.
The idea that users can reliably supply their own vendor libs/execs for applications is a bit of a fantasy. Devs working on fixing issues caused by the user having a strange issue due to the version of Python or whatever that they have installed is largely a waste of time when the application can "simply" ship with the exact dependencies it expects. This is especially true when it comes to open source work. Dealing with weird edge cases because the user has a version of FFMPEG installed that, for whatever reason, is missing h264, is work that nobody asked for. Given that the audience of this kind of app is a general one (not specific at all to devs) then it doesn't make sense to require other system packages to be present; if things like Python and FFMPEG are not required and will be downloaded anyway as part of the app install process, then there's no point in not always doing that. If you think about it, it's hardly different from any other sort of software dependency. The dependencies are just relatively bigger.
Personally, I have no desire for my applications to use other executables on my system unless I request that they do so explicitly. I'm sympathetic to the idea from a mere efficiency perspective, especially when it comes to developer tooling. But a karaoke app? No offense, but why care? A Python interpreter will be anywhere between 50 and 200 megabytes. FFMPEG is even smaller, especially if you don't enable every single feature and codec. Compared to how ridiculously bloated your average basic mobile app is (without anything like a built in JIT), bundling a desktop application with something like Python provides a lot of power relative to the number of bytes added.
> The idea that users can reliably supply their own vendor libs/execs for applications is a bit of a fantasy.
That's why package managers and OS repos exist. Users shouldn't have to even be aware of this sort of stuff. In this case, though, when the application starts trying to download and install its own dependencies at runtime, instead of everything already being sorted out at build time, the user is made aware of dependency resolution, and now has to deal with the issues involved.
> This is especially true when it comes to open source work. Dealing with weird edge cases because the user has a version of FFMPEG installed that, for whatever reason, is missing h264, is work that nobody asked for.
And that's what config tests at build time solve for, and have solved for decades.
I'm just using cross https://github.com/cross-rs/cross to build cross-arch. you can still use `cargo build`, it will just work.
It did, but then the application didn't work because it couldn't find the Python interpreter that it itself downloaded into its own config directory.
I mean... Sounds pretty clearly vibed from this description. Funnily enough this is touted as a feature, not a bug:
> Single binary > ffmpeg, Python, PyTorch, and the ML models are all bootstrapped on first launch. Nothing to install.
welcome to the millenial way of doing things
always assumes internet is connected
always assumes everything is trusted
This gave me a blast to the past to Nightingale, the media player built on top of Firefox. It was a Firefox fork that was aiming to be a more powerful alternative to iTunes/Winamp. But since it was built on Firefox, you could also use it as an all-in-one media player and web browser.
The homepage still exists, but it looks like many of the other pages like the blog and wiki are long gone. It hasn't been active in probably over a decade.
Was that a songbird followup?
Indeed it was.
> [...] users forked Songbird and created a Windows, Mac, and Linux compatible derivative under the name Nightingale.
Very nice!
I've worked on a small toy project with a similar purpose in the past [1], though it's not nearly as polished as yours, and I've made some questionable decisions here and there.
I have questions about pitch tracking. It seems you do track the pitch for scoring, and there's a line at the top of the screen that seems related but that I can't figure out. For my use case, an important feature of karaoke apps is displaying how "high" the next note should be sung, or at least some hints. Is it something your app can do and I just haven't figured it out? Or would it be a feature request?
hi there! pitch scoring is now pretty simplistic and the feature you're describing is not implemented. it's a great feature request tho!
Open source, local and passion driven. The kind of news that make me believes in humanity again. Thank you, can't wait to try it this week-end !
Looking at the commit history, this came together pretty fast. Assuming it's AI-assisted (hard to know for sure), it's a good example of the opposite of the "AI replaces developers" narrative.
AI is making whole categories of projects viable that simply weren't before. Not because they were technically impossible, but because they were too time-consuming for a niche audience to justify the effort.
Thanks for the cool project! (testing now)
I've built so many fun projects that I've been meaning to build using AI, and my boss has as well.
I do agree that AI does make a whole category of projects feasible, but I disagree that AI does not also replace developers.
Should AI replace developers? I'd argue no, and the recent Amazon outage proves that.
But is AI replacing developers? Many companies seem to be doing so, whether or not it's a good idea.
I've also subscribed to the idea that even if AI doesn't replace me, I could be replaced by a developer using AI, so it's in my best interest to understand how to use it effectively (hence the projects!)
Similar software existed at the time of windows 95. A pentium 2 could do it.
Karaoke software? Sure. Karaoke software that automatically extracts vocals? Heck no.
That removed them though…
Interesting, what was it called and how effective was it?
It was some feature in Nero, the software to burn CDs
My wife is a huge karaoke fan. I'm especially interested in the pitch scoring since we usually play the karaoke games on older consoles for that exact feature. Nobody really makes games like that anymore without a subscription (and most of these good modern karaoke platforms are exclusive to east asia anyways). If this works well this could make for some really fun social events, looking forward to trying this.
I don’t want to take away from OPs project (which seems really nice) but have you tried Ultrastar Deluxe ( https://usdx.eu/ ) in combination with USDB ( https://usdb.animux.de/ )?
There’s also a program for automatically downloading the songs: https://github.com/bohning/usdb_syncer
Many thanks for sharing, I'll have to check it out this weekend. Do you have much hands on with it? Curious how much fiddling is required when using it. Haven't had a chance to run OP's either but my dream is to find something that's as intuitive to use as the older games. Even modern karaoke software you'll get at karaoke bar usually requires me or my wife to operate it for the entire rest of the group with how complicated everything is haha.
Someone also shared https://github.com/rakuri255/UltraSinger
Which seems to output something that works with Ultrastar.
My sister loves karaoke as well and we grew up playing karaoke revolution and rock band. Shame they don't really make those anymore.
What would you consider the best "Singstar-like" alternative atm?
please try it out and share your feedback afterwards!
both transcript/alignment might not work perfectly, but it really depends on the song
This looks great, but I don't understand what it's supposed to do. I assumed the idea was "remove the lyrics" but of the 5 songs I tried (from Cry Cry Cry, Indigo Girls, and Suzanne Vega), none seemed to have any change from the original at all - it's showing the words on the screen (and the timing is perfect) but it's not removing the singing at all. How do you turn off the singing?
hey!
you can use + / - buttons on the keyboard to change the level of guidance according to your preference, generally there is a controls legend in the top right corner
Yeah, I did that. I think the problem is when there are multiple singers harmonizing it only removes one.
indeed! the stem separation model is not ideal. you can try to change the stem separation model in the settings and reanalyze the song (make sure to click the trash button and then analyze, refresh button does not refresh the stems, only the transcript/alignment)
Really nice project, I'm looking forward to trying it!
Would it be possible to process songs on one device, and then use the result in another, or even multiple? Or would it be possible to run as separate server / client?
I ask mainly because the device I connect to my TV is definitely not the most powerful one, so it would be nice if I can preprocess the songs elsewhere.
hi! answered here https://github.com/rzru/nightingale/issues/7
My sister is non-technical but loves Karaoke. I'd love to use this or similar software to build a small box attached to her TV where she can look up any song on YouTube and generate a karaoke version.
I don't expect this feature to be magically built though, so I'll have to research how everything works, and maybe I'll contribue :)
This is cool, though for me personally my minimum quality bar for the lyrics and sync accuracy in karaoke is much higher than what's achievable with WhisperX / Force Alignment. I moved on from that approach about 2 years ago in my pursuit of high quality karaoke.
If anyone's interested in tooling to generate karaoke videos which unlocks a much higher quality bar, check out my karaoke-gen project - 2 free generation credits here: https://gen.nomadkaraoke.com, open-source code here: https://github.com/nomadkaraoke/karaoke-gen
Nice work! If you are looking for ways to enhance this or complementary routes, one thing I was thinking about recently... As a musician, often I play songs I don't know the lyrics to. It would be cool to have an app that could follow along karaoke style with the words, as I sing and as the band plays. Right now I clip a phone to the mic stand, but after a lyric or two, I lose my place. This is probably multitudes more complex based on every "band/vocals" sounding different, but just something I was thinking about.
Yeah I think this is quite tricky. Even spotify screws up the sync halfway through the song somewhat frequently
Comment was deleted :(
I introduced it to my friend (who is not technical, and has a RTX 3080), his experience was: he selected a 3:32 song, it took 10-15 minutes trying to scan it. In the end it did a "decent job of removing lyrics from the audio and providing accurate ones" "It seemed to be tracking my singing but there is no mic setup or anything like it" "And I think I accidentally wiped the AI model data it took forever to generate with a single click"
In the end he's gone back to karaoke videos on youtube but it seems promising.
Big karaoke fan, so thanks for doing this. I'm processing a first test song as I write. The pitch scoring sounds really interesting as both a competitive and maybe also a training tool.
A couple of immediate small pieces of feedback:
* The colour scheme on the queue/nn% buttons is really low contrast - white on pale yellow is very hard to read
* the 'models' button (bottom left) - I assumed this would give me details about which models are available, and the sizes, but instead deleted the downloaded models without warning. Maybe add a 'are you sure you want to...' check?
Processed Kenny Rogers' The Gambler perfectly - took out the lead vocal but left the backing vocals.
hey, thanks for the feedback, both noted!
Looks amazing. I've been using Karafun [1], a paid service, but this seems promising.
Unfortunately karafun, like so many other systems, has egregious holes in its catalog. I hate playing the “this song? No… this song? No… this song? No…” game.
Psst... if you sign up to their Premium tier you can play songs from their "Community" (i.e. user uploaded songs). If you sign up to the "Pro" tier, you can play even more songs that are normally locked from their catalogue (but you'll have to manage rightsholders authorization yourself).
Karafun is a paid service? I've seen their karaoke songs on Youtube for over a decade, I thought that was their business model.
Yes, the free songs on YouTube aren't customizable: there's a low voice guide that you can't turn off (or turn up...), and you can't change tempo.
You can do this from their huge catalog of songs, using their official app or their web client: https://www.karafun.com/web/
Plus, they have music quizzes you can play with many people using smartphones as remote controls. It's super fun for parties where people don't want to sing all the time.
This is great! I thought of doing something like this for Karaoke, but was wondering about the copyright implications of doing it server-side.
We already do this for ingesting podcasts and cutting their clips with text being highlighted as people speak. AssemblyAI also supports speaker diarization.
For videos recorded using our own livestreaming studio, we can bypass all this by using Web STT and TTS APIs resulting in perfect timing and diarization without the need for server side models.
It's problematic even client side since you don't have a sync license to show words timed to the song. A bunch of other licenses are needed too for the lyrics themselves and to process the original file into the instrumental.
You’re not showing them it’s at your house
I'm not talking about public performance licenses.
No one thinks it’s legal to use this for public kareoke…
What the name reminisces me of: https://en.wikipedia.org/wiki/Nightingale_(software)
>app that works with any song on your computer
Impressive, very nice. Now let's see my death metal collection.
Just joking! Very nice, thanks for open-sourcing it.
No joke, feel free to try it! The beauty of the approach is that you don't have any limitations. Just be prepared to degraded experience, sometimes models struggle even with the simplest pop tracks :D
This looks amazing! looking forward to test it on the weekend. Does it work well on a raspberry pi with 4 gigs of ram?
oof, since it uses quite heavy models locally I would expect it to struggle on such hardware. M1+ macs & nvidia gpus work the best and fastest.
I am getting a virus detected error when downloading from Chrome.
it's a false positive, however feel free to build from the source while I'm figuring out what the virus trigger is about
So cool! I'll try it with the steam deck. Being able to go to a party with just the deck and a microphone and have a karaoke with any song people want sounds good.
I'm not sure whether the analysis would work swiftly enough on steam deck, it might be better to analyze on a desktop device and transfer ~/.nightingale/songs to the steam deck. just saying!
I haven't tried this but I would be very sceptical about the transcription of lyrics. Is there some way to correct errors?
Would it possible to port this to car play? Or does the embedded python not work for iOS apps?
Bah it has quite a few dependencies which I imagine won’t fly on iOS. More possible to build a remote streaming capability to it.
I would like the option to put the original music video as the background, like you can in Ultrastar
Aw yes, love it! Thanks for making fun stuff. Will try it out.
Very interesting! Is there any consideration to making a navidrome plugin?
This is just for English language? I have some older Japanese stuff I would love to sing!
so, as answered in one of the other threads - for me the transcript model performed poorly back in the days, however since then I've added lyrics lookup and tweaked a bunch of parameters, so it is worth trying!
This is very cool. I couldn't find it it already supports duets. I assume it won't.
stay tuned for more features!
Looks like a fun project, excited to try it out this weekend!
Cool, but on Firefox no lyrics are displayed on the website's demo.
VirusTotal says the .EXE is flagged by 1 security vendor but threat is low
hey, thanks! would be happy if you open it as issue on gh
I think you nailed it. Does it support pitch/tempo controls?
not yet, but it's a great feature request!
Amazing product. Second the pitch control feature request
Comment was deleted :(
Thank you, I wait to try it this week-end !
I hate to be that guy that points out other projects but YARG has vocals and there are a slew of songs that can be used.
i'm at the homepage of this project and i like it! will give it a try at the weekend - as a family we LOVE karaoke!
thanks, share your feedback afterwards!
This app is really amazing, congrats!!!
Can the software dump the recognised pitch and lyrics and timings to Performous text format? There's no formal specification, but examples are available on https://performous.org/songs
I just realized a couple weeks ago that the Japanese invented karaoke.
virus detected
it's a false positive, however feel free to build from the source while I'm figuring out what the virus trigger is about
[dead]
[dead]
[flagged]
[dead]
[dead]
[dead]
[flagged]
I've just made it public a few days back :)
need more pioneers to try it out and spread the word!
> separates vocals, transcribes lyrics
ML has come a long way but I have yet to encounter anything that does this reliably with speech, never mind song lyrics.
> works with any song on your computer
I'd be shocked if this is true.
Comment was deleted :(
it is as good as the models are. it is not perfect, especially for non-major languages, but it works.
Crafted by Rajat
Source Code