Everyone loves watching movies, that is, so long as you can hear what the characters on screen are saying. [GreatScott] found this second part difficult while watching through BladeRunner 2049, so he designed an automatic volume adjuster to assist.
At a high level, the solution is fairly straightforward; when there is loud music playing in a movie, turn the volume down. The challenge is how to actually achieve that. The first step was controlling the volume. To avoid having to modify or damage his sound system, [GreatScott] opted instead to mimic the volume up and down signals of his remote over IR. Using the very handy IRremote library for Arduino and its built-in decoding functionality, he was able to identify and replicate the signals with his own IR LED.
The second step in this process was measuring the volume of the movie. [GreatScott] achieved this with a microphone and amplifier circuit, that was then piped into one of the analog pins of the Arduino Pro Micro at the heart of the build. Since the audio being sampled could have a frequency as high as 20 kHz, the ADC Prescaler had to be adjusted from its standard value, which would have only permitted measurements at less than 5 kHz.
The third step was writing the algorithm to detect loud music and adjust the volume accordingly. The Arduino will measure the audio until a sound greater than the dead band value, set with one of the two onboard potentiometers, is detected. This then triggers the Arduino to start a timer, to see how frequently the upper limit is being surpassed. If it is just one or two occasionally loud noises (like a scream, a clap, whistling, etc.) the Arduino will not take any action, but multiple loud noises in rapid succession will then trigger the volume down command over the IR LED. A second potentiometer allows for adjustment of this timer’s critical value, so that you can make the system respond faster or slower depending on the movie.
Once the sound has been detected to have dropped down below a critical vaue, the Arduino assumes that the movie is back to dialog and will increase the volume by the number of times it decreased it before, leaving you back at the perfect volume.
Maybe you’re the type that cares more for the visuals of a movie, rather than the audio. In that case, this e-paper movie display will be perfect for giving you time to appreciate every frame!
How does turning the volume down help clarify dialogue? Surely this really needs to move into dynamic filtering and EQ? At which point you are esentially replicating a simple digital hearing aid (and would also need external audio amp, speakers etc.. to implement). I guess I will have to build it myself to experience the effect, but I also wonder if the response time over IR is going to impact the plan here.
Dynamic filtering and EQ does help to a point, I have a tv with some settings that do help on some shows but it seems like on some shows the composer and sound effects people really want you to notice their great work (or dont want you to notice the really bad dialogue) and run their sound levels 3 notches above any dialogue. This is really just trying to keep a constant sound level. I gave up long ago and when I get a show that does this I turn it way down and turn on the subtitles.
Inception was that way through the entire movie, at least the stereo mix on the DVD was. Perhaps with a 5.1 channel system the center front channel could have been turned up and everything else down to make the dialog audible. I had to watch it with the subtitles on.
Perhaps we are already living in “Space Merchants” world and he can’t safely say that he made a TV commercials zapper?
The point being often in movies the dynamic range is extremely high so when you set the volume to keep the explosions and loud music to a tolerable level the quiet sections with dialogue become far too quiet. With a compressor (which this is a crude implementation of), you can turn the volume up to the point where the dialogue is intelligible, and the loud sections get automatically limited to a reasonable level.
Commercially produced music has had the opposite problem (see “The Loudness War”) where the dynamic range is crushed to almost nothing, so there are no “loud” or “quiet” sections, just a constant, boring, high level.
It helps in that you can turn the volume up all the time so that the dialogue is clearly audible, but then the volume will automatically drop to keep the windows from bursting when something loud (and long) happens.
The usual way to deal with loud movies is to turn the volume down so that the loud parts aren’t too loud – but then the dialogue is too quiet.
It doesn’t help clarify dialogue, it drops the volume on high db sounds like music/fx and raises it back to the preset when the sound is back to normal.
It’s an issue of dynamic range. Movies often have extremely high dynamic range, which means the loud sections are very loud compared to the quiet sections. When you’re watching at home, often you don’t want the loud sections to rattle your eardrums or your neighbours, so you turn the volume down until they’re at an acceptable level. Unfortunately, this then makes the quiet sections which are often dialogue far too quiet to be intelligible.
This device is a crude dynamic range compressor, allowing you to raise the volume of the dialogue and have any sudden loud noises automatically limited to an acceptable level.
Commercial western music tends to have the exact opposite problem, called the “Loudness War”, where the dynamic range is compressed to almost nothing, because there’s a perverse incentive to make your song “louder” than the competition for listeners.
@Wibble, I think the issue for [GreatScott] is that the audio is too loud between the dialogue-heavy sections and needs turning down, but then the dialogue itself becomes too quiet to hear. This reduces the dynamic range so the loud parts don’t bother the neighbours but the dialogue can still be heard.
So what’s going on is that there’s a lot of dynamic range in movies. Typically, the dialogue is fairly quiet, and the action loud. It’s nice in a cinema, and they keep the overall volume high so the dialogue is perfectly audible.
But at home, perhaps you don’t want to leave the volume high. Setting the volume high enough to keep the dialogue audible could mean the loud bits are too loud – might bother the neighbours, wake the kids, or just annoy other people in the house. Potentially it might mean distorted sound from lower quality speakers, or painfully loud if the listener has reduced hearing (so needs the dialogue louder than a normal person).
So this works as a very basic compressor, with a slow response. Think of it perhaps as rather than making the dialogue more audible, it compresses the loud bits, meaning he can set the “normal” volume higher.
Yup, can definitely see it’s got it’s uses. I’ve done this manually when the kids are in bed.
most tv these days is mixed so that if you have the volume high enough to hear the dialogue then the music is deafening.
It won’t help if dialogue is spoken at the exact same time as a loud music/ noisy action sequence, but it happens many times that I crank up the volume to comfortably hear the dialogue during a ‘normal’ scene (when important plot points might be discussed) only to then find it excruciatingly loud when something exciting happens with accompanying big sfx or music. The dialogue is usually less important in those parts. I totally get it, and I don’t think the lag will be too much of an issue, since it’s designed to only respond to longer sections of loud sound.
While much of the problem relates to dynamic range, as mentioned in most of the replies above, I find that some movies/shows are incomprehensible even when characters are speaking normally with no background music.
I always thought it was me getting old, but then I was having a problem with Tenet and looked it up and… apparently there’s a move towards making dialogue less intelligible, it’s thought to be “artistic” and it adds impact to the rest of the movie.
https://www.theguardian.com/film/2020/sep/03/tenet-dialogue-christopher-nolan-sound-technology
Apparently lots of people are now watching movies with closed captioning turned on to avoid ambiguity.
I personally like to pay attention to what’s going on, and think that ambiguous dialogue detracts from the movie (and doesn’t add to it, as the “artistic” movement would argue).
I’m hoping that this movement dies out as people realize that it’s having the opposite effect.
On a positive note, apparently my misunderstanding of dialogue isn’t from me getting old.
What a stupid idea, I mean on occasion when its justified mixing the talking down so it is very hard or even deliberately impossible to really parse can make sense, it is going to draw the audience somewhat, and creates that correct feeling for the whispered conversation nobody is meant to overhear – but to do it throughout and therefore through all the important dialogue you can’t miss if the plot is to make sense (though with how shallow or predictable many plots are in movies now I’m not sure that really is much of it)..
I’d always assumed it was just downmixed awfully for broadcast/stream or mangled somewhat by the compression when it happens to me. Though I usually play audio through a decent sound system with great on the fly normalisation (if its turned on) and from a computer – so usually can implement some controls at source which means generally the issues are corrected enough that it is not that bad, but even with good ears I’m noticing it more and more recently (and my ears are still good), which I guess this explains…
I’m definitely with you this ‘movement’ shouldn’t ever have existed, the basic idea is plain stupid – its a technique that might make sense in the odd scene in a show, creating the right effect, not as the default stance!
I’d be more worried about unreliable transmission/reception – if it happens you won’t return to the original volume…
For me it is usually the downmix to stereo that is crap. Kodi has options for that. Else use a compressor.
Yes, VLC Media Player has a useful compressor / expander. I’ve run a few movies I got from dodgy sources through it, with good results (but have now deleted them after the purely scientific experiment yielded useful results). My theory is that some movies get ripped from sources intended for use in cinemas, where the massive dynamic range is used with good effect, but is not suitable for home setups.
An especially bad example is the 2 channel mix on the Inception DVD. With a 5.1 system and using the DTS track the center front channel could be turned up while turning down the “foreground music” that was obliterating the dialog.
Can’t believe that horrible sound mixing won an award.
+1 for compressor/expander, a lot more faster.
Also brings Night mode to keep the family happy at night
Thought BladerRunner 2049 was quite very clear. Perhaps better speakers and a nice listening environment are the way forward.
Can’t imagine using the IR protocol is particularly responsive. A normal compressor can usually be set to be almost un-noticeable.
If silent and loud parts switch frequent enough and this method is slow enough it could even be counter productive.
I had the same issue with the movie The Hunt for Red October, so it’s not something new.
I was gifted by the ancients a tech they called “subs” a series of symbols sometimes separated by emptiness that represented the words coming out of chris tucker’s mouth, I’ve been blessed!
Yes, useful as a crutch. Though reading subtitles distracts from the visual part of the scenes. There is a reason, why a movie normally addresses more then one sense (the eyes). It is intended to be perceived also with your ears.
Nice project!! I can relate to the issue. Plex will try to “normalize” commercial volumes but movies are left for me ro raise and lower volume throughout. the scam that loud commercials are on purpose is unfortunate but mfg’s should build better auto volume into their tv’s ( looking at you 15 year old sony)
I wonder if anyone has been fined for violations of the Commercial Audio Loudness Mitigation Act? Yes, the CALM Act. https://www.fcc.gov/media/policy/loud-commercials
My not-smart Philips TV has an option buried in menus, that activates a compressor or a volume control to average the audio output. I suppose that other brands have, hidden somewhere, this option.
nice but please add some improve
MUTE ADVERTISMENT ;-)
Interesting idea but for me thetelly insists on popping up the volume setting on-screen, so it would be highly annoying, probably more so than the volume being too loud, for a film anyway. Could be fine for normal broadcast and stupidly loud adverts though.
Create something that sorts out the sound in a Christopher Nolan film and you’ll have people lining up to buy it!
A lot of movies have some sort of surround sound garbage. Left right, front, back, center, sub-honker, etc. Dialog tends to be emphasized in the front center. When you squash that into simple left, right stereo then the dialog tends to get overwhelmed by other sounds no matter what the overall volume. Bad mixing. I have lots of movies where dialog is difficult to parse no matter how soft or loud.
… If only the dialogs were on a separate audio track that plays simultaneously…
In production the tracks aren’t mixed yet anyway and this would eliminate the requirement for any post-processing at the consumers end.
It would save space too because the different language tracks would only contain the dialog (they’d all use the same non-dialog track).
But of course establishing such a standard would be a hassle:
– New player software, firmware and maybe even hardware to play two tracks as defined in the medium/container.
– New UI configuration design to let the user properly select what the volume keys do.
– changing the volume of the separated tracks through CEC (HDMI)?
+1^
As a person who has suffered hearing damage and wears hearing aids, I would love to have the ability to bump up the dialogue track and turn down sound effects and music. What happens now is that I put the audio level where I can hear the dialogue clearly and when the explosions or rain occur I’m shaking the house. Late at night I find I use Closed Captioning to catch all the dialogue.
the problem with sound mix is that it’s subjective. What the sound designer finds easy to understand isn’t the case for every person out there. it’s the sound equivalent of the GOT episode with the dark fight scenes. It looked great in a controlled environment to that person, but once it got into the broadcast compression it didn’t fit the bill. They don’t compress audio that bad (compared to video anyway). Maybe a dialogue accentuated track as an option when watching??
AFAIK my hearing is still fine but a friend of mine is partially deaf (I think) and at a recent movie night at another friend’s home this problem came up.
Especially because you can’t just “shake the house” in a flat without disturbing the neighbors.
You can often do that a bit by messing with the7.1 / 5.1 downmix. Dialogue is almost always on the regular L/R channels, and sound effects on all including the side/back channels. So if you bring those down a bit, voice should become more clear.
It would be indeed great if effects and voice would be separately channeled but this is simply not happening.
I have mild hearing damage and started wearing hearing aids a few years ago. More recently I started using a ‘TV streamer’ to send the audio directly to the hearing aids. Helps a lot with hearing dialog. I can adjust the volume that I hear independently of the speakers. Can even watch TV with the speakers off.
Still doesn’t help with crappy downmixed films with no subtitles!
Tempted to try what others suggest. Get a surround sound processor and only feed the center channel to the streamer and see if that’s any better.
Yeah, I have considered getting a streamer and some other things, but the price is insane…I wish that Android would get their stuff together and make the bluetooth compatible with assistive hearing devices. I won’t switch to Apple to get that, but I would like to have it.
everything that’s compatible with hearing aids costs so much more.
Crap downmixes are maddening, aren’t they?
I know this is a bit excessive, but do you have a T-coil hearing aid? Installing your own induction loop in a room with cheap speaker wire/lamp cord is relatively simple, and apparently if you match the inductance right you can drive it with a relatively cheap audio amp. Not as good of a solution as android would be, but serviceable within a living room.
https://spectrum.ieee.org/a-diy-audio-induction-loop-for-the-hard-of-hearing
I know this is an expensive solution but if you have surround sound, dialog is pretty much exclusively on the center channel. Most audio receivers let you adjust the volume of each channel because they need to work for all different sizes of rooms. So I just turned up the volume on the center channel and left the rest alone. We haven’t had to turn on subtitles ever since.
There are usually similar options for this in software if you are watching something on a computer. Look for compression or a mixing matrix.
With 5.1 surround sound (Dolby Digital AC-3 or DTS), the centre front speaker “should” reproduce sound intended to come from immediately in front of the listener. Which at least in theory should be the cleanest channel for dialogue.
The problem with your idea is that on say with 5.1 system having the dialogue separate, for the most feature rich experience, would require an additional 5 audio channels for each language. And then the mixing would occur inside the end user device, with digital restriction manglement that could use a lot of additional power. I think the idea is great, and we will end up there eventually, some day in the future.
Right now I dial down the volume on the subwoofer (low-frequency effects channel), up the volume on the center channel, lower the volume on the surround channels and enable subtitles.
TV standards have long supported SAP (secondary audio program), which allows inclusion of soundtracks in multiple languages. I have wished for some time now this would be used to provide a mix which enhances the dialog relative to all the other sounds. This would be a huge help to those with aging ears or other hearing issues. Sure, some TVs and sound bars have audio settings that claim to enhance the dialog, but why not just get it right in the mix in the first place.
This would be great for day to day use, turning down the peak volume of adverts that are purposefully much louder than the programme you’re trying to watch. Sadly they still get away with claiming the average volume makes it okay.
Perhaps we could have something that watches the video and automatically cuts out the adverts from a recording, when it sees the same sequence, a symbol on the screen, the display format or colour balance change.
Loud commercials are a big problem with us frugal streamers.
Would this just level the overall volume?
Similar experience with awful TV sound here. Those small air pumps they call “speakers” will never speak clearly, no matter how much software you throw at the signal. I replaced them with external hi-fi speakers and turned off all that srs-surround-wow-dolby-whatever effects crap. Problem solved.
I have a similar feature in my TV. It is a compressor that raises the volume of quiet scenes and lowers the volume of loud scenes. This technology has been out there for at least 20 years.
Audio compression has been used for broadcasting and sound recording since the 1930s. It’s one of the ingredients that made ancient electronic media successful.
Look at all the bleeding edge tech experts who have never heard of normalization….
I like forward to the post-Google tempered responses… I’ll wait
My TV doesn’t have a setting for audio normalization & what if you were in a situation where you couldn’t modify the incoming content? I could hardly break out my handy audio/visual software to process a netflix stream could I?
Also even if you do have a sound system that does volume normalization it might not do it well, introduce lots of latency to the video perhaps… That said the IR remote method seems like it would be so slow and clunky that it doesn’t really give a good result either.
Ultimately you want to have a good listening experience you really want to be able to control the source and sink.
Why not? Netflix on the PC and on the way to the sound system (audio line) you can do what you want to the signal. Ah yes, instead of a TV I use a beamer connected to the PC. I like the big picture.
My 14y old Onkyo receiver has a night mode that fixes issue. Other names are DRC or whatever.
Dynamic. Range. Compression.
Funny enough, Hackaday actually featured pretty much the same project already back in 2013: https://hackaday.com/2013/02/11/automatic-volume-control-puts-the-kibosh-on-loud-tv-commercials/
It would help matters if the content providers would quit stripping down the data streams
(i.e. deleting nuance) with every “new and improved codec” update.
In my area, a lower range mens voice is “grainy” now. Get to the scene where the old guy is talking softly and the voice sounds like it’s at about 50~60 kbps now.
Another foul up is when they mixdown and just drop the center channel entirely…dialogue sounds like there outside of your house then.
And may I gripe about what this same data deletion also does to picture quality?
Tried watching “Epix” channel for a while. The sound was rough, but holly-f###, the video had some sort of jitter and over driven contrast (kinda like Flash vid from around 10~15 years ago)
Was a nasty headache inducer. Cable co said it’s not there fault, so we ditched the channel.
And lastly.
Can we stop cropping the top and bottom (absolutely ruining the framing) of the movies to fit cell phone aspect ratio??
we still have cable, and they are in the process of transitioning to streaming, but on some channels its not unusual to have 3 sets of bars on a show so that you end up using only about a quarter of the pixels for the show (its like watching tv in the ’80s). i can understand this on old 4:3 shows but even shows in 16:9 do this with side bars and at least 2 sets of top bars. they have been using the transition as an excuse not to fix the problem. i find myself pirating shows i could have watched legit if the cable company had just done its job.
There is a reason, why I think of this guy as [Mediocre Scott]. He never heard about compressors, automated gain control circuits, etc. NE572/SA572 or LM13700 would work much better than this idiotic solution And if he really, really wants to use a μC, he should use a dsPIC or STM32 and perform digital compression/expansion. But that would be too hard for the guy who can’t even build a spot welder…
I recognize the problem.
Not being as proficient in electronics as GreatScott, I came up with a different solution.
I turn on closed captioning or subtitles.
most useless gadget ever. just use a audio source that fits your sound system and/or calibrate your speaker setup right.
your problem is a slippery sidewalk and your solution is to offer free band aids for everybody.
If you can’t see the speakers in front of you they can not work well, not on the bottom of the screen yet alone on the back location.
No whispering allowed in the dialogue! Please.
Since most everything is fake and dubbed in anyway the dialogue is really a monologue and should be easy to channel to normalize. User selectable compression or full dynamic range was in the original specs for digital radio but was it nixed from further development, and here we are. The volume war rages on…
Music as well as cinema sound is mixed all together as though you need to hear all of whats going on as a single level event. I find that classic country is the only genre where production can put the vocal on top. I haven’t been able to hear vocals in most other music since starting with the Beatles.
I have to say that this seems like a software problem solved with hardware. Changing volume via IR just seems so slow. IMHO, a better way is to solve it via software which can preprocess the audio and compress the volume as needed.
its entirely possible my tv has a feature that does this already, but im too lazy to look for it. vlc also has a dynamic range compressor filter that you need to be an audio engineer to use correctly.
A more passive-aggressive TV-B-Gone. I like it!
good, i hate riding the remote just to watch a show at 3am without pissing off the neighbors.
I work for a cable content provider and can tell you that WE take audio seriously. We have very expensive systems in place to keep the dialnorm for our programs at a consistent level as much as possible. Since we don’t have local commercial insertion on our service the volume level is very predictable.
Cable companies need to put in some dialnorm systems so commercial insertion doesn’t blast you into next week. This also needs to be enforced for the streaming content providers…Pluto is bad at times.
Years ago (maybe 40?) I built something like this that simply connected between the preamp and power amp of my audio system. It used an FET as a volume pot that automatically adjusted based of the voltage of the audio signal coming into the circuit.
My LG soundbar has a “night mode” that keeps volume up in the quieter parts and reduces volume level of the loud stuff. It may boost the center channel a bit, too. It works pretty well, and I haven’t had any complaints from the neighbors.
So there will be “volume bars” on the screen almost all the time?