[Lambgx02] got tired of his Android device getting bogged down and decided to dig down to the cause of the issue. His investigation led him to believe that entropy is causing the slowdown. He believes that his workaround reduces 90% of the lag on the average Android device.
So how is it possible that entropy is causing the problem? It seems there is a bottleneck when an app requests a random number from the Linux kernel running at the lowest level of the device. Android is set up to use /dev/random for all random number requests, but [Lambgx02] says that location has a very shallow pool of numbers available. When they run out the kernel has to reload with a new seed and this is blocking the app that requested the data from continuing.
His solution was to write his own app that seeds /dev/random once every second using a number from /dev/urandom. He mentions that this might cause a security vulnerability as seeding the random data in this way is not quite as random. There may also be issues with battery life, so make sure to monitor performance if you give it a try.
[via Reddit]
It’s only noticeable on applications which use random numbers (SSL etc), by basically making the pool of numbers it can pick and use less random, and less secure.
No. It isn’t (in most cases.
https://code.google.com/p/android/issues/detail?id=42265#c8
From Ricardo Cerqueira:
“So to be clear… IMNSHO, the recent entropy pool fad is bullshit. The only users of /dev/random are libcrypto (used for cryptographic operations like SSL connections, ssh key generation, and so on), wpa_supplicant/hostapd (to generate WEP/WPA keys while in AP mode), and the libraries that generate random partition IDs when you do an ext2/3/4 format. None of those 3 users are in the path of app execution, so feeding random from urandom does nothing except make random… well… less random
The only conceivable reason some devices may feel faster is because by constantly polling the PRNG, it keeps the device’s I/O in constant use (which in turn, depending on device, will make the CPU stick to higher clock frequencies to keep up and/or ramp up the IO scheduler).”
https://plus.google.com/115049428938715274412/posts/GWr72W9zmY2
Not necessarily – there are a number of users who are not using adding entroy to /dev/random, but are instead rm’ing /dev/random and ln’ing /dev/urandom to /dev/random – and they report performance improvements. Using this approach, there is no polling of the PRNG, device IO is not impacted, etc
Please keep in mind that it doesn’t look like we (Google, XDA, or anyone else) fully understands the situation yet.
“Please keep in mind that it doesn’t look like we (Google, XDA, or anyone else) fully understands the situation yet.”
Android developers understand it. It’s part of the Android user community that doesn’t. How many more times does someone have to tell you that Maps doesn’t use random/urandom, and thus cannot be blocked by lack of entropy? As people have pointed out – the davlik engine doesn’t even use random/urandom; it uses a PRNG seeded from the time/date.
Considering that it took years for the Android “dev” community to figure out that the battery stats file is wiped on each boot, it would be an understatement to say they don’t understand the problem. Even today when people complain about battery life, the first thing they’re told to do is “wipe the battery stats.”
From watching the forums for more than a year, the best I can say is that the Android “dev” community by and large doesn’t understand how Android works, suffers from massive placebo issues, etc.
Maybe google (and others) is/are using encryption to constantly spy on you and log what you do.
And yes the source code might be open, but not all of it, and not all that is installed after it’s activated.
I ran into this once developing a website that sourced random data from /dev/random each request, took me a lot of debugging to work out why the entire site was so “jittery” in it’s load times on the server but fine on my workstation, of course it came down to a lack of entropy for the server’s /dev/random.
This is wrong, please look at https://code.google.com/p/android/issues/detail?id=42265#c8
Please do a bit more research next time you make such a blunt statement – consider this approach: http://forum.xda-developers.com/showpost.php?p=36226518&postcount=331 Many users are reporting that that approach results in performance improvements… and using that approach, the CPU use and governor are not affected at all. Unless you have some more data than I’ve seen (in which case, please post it ot the issue!), I don’t think you have the information necessary to back you’re simple statement that “this is wrong.”
Many users are reporting all kinds of things just like many hifi users report that their system sounds better with certain cables.
These are programmers and developers on those forums yet so far none has bothered to write an application that actually shows some hard numbers. If the problem is so well defined then it should be quite trivial to come up with a benchmark.
My experience with one thread on XDA is that every user who changed the scheduler reported that it improved the performance of their device, including those running custom ROMs using different schedulers who actually ended up returning their schedulers to the default settings.
Until someone comes up with hard numbers this is nothing more than a placebo.
EGD or mayentropy gathering daemon be a better solution.. but I like the idea..
Is entropy slowing down your Android device? Nope.
http://www.xda-developers.com/android/entropy-seed-generator-not-all-its-hacked-up-to-be/
Apparently almost nothing actually uses /dev/random, and any gains seen from this app are from keeping the CPU awake every second, which drains the battery by not letting it go into powersaving sleep mode.
Then explain why users using this method http://forum.xda-developers.com/showpost.php?p=36226518&postcount=331 are reporting performance improvements.
placebo
Sped up my gnex and touchpad both…latency wise.
After about 10 minutes of testing, I found that it actually slowed down my Nexus S 4G even worse than it already is with JB. I should have stayed with ICS and kept the awesome battery life and fluid UI.
Seems like it helps on my non-rooted GNX, but I’ll have to give it a few days and see what I think. I have been noticing some lag lately, and I couldn’t replicate it after installing this app.
Google says that your article has nothing to do with thermodynamics. Is there another definition of entropy?
http://en.wikipedia.org/wiki/Entropy_(information_theory)
*might* cause security issues? You mean *does* right?
I was waiting for someone to say this. My reply is simply: Prove it.
There has been no published exploit that determines when a system uses a reduced random number pool let alone an exploit that can figure out:
1. What that pool is
2. How it is calculated
3. Predict the calculation
4. Feed that prediction into an appropriate algorithm
5. Use that information for anything useful
In theory it is a security risk. In theory everything can be predicted with a sufficiently advanced computational system and highly accurate model. We call that system reality, maybe it would be wise to keep your thoughts within it.
*cough* OpenBSD *cough*
Your post reveals you don’t have clue one when it comes to cryptography! Poor quality (i.e. predictable or low entropy) random number generators have been responsible for some of the biggest and largest profile cryptography bugs in history.
So your example for an exploit is OpenBSD and that clearly shows I don’t have a clue? OpenBSD has nothing to do with breaking cryptography. Merely be secure out of the box.
Once again, can anyone point to an exploit that can detect when any system (/dev/random in this case) is more predictable than it normally would be and then use that information for something useful?
You want an example? Take it.
http://jblevins.org/log/ssh-vulnkey
Failed entropy. Massively reduced key generation on a bunch of systems. Much headache and paranoia caused.
Quick googling on OpenBSD pulls up this:
http://www.h-online.com/open/news/item/OpenBSD-random-number-generator-not-random-enough-734175.html
Also: ‘So your example for an exploit is OpenBSD and that clearly shows I don’t have a clue? OpenBSD has nothing to do with breaking cryptography. Merely be secure out of the box.’
Seriously? You really did miss the point on that. He’s saying OpenBSD has had this exact problem in the past. You apparently can’t read that comment.
@James
Please read again: ‘Can anyone point to an exploit that can detect when any system (/dev/random in this case) is more predictable than it normally would be and then use that information for something useful?’
All you did was equivocation and false equivalence.
By might he means in theory it is a security issue because it makes your random bytes slightly less random. In practice however there are easier exploits and/or targets.
if the ratio between “pure” random and RNG numbers is in itself random, and only a few RNG numbers are fed at a time, and the RNG feed ADDs to the “pure” random feed (not just passing numbers thru, but making them dependant on what was there before) there is absolutely no effect on security as you cannot determine the sequence and the predictable RNG sequence used is not directly fed to the output.
unpredictable number + 5 = still as unpredictable.
as long as the predictable RNG isn’t used exclusively and there are a few “pure” random bits in there it’ll be unpredictable.
add RDTSC to the mix (or the platform’s equivalent) and you got a big mess of randomness.
adding more random noise does not make the signal clearer.
Every damn Android phone is loaded by apps that ask for full access to every information even when unnecessary or they won’t install. If I still was an Android user (I stopped one year ago and never regretted) the /dev/random vs /dev/urandom would be the least of my concerns.
looks like a clever solution to me.
maybe it could be started and stopped as needed?
… for battery conservation.
is there a way to detect when its needed?
i would not think a 200$ cellphone/android would have a great random number generator anyway, so it being a small % less random wouldnt bother me much at all
even a top of the line desktop computer doesnt put out good enough random numbers for _***SERIOUS***_ cryptography. by serious i mean corporate. those guys have a “USB thing” for that.
I was wondering if there was a may to check the “volume” of /dev/random and then seed only when needed, would that help battery drain? (Or would that just be running process instead of another?)
i have seen something like this on osx it was called xpcservice it was a process launched whenever a web page loaded a video clip
i found that removing the offending file from the system stopped the cpu hogging
http://www.google.com/search?q=xpcservice+osx+cpu+hog&btnG=Search&sclient=psy-ab&hl=en&tbo=d&site=&btnK=
The usual suspect in this case was the QT xpc service. Unfortunately QT (and QTX) is a pile of sh**t, so when it crashes, it burns…
If /dev/random is running out of entropy, somebody’s probably programming with it wrong…
What you’re supposed to do is use it as a random seed for your own random number generator. Unless you’re running an insane amount of cryptographic programs at once, it shouldn’t slow you down…
LMAO @ comments! Good job HaD!
For what it’s worth, Seeder is making my Samsung Galaxy S3 run a lot smoother without any additional battery usage. As a blind test, I installed it onto my wife’s S3 without her noticing this morning, later this evening when I saw her she asked if there were any updates for the S3 today because her phone was running much better than yesterday.
If this is a placebo, it’s an amazing placebo.
Looks like another example of the “everything is a file” philosophy run amuck. Just because it can be made into a file, doesn’t mean it should be. How much extra overhead is involved in getting a random number from a file, instead of from a dedicated OS call? When the random pool is exhausted, and a single random number is requested, how many random numbers does it generate all at once to refill the file, contributing to sporadic lag? Silly.
Nice troll. I give it 8 out of 10.
Aww, I was hoping for a perfect 10.
But in all seriousness, I see no comments disputing that /dev/random is laggy. Only some that say because most apps won’t use it, the real impact on the user should be negligible.
If /dev/random is laggy, why is it so? Being a Windows and MCU developer, the concept of a laggy random number source is so alien to me, that the only reason I can figure is that it’s implemented as a file. Even a *slow* RNG is not *laggy*.
You’re completely wrong, in many, many ways. Unless you have a hardware RNG, all RNGs are laggy. They have to be. Where else do you get the entropy from? You have no entropy – therefore, you have to wait until some is added.
You’re confusing RNGs with PRNGs. PRNGs aren’t laggy, because you just generate new pseudorandom numbers. But those numbers are pseudorandom, not random. They’re not cryptographically secure – they’re predictable (and, in fact, historically using PRNGs instead of RNGs has caused problems, for instance, in TCP sequencing).
/dev/random is a source of cryptographically-secure random bits. That’s what it’s for. If you want a non-laggy source of pseudorandom bits, you use /dev/urandom.
(I won’t even mention the ‘it’s implemented as a file’ thing. I really suggest you read up on how operating systems, and Unix-like OSes in particular work before criticizing them. They are not implemented as files. Accessing them is implemented using file-like methods. But they are not ‘files’ in any common sense of the word.)
Thanks for the explanation, and for being more informative that insulting like the other replies. I’m familiar with the differences between PRNGs and RNGs, but didn’t know /dev/random wasn’t a PRNG.
I wonder if some app developers didn’t know this either. Or just always choose the better quality random number source, regardless of whether its actually needed, and without realizing the downside.
Are you really that stupid?
Unless you have a dedicated hardware random number generator (HaD had a few articles about them here, look them up), the kernel must use devices like the keyboard, touchscreen, soundcard etc.. as an entropy source. It is obvious these cannot contribute to the random poll on demand, hence the “lag”..
“Being a Windows and MCU developer,”
yeah, no wonder you sound so “clever”.
VB and Arduino…. Awww isn’t that cute!
Run along now, the adults are talking. *pats you on the head*
Before commenting on everything being a file, it’s important to understand that just because something can be accessed by what appears to be a filesystem doesn’t mean that it is a file, related to a filesystem, or has any of the normal overheads associated with accessing a filesystem.
In Unix systems there are a few mount points that provide so called special files. One of them in this case is /dev. The contents of /dev are not files, but rather simply provide an interface to a device driver. The only thing special about them is that they appear in the filesystem. In many ways accessing /dev/ttyS0 is no different from accessing COM1 in windows. In fact to really mess with with your understanding of what is a file, /dev has the block locations for all harddisks too. In the case of many linux systems the root file system “/” is actually mounted to “/dev/sda1” or something similar which if these were normal files would make it appear as a recursive loop.
I’ll believe it once I see some hard evidence that these processes access /dev/random enough to cause a serious bottleneck.
Wow… This is so wrong. IMO the fault is more on driver developers not making entropy sources. I bet mobile phones could have tons of them.
Battery tension/intensity/temperature, magnetometer/accelerometer orientation, cell codes… And so on
Instead of seeding /dev/random with urandom, try replacing random with urandom and see if you get the same benefit.
UI lag is a real issue with Android 4.2, so I tested this hack extensively on my Nexus 7. Rooted, non-rooted, with the daemon active or not. I also tried symlinking /dev/random to /dev/urandom. There was absolutely no noticeable change at all. To rule out my own subjectivity I also ran a few benchmarks, including UI responsiveness benchmarks, and no change either, except in the symlink case which was actually about 5% slower.
Let’s use this as a reminder that relying on XDA-developers and Reddit as primary source of information is not the wisest.
If this is really a placebo I am going to pretend I installed it in the hopes my phone runs “perceptively” faster. Or better yet I’ll perform a factory restore with the pledge to never again install any apps.
There’s a competing app made by one of the Seeder testers that works even better than Seeder on my Droid 3. It uses a slightly different methodology or something to use fewer resources. The explanation is a bit above me but things are noticeably snappier using it over Seeder. Here’s the market link, try it and see. I’d only recommend using one or the other, though, not both.
https://play.google.com/store/apps/details?id=com.zdvdev.sef