We’re in the last few weeks for entries in the 2016 Hackaday Prize — specifically the challenge is to show off your take on assisstive technology. This is a hugely broad category and I’ve been thinking about it for a while. I’m sure there’s a ton of low-hanging fruit that’s not obvious to everyone. This would be a great time to hit up the comments below and leave your “hey, I always thought someone should make…” ideas. I’m looking forward to reading them and it might just inspire someone to spend the next couple weeks hammering out a prototype to enter.
For me, it’s medication. I knew this can be a challenging problem having gone through a few cycles of prescription medicines in my life. But recently I helped out a family member who was suddenly on many medications taken on eight different times a day — including once, twice, three, and six times per day. This was further compounded by sleep deprivation (having to set alarms at night to take the medicine) and drowsy/woozy effects from the medicine. I can tell you first hand that this is really tough for anyone to deal with and it’s incredibly easy to make a mistake or not be able to remember if you took a dose.
Pill Organizers Do No More or Less
We’ve seen a number of pill organizers before and that’s what I reached for in this case. However, that organizer only had four slots for each day. I didn’t hack it (other than writing on the doors with a Sharpie for when to take each) but even if there were added buttons or LEDs I’m not convinced this would be a marked improvement.
What you see above is my proposal for the medicine problem. Smartphones have become ubiquitous and the processing power and cameras of even budget phones are mind blowing. I think it is entirely possible to write an app that uses computer vision to recognize pills and sync them with the schedule. This may mean whipping the phone out of your pocket, or designing a pill box that has a phone stand next to it (saying that makes me think of using RPi and a Pi camera). Grab your pills and validate them under the camera.
Useful Augmented Reality
The screen of the phone would use augmented reality to overlay information about the pills it sees — you know, like Pokemon Go but in a way that enriches your life. ‘pills, catch ’em all!’ — new pills can be learned of the fly, delivering the user to a screen to identify the pill and the dosing schedule. Taking the validation picture will record when the medicine was taken, and the natural extension of this systems is a pharmacy’s ability to push your dose schedule to your account when you pick up the prescription. A stretch goal would be keeping an eye out for interactions.
This is all very much like how hospitals do it — they’re scanning bar codes on the packaging and the patient bracelet and recording it. This would be an easier user experience and quite frankly I think companies already in this space (like Snapchat and Niantic) could whip this up in a single-day hackathon no problem.
Is it the perfect system? Maybe not. But there is no perfect system or we’d be using it by now. We need you, the world’s talent pool, to step up and make life a little better. Do it in prototype form by October 3rd and you’ll be eligible for one of twenty $1000 cash prizes and a chance at winning the Hackaday Prize. But even if you don’t build a single thing, one idea could be the spark that lets others change the world for the better. So let’s hear it!
I’ve got it. Slow mo Pac man! Getting pills taken freezes ghosties otherwise they get a move tpward you.
Actually, the gamification of taking medication is an interesting thought. It certainly would be fun to have the Pac Man sounds as prompts to take your pill. Game over sound when you’re late, gobble sound when take them, and power pellet sound when you finish for the day. Fun!
I would add to that the mobile phone for checking and informing which pills can be taken together and not wait for 15 minutes separation plus pills which should be taken thrice a day and interfering with the sleep routine, taking that info the mobile phone informing the user to take two pills with higher composition and also providing the mild negative effects of doing that. If the app on the mobile phone can link with IBM Watson, it can do far more even reduce the number of pills a person should take because there are pills which have compositions or similar compositions of two or more different medicines. I think I am taking it too far.
I don’t think you’re going to far. One of the things I don’t see often in sci-fi literature is drilling down to really granular predicitons of medical tech. If it did, I think you’d read about the system you just desribed. In fact, machine learning applied to all of the digital medical records currently being collected could uncover huge findings not immediately visible to human researchers. But as Ren mentioned below, there’s a can of worms there too.
Mike, this is great that you’ve invited everyone to tackle this. As a physician, it would be awesome to verify that meds were taken. Insurance companies may even be able to offer an incentive to help get the medihacker over the regulatory hump off day clearance in exchange for the cost savings of preventing drug resistance for infx and chronic dz complications of falling below prescriptive therapeutic levels. I would encourage those who take this on to hyper segment your target patient profile. The geriatric audience (at the risk of over generalization) is quite resistant to adopting new technology. Anecdotally, my mother- also a healthcare provider and had previously openly wished for a tech-based solution- refuses to upgrade from her decade old flip-phone sans-camera because she just doesn’t want to bother with some “new-dangled thing.”
I’m glad to hear this.
Yes, adoption is a tough thing. Although I must say, both my parents and my wife’s parents have now adopted smart phones. I do think that in the developed world we have pretty incredible levels of adoption. In the developing world much of the population are mobile-first adopters.
It is very interesting to think about getting insurance companies on board to drive innovation. Is this an issue that there is much less risk of further intervention if a reliable (and widely adopted) way of electronic remind-verify-record is found?
“Is this an issue that there is much less risk of further intervention…”
YES (if it is adhered to)! You wouldn’t believe the exponentially higher cost of stabilizing a patient who has forgotten to take their meds, or potentially as bad, forgot that they already took their meds, then double up on and overdose themselves. Just ask anyone working in the ICU!
Plus, there is a side benefit of speeding up the surveillance period of newly released (phase 4) medications. Medications are tested in controlled settings and optimized for the reduction of complications and increase in efficacy before being tested/ marketed to the general population. By verifying that the meds were taken correctly, one of the most costly investigatory tools may be ruled out (blood-drug levels) in certain circumstances. This helps speed up finding those rare un-lucky lottery losers who end up being hurt by a drug that all the scientific study hadn’t anticipated. To this regard, you’d think pharma co’s would want to get in on this too…or at least those within the FDA who have to do the after-incident investigations.
I don’t approve of the trolling.
https://xkcd.com/1425/
That’s a terrible use of computer vision. It’s also the least straightforward solution to the problem. You do realize that many pills are visually identical? And that most pills are distinguishable only by a code printed or “etched” into them? Or that this code is often proprietary? Or that many times they use a generic casing in order to turn a medication into “timed-release” or similar?
The pills are already identified. Meticulously. Over and over. Assured that they are in the right bottle. The right bottle. The right bottle. Does this make any sense to you? They are already in the right bottle. You just have to apply the right instructions to the right time as the time passes.
If you weren’t a hackaday staffer I’d just say you were totally clueless about all technology. But you MUST be trolling. Also a terrible use of AR. OMG it’s like you’re trying to make me blow a gasket. I need to sit down. Oh wait…
You got me… I wrote this article with you in mind and it seems my goal of making you “blow a gasket” has been accomplished. Look, it’s fine to point out that I’m wrong. But cool your jets.
Your opinion is that AR is horrible for this, but pills are fantastic targets for computer vision. They are the definition of uniform. And most of the time pills are easy to visually distinguish — shape, color, size. That code stamped into them should be able to be picked up by computer vision as well. Generics, custom compounds, etc could alternatively be identified by a barcode on the bottle. When the CV doesn’t recognize it could prompt for a scan of the bottle.
No plan is perfect. But this seems like a good application of the tech. When I helped that family member with pills I wrote down dose and time information on a sheet of graph paper and photographed it with the pill next to the writing. This is much easier than trying to read each and every bottle every time. And it is just a step removed from adding CV to the mix. It won’t work in all cases, but I think the majority of user could benefit from it.
Tablets (certainly from my late mums extensive medication use) seem to be grouped by function. She took a wealth of tablets and none were identical. When medication changed however (say from one type of morphine to another) the tablet was identical. I always assumed that specific tablets were grouped into a type in order to avoid having someone (perhaps visually impaired) take a 2 tablet fatal dose of morphine tablets that looked exactly like a Regular 2 x paracetamol Dose.
I’m sure that having two tablets visually exact that would be prescribed simultaneously that could give hazardous effects would lead to a bundle of litigation from families of vulnerable people.
Or am I just feeding the trolls?
Just, if it helps, occasionally doubling your dose of morphine won’t kill you. I just looked it up… “Therapeutic Index” is the ratio of lethal dose to minimum effective dose. For morphine that’s 70:1. So pretty safe. Of course she might not be on the minimum dose, but tolerance works there, the more you take, the more you can safely take.
Having taken morphine and worn a fentanyl patch for pain relief, I can say that doubling the dose of morphine once should not cause a problem; “should” because some people take oral IR morphine and their liver turns it into hydro-, or even oxymorphine. If the dose of morphine is already high enough that it causes sedation and lowered respiration then double could be fatal, but it is rarely dispensed by doctors at that level.
For me, the one time I took two 15mg instant release was, I suppose, the kind of high that I guess other people try to get. The normal 15mg dose just cut my pain to a tolerable level.
I concur with the original comment. Much easier to add a dense QR code + bluetooth LE LED to the bottle. Scan code -> BT pairing & scheduling info passed to phone app -> phone app uses timer to light LED and alert user what they need to take. No guessing, no data entry. Bottle cap could even start flashing/beeping if it’s not paired to your phone within 24 hours of being initialized. Phone could also ‘step’ through the pills to be taken at a particular time, lighting one bottle at a time, “press ok when you have taken 2 of these.”
That would be awesome. Something like this could be used to tell you if you were leaving your medication at home too. Even a clip on lid that would help to open the lids (while still leaving them child safe). If it were like the “tile” fob I have on my keys you could get it to make a racket and light an led to help you find and identify them.
Although this sounds like something that someone must already make surely.
http://www.vitality.net
Ah. They do :/
I think the barcode would be a better idea. As a kid my Mum had an encyclopaedia of medicines (I think from a book club). It let you identify pills by description. “Small, white, round” had pages and pages! I suppose you’d enter the drugs beforehand but even so there’s a LOT of ambiguity in meds. Obviously why they have labels. But the barcode should be fine. Not saying there’s no value in visioning pills themselves but it’d have problems.
Maybe just buttons would be better. The phone beeps, “time for your morphine and Lipitor”, and the user presses a button or ticks a box for each.
What’s the current state of med-reminder apps? There must already be quite a few. If you could get the app to communicate with the patient’s doctor or insurance company, you might get med or insurance companies to fund this for you, it’s ripe for branding. I wouldn’t want to see ads for pills while I’m taking my pills, ads for medicines in general are a terrible thing. No ads at all would be better, if that’s possible, as others have stated, pill compliance saves money in the long term from complications. But advertisers love to buy eyeballs.
Patients might not mind if the app communicates with their doctor in confidence. If it talks to their insurance company, they might not trust it. Health insurance companies are enemies of their customer, they have opposing drives. Don’t want your payments going up when you forget to take a pill.
In conclusion, it’s complicated!
In countries with socialised medicine, I can see it being a lot more helpful.
Sure, you aren’t going to scan a hand full of pills and have the computer vision tell you what they are and how often to take them. AR alone can’t know if the doctor ordered you to take 1 pill, or half, or 2 at a time. But paired with scanning the bottle . . .
The bottles I get from the drug store have, written on them, what the pill’s shape and imprint is. It has the dose in mg, times per day, take with or without food, with or without grapefruit juice, milk, calcium supplements, or even what other medications it interacts with. So, if all of that information were fed in to a 3D barcodes (like PM codes?) and stuck on the bottle, an AR device could have the shape and imprint of the pill without even asking a central database! That avoids a ton of medical privacy pressure, if an application can get it’s information without disclosing anything about the user.
How to go about that? AR isn’t my game, but if anyone wants to take this barcode idea and run with it, I’ve got a database of pill doses, sizes, shapes, etc that could create a ton of pre-seeded learning data.
As with anything related to patient privacy, apps need to respect HIPPA laws to prevent access from people who don’t need to know what meds a person is taking.
And I think you have touched on part of what prevents assistive technology from being more widely applied. This is serious business — miss a dose, take one early, or take the wrong amount and there are serious consequences. Add to that the need for security and the need not to collect data (or to collect it in tandem with properly secured medical records system) and the low-hanging fruit is suddenly at 10k feet.
Still, I think talking about the ideas is very important. Go check out the tech they’re using at hospitals now. It feels like a long time coming, but is worlds ahead of where they were 15-20 years ago. Good ideas proliferating now can show up commonplace in the next decade and that has a big impact.
For the most part, the assistive tech doesn’t always need to contain the PHI. A pill dispenser doesn’t need your name or birthdate. If it’s going to refill a prescription, it can send a dispenser ID to a service to do the refilling, moving the problem of securing the data to a server in a datacenter. For the tech Mike is proposing, perhaps it could still be useful outside the USA, where HIPAA doesn’t directly apply.
I always do due diligence and figure out what meds people are taking. Too many whacked-out dates and acquaintances I’ve had to deal with over the years.
HIPAA is a joke anyway. My optometrist made mention that Facebook is tracking patients using GPS and suggesting them as friends because they spend time in the same physical location together.
Then there was that time I made the mistake of asking for the veterinary records of a cat I adopted. Gotta protect that cat’s privacy — HIPAA!
HIPAA doesn’t protect animals. I don’t know what that vet or shelter was thinking. And facebook is tracking it’s user’s locations: those users need to learn to turn their tracking off or to not use their personal phone in their professional work. The BYOD trend has no place in healthcare because of that alone.
Facebook is sinister. I just about never used it on mobile, and of course with GPS turned off. As it is I barely use it anyway. It seems to bring out people’s stupidest side. Plus, y’know, sinister. My family nagged me into signing up.
Perhaps a simpler solution would be a dispenser with a number of hoppers. At the scheduled time, each pill would be dispensed, and you would be notified at the correct time to take your pills. The correct pill(s) would be sitting in a tray for you at that given time.
You’re _almost_ there.
Can’t leave them loose in a tray, too many unknowns. People, pets, environment, no confirmation of compliance, etc.
Tie it back to an app on the smartphone. App gives a reminder that you need to take a pill, proximity sensing to the dispenser unit, optionally a password, before pill actually pops out. Some type of limited override (“I’m about to leave the house, I’d like to take this 10 minutes early” or “I plan to be out all day, I’d like to take my 2:00 meds with me”)
Set it up with perhaps a dozen dispenser units, each one programmable to qty, cycles per day and a start/end day & time. Timer might need to be expanded for the more esoteric medications – I’m thinking of one that is several times day, but only for one week out of the month…
Conceivably, it could be really smart, knowing what medications the patient is taking and how to handle missed doses, watching for interactions, etc, That gets very complicated very quickly relying on the human to give it complete and accurate information as well as access to a database of pharmaceutical knowledge – and security considering the HIPPA compliance that someone else mentioned. I’m not sure there’s enough benefit to overcome the complications at this level of “smart”.
And just playing devil’s advocate here – in this scenario, it’s now possible for a malevolent hacker to potentially kill the patient…
Here are a few ideas I have for projects, mostly to help blind people as I am blind myself.
An app or device for finding dog pooh on the grass.
a self steering bicycle
A child finding device – should give a distance and direction to a toddler wearing a transmitter.
An app or device for reading LED and LCD displays. I started a project for this on hackaday.io, but don’t have time for it at the moment.
Good suggestions!
The dog poo problem, do you want to find poo your own dog just did? Or avoid poo other dogs have left?
The first one may be easier as it may utilize infrared detection while it is still warmer than the environment.
The second is hard, even for sighted people, if the grass is longer than freshly cut, they hide down in it and one doesn’t see it until it’s within 6 feet. Often in known danger areas one has to directly observe the patch of ground ones foot is going to land on and make a last second adjustment to pace to miss it. …. And sighted people can often fail to notice it in time too.
On smooth paved surfaces, one might detect suspicious objects 30 feet away, but false positives with lumps of mud, pine cones, pieces of branch or bark etc are likely until a closer observation can be made.
Hmm actually had an inkling of an idea to work with, initially to boost eyeball or camera perception of dog poo, but our area has poop and scoop bylaws so I have nowhere to test.
Trek out into your back garden at night with a roll of toilet paper…
Fall was a difficult time to clean up my yard. Not only the color, size and shape of leaves match closely with my dog’s packages. Sometimes the leaves covered them up.
If you don’t have a dog, and other people’s dogs are polluting your yard/garden, pour bacon grease on the offending material. The next visiting dog will clean it up.
The answer to this problem is in a large part not something the end use should ever need to know about, the solution is a robotic compounding chemist that produces customised pill sets that are fed into a pill reel and dispensed using system that knows the time and date. Until then the solution is, for not-entirely-competent patients, community nursing.
Where I am all the pharmacy dispensing is already fully robotic, just the compounding side of things is yet to happen, but for generic brand drugs there is nothing to stop it.
Ah, now you’re reaching into the land of pharmacology. Yes, some Rx’s can be compounded together. But even single drug-drug interactions are a challenge to identify and write algorithms for to prevent interactions. Compounds have a potential synergistic effect on other drugs that single-drug to single-drug don’t. That is a whole spectrum of headaches that pharmacologists inside the FDA get hot at just the mention of;)
Maybe tackling the easier thing of tracking one pill at a time may increase the quality and length of life- right now. And we can tackle the higher hanging fruit later?
Surely synergistic effects are going to happen regardless of how the drugs are made up? If you take 2 pills at the same time, they must mix in the stomach. Of course doctors and pharmacists keep an eye out for that, I think the prescribing software monitors for it.
The most common interaction is receptor (in-)complete inhibition which can be potentially temporary, based on blood drug levels. If molecular side arms start interacting this way, while the active arm is also acting on a separate receptor, you’ve now blocked the efficacy of the other drug that was compounded in. On the other side of the spectrum, there is receptor amplification through the same lock and key method of side arm activation. This is why we pay attention to efficacy and stagger drugs that have synergistic or competitive interactions. Add 3 drugs and you know have an exponential multiplication for each side arm each of those drugs have. And that’s assuming that every person in the world has the exact same activation, deactivation capacity of those receptors. Unfortunately, humans are analog in their biological processes- not digital. This is the reason for some having complete inhibition whereas others have incomplete.
Is your head spinning yet? This is why we don’t compound complex meds with histories of documented interactions (let alone the not-yet-documented ones).
The monitoring software is the equivalent to a 5 year old looking at fruit and asking if an orange is an apple…a long way to go before we can let the doctor or pharmacist ignore the prescribing protocols!
Using a service like https://www.pillpack.com would put the right medications together so you would just need to scan the pack’s bar code when the scheduling alert goes off on your phone to verify that you are taking the proper med at the right time. Several of the brick and mortar pharmacies in my area also offer this as an optional package.
Hey, this already exists: http://www.idmypill.com/
lol… there should be a way to search for this type of stuff on the interwebs
MoeB1 that does the lookup for the pills but after checking google play I see that there are several medication reminders on there and at least on offered integration with a packaging service like this -http://www.thriftywhite.com/images/HealthyPackRX.pdf?104129817
So your app and my app combined with a packing service does everything I believe Mike wanted plus some.