Humans And Balloon Hands Help Bots Make Breakfast

September 24, 2023 by Kristina Panos 7 Comments

Breakfast may be the most important meal of the day, but who wants to get up first thing in the morning and make it? Well, there may come a day when a robot can do the dirty work for you. This is Toyota Research Institute’s vision with their innovatively-trained breakfast bots.

Going way beyond pick and place tasks, TRI has, so far, taught robots how to do more than 60 different things using a new method to teach dexterous skills like whisking eggs, peeling vegetables, and applying hazelnut spread to a substrate. Their method is built on generative AI technique called Diffusion Policy, which they use to create what they’re calling Large Behavior Models.

Instead of hours of coding and debugging, the robots learn differently. Essentially, the robot gets a large flexible balloon hand with which to feel objects, their weight, and their effect on other objects (like flipping a pancake). Then, a human shows them how to perform a task before the bot is let loose on an AI model. After a number of hours, say overnight, the bot has a new working behavior.

Now, since TRI claims that their aim is to build robots that amplify people and not replace them, you may still have to plate your own scrambled eggs and apply the syrup to that short stack yourself. But they plan to have over 1,000 skills in the bag of tricks by the end of 2024. If you want more information about the project and to learn about Diffusion Policy without reading the paper, check out this blog post.

Perhaps the robotic burger joint was ahead of its time, but we’re getting there. How about a robot barista?

Continue reading “Humans And Balloon Hands Help Bots Make Breakfast” →

Programming A Poker Game With GPT Help

September 7, 2023 by Bryan Cockfield 10 Comments

Although ChatGPT generated a huge amount of hype around replacing white collar workers completely when it was first released to the public, the general consensus now is that it won’t outright replace anyone yet, but rather people who know how to use it as a tool will replace those who don’t. Getting started with it is not too hard, either, but you’ll of course need a project to work on to familiarize yourself with the tool. [Volos Projects] gave himself the challenge of writing a poker game using ChatGPT not as the opposing player, but as a co-designer in order to learn more about it as an assistant.

The poker game is being built on an ESP32 board with a built-in AMOLED screen. Five buttons are wired to the microcontroller to allow the player to select which cards to discard and which to keep. The bet for each hand can be raised or lowered much like the tabletop poker games often seen in bars and restaurants. To program it, though, ChatGPT was used to help design the code at each step of the way, first describing the overall goal and then building each function one-by-one like shuffling the deck, dealing the hand, and then replacing and dealing new cards.

For anyone who hasn’t yet explored using ChatGPT to help design their programming projects, this effort goes a long way to showing just how useful a tool it can be. For more complex tasks, though, it does take a little bit of knowledge on the part of the user because ChatGPT can often turn out nonsense or factually inaccurate information, but at least in a programming environment you’ll generally find out quickly when that happens. It’s not just a useful tool for writing programs, either. It can accomplish a lot of ancillary tasks related to programming as well, even if it’s not writing the code directly.

Thanks to [Peter] for the tip!

Continue reading “Programming A Poker Game With GPT Help” →

Next-Gen Autopilot Puts A Robot At The Controls

August 25, 2023 by Bryan Cockfield 19 Comments

While the concept of automotive “autopilots” are still in their infancy, pretty much any aircraft larger than an ultralight will have some mechanism to at least hold a fixed course and altitude. Typically the autopilot system is built into the airplane’s controls, but this new system replaces the pilot themselves in a manner reminiscent of the movie Airplane.

The robot pilot, known as PIBOT, uses both AI and robotics technology to fly the airplane without altering the aircraft. Unlike a normal autopilot system, this one can be fed the aircraft’s manuals in natural language, understand them, and use that information to fly the airplane. That includes operating any of the aircraft’s cockpit controls, not just the control column and pedal assembly. Supposedly, the autopilot can handle everything from takeoff to landing, and operate capably during heavy turbulence.

The Korea Advanced Institute of Science and Technology (KAIST) research team that built the machine hopes that it will pave the way for more advanced autopilot systems, and although this one has only been tested in simulators so far it shows enormous promise, and even has certain capabilities that go far beyond human pilots’ abilities including the ability to remember a much wider variety of charts. The team also hopes to eventually migrate the technology to the land, especially military vehicles, although we’ve seen how challenging that can be already.

Automated Drone Takes Care Of Weeds

August 10, 2023 by Bryan Cockfield 38 Comments

Commercial industrial agriculture is responsible for providing food to the world’s population at an incredibly low cost, especially when compared to most of human history when most or a majority of people would have been involved in agriculture. Now it’s a tiny fraction of humans that need to grow food, while the rest can spend their time in cities and towns largely divorced from needing to produce their own food to survive. But industrial agriculture isn’t without its downsides. Providing inexpensive food to the masses often involves farming practices that are damaging to the environment, whether that’s spreading huge amounts of synthetic, non-renewable fertilizers or blanket spraying crops with pesticides and herbicides. [NathanBuildsDIY] is tackling the latter problem, using an automated drone system to systemically target weeds to reduce his herbicide use.

The specific issue that [NathanBuildsDIY] is faced with is an invasive blackberry that is taking over one of his fields. To take care of this issue, he set up a drone with a camera and image recognition software which can autonomously fly over the field thanks to Ardupilot and a LiDAR system, differentiate the blackberry weeds from other non-harmful plants, and give them a spray of herbicide. Since drones can’t fly indefinitely, he’s also build an automated landing pad complete with a battery swap and recharge station, which allows the drone to fly essentially until it is turned off and uses a minimum of herbicide in the process.

The entire setup, including drone and landing pad, was purchased for less than $2000 and largely open-source, which makes it accessible for even small-scale farmers. A depressing trend in farming is that the tools to make the work profitable are often only attainable for the largest, most corporate of farms. But a system like this is much more feasible for those working on a smaller scale and the automation easily frees up time that the farmer can use for other work. There are other ways of automating farm work besides using drones, though. Take a look at this open-source robotics platform that drives its way around the farm instead of flying.

Thanks to [PuceBaboon] for the tip!

Continue reading “Automated Drone Takes Care Of Weeds” →

AI Learns To Walk In 3D Training Grounds

July 21, 2023 by Lewin Day 13 Comments

AI agents are learning to do all kinds of interesting jobs, even the creative ones that we quite prefer handling ourselves. Nevertheless, technology marches on. Working in this area is YouTuber [AI Warehouse], who has been teaching an AI to walk in a simulated environment.

Albert needed some specific guidance to learn how to walk upright, something that humans tend to figure out innately.

The AI controls a vaguely humanoid-like creature, albeit with a heavily-simplified body and limbs. It “lives” in a 3D environment created in the Unity engine, which provides the necessary physics engine for the work. Meanwhile, the ML-Agents package is used to provide the brain for Albert, the AI charged with learning to walk.

The video steps through a variety of “deep reinforcement learning” tasks. In these, the AI is rewarded for completing goals which are designed to teach it how to walk. Albert is given control of his limbs, and simply charged with reaching a button some distance away on the floor. After many trials, he learns to do the worm, and achieves his goal.

Getting Albert to walk upright took altogether more training. Lumpy ground and walls in between him and his goal were used to up the challenge, as well as encouragements to alternate his use of each foot and to maintain an upright attitude. Over time, he was able to progress through skipping and to something approximating a proper walk cycle.

One may argue that the teaching method required a lot of specific guidance, but it’s still a neat feat to achieve nonetheless. It’s altogether more complex than learning to play Trackmania, we’d say, and that was impressive enough in itself. Video after the break.

Continue reading “AI Learns To Walk In 3D Training Grounds” →

Contrary View: Chatbots Don’t Help Programmers

June 8, 2023 by Al Williams 67 Comments

[Bertrand Meyer] is a decided contrarian in his views on AI and programming. In a recent Communications of the ACM blog post, he reveals that — unlike many others — he thinks AI in its current state isn’t very useful for practical programming. He was responding, in part, to another article from the ACM entitled “The End of Programming,” which, like many other articles, is claiming that, soon, no one will write software the way we do and have done for the last few decades. You can see [Matt Welsh] describe his thoughts on this in the video below. But [Bertrand] disagrees.

As we have also noted, [Bretrand] says:

“AI in its modern form, however, does not generate correct programs: it generates programs inferred from many earlier programs it has seen. These programs look correct but have no guarantee of correctness.”

Continue reading “Contrary View: Chatbots Don’t Help Programmers” →

Prompt Injection: An AI-Targeted Attack

May 19, 2023 by Bryan Cockfield 12 Comments

For a brief window of time in the mid-2010s, a fairly common joke was to send voice commands to Alexa or other assistant devices over video. Late-night hosts and others would purposefully attempt to activate voice assistants like these en masse and get them to do ridiculous things. This isn’t quite as common of a gag anymore and was relatively harmless unless the voice assistant was set up to do something like automatically place Amazon orders, but now that much more powerful AI tools are coming online we’re seeing that joke taken to its logical conclusion: prompt-injection attacks. Continue reading “Prompt Injection: An AI-Targeted Attack” →