We love the intersection between art and technology, and a video made by an AI (Stable Diffusion) imagining a journey through time (Nitter) is a lovely example. The project is relatively straightforward, but as with most art projects, there were endless hours of [Xander Steenbrugge] tweaking and playing with different parts of the process until it was just how he liked it. He mentions trying thousands of different prompts and seeds — an example of one of the prompts is “a small tribal village with huts.” In the video, each prompt got 72 frames, slowly increasing in strength and then decreasing as the following prompt came along.
There are other AI videos on YouTube, often putting the lyrics of a song into AI-generated form. But if you’ve worked with AI systems, you’ll notice that the background stays remarkably stable in [Xander]’s video as it goes through dozens of feedback loops. This is difficult to do as you want to change the image’s content without changing the look. So he had to write a decent amount of code to try and maintain visual temporal cohesion over time. Hopefully, we’ll see an open-source version of some of his improvements, as he mentioned on Twitter.
In the meantime, we get to sit back and enjoy something beautiful. If you still aren’t convinced that Stable Diffusion isn’t a big deal, perhaps we can do a little more to persuade your viewpoint.
As anyone who has taken care of chickens or other poultry before will tell you, it can be backbreaking work. So why not build a robot to do all the hard work for us? That’s precisely what [Aktar Kutluhan] demonstrated with an AI-powered IoT system that automatically feeds chicks and monitors unhatched eggs.
Make no mistake, hens are adorable, feathered creatures, but they can be quite finicky. An egg’s weight, size, and frequency can determine the overall health of a hen, and they can stop laying eggs altogether if something as simple as their feeding schedule is too sporadic. This is precisely what inspired [Aktar] to create a system that can feed hens at a consistent time every day while keeping track of the eggs laid to ensure the coop is happy and healthy.
What’s so impressive about this build isn’t just the clever automation that scratches off a daily chore, it’s built completely with IoT devices, including the AI. The setup uses Edge Impulse as an object detection model on an OPenMV Cam H7 microcontroller to recognize eggs in the coop. From there, an WizFi360-EVB-Pico board was attached so data could be sent over WiFi, with a DHT22 thrown in to monitor and record the overall temperature of the coop.
This is already an amazing setup, but when it comes to IoT devices, the sky’s the limit. You could control heat lamps in larger coops, automatically refill a water bowl if the hens’ water is low, or even build a hands-off incubator. We’re only just beginning to see the clever ways with which AI can help monitor our pet’s health. Just look at how another hacker used AI to monitor cat poop to make sure their furry friend wasn’t eating plastic. Thanks to [Aktar Kutluhan] for showing us more ways we can use AI to help our pets!
What do SQL injection attacks have in common with the nuances of GPT-3 prompting? More than one might think, it turns out.
Many security exploits hinge on getting user-supplied data incorrectly treated as instruction. With that in mind, read on to see [Simon Willison] explain how GPT-3 — a natural-language AI — can be made to act incorrectly via what he’s calling prompt injection attacks.
This all started with a fascinating tweet from [Riley Goodside] demonstrating the ability to exploit GPT-3 prompts with malicious instructions that order the model to behave differently than one would expect.
This modern era of GPU-accelerated AI applications have their benefits. Pulling useful information out of mountains of raw data, alerting users to driving hazards, or just keeping an eye on bee populations are all helpful. Lately there has been a rise in attempts at producing (or should that be curating?) works of art out of carefully sculpted inputs.
One such AI art project is midjourney, which can be played with via a Discord integration bot. That bot takes some textual input, then “dreams” with it, producing sometime uncanny, often downright disturbing images.
[Vuong Nguyen] clearly knows his way around artificial intelligence accelerator hardware, creating ztachip: an open source implementation of an accelerator platform for AI and traditional image processing workloads. Ztachip (pronounced “zeta-chip”) contains an array of custom processors, and is not tied to one particular architecture. Ztachip implements a new tensor programming paradigm that [Vuong] has created, which can accelerate TensorFlow tasks, but is not limited to that. In fact it can process TensorFlow in parallel with non-AI tasks, as the video below shows.
A RISC-V core, based on the VexRiscV design, is used as the host processor handling the distribution of the application. VexRiscV itself is quite interesting. Written in SpinalHDL (a Scala variant), it’s super configurable, producing a Verilog core, ready to drop into the design.
A Digilent Arty-A7, Arducam and a VGA PMOD is all you need
From a hardware design perspective the RISC-V core hooks up to an AXI crossbar, with all the AXI-lite busses muxed as is usual for the AMBA AXI ecosystem. The Ztachip core as well as a DDR3 controller are also connected, together with a camera interface and VGA video.
Other than providing an FPGA-specific DDR3 controller and AXI crossbar IP, the rest of the design is generic RTL. This is good news. The demo below deploys onto an Artix-7 based Digilent (Arty-A7) with a VGA PMOD module, but little else needed. Pre-build Xilinx IP is provided, but targeting a different FPGA shouldn’t be a huge task for the experienced FPGA ninja.
Ztachip top level architecture
The magic happens in the Ztachip core, which is mostly an array of Pcores. Each Pcore has both vector and scalar processing capability, making it super flexible. The Tensor Engine (internally this is the ‘dataplane processor’) is in charge here, sending instructions from the RISC-V core into the Pcore array together with image data, as well as streaming video data out. That camera is only a 0.3 MP Arducam, and the video is VGA resolution, but give it a bigger FPGA and those limits could be raised.
This domain-specific approach uses a highly modified C-like language (with a custom compiler) to describe the application that is to be distributed across the accelerator array. We couldn’t find any documentation on this, but there are a few example algorithms.
The demo video shows a real-time mix of four algorithms running in parallel; one object classification (Google’s Tensorflow mobilenet-ssd, a pre-trained AI model) canny edge detection, a Harris corner detection, and Optical flow which gives it a predator-like motion vision.
[Vuong] reckons, efficiency wise it is 5.5x more computationally efficient than a Jetson Nano and 37x more than Google’s TPU edge. These are bold claims, to say the least, but who are we to argue with a clearly incredibly talented engineer?
You might not have heard about Stable Diffusion. As of writing this article, it’s less than a few weeks old. Perhaps you’ve heard about it and some of the hubbub around it. It is an AI model that can generate images based on a text prompt or an input image. Why is it important, how do you use it, and why should you care?
This year we have seen several image generation AIs such as Dall-e 2, Imagen, and even Craiyon. Nvidia’s Canvas AI allows someone to create a crude image with various colors representing different elements, such as mountains or water. Canvas can transform it into a beautiful landscape. What makes Stable Diffusion special? For starters, it is open source under the Creative ML OpenRAIL-M license, which is relatively permissive. Additionally, you can run Stable Diffusion (SD) on your computer rather than via the cloud, accessed by a website or API. They recommend a 3xxx series NVIDIA GPU with at least 6GB of RAM to get decent results. But due to its open-source nature, patches and tweaks enable it to be CPU only, AMD powered, or even Mac friendly.
This touches on the more important thing about SD. The community and energy around it. There are dozens of repos with different features, web UIs, and optimizations. People are training new models or fine-tuning models to generate different styles of content better. There are plugins to Photoshop and Krita. Other models are incorporated into the flow, such as image upscaling or face correction. The speed at which this has come into existence is dizzying. Right now, it’s a bit of the wild west. Continue reading “Stable Diffusion And Why It Matters”→
[Estefannie] is a proud cat owner, but one of her cats has a bad habit of eating plastic. That means she needs to keep an eye on that cat’s bowel movements, but with two cats in the house, it’s difficult to know who did what. Thus, she whipped up an AI system to log her cats bathroom visits and give her peace of mind.
It’s not the most glamorous project — [Estefannie] notes she took over 50,000 pictures of her cats using the litterbox to train Microsoft Azure’s Custom Vision model. But after some work, it could readily identify which cat was using the litter box when fed images from a NoIR camera. The system then differentiates between number 1 and number 2 via the time the cat spends in the litter box. It’s not perfect, but it works.
The Raspberry Pi runs a Node.JS server to collate the results, paired with a website front-end for easy data display. That way, anyone on [Estefannie’s] WiFi network can see who did what from a browser. We’ve seen cat litter boxes put on the Internet of Things before, and we’ve even seen people hack litterbox DRM, too.
