Pitching a baseball is about accuracy and speed. A swift ball on target is the goal, allowing the pitcher to strike out the batter. [Nick Bild] created an AI system that can determine a ball’s trajectory in mid-flight, based on a camera feed.
The system uses an NVIDIA Jetson AGX Xavier, fitted with a USB camera running at 100FPS. A Nerf tennis ball launcher is used to fire a ball towards the batter. Once triggered, the AI uses the camera to capture two successive images of the ball in flight. These images are fed into a convolutional neural network (CNN), and the software determines whether the ball is heading for the strike zone, or moving off-target. It uses this information to light a green or red LED respectively to alert the batter.
While such a system is unlikely to appear in professional baseball anytime soon, it shows the sheer capability of neural network systems to quickly and effectively analyse data in ways simply impossible for mere humans. [Nick]’s future goals involve running the system on faster hardware, and expanding it to determine effects like spin and more accurate positioning within the strike zone.
We’ve seen CNNs do everything from naming tomatoes to finding parking spaces. Video after the break.
There is problem with your Nerf tennis ball launcher not working properly
Do you really need a NN for this? Isn’t this something that Physics can solve?
Also, from the article, 2 pictures are compared, so it is a linear interpolation, I assume real human throwers can give the ball a spin and make it follow a curve where linear interpolation may not be good enough.
I bet that it would even be more accurate w just simple kinematics. Camera takes 2 photos let’s say 1/10 sec apart. Change in position is extrapolated and then velocity calculated. You could even accuratly estimate where it’s going to hit.
Don’t know what nividia Jetsons run for currently but you could do this for < $50
Baseball already uses a neural net and deep learning to solve these problems in real time during games.
Its called a ‘batter’.
Awesome, the AI equivalent of using an arduino to blink an LED, considering that pre-WWII electronic analog computers (Artillery computers) and early digital computers were used on this sort of thing.
That’s a strange comparison. If you know all of the parameters going into firing a projectile, as well as environmental parameters, sure you can determine where it will go. How does that translate to baseball when you know none of these parameters?
How does this indoor nerf gun model translate to baseball?
Go outside, replace the ball launcher with a pitcher, replace the tennis ball with a baseball.
I was trying to point out that the accuracy would not be what you expect. Yes neural networks can be trained to approximate functions. But you need a lot of data. As you mentioned there are a lot parameters involved if we were to create a mathematical model. You can not simply wish away that complexity. Your indoor model does not know about wind just as an example.
That’s the beauty of ML. I don’t need to know about any of those factors, and I don’t need to change a line of code to support them. Of course you need to train the model with data appropriate for your use case, that’s not surprising. I’ve built a proof of concept device — but you have to BYOD (Bring Your Own Data). :)
Glad you understand.
The ball positions are just dots on the tickertape of your newtonian physics 101 labs. This statement is false or imcomplete “effectively analyse data in ways simply impossible for mere humans.” only need the first few frames of the ball out of the gun and slog away with your s=ut+1/2at^2 stuff, taking wayyy too long of course, but you can mechanise that very easily, it’s mechanical not AI intensive.
I think you’d have a difficult time finding all of the parameters (and the interactions between them) that influence the final destination of the ball. Not to mention actually extracting those features from images, then doing the calculations and having a result within a few 10s of milliseconds.
Machine learning isn’t magic, it’s determining which features matter and tuning a mathematical function. So I suppose with enough effort, the same function could be determined in some other way. But is that other way better/easier/faster? If so, go with it. If not, why choose a worse tool for the job?
I think ML is the right tool for this job. It works, it was fast to develop, and the processing is fast enough to be useful for the task. There’s always another way, but is it better? Or even adequate? I’m not quite sure what you’re proposing, but I suspect the game may be over before we have a prediction about where the opening pitch will end up.
As you say, there are much simpler ways you could do this, but as far as I know those early artillery computers were only used to determine where you *should* fire – they had no way to figure out where your shot actually went (though they could calculate corrections for future shots if you told them).
Being able to determine that your shot is going to miss (before it even reaches the target!) is still pretty cool in my book.
“…considering that pre-WWII electronic analog computers (Artillery computers) and early digital computers were used on this sort of thing.”
They were used to compile “artillery tables” – paper workbooks for artillery crews to estimate where the shells might go under ideal circumstances given a large range of variables. They required a nation’s resources, whole buildings and many personnel taking months to years of time to finish. This was considered a huge improvement over trial-and-error or simple parabolic projections which were used for this sort of thing previously.
This article’s setup projects a trajectory in real time, in a living room, with one person’s efforts and didn’t require a war bond issue to build.
I’d say that’s a significant improvement.
Nope those existed from late 19th C calculating engines forward, but these worked real time, and were integrated into first radar fire control systems in WWII that detected track of aircraft and predicted where they ought to be when the shot got there.
Impressed that you don’t have any latency issues using a USB camera.
I run the camera captures in a separate thread, so there are always images waiting to be processed, and I’m using a 100FPS camera.
Good to know. I’m about to use a Logitech Brio with the Jetson Nano, however my main camera will be a CSI/MIPI 2-lane job to hopefully keep latency right down.
CSI is definitely the way to go; that’s what I use with my Nano. The Xavier I used here is far more powerful, but the camera options are fairly expensive — hence the USB cam. Not the ideal choice, but more budget-friendly for a hobbyist.
Appreciate the reply. That’s really good to know. Just a shame the development board for the Nano is so crippled regarding camera I/O.
I’m guessing this is more of a starter project for you since the Xavier is so powerful compared to the Nano and your baseball project would probably run okay on the Nano (I guess).