[Jamie] aka [vector76] hit us with a line-tracing plugin for OctoPrint that cuts out whatever 2D shape you draw on a piece of wood. The plugin lets you skip the modeling step entirely, going straight from a CNC-mounted webcam that reads your scribbles and gives you a Gcode toolpath in return. The code is on GitHub and there’s a demo video embedded below.
Under the hood, OpenCV is doing a lot of the image processing, including line detection, and the iterative “find the line” and “move the toolhead” steps really show off what computer vision can do. It starts off with a fiducial arrow for scale and orientation, then it mores the webcam around the scene. The user can enter the usual milling parameters: speeds, feeds, depth of cut, tool offset, milling direction, etc. And then it gets to work.
Right now, it’s limited to paths with non-crossing lines, and probably with good contrast and a nice dark line — all the usual CV restrictions. But mounting a webcam to a CNC toolhead and using it for various pathing problems really opens up tons of possibilities: visual homing, workpiece edge finding, copying parts, custom fitting odd shapes, and more. This project is clearly an invitation to keep on hacking, an appetizer. Once you see the girl pirate robot that [Jamie]’s daughter made, you’ll get the idea.
We’ve seen a similar OpenCV approach used for center-finding bore holes, but while we’ve seen a few webcams used with laser cutters, the CNC mill applications seem largely untapped. Let us know in the comments if you’ve got some other good examples.
Thanks [Jeff] for the tip!
LinuxCNC has supported webcams for quite some time, but there is nothing like this level of integration.
https://forum.linuxcnc.org/qtvcp/40032-qtvcp-cam-view-dialog
However, I saw a demo of a Datron mill on YouTube that seems to integrate a camera really rather neatly.
https://youtu.be/-nT_o4icilA
Though it isn’t necessariy using computer vision.
That Datron interface is rad. “Here’s the corner, sample at these points” is pretty awesome.
I’m not going to look at the price tag.
Cool!
Computer vision is just a means to an end. Even if the Datron tool were to automatically extract a 3D model, it’s still up to the human to judge what features to probe and how, so not a big deal to make the human do the vision part, and adds robustness to how the part appears. A neat solution!
Not quite as exciting as openCV, to some extent this has happened for a long time… actually getting the gcode out of the machine is pretty useful. Scaling and modifying without redrawing the entire drawing.
https://www.youtube.com/watch?v=6SfjdU06Wwo
I think I will probably add an option to extract an SVG instead of gcode, for scaling or modifying digitally. And also I don’t think I’ll ever be able to support all the options that a full digital workflow or CAM package can offer, like holding tabs, finishing passes, pockets, or cutting out ten copies.
Thanks for sharing the video. It’s not surprising that some degree of optical input has existed for some time, and I would not claim to be first. What I find more surprising is that, given the current technology that’s easily accessible, it seems not to be very common. Maybe that will change.
.SVG output would be really useful. I can see how students could draw by hand, then modify on a computer (or depending age/skill, have the teacher do it) then cut by CNC and assemble/finish/paint.
Really nice work, I’ve bookmarked this one :-)
Organising the arucoboard markers into an arrow shape is a really cute trick; there’s no ambiguity about what orientation it’s encoding.
Love it! Good work with getting it to see the lines! Love the charuco markers with openCV, tons of potential.
I think taking what you did and putting into a g code sender (gSender maybe) would be amazing.
Here is another aspect of table top CNC that I am hoping can get into sender too (Generating a cut preview with a cell phone picture of cnc bed).
https://www.youtube.com/watch?v=Ghqjpu9la84&t=31s
MDF dust flung at that camera lens at 30mph will make for a fairly short lifespan. Might want to incorporate the camera into some kind of a dust shoe.
Love it! Good work with getting it to see the lines! Love the charuco markers with openCV, tons of potential.
I think taking what you did and putting into a g code sender (gSender maybe) would be amazing.
Here is another aspect of table top CNC that I am hoping can get into sender too (Generating a cut preview with a cell phone picture of cnc bed).
https://www.youtube.com/watch?v=Ghqjpu9la84&t=31s
Hi Tony, I think what you are showing has a lot of potential. In some ways your application is entirely different, but I feel like it is in a similar spirit. There are a lot of applications where imaging and computer vision can add a lot of value without particularly advanced technology. The techniques that you and I are using are largely off-the-shelf existing techniques, but in the domain of CNC it is relatively uncommon. There is a lot of bang-for-the-buck to be had, where low-cost hardware can open whole worlds of functionality.
The line follower is an OctoPrint plugin, and OctoPrint is already a gcode sender of sorts. The line follower operates within the gcode sender, so the movements work through the gcode sender, and not as a separate application. This integration was the hardest part of the line follower, largely because I am still a novice at understanding how these plugins work.
For integrating a camera feed into the toolpath visualizer, I would think that should be doable, and definitely valuable for all the reasons you pointed out. Adjusting a segment of the toolpath should be possible in principle, but that is pretty advanced and that would be pretty challenging.
I am contemplating a dual-camera setup, with one wide-field that can capture the entire workspace with not too many pictures, and another narrow-field, high resolution camera, for more precise locations. Tracing multiple contours becomes possible, by marking each with a tag, whereas with only one narrow-field camera, it’s not really feasible to scan the whole workspace. But multiple contours is not even the most useful application. Rendering the toolpath onto the workpiece has everyday utility, and there are tons of other possibilities as well.
Love it! Good work with getting it to see the lines! Love the charuco markers with openCV, tons of potential.
I think taking what you did and putting into a g code sender (gSender maybe) would be amazing.
Here is another aspect of table top CNC that I am hoping can get into sender too…uses charcoal markers too (Generating a cut preview with a cell phone picture of cnc bed).
https://www.youtube.com/watch?v=Ghqjpu9la84&t=31s
Seems like these ideas are becoming hot and agree, this has been possible for some time and neat to see movement in this area.
Wow, that’s awesome. Makes me want to build a new CNC.