Augmented Reality with an FPGA

 

bruceinabox

 

[Julie Wang] has created an augmented reality system on a Field Programmable Gate Array (FPGA). Augmented reality is nothing new – heck, these days even your tablet can do it. [Julie] has taken a slightly different approach though. She’s not using a processor at all. Her entire system, from capture, to image processing, to VGA signal output, is all instantiated in a FPGA.

Using the system is as simple as holding up a green square of cardboard. Viewing the world through an old camcorder, [Julie's] project detects and tracks the green square. It then adds a 3D image of Cornell’s McGraw Tower on top of the green. The tower moves with the cardboard, appearing to be there. [Julie] injected a bit of humor into the project through the option of substituting the tower for an image of her professor, [Bruce Land].

[Julie] started with an NTSC video signal. The video is captured by a DE2-115 board with an Altera Cyclone IV FPGA. Once the signal was inside the FPGA, [Julie's] code performs a median filter. A color detector finds an area of green pixels which are passed to a corner follower and corner median filter. The tower or Bruce images are loaded from ROM and overlaid on the video stream, which is then output via VGA.

The amazing part is that there is no microprocessor involved in any of the processing. Logic and state machines control the show. Great work [Julie], we hope [Bruce] gives you an A!

Comments

  1. echodelta says:

    What would happen if you went outside into the green world? Crazy!
    Chroma key picks the colour least likely to be in the unnatural world.
    FPGA, the closest thing yet on the brain.

    • Philidactyl says:

      Could chroma key be improved, by using a something switches between two states in short intervals to identify itself so that it may be used between things that even include it’s own colour? I’m thinking of a solution based on a LCD panel that switches between black and white or maybe some sort of IR beacon solution, any ideas?

    • Jon says:

      QR code or QR code style patterns seems to be a popular solution for augmented reality.
      I am not sure how difficult it will be, it might even be easier.
      But this is excellent as a prove of concept and senior project.

  2. ian says:

    Really cool! And she is very cute too :)

    • MoJo says:

      It’s no wonder women are put off this site and engineering in general when the 2nd or 3rd comment is always from some perv.

    • Skeptical says:

      As a woman I just want to point out that the reason why we get irked when people are passing a complement, is because those comments don’t exist in a vacuum.

      When it turns into a pattern of people commenting on your appearance, it just reinforces the idea that we’re judged to varying degrees on our appearance.

      I say this because some men seem genuinely confused when women react negatively to what men think are harmless comments. It’s a “death by a thousand paper cuts” sort of situation.

      I guess it’s also kind of weird? I don’t feel this urge to comment on the appearance of men as often as some men do.

      Hackaday has a serious problem if I can look at the opening of an article and make solid predictions about what I’m going to find in the comments.

  3. koswix says:

    Hacking and making is such a wonderfully open community, where ideas and knowledge are shared without prejudice, for the simple joys of learning and understanding. It’s really a remarkable community, and often represents the very best of what the human race is capable of.

    And then there’s your comment. Pathetic, narrow minded racism. Real bottom of the barrel stuff. This has no place in any community, especially one devoted to sharing ideas.

    If you were trying to be funny, you not only failed but also managed to be offensive and repulsive. If this is a genuinely held belief, then please feel free to FOAD. Your views are not welcome in the hacker community.

  4. Urop says:

    Interesting, I learned a new word today.

    As a European, I am always surprised how racist the US is. I did not learn about the word “wetback” until I went out with a mexican born professor in the US, who was heckled from across the street.

  5. BBeyes says:

    Errrrrm, where was the racist comment?

  6. Adobe/Flash hater says:

    I always try to at least skim this site, no matter how tired or rushed I may be.
    I saw and wondered if that remark was a sly use of wording related to the article to get in a ugly thought
    Or was i just missing some “tech jargon”
    As one of the proponents of Hackadays light handed moderation
    I don’t envy the staffs task of choosing when it’s time to remove
    one of our missives.
    may I say that I do wish there was a practical way to leave something of a placeholder to make it easier to follow the flow of what happened
    in the comments section.
    Perhaps a plain block with the routine “comment removed”
    and the offending name intact for the rest of us to see who
    was trying to ruin things for everyone else?
    It’s almost funny how I was wondering if we’d see some sophomoric
    comments about the camera angle…..
    ( a fellows never to old for hormones ,ehh?)
    But instead it goes this way.

    Damn…..
    – – –
    p.s. I hope the “Shatnerizer” option hasn’t mangled my post once again.

  7. eldphm says:

    Awesome project, at first I was thinking: Why would someone want to do that with a FPGA?, as it is much easier to do with a microprocessor.
    But then I realized that it was BECAUSE it is difficult she made this choice.
    The easy path seldom leads to greatness.
    If Julie doesn’t already has a job waiting for her (she is about to graduate soon if I understand things correctly), I do not think she has to worry about that for very long.

  8. Farbro says:

    That’s amazing! I knew only a tiny bit about FPGAs and had no idea you could do this sort of thing with them.

  9. michal says:

    A microprocessor is also logic and state machines, however already connected. It is possible to put microprocessor’s net into FPGA creating a very expensive variant of the processor (FPGA prices are much higher than simple microprocessors/micro controllers — actually this is the reason we have them — they are much more economical to use). Therefore, I miss a little bit the purpose of this achievement. Looks like a student exercise, luckily the FPGA can be easily reprogrammed to do the stuff that general purpose microprocessor can’t.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 91,872 other followers