Duplex technology for Google Assistant

Google’s Duplex AI Has Conversation Indistinguishable From Human’s

First Google gradually improved its WaveNet text-to-speech neural network to the point where it sounds almost perfectly human. Then they introduced Smart Reply which suggests possible replies to your emails. So it’s no surprise that they’ve announced an enhancement for Google Assistant called Duplex which can have phone conversations for you.

What is surprising is how well it works, as you can hear below. The first is Duplex calling to book an appointment at a hair salon, and the second is it making reservation’s with a restaurant.

Note that this reverses the roles when talking to a computer on the phone. The computer is the customer who calls the business, and the human is on the business side. The goal of the computer is to book a hair appointment or reserve a table at a restaurant. The computer has to know how to carry out a conversation with the human without the human knowing that they’re talking to a computer. It’s for communicating with all those businesses which don’t have online booking systems but instead use human operators on the phone.

Not knowing that they’re talking to a computer, the human will therefore speak as it would with another human, with all the pauses, “hmm”s and “ah”s, speed, leaving words out, and even changing the context in mid-sentence. There’s also the problem of multiple meanings for a phrase. The “four” in “Ok for four” can mean 4 pm or four people.

The component which decides what to say is a recurrent neural network (RNN) trained on many anonymized phone calls. The input is: the audio, the output from Google’s automatic speech recognition (ASR) software, and context such as the conversation’s history and the parameters of the conversation (e.g. book places at a restaurant, for how many, when), and more.

Producing the speech is done using Google’s text-to-speech technologies, Wavenet and Tacotron. “Hmm”s and “ah”s are inserted for a more natural sound. Timing is also taken into account. “Hello?” gets an immediate response. But they introduce latency when responding to more complex questions since replying too soon would sound unnatural.

There are limitations though. If it decides it can’t complete a task then it hands the conversation over to a human operator. Also, Duplex can’t handle a general conversation. Instead, multiple instances are trained on different domains. So this isn’t the singularity which we’ve talked about before. But if you’re tired of talking to computers at businesses, maybe this will provide a little payback by having the computer talk to the business instead.

On a more serious note, would you want to know if the person you were speaking to was in fact a computer? Perhaps Google should preface each conversation with “Hi! This is Google Assistant calling.” And even knowing that, would you want to have a human conversation with a computer, knowing that it’s “um”s were artificial? This may save time for the person whom the call is on behalf of, but the person being called may wish the computer would be a little more computer-like and speak more efficiently. Let us know your thoughts in the comments below. Or just check out the following Google I/O ’18 keynote presentation video where all this was announced.

Continue reading “Google’s Duplex AI Has Conversation Indistinguishable From Human’s”

Build Your Own Android Smartphone

Let’s get this out of the way first – this project isn’t meant to be a replacement for your regular smartphone. Although, at the very least, you can use it as one if you’d like to. But [Shree Kumar]’s Hackaday Prize 2018 entry, the Kite : Open Hardware Android Smartphone aims to be an Open platform for hackers and everyone else, enabling them to dig into the innards of a smartphone and use it as a base platform to build a variety of hardware.

When talking about modular smartphones, Google’s Project Ara and the Phonebloks project immediately spring to mind. Kite is similar in concept. It lets you interface hacker friendly modules and break out boards – for example, sensors or displays – to create your own customized solutions. And since the OS isn’t tied to any particular brand flavor, you can customize and tweak Android to suit specific requirements as well. There are no carrier locks or services to worry about and the bootloader is unlocked.

Hackaday Show-n-Tell in Bangalore

At the core of the project is the KiteBoard – populated with all the elements that are usually stuffed inside a smartphone package – Memory, LTE/3G/2G radios, micro SIM socket, GPS, WiFi, BT, FM, battery charging, accelerometer, compass, gyroscope and a micro SD slot. The first version of  KiteBoard was based around the Snapdragon 410. After some subtle prodding at a gathering of hackers in Bangalore, [Shree] moved over to the light side, and decided to make the KiteBoard V2 Open Source. The new board will feature a Snapdragon 450 processor among many other upgrades. The second PCB in the Kite Project is a display board which interfaces the 5″ touchscreen LCD to the main KiteBoard. Of Hacker interest is the addition of a 1080p HDMI output on this board that lets you hook it up to external monitors easily and also allows access to the MIPI DSI display interface.

Finally, there’s the Expansion Board which provides all the exciting hacking possibilities. It has a Raspberry Pi compatible HAT connector with GPIO’s referenced to 3.3 V (the KiteBoard works at 1.8 V). But the GPIO’s can also be referenced to 5 V instead of 3.3 V if you need to make connections to an Arduino, for example. All of the other phone interfaces are accessible via the expansion board such as the speaker, mic, earpiece, power, volume up / down for hacking convenience. The Expansion board also provides access to all the usual bus interfaces such as SPI, UART, I²C and I²S.

To showcase the capabilities of the Kite project, [Shree] and his team have built a few phone and gadget variants. Build instructions and design files for 3D printing enclosures and other parts have been documented in several of his project logs. A large part of the BoM consists of off-the-shelf components, other than the three Kite board modules. If you have feature requests, the Kite team is looking to hear from you.

When it comes to smartphone design, Quantity is the name of the game. Whether you’re talking to Qualcomm for the Snapdragon’s, or other vendors for memory, radios, displays and other critical items, you need to be toeing their line on MOQ’s. Add to this the need to certify the Kite board for various standards around the world, and one realizes that building such a phone isn’t a technical challenge as much as a financial one. The only way the Kite team could manage to achieve their goal is to drum up support and pledges via a Kickstarter campaign to ensure they have the required numbers to bring this project to fruition. Check them out and show them some love. The Judges of the Hackaday Prize have already shown theirs by picking this project among the 20 from the first round that move to the final round.

Continue reading “Build Your Own Android Smartphone”

Simple Ethereum Vending Machines With NodeMCU

Recently, we covered how to use the Etherscan API to query data (a wallet balance) from the Ethereum blockchain with NodeMCU. It’s a very useful method for retrieving information from a blockchain on embedded systems where storage and memory are an issue.

It has some limitations though. Most notably, it’s polling the API at some interval to retrieve information whether it has changed or not. I would like to be able to receive data more efficiently than this, and quickly enough to make simple vending machines possible. While we’ve seen videos of Bitcoin-based Red Bull vending machines before, they required an NFC card to use.

If we could receive information about Ethereum transactions quickly and reliably enough, we could build a similar vending machine without requiring an NFC card as an intermediary. Simply send to an address via some method, and receive goods!

It turns out we can do exactly that with NodeMCU using WebSocket. Like HTTP, WebSocket is a communications protocol that uses TCP connections (typically over port 80), but it allows full-duplex communication. In other words, you can establish a connection to a server, and send/receive messages without needing to poll the server.

As in the previous example, we’ll use a NodeMCU running Lua. You may wish to refer to it for compile options and information about the screen, which will be the same in this case. Unlike the previous article, you will not need an API key from Etherscan to use this service (not yet, anyway). As usual, we’ll start off by connecting to WiFi:

wifi.setmode(wifi.STATION)
wifi.setphymode(wifi.PHYMODE_B)
station_cfg={}
station_cfg.ssid="Your SSID"
station_cfg.pwd="Your Password"
station_cfg.save=true
wifi.sta.config(station_cfg)

Connecting to a server with WebSockets is easy, but since we’re not using HTTP, we’ll have to remove the https:// and replace that with ws://. (Note: not wss:// because we’ve not enabled encryption yet.)

ws:connect(‘ws://socket.etherscan.io/wshandler’)

Next, we need to report back when the connection is established as the trigger to run additional code. It will return an error code if the connection fails to be established. Handling these error codes in a sensible way is an excellent feature, but we’ll handle that later:

ws:on("connection", function(ws)
    print('got ws connection')
    end)

Now, we need to extend the above to subscribe to an Eth address, and add some new code to do something when a transaction occurs. Note that the API requires that you subscribe to an address within 60 seconds of connecting. It also states that you have to send a ping event to the server every 20 seconds to keep the connection alive, so we’ll need to set a recurring timer for that.

If you’re using ESPlorer, you can send the ping request manually by entering =ws:send('{"event": "ping"}') and pressing Send. This is a useful way to test the connection status.

The address I used seems to have frequent transactions so is reasonable for testing. Be advised though that sitting and waiting for a transaction to happen to test the code creates a slow development cycle so some patience is necessary here.

ws = websocket.createClient()
ws:on("connection", function(ws)
    print('got ws connection')
    ws:send('{"event": "txlist", "address": "0x2a65aca4d5fc5b5c859090a6c34d164135398226"}')
    end)

ws:on("receive", function(_, msg, opcode)
    print('got message:', msg, opcode)
    end)

You should see something like what follows below. The first message is a simple confirmation of connection, the second confirms your subscription to an address, and the third is what you get sent when a transaction occurs. You can subscribe to up to 30 addresses with a single connected device! Note that the data is all in JSON format, which is something we’ll take advantage of later.

got message: {"event":"welcome"} 1
got message: {"event":"subscribe-txlist", "status":"1", "message":"OK, 0x2a65aca4d5fc5b5c859090a6c34d164135398226"} 1
got message: {"event":"txlist","address":"0x2a65aca4d5fc5b5c859090a6c34d164135398226","result":[{"blockNumber":"5532531","timeStamp":"1525098009","hash":"0xe5ec497cb5b38811e8bf5db67a056a2bdd4aa9b68df5c8e8225cb300cbcfa413","nonce":"3363391","blockHash":"0xf446f77d92ed29c221e8451b8048113969ed305a7dd49177e10b422e8e2c4bda","transactionIndex":"172","from":"0x2a65aca4d5fc5b5c859090a6c34d164135398226","to":"0xec5fdfba35c01c6ad7a00085e70e8f30cd121597","value":"24418350000000000","gas":"50000","gasPrice":"4000000000","input":"0x","contractAddress":"","cumulativeGasUsed":"7896403","gasUsed":"21000","confirmations":"1"}]} 1

That’s quite a mess of transaction data, and unfortunately the datum of interest is in the ‘result’ field – which is nested JSON. In the last article, we converted simple JSON to a Lua table using the excellent sjson module. We’ll do the same here after verifying the message type is a transaction (txlist).

ws:on("receive", function(_, msg, opcode)
    print('got message:', msg, opcode)
    ok, ethdata = pcall(sjson.decode, msg)
    if ok then
        msgtype = (ethdata["event"])
        if msgtype == "txlist" then
...

The NodeMCU documentation specifically notes that nested JSON can cause out-of-memory errors. For that reason we use pcall (protected call) to contain any such errors when decoding our JSON message. Next, we extract the contents of the ‘value’ field, nested within the ‘result’ field:

if msgtype == "txlist" then
    wei = ethdata.result[1].value
    print (wei)
    eth = wei/1000000000000000000
    print (eth)
    end

It took me a few hours to figure out how to deal with nested tables, but in the end it was actually quite clean and easy — I was just being dense. Now, we need to add a basic provision to handle errors when the websocket is closed:

ws:on("close", function(_, status)
    print('connection closed', status)
    print('Reconnecting...')
    ws = nil -- required to Lua gc the websocket client
    tmr.alarm(0,4000,tmr.ALARM_SINGLE,transact) -- This reconnects after 4 seconds
end)

To wrap it all up, we encase the code in a couple of functions — first, one to establish a connection, subscribe to the right address, and notify when there is a transaction. Next we need one to display the amount of Eth transferred. Finally, we need a ‘ping’ function to call every 20 seconds or less to keep the connection alive. Overall this turned out to be more robust than expected and has yet to encounter an error. Check out the full code listing here. Note that I’ve also added a little code above to interface with a 128×32 OLED screen, the same one we used previously.

Now that it works, let’s consider im/practical applications. It’s a neat way to display Ethereum transactions in real-time, say if you do livestreaming and accept Eth donations and want them to trigger something fancy. Or, you could make a somewhat insecure vending machine. Clearly, getting a secure WebSocket up and running is the next order of business.

You could also set a timer where the length depends on the amount of Eth received. This would allow for things like public advertisements that go away for a while if someone pays a fee. (Please don’t do this!) Maybe a conference room for rent with the power controlled this way? Hackerspace membership payment? An electric bicycle that charges you for power used?

In any case, it’s not legal to use cryptocurrency as a form of payment in my country so I can’t implement any of the above examples at this time. If you’ve got a better application, please share it in the comments!

VCF East XIII Is Right Around The Corner

The middle of May is a very special time at the Jersey Shore. It’s finally warm enough that business owners decide they might as well unlock their doors, but still cool enough that you can go on the boardwalk without having to bump into sweaty strangers. It’s a time for saltwater taffy without the risk of skin cancer, for hot dogs without seagulls trying to steal them from you. You get the idea.

This year it’s also when the Vintage Computer Festival East will be held in Wall, New Jersey. Running from May 18th to the 20th, VCF East XIII will play host to talks, workshops, and demonstrations focusing on the storied days before we all started carrying supercomputers in our pants. Of course it wouldn’t be a computer festival without vendor tables, and there’s even a consignment area where VCF staffers will sell your old gear while you peruse the show.

Here’s hoping that [Don Eyles] brings his hardcopy of the lunar module source code with him. Our own [Greg Charvat] got a close look at this incredible piece of history.
A trio of keynote speakers will help set the tone of VCF East. On Friday [Bill Dromgoole] will discuss the ongoing restoration of a UNIVAC 1219-B military mainframe. Once designed to handle the radar on a Navy destroyer, thanks to the tireless efforts of [Bill] and his merry band, it’s currently on the hunt for the Wumpus. Saturday will see NASA contractor [Don Eyles] explain how he hacked his way around a stuck “Abort” button on Apollo 14, deftly avoiding the kind of problems the previous guys had in getting to the Moon. Finally on Sunday [Dave Walden] will talk about his work in Ye Olden Days of the Internet: programming the very first routers, known as “Internet Message Processors” or IMPs.

If you’re looking to get up close and personal with tech of yesteryear, you’ll definitely want to check out the exhibit area. Arguably the largest single draw of VCF, this is where you’ll see everything from 1980’s robots to software defined radio on the MITS Altair 8800. In previous years we’ve seen WWII Enigma machines, as well all the cool toys that kids had back in the day.

As usual we’ll be there to take plenty of pictures and dive into the sundry amusements offered by the show and the InfoAge Museum which plays host to it. Hackaday is once again a proud sponsor of VCF East and their continuing efforts to preserve technology history for future generations. We’d love to see you there and we’re always looking for the inside scoop on interesting hardware so don’t be shy about tracking me down with your story!

Microsoft Kinect Episode IV: A New Hope

The history of Microsoft Kinect has been of a technological marvel in search of the perfect market niche. Coming out of Microsoft’s Build 2018 developer conference, we learn Kinect is making another run. This time it’s taking on the Internet of Things mantle as Project Kinect for Azure.

Kinect was revolutionary in making a quality depth camera system available at a consumer price point. The first and second generation Kinect were peripherals for Microsoft’s Xbox gaming consoles. They wowed the world with possibilities and, thanks in large part to an open source driver bounty spearheaded by Adafruit, Kinect found an appreciative audience in robotics, interactive art, and other hacking communities. Sadly its novelty never translated to great success in its core gaming market and Kinect as a gaming peripheral was eventually discontinued.

For its third-generation, Kinect retreated from gaming and found a role in Microsoft’s HoloLens AR headset running “backwards”: tracking user’s environment instead of user’s movement. The high cost of a HoloLens put it out of reach of most people, but as a head-mounted battery-powered device, it pushed Kinect technology to shrink in physical size and power consumption.

This upcoming fourth generation takes advantage of that evolution and the launch picture is worth a thousand words all on its own: instead of a slick end-user commercial product, we see a populated PCB awaiting integration. The quoted power draw of 225-950mW is high by modern battery-powered device standards but undeniably a huge reduction from previous generations’ household AC power requirement.

Microsoft’s announcement heavily emphasized how this module will work with their cloud services, but we hope it can be persuaded to run independently from Microsoft’s cloud just as its predecessors could run independent of game consoles. This will be a big factor for adoption by our community, second only to the obvious consideration of price.

[via Engadget]

Modular Robotics: When You Want More Robots In Your Robot

While robots have been making our lives easier and our assembly lines more efficient for over half a century now, we haven’t quite cracked a Jetsons-like general purpose robot yet. Sure, Boston Dynamics and MIT have some humanoid robots that are fun to kick and knock over, but they’re far from building a world-ending Terminator automaton.

But not every robot needs to be human-shaped in order to be general purpose. Some of the more interesting designs being researched are modular robots. It’s an approach to robotics which uses smaller units that can combine into assemblies that accomplish a given task.

We’ve been immersing ourselves in topics like this one because right now the Robotics Module Challenge is the current focus of the Hackaday Prize. We’re looking for any modular designs that make it easier to build robots — motor drivers, sensor arrays, limb designs — your imagination is the limit. But self contained robot modules that themselves make up larger robots is a fascinating field that definitely fits in with this challenge. Join me for a look at where modular robots are now, and where we’d like to see them going.

Continue reading “Modular Robotics: When You Want More Robots In Your Robot”

1960’s Console Stereo Gets Raspberry Pi Touch Screen

When he was but a wee hacker, [WhiskeyDrinker] loved to play with the big console stereo his grandparents had. The idea of a functional piece of furniture always appealed to him, and he decided that when he grew up and had a place of his own he’d get a similar stereo. Fast forward to the present, and a Craigslist ad for a working Penncrest stereo seemed to be a dream come true. Until it wasn’t.

The original physical controls are connected to the Pi’s GPIO

As difficult as it might be to believe, sometimes things we read on the Internet are not true. The “working” Penncrest radio turned out to be a dud. But realizing that the look of the cabinet was more important to him than historical accuracy, [WhiskeyDrinker] decided to outfit it with a Raspberry Pi powered touch screen that would look as close to stock hardware as possible.

The final result really does look like some kind of alternate timeline piece of consumer electronics: where chunky physical buttons and touch screens coexisted in perfect harmony. The vintage stereo aficionados will probably cry foul, but let them. [WhiskeyDrinker] did a fantastic job of blending old and new, being respectful to the original hardware and aesthetic where it made sense, and clearing house where only nostalgia had lease.

A HiFiBerry DAC+ Pro is used to get some decent audio out of the Raspberry Pi, and the touch screen interface is provided by Volumio. [WhiskeyDrinker] mentions that it even has a GPIO plugin which he successfully used to handle getting the physical buttons to play nice with their digital counterparts.

Updating old audio gear is always a sensitive subject around these parts. Sometimes they go so far that the original hardware is almost an afterthought. On the other side of the spectrum are the projects which try to take modern gear and mimic the look of the classics. In any event, one thing is clear: they don’t make ’em like they used to.

Continue reading “1960’s Console Stereo Gets Raspberry Pi Touch Screen”