[Konstantinos] wrote in to tell us about his CDW project: a digital encoding scheme for ham radio that uses CW (continuous wave) Morse code for digital data transfer. [Link updated 1/5/16] CW operation with Morse code is great for narrow-bandwidth low-speed communication over long distances. To take advantage of this, [Konstantinos] developed a program that takes binary or text files, compresses them, and translates them to a series of letters and numbers that can be represented with Morse code.
The software translates the characters into sequences of Morse code pulses, and plays an audio stream of the result. His software doesn’t support decoding Morse from an audio stream, so [Konstantinos] recommends using one of many existing programs to get the job done. Alternatively those with a good ear and working knowledge of Morse can transcribe the characters by hand.
After receiving a broadcast, the user pastes received characters back in the software. The software re-assembles the binary file from the Morse characters and decompresses the result. [Konstantinos] also added a simple XOR encryption feature, but keep in mind that using encryption on ham radio bands is technically illegal.
What is the data transfer rate? Is it at normal Morse speed?
There are of course far better encoding schemes than Morse for non-text information, and transmission of data using CW is far from new, but as a project it’s nicely done and demonstrates that it’s not difficult to write software to do this sort of thing.
Quote: “CW is far from new”.
If my memory serves me correctly carrier wave (CW) was the very first form of RF transmission so I recon you have got your bases covered there.
That’s nice, but I was told that ruggedness and reliability of Morse Code keyed CW transmission comes from natural noise-suppression and pattern matching abilities of human brain and hearing. If that’s true, then if there is no human in the signal loop, perhaps some clever coding scheme and DSP magic would be better suited for data transfer task
Encryption for purposes of concealment is illegal, but the well known WiFI uses encryption as a normal means of data transfer. Wifi encryption is allowed.
Comment is bullshit without including the jurisdiction.
When using Ham Radio stations, encryption is illegal. WiFi is a whole different category and class of signal. So HAD was right when they say that the XOR encryption is technically illegal, and Jeff is right when he says WiFi can transmit encrypted. This is true anywhere the American law touches, regardless of jurisdiction (ie Army bases, territories, districts, etc).
As long as you follow all the consumer Part 98 rules when transmitting, Wifi encryption is fine. If you are transmitting under amateur radio rules (Part 97) with higher ERP or modified transmitters then encryption is illegal.
Part 15, not Part 98.
@NorseEngineering
Not all jurisdictions are American. That was my whole point.
If you are talking about American rules, please give us the Part 97 rule number that says this. If you feel it is legal in America, you are misinformed.
I’m assuming US laws simply because that is where I am. I know nothing about laws elsewhere.
“Encryption for purposes of concealment is illegal” –
Yes that is what the law actually says as written. I think this is a valid distinction to make. I’ve long thought that this means it should be ok to use public/private key encryption in ham radio to prove identy. This could be useful for things like repeater control and other R/C applications. I would think that actually sending the public key every so that anyone can decode your message even though they can’t actually send a message themselves would be necessary. Since the law doesn’t really say anything about this I’m thinking every 10 minutes (since that’s how often you have to ID) makes sense.
I’ve never heard of anyone actually using encryption this way in ham radio though though. I’d be a bit nervous about trying it before the FCC clarifies that this interpretation is correct. If nothing else it would be a pain to have a bunch self-apointed “ham police” complaining all the time even if the FCC does reply with ‘technically that’s ok’ in the end.
“WiFI uses encryption as a normal means of data transfer. Wifi encryption is allowed.”
Ummm, where did you get that idea? Encryption in wifi is optional. On any wifi device I have ever seen it can be turned off. Yes, hams do use Wifi. The Wifi bands are also ham bands and as long as we ID we get to use greater power and more channels. I’ve never heard of hams doing this with the encryption turned on. I’m sure that is illegal here in the US and I really doubt it is legal in many other countries.
People could argue all day about wether encryption SHOULD be legal. I’ve seen some people rant here before thinking it is some great affront to free speech that it is not. Think about this though, if you could do encryption you could set up an over-the air vpn back to your home. That would be awesome for you! It would be like having free unlimitted cellular data.
But… so could everyone else. It would be awesome for them too. And… ham bands only have so much spectrum available. How many such users could it support before there is no room for anything else? How many users before there is so much interference that nobody gets a connection? That’s why we need this law.
Some people also argue that encryption should be allowed for specific limited uses. But.. how would that be enforced? If it’s encrypted how would you know if it’s being abused or not?
I’ve long thought that this means it should be ok to use public/private key encryption in ham radio to prove identy.
You can utilize digital signatures to prove identity without encrypting the payload.
Yes. I did some research into this when trying to work out if I could transmit Bitcoin transactions under my license. The general answer seemed to be “yes”, once people understood that all I was talking about was machine code and cryptographic signatures. You could be using the digital signatures as a cover for encrypted data though, and people wouldn’t be able to prove you wrong if that was the case. It’s a mildly interesting problem with a real world-chance of being a problem equivalent to zero.
It’s only “legal” on ham bands for satellite command&control comms (in US). Which is good as otherwise all bands will be all garbage. Any encoding scheme needs to be public too.
Actually, HF communications in the Ham bands pretty much cannot be legally encrypted anywhere in the world, by ITU regulations. As for encoding, what specifically are you referring to? Most every modulation scheme, even the proprietary versions of Pactor, are publicly documented somewhere; what isn’t legal is to use the information that is available publicly to write software to decode the proprietary modulation and redistribute it.
Yeah Ham laws are pretty similar, except it’s practically impossible to enforce laws over all spectrum 24/7. So honestly if you use a rare band (if you must use ham bands) and do it quick, and by quick I mean no more than 10 seconds tops (plenty of time for a decent-sized message), I personally don’t care if you tell your baby mama how much you love her or that cocaine shipment you got coming in.
By encoding, I mean what you’re getting at, modes and methods of transmitting the data; protocols, etc. Again, if you invented some obscure method of TX/RX no one else can even receive, few to no one is going to hunt you down. Likewise w/ “encrypting” by using essentially OTP’s to prescribe completely different meaning to normal terms; probably won’t be found.
These days you can set up a computer radio station remotely, so theoretically you could VPN into a station on the other side of the world via a wifi network you hacked and a liveCD w/ another VM; you could even leave it completely open to be hacked to sh*t and give deniability.
What does WiFi have to do with anything? WiFi doesn’t operate on the Ham bands and isn’t regulated by part 97. Encryption is not allowed on the Ham bands, with the exception of satellite control. You are allowed to use encoding schemes as long as they are publicly known (i.e., you can transmit a binary message using error correcting codes).
channels 1-8 of the 802.11B/G standard fall completely within the amatuer radio band in the United States when operated with a 20MHz bandwith (standard). More channels do at the narrower 10 or 5MHz bandwiths. Additionally, with some firmware modifications, many will also operate at negative channel numbers that are ONLY in the ham band, not the ISM band.
My mistake, you’re right, WiFi does share frequencies with Ham radio. It operates under part 15 rather than 97, which is why encryption is allowed.
He was referring to amateur radio regulations. Encryption is now allowed on amateur radio bands. Encryption is allowed on all other radio bands, such as the ISM bands, or on privately licensed bands.
You can’t call “xor-with-something” encryption. (that is when it fits the “simple xor” phrase).
Xor is the key of most encryption. The one-time-pad is the most practicable (but still not practicable) cypher that has perfect forward secrecy and it is “simply xoring” with a truly random one time key. And Stream cypher are “simple xoring” too. The statistic characteristics of the Xor-operation make it perfect for encryption.
Why? Even a simple Caesar Cypher is encryption. Maybe not good encryption, but encryption.
Philosophically if not legally, Morse code itself is encryption if you can’t (literally) decipher it. So is English.
Consider the basic encryption model:
plaintext –>[encryption]–> cipher text=>{not plainly understood communication}=>cipher text -> [decryption] ->plaintext
If the encryption/decryption model is a “foreign” (not understood) language, and that includes things like dots and dashes to non-telegraphers, then you have encryption.
Consider the Apache “code talkers” that had both encryption in their language (which the German “naturalists”, who aggressively studied previously-used Native America languages after the First World War, had missed) and encoding in their terms (eg: “Tortoise” (pronounced ” tɔːtəʃel “) = tank)
I don’t think that holds true in all cases, otherwise you could say that a one time pad isn’t encryption, which we both know isn’t true.
PSK31 is more robust for machine to machine than morse.
Bingo. Or if you don’t like that for a variety of reasons, there’s a whole slew of RTTY transmission modes you can use. And with a technician license (as opposed to general) its one of the easiest ways to get on the air, without learning CW that is.
I could see this being interesting for a beacon. Put the latitude and longitude and maybe some weather data as well and it could be useful for navigation. Morse is handy because it is both human and machine readable. It would make a nice target for people that are into DXing
This is for sending binary files though. This is NOT human readable stuff. All that kind of information you mentioned would could be sent better as plain text. To keep with the nature of your example this would become useful if you wanted to send a map or radar images along with your weather information.
Actually I was thinking of a small remote beacon using a small micro and solar. It would probably not have any radar.
I could see this for binary to send updates to that station. Sending binary of ctty would seem pretty simple to me. dot 0 dash 1 compress the file first and use something like a kermit protocol since kermit can use half duplex links.
I suppose you could also send the morse data via email, why I don’t know, but it could be done.
I think the illegal thing only counts in the US that considers encryption munitions, that is, instruments of war. I never understood this and it seems to go perpendicular to freedom of speech. This is something I would expect of China, but not the US.
Nope, there are specific rules in the amateur radio service rules (part 97 of FCC rules) that forbid encryption except for one very specific exception. It has nothing to do with “munitions”. Feel free to look it up.
Two specific exceptions: command transmissions to spacecraft, and command transmissions to model aircraft. Look it up.
No, it’s pretty much a universal rule, and one Hams like, for a few reasons:
1) Ham is supposed to be for the benefit of everyone looking to learn about or advance radio. You can pick up transmissions, decode them, etc. If they were encrypted, they would be useless.
2) Ham is for noncommercial use only. If encrypted messages were allowed, that would be impossible to enforce.
3) Many governments in countries with less-than-stellar human rights still allow ham radio. Political discussions are not permitted. If governments were worried about people using the ham bands to undermine them, they would ban ham in their countries.
No, Part 97.113(a)(4) says “No amateur station shall transmit messages encoded for the purpose of obscuring their meaning, except as otherwise provided herein”.
So if I make an arrangement with another ham, saying, for example, if I chat about my pet dog, that will mean “buy the stock, the deal is going through”, whereas if I chat about my pet cat, that will mean “sell the stock, the deal is off”, then getting on the air in ordinary analog FM or SSB voice and chatting in plain English about my pets could be illegal. To be sure, it would be difficult to detect and prove that violation. But the point is, the law doesn’t make encoding by certain mechanisms illegal, it makes encoding FOR THE PURPOSE OF OBSCURING MEANING illegal, regardless of the mechanism.
This is the rule according to the USA FCC, anywhere they have jurisdiction. I believe many other countries have similar rules, but I’m not certain.
73 de Rich/AG6QR
Interesting project, but there are better ways of transmitting data reliably using very little power on the HF bands. PSK31 is a very good choice, but if you don’t care about data rate and just want to get your message through error-free, check out the various JT modes from Joe Taylor K1JT.
There are hundrends of other digital modes. However, this is the only data transfer scheme I am aware of, that can be UNDERSTOOD by humans and this is because it uses MORSE code (please correct me if not). Well, the data stream is understood, but the actual files can displayed only in a computer as you expect.
As I explained in the website, this is not exactly a mode, it is an encoding scheme. It ENCODES files into raw characters, compresses them and sends them.
It is not meant to replace other modes, each mode has it’s own characteristics and the characteristic of this mode is that it can transfer data through morse code (or phonetic alphabet) and you do not need a computer to receive this data, but only to decode it.
Neat project, well presented.
In light of the whole page change, I must say, love the project site, simple and effective.
I’m going to try to handle a few questions all in one post.
1) Morse Code – no, it’s not all that great as far as signal-to-noise ration requirements go, PSK is superior. However, in the US, 50.0-50.1 and 144.0-144.1 MHz are Morse Code Only – no other digital modulation techniques are allowed there. Your jurisdiction may vary. I’m perfectly aware that these CW-only slices are frequently used for Earth-Moon-Earth (moonbounce) communications.
2) Don’t confuse Encryption with Encoding. Encryption is intended to hide the meaning of your message. Encoding is how you get your message through the channel. Encoding are legal, and indeed required, whereas encrytion is illegal in the amateur service in the US. PSK31 is an encoding, as is 45 Baud Radioteletype, Morse Code, and Navaho-language voice. Or English-language voice, or even expressive sighing. Incidentally, non-English voice is legal in the US as log as you identify your station with English-language characters or with ITU Phonetics. The FCC may, in the US and at their discretion, ask you to document your encoding method if it is not widely known. This has been the case for at least 8 years, maybe 15 or so, since they opened up spread spectrum to more than just a handful of permitted spreading polynomials.
Frequency hopping on the 70cm band is a fun way to mess with the local Irritable Old Dudes, by the way. They can just barely hear you, sometimes, for a fraction of a second.
3) As has been pointed out, this would be the easiest way for US hams with basic Technician licenses to use the bands below 28 MHz without having to learn Morse Code.
–… …–
Actually, Part 97.113(a)(4) prohibits “message encoded for the purpose of obscuring their meaning”. They use the word “encoded”, not “encrypted”. But yes, the issue isn’t the means of encoding used, what’s prohibited is the idea of trying to hide the meaning of a message.
73 de AG6QR
I think it’s safe to say that “messages encoded for the purposes of obscuring their meaning” is a practical definition of encryption.
Ha, yeah there’s a local group on 2m; this frickin’ guy w/ his call sign is the most annoying thing ever, says it sooo slow. But they aren’t practicing Ham moral code and being douches to newbies, just ignoring him and essentially trying to claim this band for themselves.
Well, apparently some other people caught on to this douche behavior (or they’re going after each other) and just hold the PTT button down and they start getting pissed and bitching at each other lol.
MOVIE SPOILER ALERT
MOVIE SPOILER ALERT
MOVIE SPOILER ALERT
It worked in “Interstellar”. They transmitted morese code from a black hole.
behind the book shelf! hehehe they travelled so far, crossed wormhole and got eaten by a blackhole just to end up behind the bookshelf :)
If you want to use this method of transferring data (there are indeed better modulation schemes but that is totally besides the point here), there is a further optimization that can be done: the zip & zbase32 encoding step will produce a string where each symbol will appear in equal amounts. But morse symbols do not all have the same transmit time (eg. ‘e’ is ‘.’ while 0 is ‘- – – – -‘). It is possible to design a scheme where the short symbols are preferred, but that makes it less of a “lets string some utilities together” project and more into custom software. You’d have to design a custom compression program that outputs the string to be sent in one step, where ‘e’ is more prevalent than ‘0’).
Thanks for your idea. I was thought of it at the beginning, they do this in PSK. I will try to see how this could be implemented. For english text on CW that makes sense but for binary data, I do not know if it matters. This is something I have to consider. Do not forget to read at the core of the scheme which has been implemented by another enthusiast http://qrp.gr/cdw/human-oriented-base-32-encoding.txt
I would think hexadecimal in Morse Code is better. It would make more sense and is more or less universal.
Binary to UUENCODE to Morse to UUDECODE to Binary.
Yup, already been done, and already a standard, already freely available. Commence reinvention.
Oh. Except that uuencode, base64, and other similar schemes expect that upper and lowercase letters are distinct. Fail.
On the other hand, there’s an open-source *nix program, codegroup, that turns binary data into 5-letter codegroups suitable for CW transmission. So at least in theory you could take a pdf file, run it through bzip2 or other suitable compressor, then codegroup, then send it by hand or by automatic keyer, and go the other way on receive.
The problem with these (and I believe also with [Konstantinos]’ CDW) is that they’re not error-tolerant methods. For example, codegroups does a CRC to check for errors, but offers no error correction. So you need another layer on top of THAT in order to handle burst errors that are likely to come with CW transmission.
I guess if you had interference, you could say, “please resend after GESXV and up to ESEIS”, but still I see problems with getting any reasonable-sized file (say 1-2k) across accurately.
There IS something to be said for extending the usefulness of CW, since CW equipment can be simple and reliable, and CW is inherently low-bandwidth.
Appreciating the commitment you put into your website and in depth information you provide.
It’s awesome to come across a blog every once in a while that isn’t the same unwanted rehashed
information. Great read! I’ve saved your site and I’m including your RSS feeds to my
Google account.
Somewhat off topic, but I bet someone here will know. I recall reading about a communications technique using extremely low bitrate transmission (like a bit every few seconds), allowing the receiving end to recover the data despite weak signals, even those down in the noise. But I can’t seem to find the magic keywords to Google it. Would like to do some experimentation with this, any references would be much appreciated.
Yeah your right… thought I read it here… or maybe it was a defcon talk… googles not playing nice or I’d find u the link!
try wspr or qrss
Maybe some WSJT mode?
Re: encryption by Hams.
This was posted a few days ago: http://www.qsl.net/kb9mwr/projects/wireless/Data%20Encryption%20is%20Legal.pdf
For your and your attorney’s consideration.
Take a look at the worlds smallest HF transmitter!
https://www.etsy.com/shop/QuirkyQRPHamRadios
Great information. Me and my buddies are trading using RobotFX EA. They are commercializing useful expert advisors (forex trading robots or robot fx) and utilitarian MT4 technical indicators based on successful forex strategies. The robotfx trading tools are competently documented so every expert advisor trading trader can understand the trading idea that supports it. The RobotFX EAs are also available as demo version that anyone can download for free.
psar trailing stop http://robotfx.ro/