SQLite On The Web: Absurd-sql

Love it or hate it, the capabilities of your modern web browser continuously grow in strange and wild ways. The ability for web apps to work offline requires a persistent local storage solution and for many, IndexedDB is the only choice as it works across most browsers and provides a database-like interface. However, as [James Long] found, IndexedDB is painfully slow on chrome and limited in querying ability. He set out to bring a tool he was familiar with, SQLite, and bring it to the web browser as absurd-sql.

Why absurd? Partially because most browsers (not chrome) implement IndexedDB on top of SQLite. So for many browsers, it is just SQLite on top of IndexedDB on top of SQLite. Luckily for [James] there already was a project known as sql.js that uses emscripten to compile the C-based SQLite into WebAssembly. However, sql.js uses an in-memory storage backing and all data is lost when refreshing the page. [James] tweaked SQLite’s method of reading and writing blocks. Instead of being memory backed, he added a layer to read and write blocks from IndexedDB. This means that only sections of the database need to be read in, bringing in huge performance gains.

a graph showing absurd-sql beating IndexDB on every benchmarkThat brings us to the other reason why it’s absurd. On chrome (as well as Firefox), absurd-sql beats IndexedDB on almost every benchmark. A query like SELECT SUM(*) FROM kv led to stunning results.

So what’s the downside? Other than a somewhat large WebAssembly file that needs to get downloaded (409KB) and cached, there really isn’t. Of course, it’s not all roses when it comes to web development. Native SQLite runs 2-3 times faster than absurd-sql, which demonstrates how slow IndexedDB really is.

There are other storage standards on the horizon for web browsers, but locking becomes an issue. SQLite expects synchronous reads and writes because it’s just simple C. IndexedDB and other storage solutions are asynchronous as the event loop of Javascript lends itself well to that model. Absurd-sql gets around that by creating a SharedArrayBuffer that is shared with a worker process. The atomics API is used to communicate with the buffer. In particular, atomics.wait() allows the worker to block main thread execution until the read or write has finished. From the perspective of SQLite, the operations are synchronous. IndexedDB provides transactions so multiple connections can happen (for example multiple tabs open). Multiple readonly transactions can occur in parallel but only one readwrite transaction can be in flight.

Why not pull up your browser and start playing around with it? You’re already doomed to learn WebAssembly anyway.

You Are Doomed To Learn WebAssembly

At first, Web browsers displayed HTML pages. But then people wanted those pages to do something. So we got — among other things — JavaScript. Then people wanted to do super complicated and compute-intensive things. So now we have WebAssembly. If you want to learn it, [diekmann] has a 4-part series that covers everything from getting started to porting Doom into your browser.

Paradoxically, instead of using a browser, he uses the wasm binary toolkit to run code more like a standard assembler. And wasm — what most people call WebAssembly — isn’t like most assemblers you know. Instead of labels, there are blocks that work much more like high-level language constructs such as while loops in C.

Continue reading “You Are Doomed To Learn WebAssembly”

Web Assembly, Music Synthesis, And The Beauty Of Math

The electronics hobby has changed a lot since the advent of the microprocessor. Before that — and with the lack of large-scale integrated circuits — projects in magazines tended to be either super simple or ultra complex. However, one popular type of project dealt with music synthesis. Fairly simple circuits could combine to make a complex synthesizer so it was sort of the best of both worlds. Nowadays, you are more likely to tackle a music synthesizer in software like [Tim] did when he created Abelton in Web Assembly and C++. Along the way, he learned a lot about the relationship between math and music.

[Tim] covers what he learned about the Nyquist theorem and how to keep synthesis data flowing in real time with buffers. However, there are some problems trying to do all this in a cross-browser context. The AudioWorklet class appears to have widespread support, though, and [Tim] managed to get that working.

Continue reading “Web Assembly, Music Synthesis, And The Beauty Of Math”

C++ Compiler Targets The Web

It is a common problem these days. You have a piece of code in C or C++. Maybe it is older code. Or maybe you prefer prototyping your ideas using C. But, inevitably, someone now wants your code to run in a Web browser. The options for making this happen have expanded quite a bit lately and one possibility is Cheerp, an open-source compiler that handles up to C++ 17 and can output to WebAssembly, JavaScript, or asm.js.

The compiler is free to use for GPLv2 projects. If you aren’t open yourself, it looks like you have to cut a deal to use Cheerp with its maker, Learning Technologies.

Continue reading “C++ Compiler Targets The Web”

Breadboard Computer Plays Snake On Character Display; Also In A Browser!

If building a homebrew computer on a breadboard is your thing, you’re most certainly familiar with [Ben Eater], whose design of using nothing but logic gates has served as inspiration for many replicas over the years. [visrealm] took the concept and expanded upon it, even adding a 16×2 LCD that let’s you play Snake by moving a single pixel on the character display!

Making the most of tiny resolution is impressive — it’s a difficult constraint for the game field. But there are other tricks at work as well. [visrealm] uses different intensities to distinguish between the snake and its food which is kind of a dark pixel in the demo shown after the break. But what stands out most is that the breadboard build is really only half of the story. In addition, [visrealm] built an entire emulator that resembles his actual breadboard design, which can be programmed and used via browser, giving WebAssembly a whole new meaning. While that’s convenient for anyone interested to play around with these breadboard computers, but lacks the patience to build one themselves, it also functions as the real one’s programming environment. In addition, an ESP8266 is used to load a new program directly via WiFi.

All the code and some build notes are available on GitHub, and if you’re looking for a nifty LCD emulator for your web site, there’s a standalone repository for that as well. But in case you need a better display option for your own breadboard computer, how about adding a VGA connector? And if you don’t build your own yet, it’s never too late to start.

Continue reading “Breadboard Computer Plays Snake On Character Display; Also In A Browser!”

Examine Source Code To Assembly Mapping With PenguinTrace

C-programmers who don’t have a mental model of what’s going on underneath their thin veneer of abstraction above assembly code are destined for trouble. In order to provide a convenient way to understand what C-code gets compiled to and how it runs on the machine, [Alex Beharrell] has created penguinTrace, a program which allows you to see what instructions your code compiles to, and examine how it executes.

While you can get somewhat similar functionality out of standard debuggers, penguinTrace was purpose-built to facilitate exploration of how the whole process works. You can single-step through the instructions your code compiled to, examine variables, and look at the stack — the usual debugger stuff — but structured more for exploration and learning than full-on debugging. Based on our experiences when we learned low-level programming, anything that can help novices build that all-important mental picture of what’s going on underneath is a good thing. But, since it was written with a secondary purpose of learning how debuggers themselves work, it’s a great opportunity for exploring that space, too.

The UI harnesses CodeMirror to provide a browser-based interface, and is configurable to use Clang or GCC for compilation. It supports AMD64/X86-64 and AArch64 architectures, and will run on Windows using WSL: if you’ve got a PC running Linux, a Raspberry Pi, or a Windows box, you’re good to go. The code is AGPL-licensed and available on GitHub. So, if you want to gain a better understanding of what happens when you compile and run “hello, world,” grab a copy and start exploring.

This isn’t the only way to debug, though – we previously featured an application that allows a type of debugging for the Arduino platform.

 

 

WebAssembly: What Is It And Why Should You Care?

If you keep up with the field of web development, you may have heard of WebAssembly. A relatively new kid on the block, it was announced in 2015, and managed to garner standardised support from all major browsers by 2017 – an impressive feat. However, it’s only more recently that the developer community has started to catch up with adoption and support.

So, what is it? What use case is so compelling that causes such quick browser adoption? This post aims to explain the need for WebAssembly, a conceptual overview of the technical side, as well as a small hands-on example for context.

Continue reading “WebAssembly: What Is It And Why Should You Care?”