Parsing Base64 Encoded Binary PNG Images in JavaScript

The other day David Walsh was experimenting with rendering images in the browser using regular tags as pixels. Valerio picked up the idea and made some enhancements. A server-side script transformed PNG files into a JSON image format for easy parsing on the client. That raised the question… How difficult would it be to do that parsing on the client instead?

Why PNG? Well, other than becoming the new defacto standard for graphics it’s a very simple format. It’s also free of patents and uses only simple well known techniques. It makes it very easy to work with. This post is about parsing raw PNG image data in pure JavaScript. It has nothing to do with built in browser support for the format.

Base64 Encoding

JavaScript doesn’t allow us to work with binary data directly. Even with XHR we can’t work with the raw binary data because JavaScript doesn’t currently have a concept of raw bytes. Instead we have to get the bytes from a character representation of the data.

Luckily there’s already a standard transfer encoding already heavily in use in various places of the W3C standards… Base64! You can use the data: URI scheme to embed image data in your HTML or CSS documents. It’s also heavily used for binary data in e-mails.

We can get the data either from an XHR request, from a src attribute or just statically embedded in your JavaScript file. So, now we have our data as Base64 encoded string.

To work with the raw data we need a way to represent bytes. If you’re working with ASCII data you can just stick to string representations. But since we’re going to be working binary data the most useful way seems to be simple Numbers. That allows you to do bitwise operations and easily convert them to and from ASCII. It’s also provides better performance than representing the bytes as Objects.

Now we need a parser. I went with a sample parser by some guy named notmasteryet. There are others but this seems like a pretty solid implementation and allows us to work with bytes as Numbers. It also works as a reader that lets us read our data piece by piece instead of filling our memory.

DEFLATE

The current PNG standard only uses the DEFLATE algorithm for compression. It’s the same algorithm used in ZIP, GZIP, zlib, etc. So it’s a very common format.

Luckily for us, notmasteryet’s sample also includes a DEFLATE decompressor. It also works as a piece by piece reader which makes it more memory efficient to work with. The reader pattern is a great way to read data in nested formats.

PNG

The PNG format consists of a set of named chunks. A set of “IDAT” chunks makes up the main image data. The total data stream is compressed using DEFLATE. The uncompressed data is filtered using one of 5 simple delta compression filters for each line of pixels.

Notice that we haven’t yet touched any image-processing specific logic. DEFLATE and delta compression is used for text and other data as much as anything else.

The raw data consists of a color for each pixel. This can be either grayscale, RGB or a reference to a palette color. This is what we really want.

The PNG format is open and well documented. So I’m not going to cover it in any more detail.

Proof of Concept

Since we’re doing a lightweight JavaScript parser and probably have some control over the image data, we can skip some of the more outlandish features of the specification. We can also skip the verification parts. We’ll just skip the file headers and CRC checks.

I decided on an a simple API that reads each line of pixels as an array of RGB colors represented as a number.

var image = new PNG(base64data);
image.width; // Image width in pixels
image.height; // Image height in pixels
var line;
while(line = image.readLine()){
  for(var x = 0;x < line.length;x++){
    var px = line[x]; // Pixel RGB color as a single numeric value
    // white pixel == 0xFFFFFF
  }
}

I then took that RGB data and inserted the pixels into my document as DIV tags with a background-color.

In less than 3 hours I had a working Proof of Concept of a format I had never worked with before.

I skipped interlacing, alpha and some of the filters for the demo. It’s not meant to be a fully working prototype nor a reference library in any way.

Now What?

You could…

• Display the image using a regular rendering method but use the PNG parser to extract colors using a Color Picker.
• Add obfuscation or cryptographic layers to render images that can't be easily ripped by bots or downloaded by users.
• Render embedded PNG images using VML in Internet Explorer (which lacks data: URI support) with full alpha support. Don't expect this method to become the new hack for PNG or embedded images in Internet Explorer. The rendering methods here are probably too slow for that. You could do some nice stuff with CANVAS though. However, I have demonstrated that it is possible to work with binary formats in JavaScript. We shouldn't be afraid of utilizing existing binary standards (PNG, GZIP, SVGZ, SWF, TTF...). We shouldn't always fallback to our comfortable old JSON format and reinvent the wheel for every client-side need. Relevant Projects The MooTools team is working on a tool set for vector graphics in the web browser, A.R.T. You could use binary formats to embed your vector based graphics in formats like... TrueType! The APE (Ajax Push Engine) project brings socket programming to the JavaScript platform. Digg's MXHR stream parses multipart encoded data and extracts the parts for various uses. This could provide a packaging model for various widgets or data packets.