CalyptusAdvanced web application development tactics – from scalability to UI designJekyll2014-02-09T19:39:33+01:00http://blog.calyptus.eu/http://blog.calyptus.eu/info@calyptus.euhttp://blog.calyptus.eu/seb/2011/12/custom-datetime-json-serialization2011-12-28T09:21:32Z2011-12-28T09:21:32Zhttp://blog.calyptus.euinfo@calyptus.eu<p>The DateTime serialization format in the built-in JSON serializer in .NET is particularly inconvenient as it requires a regular expression to parse and is not human-readable which makes debugging a pain.</p>
<h2 id="iso-8601">ISO 8601</h2>
<p>JSON doesn’t have a standard Date format per say. By design. Instead, the JavaScript defines toJSON methods so that objects can convert themselves to a JSON compatible format. The ECMAScript specification 5th Edition (15.9.5.4) defines that toJSON of a Date object should return a ISO 8601 formatted string in universal time (UTC). E.g: “2011-12-28T08:20:31.917Z”</p>
<p>This is a convenient format that specifically avoids confusion and ambiguity when passing it between systems. To turn it back into a Date object in JavaScript we can simply pass it to the Date constructor: E.g: Date(“2011-12-28T08:20:31.917Z”)</p>
<h2 id="custom-serialization-using-javascriptconverter">Custom Serialization Using JavaScriptConverter</h2>
<p>.NET’s JavaScriptSerializer does provide an extension point for custom serialization. By calling RegisterConverters and passing custom JavaScriptConverter instances you can control how custom classes are serialized. APIs such as Script Services allow you to register such converters in Web.config. (AFAIK ASP.NET MVC doesn’t yet provide a hook to add JavaScriptConverters so you have to use <a href="http://weblogs.asp.net/rashid/archive/2009/03/23/submitting-my-first-bug-after-asp-net-mvc-1-0-rtm-release.aspx">a custom ActionResult</a>.)</p>
<p>Unfortunately the JavaScriptConverter API will only allow you to convert an object into another object (IDictionary<string, object>). Not into a custom string format.</p>
<p>It is at this point you might be looking for other serializers such as Json.NET. There are certainly many much faster and less crappy serializers out there. However, there is a benefit to using the built-in one. Mainly it may be already integrated into existing system that you can’t change. Luckily, as long as they allow you to register custom JavaScriptConverters I have the solution for you.</p>
<h2 id="the-hack">The Hack</h2>
<p>The implementation of JavaScriptSerializer does string serialization for several types but Uri in particular. Uri is a class instead of a struct and fortunately is not sealed. That means we can inherit from it to implement the IDictionary<string, object> interface. That will allowing us to pass it in our custom JavaScriptConverter. Since it inherits from Uri it will be serialized as a string. It will do some URI encoding but this doesn’t affect the ISO 8601 format.</p>
<p>Deserialization is handled by the default implementation.</p>
<p>You can also use this hack to serialize custom enum types as strings instead of numbers or objects.</p>
<h2 id="the-code">The Code</h2>
<div class="highlight"><pre><code class="c#"><span class="k">public</span> <span class="k">class</span> <span class="nc">DateTimeJavaScriptConverter</span> <span class="p">:</span> <span class="n">JavaScriptConverter</span>
<span class="p">{</span>
<span class="k">public</span> <span class="k">override</span> <span class="kt">object</span> <span class="nf">Deserialize</span><span class="p">(</span><span class="n">IDictionary</span><span class="p"><</span><span class="kt">string</span><span class="p">,</span> <span class="kt">object</span><span class="p">></span> <span class="n">dictionary</span><span class="p">,</span> <span class="n">Type</span> <span class="n">type</span><span class="p">,</span> <span class="n">JavaScriptSerializer</span> <span class="n">serializer</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">return</span> <span class="k">new</span> <span class="nf">JavaScriptSerializer</span><span class="p">().</span><span class="n">ConvertToType</span><span class="p">(</span><span class="n">dictionary</span><span class="p">,</span> <span class="n">type</span><span class="p">);</span>
<span class="p">}</span>
<span class="k">public</span> <span class="k">override</span> <span class="n">IDictionary</span><span class="p"><</span><span class="kt">string</span><span class="p">,</span> <span class="kt">object</span><span class="p">></span> <span class="n">Serialize</span><span class="p">(</span><span class="kt">object</span> <span class="n">obj</span><span class="p">,</span> <span class="n">JavaScriptSerializer</span> <span class="n">serializer</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">if</span> <span class="p">(!(</span><span class="n">obj</span> <span class="k">is</span> <span class="n">DateTime</span><span class="p">))</span> <span class="k">return</span> <span class="k">null</span><span class="p">;</span>
<span class="k">return</span> <span class="k">new</span> <span class="nf">CustomString</span><span class="p">(((</span><span class="n">DateTime</span><span class="p">)</span><span class="n">obj</span><span class="p">).</span><span class="n">ToUniversalTime</span><span class="p">().</span><span class="n">ToString</span><span class="p">(</span><span class="s">"O"</span><span class="p">));</span>
<span class="p">}</span>
<span class="k">public</span> <span class="k">override</span> <span class="n">IEnumerable</span><span class="p"><</span><span class="n">Type</span><span class="p">></span> <span class="n">SupportedTypes</span>
<span class="p">{</span>
<span class="k">get</span> <span class="p">{</span> <span class="k">return</span> <span class="k">new</span><span class="p">[]</span> <span class="p">{</span> <span class="k">typeof</span><span class="p">(</span><span class="n">DateTime</span><span class="p">)</span> <span class="p">};</span> <span class="p">}</span>
<span class="p">}</span>
<span class="k">private</span> <span class="k">class</span> <span class="nc">CustomString</span> <span class="p">:</span> <span class="n">Uri</span><span class="p">,</span> <span class="n">IDictionary</span><span class="p"><</span><span class="kt">string</span><span class="p">,</span> <span class="kt">object</span><span class="p">></span>
<span class="p">{</span>
<span class="k">public</span> <span class="nf">CustomString</span><span class="p">(</span><span class="kt">string</span> <span class="n">str</span><span class="p">)</span>
<span class="p">:</span> <span class="k">base</span><span class="p">(</span><span class="n">str</span><span class="p">,</span> <span class="n">UriKind</span><span class="p">.</span><span class="n">Relative</span><span class="p">)</span>
<span class="p">{</span>
<span class="p">}</span>
<span class="k">void</span> <span class="n">IDictionary</span><span class="p"><</span><span class="kt">string</span><span class="p">,</span> <span class="kt">object</span><span class="p">>.</span><span class="n">Add</span><span class="p">(</span><span class="kt">string</span> <span class="n">key</span><span class="p">,</span> <span class="kt">object</span> <span class="k">value</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">throw</span> <span class="k">new</span> <span class="nf">NotImplementedException</span><span class="p">();</span>
<span class="p">}</span>
<span class="kt">bool</span> <span class="n">IDictionary</span><span class="p"><</span><span class="kt">string</span><span class="p">,</span> <span class="kt">object</span><span class="p">>.</span><span class="n">ContainsKey</span><span class="p">(</span><span class="kt">string</span> <span class="n">key</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">throw</span> <span class="k">new</span> <span class="nf">NotImplementedException</span><span class="p">();</span>
<span class="p">}</span>
<span class="n">ICollection</span><span class="p"><</span><span class="kt">string</span><span class="p">></span> <span class="n">IDictionary</span><span class="p"><</span><span class="kt">string</span><span class="p">,</span> <span class="kt">object</span><span class="p">>.</span><span class="n">Keys</span>
<span class="p">{</span>
<span class="k">get</span> <span class="p">{</span> <span class="k">throw</span> <span class="k">new</span> <span class="n">NotImplementedException</span><span class="p">();</span> <span class="p">}</span>
<span class="p">}</span>
<span class="kt">bool</span> <span class="n">IDictionary</span><span class="p"><</span><span class="kt">string</span><span class="p">,</span> <span class="kt">object</span><span class="p">>.</span><span class="n">Remove</span><span class="p">(</span><span class="kt">string</span> <span class="n">key</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">throw</span> <span class="k">new</span> <span class="nf">NotImplementedException</span><span class="p">();</span>
<span class="p">}</span>
<span class="kt">bool</span> <span class="n">IDictionary</span><span class="p"><</span><span class="kt">string</span><span class="p">,</span> <span class="kt">object</span><span class="p">>.</span><span class="n">TryGetValue</span><span class="p">(</span><span class="kt">string</span> <span class="n">key</span><span class="p">,</span> <span class="k">out</span> <span class="kt">object</span> <span class="k">value</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">throw</span> <span class="k">new</span> <span class="nf">NotImplementedException</span><span class="p">();</span>
<span class="p">}</span>
<span class="n">ICollection</span><span class="p"><</span><span class="kt">object</span><span class="p">></span> <span class="n">IDictionary</span><span class="p"><</span><span class="kt">string</span><span class="p">,</span> <span class="kt">object</span><span class="p">>.</span><span class="n">Values</span>
<span class="p">{</span>
<span class="k">get</span> <span class="p">{</span> <span class="k">throw</span> <span class="k">new</span> <span class="n">NotImplementedException</span><span class="p">();</span> <span class="p">}</span>
<span class="p">}</span>
<span class="kt">object</span> <span class="n">IDictionary</span><span class="p"><</span><span class="kt">string</span><span class="p">,</span> <span class="kt">object</span><span class="p">>.</span><span class="k">this</span><span class="p">[</span><span class="kt">string</span> <span class="n">key</span><span class="p">]</span>
<span class="p">{</span>
<span class="k">get</span>
<span class="p">{</span>
<span class="k">throw</span> <span class="k">new</span> <span class="nf">NotImplementedException</span><span class="p">();</span>
<span class="p">}</span>
<span class="k">set</span>
<span class="p">{</span>
<span class="k">throw</span> <span class="k">new</span> <span class="nf">NotImplementedException</span><span class="p">();</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="k">void</span> <span class="n">ICollection</span><span class="p"><</span><span class="n">KeyValuePair</span><span class="p"><</span><span class="kt">string</span><span class="p">,</span> <span class="kt">object</span><span class="p">>>.</span><span class="n">Add</span><span class="p">(</span><span class="n">KeyValuePair</span><span class="p"><</span><span class="kt">string</span><span class="p">,</span> <span class="kt">object</span><span class="p">></span> <span class="n">item</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">throw</span> <span class="k">new</span> <span class="nf">NotImplementedException</span><span class="p">();</span>
<span class="p">}</span>
<span class="k">void</span> <span class="n">ICollection</span><span class="p"><</span><span class="n">KeyValuePair</span><span class="p"><</span><span class="kt">string</span><span class="p">,</span> <span class="kt">object</span><span class="p">>>.</span><span class="n">Clear</span><span class="p">()</span>
<span class="p">{</span>
<span class="k">throw</span> <span class="k">new</span> <span class="nf">NotImplementedException</span><span class="p">();</span>
<span class="p">}</span>
<span class="kt">bool</span> <span class="n">ICollection</span><span class="p"><</span><span class="n">KeyValuePair</span><span class="p"><</span><span class="kt">string</span><span class="p">,</span> <span class="kt">object</span><span class="p">>>.</span><span class="n">Contains</span><span class="p">(</span><span class="n">KeyValuePair</span><span class="p"><</span><span class="kt">string</span><span class="p">,</span> <span class="kt">object</span><span class="p">></span> <span class="n">item</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">throw</span> <span class="k">new</span> <span class="nf">NotImplementedException</span><span class="p">();</span>
<span class="p">}</span>
<span class="k">void</span> <span class="n">ICollection</span><span class="p"><</span><span class="n">KeyValuePair</span><span class="p"><</span><span class="kt">string</span><span class="p">,</span> <span class="kt">object</span><span class="p">>>.</span><span class="n">CopyTo</span><span class="p">(</span><span class="n">KeyValuePair</span><span class="p"><</span><span class="kt">string</span><span class="p">,</span> <span class="kt">object</span><span class="p">>[]</span> <span class="n">array</span><span class="p">,</span> <span class="kt">int</span> <span class="n">arrayIndex</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">throw</span> <span class="k">new</span> <span class="nf">NotImplementedException</span><span class="p">();</span>
<span class="p">}</span>
<span class="kt">int</span> <span class="n">ICollection</span><span class="p"><</span><span class="n">KeyValuePair</span><span class="p"><</span><span class="kt">string</span><span class="p">,</span> <span class="kt">object</span><span class="p">>>.</span><span class="n">Count</span>
<span class="p">{</span>
<span class="k">get</span> <span class="p">{</span> <span class="k">throw</span> <span class="k">new</span> <span class="n">NotImplementedException</span><span class="p">();</span> <span class="p">}</span>
<span class="p">}</span>
<span class="kt">bool</span> <span class="n">ICollection</span><span class="p"><</span><span class="n">KeyValuePair</span><span class="p"><</span><span class="kt">string</span><span class="p">,</span> <span class="kt">object</span><span class="p">>>.</span><span class="n">IsReadOnly</span>
<span class="p">{</span>
<span class="k">get</span> <span class="p">{</span> <span class="k">throw</span> <span class="k">new</span> <span class="n">NotImplementedException</span><span class="p">();</span> <span class="p">}</span>
<span class="p">}</span>
<span class="kt">bool</span> <span class="n">ICollection</span><span class="p"><</span><span class="n">KeyValuePair</span><span class="p"><</span><span class="kt">string</span><span class="p">,</span> <span class="kt">object</span><span class="p">>>.</span><span class="n">Remove</span><span class="p">(</span><span class="n">KeyValuePair</span><span class="p"><</span><span class="kt">string</span><span class="p">,</span> <span class="kt">object</span><span class="p">></span> <span class="n">item</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">throw</span> <span class="k">new</span> <span class="nf">NotImplementedException</span><span class="p">();</span>
<span class="p">}</span>
<span class="n">IEnumerator</span><span class="p"><</span><span class="n">KeyValuePair</span><span class="p"><</span><span class="kt">string</span><span class="p">,</span> <span class="kt">object</span><span class="p">>></span> <span class="n">IEnumerable</span><span class="p"><</span><span class="n">KeyValuePair</span><span class="p"><</span><span class="kt">string</span><span class="p">,</span> <span class="kt">object</span><span class="p">>>.</span><span class="n">GetEnumerator</span><span class="p">()</span>
<span class="p">{</span>
<span class="k">throw</span> <span class="k">new</span> <span class="nf">NotImplementedException</span><span class="p">();</span>
<span class="p">}</span>
<span class="n">IEnumerator</span> <span class="n">IEnumerable</span><span class="p">.</span><span class="n">GetEnumerator</span><span class="p">()</span>
<span class="p">{</span>
<span class="k">throw</span> <span class="k">new</span> <span class="nf">NotImplementedException</span><span class="p">();</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div>
<p><a href="http://blog.calyptus.eu/seb/2011/12/custom-datetime-json-serialization/">Custom DateTime JSON Format for .NET JavaScriptSerializer</a> was originally published by at <a href="http://blog.calyptus.eu">Calyptus</a> on December 28, 2011.</p>http://blog.calyptus.eu/seb/2011/10/choosing-a-javascript-module-syntax2011-10-08T12:39:32Z2011-10-08T12:39:32Zhttp://blog.calyptus.euinfo@calyptus.eu<p><em>TLDR This post is mainly a way for me to structure arguments and counter-arguments for various JavaScript module systems. You can jump to the bottom of the page for a matrix mapping features to various module systems. I’ll keep this page updated as new arguments are accepted or new module systems are invented, if they add new benefits.</em></p>
<p><em>I make the assumption that you should always compile scripts together and gzip in production (possibly to multiple files). I make a second assumption that the complexity of such a build tool is irrelevant once it’s already developed. If you don’t accept these assumptions, the arguments below might be irrelevant or incomplete.</em></p>
<h2 id="introduction">Introduction</h2>
<p>JavaScript originally didn’t specify a way to work with source code in multiple different files, or across different modules in the same file. Recently we have seen many different ways to solve this approach mainstream JavaScript development. I’ll try to break them down and explain the reasoning for choosing between them in today’s JavaScript environments.</p>
<p>There are three primary attributes that a module system might include:</p>
<p><strong>Module Isolation</strong> - Code within a module should not leak in to the global scope shared by multiple modules.</p>
<p><strong>Inter-module Dependency Definition</strong> - There should be some syntax for defining dependencies between multiple source code files.</p>
<p><strong>Script Loader</strong> - Some how the application code from all relevant files must be loaded at runtime. The module system might use the same technique during development and release, but often you should treat them differently. In a production environment, your focus is on fast loading. In debug, your focus is on ease of use and mapping VM breakpoints and errors to readable source code.</p>
<p><strong>Sandboxing and Inversion of Control</strong> - This is currently a rarely used feature. It involves a way to load a module dependency graph in a different context or sandbox. It is an important part of dependency loading. It’s beyond the scope of this post, but it can be implemented together with any custom script loader.</p>
<h2 id="the-traditional-javascript-library">The Traditional JavaScript Library</h2>
<p>Traditionally JavaScript libraries handled module isolation by wrapping source code in a (function(){ … })() closure.</p>
<p>Inter-module Dependencies didn’t have a specific syntax. One module gained access to another through a namespace. A namespace was defined in the global scope. Modules could collide if the same namespace was used by two different modules.</p>
<p>On the web, loading of dependent files was managed by defining all application dependencies in a long list using <script> tags.</p>
<p>Note: If one <script> tag fails to load, others may still run. That means this model is fragile in a production environment. If one of your dependencies isn’t loaded, other code may still run. It might not be noticeable at first. You can see this with Chrome sometime. It uses various “smart” network techniques that sometimes fail to load a script dependency on a page, causing some features to break, sometimes in very bad ways.</p>
<p>In production, the best practice is to compile all these code files into a single minified and compressed JavaScript file. This ensures that all dependencies are loaded together, or not at all. It also makes sure the script is small by minifying the source. Putting it all into a single file makes good uses of GZIP compression and ensures minimal overhead from the transfer protocols.</p>
<p>The problem with the traditional approach is that there isn’t a specified way to define dependencies between various script files. Therefore it’s difficult to know what scripts need to be included to run a program and in what order they need to run. Together they form a dependency graph.</p>
<h2 id="externally-defined-dependencies">Externally Defined Dependencies</h2>
<p>The module isolation is done as in traditional JavaScript libraries. The creation of the dependency graph and script loader is a little different though.</p>
<p>We can define the dependency graph in an external file. E.g. we can include a package file which defines references to all the files in a packages and how they relate to each other.</p>
<div class="highlight"><pre><code class="js"><span class="p">{</span>
<span class="s1">'/MyDependentModule.js'</span><span class="o">:</span> <span class="p">[</span><span class="s1">'/lib/StandAloneModule.js'</span><span class="p">],</span>
<span class="s1">'/lib/StandAloneModule.js'</span><span class="o">:</span> <span class="p">[]</span>
<span class="p">}</span>
</code></pre></div>
<p>A custom script loader would first load a package file, then load and execute all scripts in the order they need to be executed. We can also add some asynchronous loading magic. This is the most efficient way to load scripts and easy to do.</p>
<p>It is easier to maintain than list of scripts because we don’t have to worry about the ordering of the scripts. The loader calculates that. On the down side, it’s still rather painful to maintain since you have a separate file that needs to be maintained when dependencies change.</p>
<h2 id="inferred-dependencies">Inferred Dependencies</h2>
<p>You can try to infer the dependency graph by parsing the code and infer what global namespaces are defined in one file and what namespaces are used by another.</p>
<div class="highlight"><pre><code class="js"><span class="kd">var</span> <span class="nx">MyGlobalModule</span> <span class="o">=</span> <span class="p">(</span><span class="kd">function</span><span class="p">(){</span>
<span class="kd">var</span> <span class="nx">imported</span> <span class="o">=</span> <span class="nx">MyOtherModule</span><span class="p">;</span>
<span class="p">...</span>
<span class="k">return</span> <span class="nx">exports</span><span class="p">;</span>
<span class="p">})();</span>
</code></pre></div>
<p>Since we’re not in a “with” statement we can infer the MyOtherModule must be in the global scope. Since we’re not defining it, someone else must be. Therefore we have a dependency on what ever file defines MyOtherModule.</p>
<p>Modules that don’t define a new object but rather extend existing ones (monkey patching) doesn’t necessarily have an exports. You can easily add an empty object to create a dependency on these pseudo-modules.</p>
<p>One problem with this approach is that we need to have a list of all files available. Then pre-parse them so that we can see which files are used and in what order they need to be executed. This is easy enough for a local compiler. It can just scan all files in a directory. An in-browser script loader for use during development can’t list files though.</p>
<p>One way to solve this is to have a convention where the name of a global variable maps to the name of a file. That way we can also infer the name and location of the file.</p>
<p>This technique hasn’t gained much traction. Possibly because it takes some effort to make the tools. Once the tooling is done, that’s not an argument not to use it.</p>
<h2 id="comment-defined-dependencies">Comment Defined Dependencies</h2>
<p>Instead of inferring our dependencies using a convention, we define them in a comment within the source file.</p>
<div class="highlight"><pre><code class="js"><span class="c1">// @requires /lib/MyOtherModule.js</span>
<span class="kd">var</span> <span class="nx">MyGlobalModule</span> <span class="o">=</span> <span class="p">(</span><span class="kd">function</span><span class="p">(){</span>
<span class="kd">var</span> <span class="nx">imported</span> <span class="o">=</span> <span class="nx">MyOtherModule</span><span class="p">;</span>
<span class="p">...</span>
<span class="k">return</span> <span class="kr">export</span><span class="p">;</span>
<span class="p">})();</span>
</code></pre></div>
<p>The script loader can load the script file using XHR. Then parse the file’s comments to see what dependencies it has. Then load the dependency using XHR etc. After the dependency graph has been loaded this way, they can be executed in order using <script> tags.</p>
<p>This has the added benefit that the file names doesn’t necessarily need to correspond to global object names.</p>
<p>This is also easy to implement which is why it has gained more traction than inferred dependencies.</p>
<h2 id="synchronous-module-definition-commonjs">Synchronous Module Definition (CommonJS)</h2>
<p>The CommonJS wiki originally defined a synchronous API to load dependencies in non-browser environments. Module isolation between files is built-in. So, the global scope isn’t shared. Only exported properties are shared.</p>
<p>Inter-module dependencies are defined using a synchronous API. You call the global require function passing a string argument representing the name of another source file.</p>
<div class="highlight"><pre><code class="js"><span class="kd">var</span> <span class="nx">imported</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="s1">'lib/MyOtherModule'</span><span class="p">);</span>
<span class="p">...</span>
<span class="nx">exports</span><span class="p">.</span><span class="nx">myExport</span> <span class="o">=</span> <span class="p">...;</span>
</code></pre></div>
<p>These modules can be used as is, in Node.js and other CommonJS compatible implementations. (Much fewer than JavaScript compatible environments since CommonJS is not a formal standard.)</p>
<p>This model can be used in a browser environment using a custom script loader. The script loader can use XHR to load the script, parse the require statements, load the next script, parse it and build the dependency graph.</p>
<p>To actually execute modules in isolation, you need to wrap them in some custom closure. Then the code is executed by inject the new code into script tags.</p>
<h2 id="commonjs-hybrids">CommonJS Hybrids</h2>
<p>To make a script compatible with both CommonJS and the traditional model you can use a hybrid model. Your source file checks for existence of CommonJS require/exports and uses them if available, otherwise it falls back on the traditional model. This makes source code directly compatible with CommonJS and the traditional model out-of-the-box.</p>
<h2 id="define-block">Define Block</h2>
<div class="highlight"><pre><code class="js"><span class="nx">define</span><span class="p">(</span><span class="kd">function</span><span class="p">(</span><span class="nx">require</span><span class="p">,</span> <span class="nx">exports</span><span class="p">,</span> <span class="nx">module</span><span class="p">){</span>
<span class="kd">var</span> <span class="nx">imported</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="s1">'lib/MyOtherModule'</span><span class="p">);</span>
<span class="p">...</span>
<span class="nx">exports</span><span class="p">.</span><span class="nx">myExport</span> <span class="o">=</span> <span class="p">...;</span>
<span class="p">});</span>
</code></pre></div>
<p>The define() statement that enables async running. That’s half way to AMD. This solves some debugging issues in current browsers. However, it doesn’t support defining the dependency list before running the code. Therefore the loaders still won’t work automatically in browser environments.</p>
<p><strong>UPDATE: This was supported in Node.js for a while but is no longer supported in 0.5.x.</strong></p>
<h2 id="asynchronous-module-definition-amd">Asynchronous Module Definition (AMD)</h2>
<p>Asynchronous Module Definition combines module-isolation, dependency definitions and script-loading into one tool, to make it easier to use the code in the browser.</p>
<div class="highlight"><pre><code class="js"><span class="nx">define</span><span class="p">([</span><span class="s1">'MyOtherModule'</span><span class="p">],</span> <span class="kd">function</span><span class="p">(</span><span class="nx">imported</span><span class="p">){</span>
<span class="p">...</span>
<span class="k">return</span> <span class="nx">exports</span><span class="p">;</span>
<span class="p">});</span>
</code></pre></div>
<p>This requires a custom loader. The custom loader is efficient than the custom loaders used by Infered Dependencies, Comment Defined Dependencies and Synchronous Module Definitions. It can also work with Cross-Site Scripting (XSS) dependencies without the Cross-Origin Resource Sharing protocol (CORS). However, the Externally Defined dependency graph is even more efficient.</p>
<p>It is possible to create and AMD/CommonJS/Traditional hybrid that supports all formats directly from source. This requires a lot of boilerplate code and is not easy to maintain.</p>
<h2 id="ecmascriptnext-modules">ECMAScript.Next Modules</h2>
<p>The ECMAScript Harmony proposals include a new syntax for defining modules. This will probably be in ECMAScript.Next (probably ECMAScript 6).</p>
<p>All module scope will be isolated by default. Dependencies are defined using a new syntax. The new syntax ensures that module dependencies can be statically inferred (without relying on convention).</p>
<div class="highlight"><pre><code class="js"><span class="nx">module</span> <span class="nx">importedModule</span> <span class="nx">from</span> <span class="s2">"MyOtherModule.js"</span><span class="p">;</span>
<span class="p">...</span>
<span class="kr">export</span> <span class="nx">myExport</span><span class="p">;</span>
</code></pre></div>
<p>Just like Synchronous Module Definitions, this requires code rewriting. The custom script loader will execute the rewritten code after all dependencies has been loaded.</p>
<p>This syntax has the added advantage of all the other ECMAScript.Next syntax that you can use as well.</p>
<h2 id="static-compilation">Static Compilation</h2>
<p>All systems except (A)synchronous Module Definitions and Infered Dependencies allow dynamic naming of modules based on some programmatic logic. I.e. you can call the require function with a variable require(selectedModule + ‘/subComponent’); Don’t do this. If you do, you can’t statically infer the dependencies and you can’t compile your script together without executing it. I’m going to assume the convention to always define a constant string.</p>
<p>Regardless of what syntax you use for defining dependencies, you can use a compiler and minifier to package them up. There are some benefits of using a custom script loader even in production because you can load scripts in parallell and from several servers. The best way is to use an Externally Defined Dependency Graph. That way, you can start loading files without first loading the files that dependent upon them.</p>
<p>Asynchronous Module Definition (AMD) by definition includes a fairly efficient script loader for the browser. You may be tempted to use it as-is in production. However, you should package related files together and at least minify them (as stated above). Therefore you’re already introducing a compilation step. That compilation step can be used to turn any other syntax into AMD or better yet, generate an Externally Defined Dependency Graph.</p>
<p>Because you should <strong>never</strong> put raw source code in a production web app, all syntaxes are equals in terms of production browser script loading. However, during development you may well want to load source code without a compilation step and different syntaxes can provide some advantages there.</p>
<h2 id="choosing-module-dependency-syntax">Choosing Module Dependency Syntax</h2>
<table class="compatibility" style="margin: auto;" border="0" cellspacing="1" cellpadding="0">
<tbody>
<tr class="top-head">
<td></td>
<th><span>Traditional</span></th>
<th><span>External</span></th>
<th><span>Comment</span></th>
<th><span>Inferred</span></th>
<th><span>CommonJS</span></th>
<th><span>Hybrid</span></th>
<th><span>Define Block</span></th>
<th><span>AMD</span></th>
<th><span>AMD Hybrid</span></th>
<th><span>ES.Next</span></th>
</tr>
<tr>
<th><span>Easily Maintained / Clean Syntax</span></th>
<td style="background: red;"></td>
<td style="background: red;"></td>
<td style="background: lime;"></td>
<td style="background: lime;"></td>
<td style="background: lime;"></td>
<td style="background: yellow;"></td>
<td style="background: lime;"></td>
<td style="background: lime;"></td>
<td style="background: red;"></td>
<td style="background: lime;"></td>
</tr>
<tr>
<th><span>Source Compatible with Traditional Model</span></th>
<td style="background: lime;"></td>
<td style="background: lime;"></td>
<td style="background: lime;"></td>
<td style="background: lime;"></td>
<td style="background: red;"></td>
<td style="background: lime;"></td>
<td style="background: red;"></td>
<td style="background: red;"></td>
<td style="background: lime;"></td>
<td style="background: red;"></td>
</tr>
<tr>
<th><span>Source Compatible with Node.js (Out-of-the-box)</span></th>
<td style="background: red;"></td>
<td style="background: red;"></td>
<td style="background: red;"></td>
<td style="background: red;"></td>
<td style="background: lime;"></td>
<td style="background: lime;"></td>
<td style="background: red;"></td>
<td style="background: red;"></td>
<td style="background: lime;"></td>
<td style="background: red;"></td>
</tr>
<tr>
<th><span>Analysis/Completion Tooling in Current IDEs</span></th>
<td style="background: lime;"></td>
<td style="background: lime;"></td>
<td style="background: lime;"></td>
<td style="background: lime;"></td>
<td style="background: red;"></td>
<td style="background: yellow;"></td>
<td style="background: red;"></td>
<td style="background: red;"></td>
<td style="background: yellow;"></td>
<td style="background: red;"></td>
</tr>
<tr>
<th><span>Debugging in WebKit</span></th>
<td style="background: lime;"></td>
<td style="background: lime;"></td>
<td style="background: lime;"></td>
<td style="background: lime;"></td>
<td style="background: red;"></td>
<td style="background: lime;"></td>
<td style="background: lime;"></td>
<td style="background: lime;"></td>
<td style="background: lime;"></td>
<td style="background: red;"></td>
</tr>
<tr>
<th><span>Debugging in Other Browsers</span></th>
<td style="background: lime;"></td>
<td style="background: lime;"></td>
<td style="background: lime;"></td>
<td style="background: lime;"></td>
<td style="background: yellow;"></td>
<td style="background: lime;"></td>
<td style="background: lime;"></td>
<td style="background: lime;"></td>
<td style="background: lime;"></td>
<td style="background: yellow;"></td>
</tr>
<tr>
<th><span>XSS Development Loader using CORS</span></th>
<td style="background: lime;"></td>
<td style="background: lime;"></td>
<td style="background: lime;"></td>
<td style="background: lime;"></td>
<td style="background: lime;"></td>
<td style="background: lime;"></td>
<td style="background: lime;"></td>
<td style="background: lime;"></td>
<td style="background: lime;"></td>
<td style="background: lime;"></td>
</tr>
<tr>
<th><span>XSS Development Loader without CORS</span></th>
<td style="background: lime;"></td>
<td style="background: yellow;"></td>
<td style="background: red;"></td>
<td style="background: red;"></td>
<td style="background: red;"></td>
<td style="background: red;"></td>
<td style="background: red;"></td>
<td style="background: lime;"></td>
<td style="background: lime;"></td>
<td style="background: red;"></td>
</tr>
<tr>
<th><span>file:// Protocol Development Loader</span></th>
<td style="background: lime;"></td>
<td style="background: lime;"></td>
<td style="background: red;"></td>
<td style="background: red;"></td>
<td style="background: red;"></td>
<td style="background: red;"></td>
<td style="background: red;"></td>
<td style="background: lime;"></td>
<td style="background: lime;"></td>
<td style="background: red;"></td>
</tr>
<tr>
<th><span>Fast Development Loader (Large Code Base, Cold Start)</span></th>
<td style="background: lime;"></td>
<td style="background: lime;"></td>
<td style="background: yellow;"></td>
<td style="background: red;"></td>
<td style="background: red;"></td>
<td style="background: red;"></td>
<td style="background: red;"></td>
<td style="background: lime;"></td>
<td style="background: lime;"></td>
<td style="background: red;"></td>
</tr>
<tr>
<th><span>Fast Production Loader</span></th>
<td style="background: lime;"></td>
<td style="background: lime;"></td>
<td style="background: lime;"></td>
<td style="background: lime;"></td>
<td style="background: lime;"></td>
<td style="background: lime;"></td>
<td style="background: lime;"></td>
<td style="background: lime;"></td>
<td style="background: lime;"></td>
<td style="background: lime;"></td>
</tr>
<tr>
<th><span>ES.Next Features (Classes, Destructuring etc.)</span></th>
<td style="background: red;"></td>
<td style="background: red;"></td>
<td style="background: red;"></td>
<td style="background: red;"></td>
<td style="background: red;"></td>
<td style="background: red;"></td>
<td style="background: red;"></td>
<td style="background: red;"></td>
<td style="background: red;"></td>
<td style="background: lime;"></td>
</tr>
</tbody>
</table>
<p>For me, maintainability and less boilerplate is important. Therefore the Traditional, External and AMD Hybrid model are not acceptable.</p>
<p>The main reason to choose the AMD syntax is if you have a very large code base. For those cases, pre-parsing may be slow during development. You can also cache pre-parsed dependencies so that only changed files are updated. This means the others can be fast the second time you run a large code base in development.</p>
<p>Another case to use AMD is if you need cross-site scripting dependency support in IE6/7 or against a third-party server without CORS support, during development. This is a much more esoteric use-case. I’ve had the need for it, but not during development. Only in production, and then we’re only using compiled code.</p>
<p>If you do need source compatibility with the traditional model and/or CommonJS, then choose either Inferred, CommonJS or the Hybrid model. This doesn’t require your users to download a separate bootstrapper to run your source.</p>
<p>If you accept that you require a bootstrapper. Then you can use the ES.Next model which gives you the added benefit of many new syntax features. There is a problem debugging code that is rewritten on-the-fly in some WebKit Inspector implementations. This is a passing problem and we’ll soon have full symbol mapped debugging for new source languages.</p>
<p>In fact, all the problems with the ES.Next model are likely a passing problems while the other’s remain. It’s going to be the model to replace them all in the long term.</p>
<p>If you have more module syntaxes or more arguments for one model over another, please drop a comment. A lot of times we base our decisions on intuition rather than reasoned arguments. I’ve tried to break it down without intuitive arguments (like “it feels cleaner or feels simpler”).</p>
<p><a href="http://blog.calyptus.eu/seb/2011/10/choosing-a-javascript-module-syntax/">Choosing JavaScript Module Dependency Syntax</a> was originally published by at <a href="http://blog.calyptus.eu">Calyptus</a> on October 08, 2011.</p>http://blog.calyptus.eu/seb/2011/09/simple-browser-icons2011-09-13T20:25:14Z2011-09-13T20:25:14Zhttp://blog.calyptus.euinfo@calyptus.eu<p>For the <a href="/seb/2011/09/csspathanimation/">conversion tool from SVG path to CSS animations</a> I needed a selector for vendor prefixes. I decided to use the browser’s logos.</p>
<p>High resolution and vector versions are readily available for all the browser logos. Unfortunately they don’t look very good in small icons. They’re also very colorful which tend to attract the eye of the user. Especially in a simple design.</p>
<p>I couldn’t find any simplified versions of the logos so I decided to roll my own. You’re welcome to use it if you want.</p>
<p>Here’s the result:</p>
<p><a href="/post-content/browser-logos.svg"><img class="alignnone" title="Browser Logos" src="/post-content/browser-logos.png" alt="" width="344" height="61" /></a></p>
<p><a href="/post-content/browser-logos.svg">Download browser-logos.svg</a></p>
<p><a href="http://blog.calyptus.eu/seb/2011/09/simple-browser-icons/">Simple Browser Icons</a> was originally published by at <a href="http://blog.calyptus.eu">Calyptus</a> on September 13, 2011.</p>http://blog.calyptus.eu/seb/2011/09/csspathanimation2011-09-12T14:16:14Z2011-09-12T14:16:14Zhttp://blog.calyptus.euinfo@calyptus.eu<p><strong><a href="http://csspathanimation.calyptus.eu/">Click Here to Test The Tool</a></strong></p>
<p>It’s not easy to generate keyframes for <a href="http://www.w3.org/TR/css3-animations/">CSS3 Animations</a>. <a href="http://csspathanimation.calyptus.eu/">This new CSS Path Animation tool</a> can help you convert SVG paths into CSS keyframes.</p>
<h2 id="svg">SVG</h2>
<p>Using SVG you can use the <a href="http://www.w3.org/TR/SVG/animate.html#AnimateMotionElement">animateMotion</a> element to move and rotate content following along a complex curved path. However, some vendors have turned away from SVG for animation and prefer CSS as the way to do animations. For HTML content it tends to be very invasive. Using SVG with foreign objects tend to be poorly implemented as well.</p>
<h2 id="javascript">JavaScript</h2>
<p>Most JavaScript frameworks for vector graphics (<a href="http://github.com/calyptus/art">including ART</a>) can do the same thing with JavaScript animations.</p>
<p>Some people think that CSS animations inherently provide better performance than JavaScript animations. This is not strictly true. It is the hardware compositing step that’s the important part. You can use JavaScript animations together with requestAnimationFrame and 3D transforms to trigger hardware compositing. You get the same level of performance as with CSS.</p>
<p>However, this won’t work in JavaScript-free environments, such as some banner ad sandboxes. CSS animations provide a nice separation of concerns and easy of use.</p>
<p>NOTE: If available, JavaScript is the preferred way to do this kind of animation efficiently. In a later post, I’ll show you how to do this with <a href="http://github.com/calyptus/art">ART</a>.</p>
<p>Another aspect is that the JavaScript to run this animation can be very small.</p>
<h2 id="css">CSS</h2>
<p>If you prefer <a href="http://www.w3.org/TR/css3-animations/">CSS Animations</a>, how do you create the keyframes to animate an object along a complex path?</p>
<p>If you only have straight lines, it is easy. Just apply a <a href="http://www.w3.org/TR/css3-2d-transforms/">CSS Transform</a> to the X and Y coordinates of each point along the path. The rotation can be adjusted based on the angle of the line if you need it.</p>
<h2 id="css-curve-animation">CSS Curve Animation</h2>
<p>Things get much more complicated when you consider smooth curved paths. I.e. bezier curves or arcs.</p>
<p>The typical way to handle curves, is to subdivide them into very small straight line segments. However, if you generate this many CSS keyframes, the document would be huge. That’s not an option. We need a better technique.</p>
<h2 id="the-arc-rotation-technique">The Arc Rotation Technique</h2>
<p>For arcs, you can adjust the transform origin and rotate the element around the arc’s center point. This is the optimal technique for simple arcs such as circles and ellipses. You can approximate bezier curves using several arc segments. This can generate very small files for simple curves, or very big files for complex curves. This means it’s unreliable for an automated conversion tool.</p>
<p>If you want to avoid rotating the element and only move along a path, then you need an additional nested element to counter act the rotation.</p>
<h2 id="the-timing-function-technique">The Timing Function Technique</h2>
<p>A future new best friend of mine pointed out to me that you can animate the X and Y axis using different easing functions. That way the timing will cause the animation to be a smooth curve. The CSS animation standard specifies a timing function based on a cubic bezier. This gives us a lot of flexibility to define an easing function.</p>
<p>The trick is in defining the timing function used. A simple implementation would specify bezier curve control points on the second and forth argument to the timing function, leaving the first and third argument at 0 and 1 respectively. This gives a perfect motion along the path. Unfortunately, it doesn’t move at a constant speed.</p>
<p>To get the animation to move across the curve at a constant speed we have to use a little black magic math to approximate the motion at constant speed.</p>
<p>Not all curves can be animated using this technique due to limitations placed on the timing function. If a complex curve is found, we split it into two, until we have curves that can be animated this way. This provides us with a generic technique that can be used reasonably well for any path.</p>
<h2 id="hardware-acceleration">Hardware Acceleration</h2>
<p>The arc technique uses a single CSS transform and is therefore easily hardware accelerated.</p>
<p>The timing function technique, requires different easing functions for each axis. That means that each axis needs it’s own animation.</p>
<p>Luckily we can apply multiple animations on a single element by animating the “left” and “top” properties independently. This is simple to use and non-invasive. However, this doesn’t currently provide the hardware compositing with sub-pixel accuracy, like the “transform” property does.</p>
<p>Unfortunately, we can’t animate each axis of a transform independently. The way to solve this is to use three nested elements and apply one transform for each element. This is a bit more invasive but provides the best visuals.</p>
<p>(UPDATE: The IE10 preview has a bug in nested transforms. At least in software mode. The previous method works well though and it’s hardware accelerated in IE. If the bug remains, I’ll update the converter to take this into account.)</p>
<h2 id="conclusion">Conclusion</h2>
<p>In <a href="http://csspathanimation.calyptus.eu/">the conversion tool</a> I use the timing function technique with three nested elements. This gives us a generic solution that is also hardware accelerated in current Webkit and Mozilla browsers.</p>
<p>The resulting CSS is very bloated. GZIP:ing does a lot to the size but it’s still fairly big. There are a few action points browser vendors and W3C people could do to help:</p>
<p>1) Hardware accelerate left/top absolute positioning with sub-pixel precision. IE9 did this from the start. This is where the full(er) hardware pipeline of IE9 shines. This would allow us to use a single element for this hack.</p>
<p>2) Drop vendor prefixes on specs that are far along. CSS keyframes can be very bloated and they have to be multiplied in full for each vendor.</p>
<p>3) Standardize something like SVG’s animateMotion, but for CSS. SVG isn’t less verbose but there’s a specialized element to handle this scenario. This same hack would be even more bloated in SVG.</p>
<p>A CSS rule could be used to name a SVG-style path. This name can then be referenced by the “animation-name” property to apply a transform to an element along a path. The current hack can be used to provide backward compatibility.</p>
<div class="highlight"><pre><code class="css"><span class="nf">#target</span> <span class="p">{</span>
<span class="n">animation</span><span class="o">:</span> <span class="n">myMotion</span> <span class="m">1s</span> <span class="n">infinite</span><span class="p">;</span>
<span class="p">}</span>
<span class="k">@motion</span> <span class="nt">myMotion</span> <span class="p">{</span>
<span class="nt">path</span><span class="o">:</span> <span class="s1">'m0,0 l100,100 ...'</span><span class="o">;</span>
<span class="p">}</span>
</code></pre></div>
<p>I think I’ll prototype something like this for LESS.js.</p>
<p><strong>UPDATE:</strong>
I’ve started <a href="http://lists.w3.org/Archives/Public/www-style/2011Sep/0295.html">a discussion about standardizing this pattern on the www-style mailing list</a>.</p>
<p>The current idea is to add a “motion” property that’s valid only within the context of keyframes.</p>
<div class="highlight"><pre><code class="css"><span class="nf">#target</span> <span class="p">{</span>
<span class="n">animation</span><span class="o">:</span> <span class="n">myMotion</span> <span class="m">1s</span> <span class="n">infinite</span><span class="p">;</span>
<span class="p">}</span>
<span class="k">@keyframes</span> <span class="nt">myMotion</span> <span class="p">{</span>
<span class="nt">0</span><span class="o">%</span> <span class="p">{</span>
<span class="n">motion</span><span class="o">:</span> <span class="n">m0</span><span class="o">,</span><span class="m">0</span> <span class="n">l100</span><span class="o">,</span><span class="m">100</span> <span class="o">...</span><span class="p">;</span>
<span class="p">}</span>
<span class="nt">50</span><span class="o">%</span> <span class="p">{</span>
<span class="n">motion</span><span class="o">:</span> <span class="n">M100</span><span class="o">,</span><span class="m">100</span> <span class="n">L0</span><span class="o">,</span><span class="m">0</span> <span class="o">...</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div>
<p><a href="http://blog.calyptus.eu/seb/2011/09/csspathanimation/">Convert a SVG Path to CSS Animation Keyframes</a> was originally published by at <a href="http://blog.calyptus.eu">Calyptus</a> on September 12, 2011.</p>http://blog.calyptus.eu/seb/2011/01/javascript-call-performance-just-inline-it2011-01-24T23:46:49Z2011-01-24T23:46:49Zhttp://blog.calyptus.euinfo@calyptus.eu<p>There are a couple of well-known micro optimization techniques when you need that extra speed within an expensive loop. It generally comes down to eliminating function calls and flattening the closure scope.</p>
<p>Intuitively you don’t want to bloat your download size with repeated code. You also want a clear separation of concerns to make your code understandable and managable. You want to separate common code into sub-routines (functions).</p>
<h2 id="intra-object-method-calls">Intra-object method calls</h2>
<p>I’ve done <a href="http://jsperf.com/classes-with-inner-function-calls-instantiation">a number</a> <a href="http://jsperf.com/classes-with-inner-function-calls">of tests</a> <a href="http://jsperf.com/classes-with-many-inner-function-calls">on repeated calls</a> within a single object. This is typically where an expensive loop would occur.</p>
<p>Now you might think that property look-ups are expensive and should be avoided. Since in a naive implementation they would be continued hash-table look-ups. However, traversing closure scopes and .call(bind) invocations tend to be more expensive in real world engines. Generally, you are better off invoking the function, by property name, on the same prototype - continuously. Engines optimize for this pattern. In the tests you can see that this is even more prevalent in recent optimizations.</p>
<h2 id="superbase-method-calls">Super/base method calls</h2>
<p>In Class-oriented/OOP patterns, you often see the need to override an inherited method. However, while doing so you wish to call the overridden (super/base/parent) function from within the new one.</p>
<p>The most common way of doing this is by storing a reference to the prototype object, then invoking the function using .call() or .apply() to set the “this” context to the right value. However, as I’ve shown, navigating a closure and invoking the function using .call() is more expensive than invoking prototype methods.</p>
<p>The benefit of storing a reference to the prototype is that you get a live property that can be monkey patched with a new super function on the fly. However, relying on this pattern can be tricky.</p>
<p>You’re better off having a module system that supports proper file ordering of monkey patches. Where monkey patches are applied before any child modules inherit from their parent. This will enable better optimization on engines that depend on declaration order to optimize their hidden classes (like V8). It also enables monkey patching of mixins that are copied. I.e. has no live inheritance (since JS doesn’t support multiple inheritance).</p>
<p>If we can assume that we don’t need on-the-fly updates of super functions, then we can optimize this further. Again, <a href="http://jsperf.com/super-call">my tests</a> show, that it’s faster to invoke a method on the same prototype than finding a function reference and invoking .call().</p>
<p>I’ve toyed with some <a href="https://gist.github.com/790455">syntax experiments</a> to make this prettier.</p>
<h2 id="just-inline-it">Just Inline It</h2>
<p><a href="http://twitter.com/steida">Daniel Steigerwald</a> pointed out that once you make the assumption that we don’t need on-the-fly monkey patches, you should just inline the code. This is always faster.</p>
<p>Let’s take a look at some nonsensical code. The function bar calls the function foo four times.</p>
<div class="highlight"><pre><code class="js"><span class="kd">function</span> <span class="nx">foo</span><span class="p">(</span><span class="nx">state</span><span class="p">){</span>
<span class="kd">var</span> <span class="nx">my</span> <span class="o">=</span> <span class="s1">'code'</span><span class="p">;</span>
<span class="k">for</span> <span class="p">(</span><span class="kd">var</span> <span class="nx">i</span> <span class="o">=</span> <span class="mi">0</span><span class="p">,</span> <span class="nx">l</span> <span class="o">=</span> <span class="nx">my</span><span class="p">.</span><span class="nx">length</span><span class="p">;</span> <span class="nx">i</span> <span class="o"><</span> <span class="nx">l</span><span class="p">;</span> <span class="nx">i</span><span class="o">++</span><span class="p">)</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">i</span> <span class="o">%</span> <span class="mi">10</span> <span class="o">==</span> <span class="mi">2</span><span class="p">)</span>
<span class="nx">state</span><span class="p">.</span><span class="nx">push</span><span class="p">(</span><span class="nx">my</span><span class="p">[</span><span class="nx">i</span><span class="p">]);</span>
<span class="k">else</span> <span class="k">if</span> <span class="p">(</span><span class="nx">i</span> <span class="o">==</span> <span class="mi">1</span><span class="p">)</span>
<span class="nx">state</span><span class="p">.</span><span class="nx">push</span><span class="p">(</span><span class="nx">my</span><span class="p">[</span><span class="mi">0</span><span class="p">]);</span>
<span class="k">else</span> <span class="k">if</span> <span class="p">(</span><span class="nx">l</span> <span class="o">==</span> <span class="mi">2</span> <span class="o">&&</span> <span class="nx">i</span> <span class="o">==</span> <span class="mi">3</span><span class="p">)</span>
<span class="nx">state</span><span class="p">.</span><span class="nx">push</span><span class="p">(</span><span class="s1">'foo'</span><span class="p">);</span>
<span class="k">else</span>
<span class="nx">state</span><span class="p">.</span><span class="nx">push</span><span class="p">(</span><span class="s1">'else'</span><span class="p">);</span>
<span class="k">return</span> <span class="nx">my</span><span class="p">;</span>
<span class="p">}</span>
<span class="kd">function</span> <span class="nx">bar</span><span class="p">(</span><span class="nx">condition</span><span class="p">,</span> <span class="nx">state</span><span class="p">){</span>
<span class="kd">var</span> <span class="nx">complexCondition</span> <span class="o">=</span> <span class="o">!</span><span class="nx">condition</span> <span class="o">&&</span> <span class="nx">state</span><span class="p">.</span><span class="nx">length</span> <span class="o">==</span> <span class="mi">3</span><span class="p">;</span>
<span class="kd">var</span> <span class="nx">value</span> <span class="o">=</span> <span class="nx">state</span><span class="p">[</span><span class="mi">0</span><span class="p">];</span>
<span class="k">while</span> <span class="p">(</span><span class="nx">condition</span><span class="p">){</span>
<span class="nx">value</span> <span class="o">=</span> <span class="nx">foo</span><span class="p">(</span><span class="nx">state</span><span class="p">);</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">complexCondition</span><span class="p">){</span>
<span class="nx">state</span><span class="p">.</span><span class="nx">push</span><span class="p">(</span><span class="s1">'someData'</span><span class="p">);</span>
<span class="nx">value</span> <span class="o">=</span> <span class="nx">foo</span><span class="p">(</span><span class="nx">state</span><span class="p">);</span>
<span class="p">}</span> <span class="k">else</span> <span class="k">if</span> <span class="p">(</span><span class="nx">value</span><span class="p">.</span><span class="nx">length</span> <span class="o">==</span> <span class="mi">5</span><span class="p">){</span>
<span class="nx">value</span> <span class="o">=</span> <span class="nx">foo</span><span class="p">([]);</span>
<span class="p">}</span> <span class="k">else</span> <span class="k">if</span> <span class="p">(</span><span class="nx">value</span> <span class="o">==</span> <span class="s1">'else'</span><span class="p">){</span>
<span class="nx">value</span> <span class="o">=</span> <span class="nx">foo</span><span class="p">(</span><span class="nx">state</span><span class="p">)</span>
<span class="nx">state</span><span class="p">.</span><span class="nx">push</span><span class="p">(</span><span class="s1">'test'</span><span class="p">);</span>
<span class="p">}</span>
<span class="nx">condition</span> <span class="o">=</span> <span class="p">(</span><span class="nx">value</span> <span class="o">==</span> <span class="s1">'foo'</span><span class="p">);</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div>
<p>This file weighs in at about 300 bytes gzipped.</p>
<p>If we instead inline (copy) the foo code into all four places in the bar function. We get something like this.</p>
<div class="highlight"><pre><code class="js"><span class="kd">function</span> <span class="nx">bar</span><span class="p">(</span><span class="nx">condition</span><span class="p">,</span> <span class="nx">state</span><span class="p">){</span>
<span class="kd">var</span> <span class="nx">complexCondition</span> <span class="o">=</span> <span class="o">!</span><span class="nx">condition</span> <span class="o">&&</span> <span class="nx">state</span><span class="p">.</span><span class="nx">length</span> <span class="o">==</span> <span class="mi">3</span><span class="p">;</span>
<span class="kd">var</span> <span class="nx">value</span> <span class="o">=</span> <span class="nx">state</span><span class="p">[</span><span class="mi">0</span><span class="p">];</span>
<span class="k">while</span> <span class="p">(</span><span class="nx">condition</span><span class="p">){</span>
<span class="kd">var</span> <span class="nx">stack</span> <span class="o">=</span> <span class="nx">state</span><span class="p">;</span>
<span class="kd">var</span> <span class="nx">my</span> <span class="o">=</span> <span class="s1">'code'</span><span class="p">;</span>
<span class="k">for</span> <span class="p">(</span><span class="kd">var</span> <span class="nx">i</span> <span class="o">=</span> <span class="mi">0</span><span class="p">,</span> <span class="nx">l</span> <span class="o">=</span> <span class="nx">my</span><span class="p">.</span><span class="nx">length</span><span class="p">;</span> <span class="nx">i</span> <span class="o"><</span> <span class="nx">l</span><span class="p">;</span> <span class="nx">i</span><span class="o">++</span><span class="p">)</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">i</span> <span class="o">%</span> <span class="mi">10</span> <span class="o">==</span> <span class="mi">2</span><span class="p">)</span>
<span class="nx">stack</span><span class="p">.</span><span class="nx">push</span><span class="p">(</span><span class="nx">my</span><span class="p">[</span><span class="nx">i</span><span class="p">]);</span>
<span class="k">else</span> <span class="k">if</span> <span class="p">(</span><span class="nx">i</span> <span class="o">==</span> <span class="mi">1</span><span class="p">)</span>
<span class="nx">stack</span><span class="p">.</span><span class="nx">push</span><span class="p">(</span><span class="nx">my</span><span class="p">[</span><span class="mi">0</span><span class="p">]);</span>
<span class="k">else</span> <span class="k">if</span> <span class="p">(</span><span class="nx">l</span> <span class="o">==</span> <span class="mi">2</span> <span class="o">&&</span> <span class="nx">i</span> <span class="o">==</span> <span class="mi">3</span><span class="p">)</span>
<span class="nx">stack</span><span class="p">.</span><span class="nx">push</span><span class="p">(</span><span class="s1">'foo'</span><span class="p">);</span>
<span class="k">else</span>
<span class="nx">stack</span><span class="p">.</span><span class="nx">push</span><span class="p">(</span><span class="s1">'else'</span><span class="p">);</span>
<span class="nx">value</span> <span class="o">=</span> <span class="nx">my</span><span class="p">;</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">complexCondition</span><span class="p">){</span>
<span class="nx">state</span><span class="p">.</span><span class="nx">push</span><span class="p">(</span><span class="s1">'someData'</span><span class="p">);</span>
<span class="kd">var</span> <span class="nx">stack</span> <span class="o">=</span> <span class="nx">state</span><span class="p">;</span>
<span class="kd">var</span> <span class="nx">my</span> <span class="o">=</span> <span class="s1">'code'</span><span class="p">;</span>
<span class="k">for</span> <span class="p">(</span><span class="kd">var</span> <span class="nx">i</span> <span class="o">=</span> <span class="mi">0</span><span class="p">,</span> <span class="nx">l</span> <span class="o">=</span> <span class="nx">my</span><span class="p">.</span><span class="nx">length</span><span class="p">;</span> <span class="nx">i</span> <span class="o"><</span> <span class="nx">l</span><span class="p">;</span> <span class="nx">i</span><span class="o">++</span><span class="p">)</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">i</span> <span class="o">%</span> <span class="mi">10</span> <span class="o">==</span> <span class="mi">2</span><span class="p">)</span>
<span class="nx">stack</span><span class="p">.</span><span class="nx">push</span><span class="p">(</span><span class="nx">my</span><span class="p">[</span><span class="nx">i</span><span class="p">]);</span>
<span class="k">else</span> <span class="k">if</span> <span class="p">(</span><span class="nx">i</span> <span class="o">==</span> <span class="mi">1</span><span class="p">)</span>
<span class="nx">stack</span><span class="p">.</span><span class="nx">push</span><span class="p">(</span><span class="nx">my</span><span class="p">[</span><span class="mi">0</span><span class="p">]);</span>
<span class="k">else</span> <span class="k">if</span> <span class="p">(</span><span class="nx">l</span> <span class="o">==</span> <span class="mi">2</span> <span class="o">&&</span> <span class="nx">i</span> <span class="o">==</span> <span class="mi">3</span><span class="p">)</span>
<span class="nx">stack</span><span class="p">.</span><span class="nx">push</span><span class="p">(</span><span class="s1">'foo'</span><span class="p">);</span>
<span class="k">else</span>
<span class="nx">stack</span><span class="p">.</span><span class="nx">push</span><span class="p">(</span><span class="s1">'else'</span><span class="p">);</span>
<span class="nx">value</span> <span class="o">=</span> <span class="nx">my</span><span class="p">;</span>
<span class="p">}</span> <span class="k">else</span> <span class="k">if</span> <span class="p">(</span><span class="nx">value</span><span class="p">.</span><span class="nx">length</span> <span class="o">==</span> <span class="mi">5</span><span class="p">){</span>
<span class="kd">var</span> <span class="nx">stack</span> <span class="o">=</span> <span class="p">[];</span>
<span class="kd">var</span> <span class="nx">my</span> <span class="o">=</span> <span class="s1">'code'</span><span class="p">;</span>
<span class="k">for</span> <span class="p">(</span><span class="kd">var</span> <span class="nx">i</span> <span class="o">=</span> <span class="mi">0</span><span class="p">,</span> <span class="nx">l</span> <span class="o">=</span> <span class="nx">my</span><span class="p">.</span><span class="nx">length</span><span class="p">;</span> <span class="nx">i</span> <span class="o"><</span> <span class="nx">l</span><span class="p">;</span> <span class="nx">i</span><span class="o">++</span><span class="p">)</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">i</span> <span class="o">%</span> <span class="mi">10</span> <span class="o">==</span> <span class="mi">2</span><span class="p">)</span>
<span class="nx">stack</span><span class="p">.</span><span class="nx">push</span><span class="p">(</span><span class="nx">my</span><span class="p">[</span><span class="nx">i</span><span class="p">]);</span>
<span class="k">else</span> <span class="k">if</span> <span class="p">(</span><span class="nx">i</span> <span class="o">==</span> <span class="mi">1</span><span class="p">)</span>
<span class="nx">stack</span><span class="p">.</span><span class="nx">push</span><span class="p">(</span><span class="nx">my</span><span class="p">[</span><span class="mi">0</span><span class="p">]);</span>
<span class="k">else</span> <span class="k">if</span> <span class="p">(</span><span class="nx">l</span> <span class="o">==</span> <span class="mi">2</span> <span class="o">&&</span> <span class="nx">i</span> <span class="o">==</span> <span class="mi">3</span><span class="p">)</span>
<span class="nx">stack</span><span class="p">.</span><span class="nx">push</span><span class="p">(</span><span class="s1">'foo'</span><span class="p">);</span>
<span class="k">else</span>
<span class="nx">stack</span><span class="p">.</span><span class="nx">push</span><span class="p">(</span><span class="s1">'else'</span><span class="p">);</span>
<span class="nx">value</span> <span class="o">=</span> <span class="nx">my</span><span class="p">;</span>
<span class="p">}</span> <span class="k">else</span> <span class="k">if</span> <span class="p">(</span><span class="nx">value</span> <span class="o">==</span> <span class="s1">'else'</span><span class="p">){</span>
<span class="kd">var</span> <span class="nx">stack</span> <span class="o">=</span> <span class="nx">state</span><span class="p">;</span>
<span class="kd">var</span> <span class="nx">my</span> <span class="o">=</span> <span class="s1">'code'</span><span class="p">;</span>
<span class="k">for</span> <span class="p">(</span><span class="kd">var</span> <span class="nx">i</span> <span class="o">=</span> <span class="mi">0</span><span class="p">,</span> <span class="nx">l</span> <span class="o">=</span> <span class="nx">my</span><span class="p">.</span><span class="nx">length</span><span class="p">;</span> <span class="nx">i</span> <span class="o"><</span> <span class="nx">l</span><span class="p">;</span> <span class="nx">i</span><span class="o">++</span><span class="p">)</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">i</span> <span class="o">%</span> <span class="mi">10</span> <span class="o">==</span> <span class="mi">2</span><span class="p">)</span>
<span class="nx">stack</span><span class="p">.</span><span class="nx">push</span><span class="p">(</span><span class="nx">my</span><span class="p">[</span><span class="nx">i</span><span class="p">]);</span>
<span class="k">else</span> <span class="k">if</span> <span class="p">(</span><span class="nx">i</span> <span class="o">==</span> <span class="mi">1</span><span class="p">)</span>
<span class="nx">stack</span><span class="p">.</span><span class="nx">push</span><span class="p">(</span><span class="nx">my</span><span class="p">[</span><span class="mi">0</span><span class="p">]);</span>
<span class="k">else</span> <span class="k">if</span> <span class="p">(</span><span class="nx">l</span> <span class="o">==</span> <span class="mi">2</span> <span class="o">&&</span> <span class="nx">i</span> <span class="o">==</span> <span class="mi">3</span><span class="p">)</span>
<span class="nx">stack</span><span class="p">.</span><span class="nx">push</span><span class="p">(</span><span class="s1">'foo'</span><span class="p">);</span>
<span class="k">else</span>
<span class="nx">stack</span><span class="p">.</span><span class="nx">push</span><span class="p">(</span><span class="s1">'else'</span><span class="p">);</span>
<span class="nx">value</span> <span class="o">=</span> <span class="nx">my</span><span class="p">;</span>
<span class="nx">state</span><span class="p">.</span><span class="nx">push</span><span class="p">(</span><span class="s1">'test'</span><span class="p">);</span>
<span class="p">}</span>
<span class="nx">condition</span> <span class="o">=</span> <span class="p">(</span><span class="nx">value</span> <span class="o">==</span> <span class="s1">'foo'</span><span class="p">);</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div>
<p>This file weighs about 290 bytes gzipped. Wait, what? We more than doubled the original file size but the result is smaller?</p>
<p>Yes, the GZIP compression uses the <a href="http://en.wikipedia.org/wiki/LZ77">LZ77</a> algorithm to find repeated byte sequences. This means that inlining your code may actually result in smaller download sizes since some plumbing code is removed.</p>
<p>Is it cheating to compare gzipped file size instead of originals? No. You should always send your static uncompressed content using the GZIP compression. It’s cheap. This is the real world baseline. You should optimize for this.</p>
<p>There are some quirks to look out for. Some minifiers rename variables inconsistently. This may result in inconsistent sequences. You should look out for this, since that would cause a larger file size.</p>
<p>The larger code base may cause a slightly slower start up, since the JS engine needs to parse and compile more code. This should be very very minimal though. Remember, we’re optimizing for tight loops here.</p>
<h2 id="js-to-js-compilers">JS-to-JS Compilers</h2>
<p>Our inlined code is ugly and unmaintainable. Luckily there are tools that can inline pretty source code for us. E.g. the <a href="http://closure-compiler.appspot.com">Google Closure Compiler</a>. Unfortunately the Closure Compiler doesn’t seem to inline functions that are called more than once. Neither does it use these optimization techniques for super calls. Perhaps there are settings that I’m not aware of.</p>
<p>Recursive functions may be difficult to inline but there are tools that can rewrite these and do tail call optimizations as well.</p>
<p>However, this is just a hint of the awesomeness that could be achieved by using a custom JS-to-JS compiler. More on that later.</p>
<p><a href="http://blog.calyptus.eu/seb/2011/01/javascript-call-performance-just-inline-it/">JavaScript Call Performance - Just Inline It</a> was originally published by at <a href="http://blog.calyptus.eu">Calyptus</a> on January 24, 2011.</p>http://blog.calyptus.eu/seb/2010/11/webgl-not-just-for-3d2010-11-18T04:11:28Z2010-11-18T04:11:28Zhttp://blog.calyptus.euinfo@calyptus.eu<p><strong>UPDATED: The demos are now updated to work around some bugs in the ANGLE implementation currently in Chrome 9 beta and Firefox 4 beta 8.</strong></p>
<p>Those of you who are <a href="http://twitter.com/sebmarkbage">following me on Twitter</a> know that I’ve been really getting into <a href="http://www.khronos.org/webgl/">WebGL</a> lately. Basically it’s 1:1 JavaScript bindings to OpenGL ES 2.0 including GLSL (OpenGL Shading Language). This will allow you to access the power of GPU hardware directly in your website.</p>
<p>Safari, Chrome and Firefox all have WebGL enabled in their latest betas/nightly builds. Opera is on the working group as well. Additionally modern iOS and Android devices have OpenGL ES support so mobile support is probably not far away. Firefox already has WebGL support on Maemo.</p>
<h2 id="microsoft-is-not-yet-on-board">Microsoft is not yet on board</h2>
<p>While Microsoft is constantly bragging about the fully hardware accelerated rendering in IE9 using DirectX. They’re the only one of the major players not yet in the WebGL working group. WebGL is supposedly designed so that it can be implemented on top of DirectX. This allows for stronger implementations on Windows where OpenGL support is poor. Mozilla and Google are already <a href="http://news.softpedia.com/news/Google-s-ANGLE-Project-WebGL-Based-on-DirectX-137892.shtml">doing this</a> in their implementations.</p>
<p>Microsoft obviously have a lot invested in DirectX and may not want to support an API so close to the competing OpenGL. However, given their recent effort to support standards I’m hopeful they’ll get on board for IE 10.</p>
<h2 id="fallbacks">Fallbacks</h2>
<p>Given that no browser supports WebGL yet, other than in beta versions. We will obviously need some fallback.</p>
<p><a href="http://mrdoob.com/">Mr. Doob</a> is doing some great progress on a 3D API with Canvas 2D and SVG fallback support - called <a href="https://github.com/mrdoob/three.js/">three.js</a>. However, I think this will be of limited use in most real-world scenarios.</p>
<p>For any proper 3D beyond gimmicks you’ll want to use hardware acceleration. The exception being data visualization.</p>
<p>Flash and Silverlight already have pixel shader support. This doesn’t help 3D much but they do help for 2D special effects etc. Although these are not hardware accelerated, they can run multi-core and they’re faster than anything you can do with 2D canvas.</p>
<p>Additionally Adobe has announced hardware accelerated 3D support through the <a href="http://labs.adobe.com/technologies/flash/molehill/">Molehill APIs</a>. I’m sure Microsoft is not far behind with Silverlight. Once these get out to IE users, we’ll have direct hardware capabilities on all the major platforms.</p>
<h2 id="abstractions">Abstractions</h2>
<p>WebGL is a horrible looking API. It’s intentionally a direct translation of the C interface. The intension is that people will provide abstractions on top of it. E.g. a general game engine or a data visualization API built on top of it.</p>
<p>Some have tried to make thin wrappers around it. Like <a href="http://mootools.net">MooTools</a> or <a href="http://jquery.com">jQuery</a> wraps the DOM in a thin API abstraction. I think that is a mistake. You really need to optimize for performance and to do that you need a higher level abstraction that works directly to the WebGL API.</p>
<p>I have no intension of making a thin GL wrapper nor a single generic 3D API. Although…</p>
<h2 id="its-not-just-for-3d">It’s not just for 3D</h2>
<p>Those of you <a href="http://github.com/calyptus">following me</a> on <a href="http://github.com/">GitHub</a> know that I’ve been doing a lot of work on the <a href="http://mootools.net/">MooTools</a> vector graphics library - <a href="http://github.com/calyptus/art">ART</a>. We already have great SVG and VML support. Next for me is to focus on high performance scenarios.</p>
<p>To start off, I’ve implemented two 2D use cases that are otherwise difficult to do properly using CPU only.</p>
<p><a href="http://labs.calyptus.eu/diffusioncurves/">Diffusion Curves</a></p>
<p><a href="http://labs.calyptus.eu/diffusioncurves/"><img alt="" src="http://blog.calyptus.eu/wp-content/uploads/diffusion-curves.jpg" title="Diffusion Curves" class="alignnone" width="512" height="512" /></a></p>
<p>A Diffusion Curve is a new vector primitive. To understand what they are and why they’re incredibly cool, I recommend watching <a href="http://artis.imag.fr/Publications/2008/OBWBTS08/">the video from the authors of the original research paper</a>. Implementing a rasterizer of Diffusion Curves involves a diffusion and blur step that are both very computationally expensive. However, using a clever implementation you can take advantage of GPU hardware acceleration to get acceptable performance. There have been talks about introducing diffusion curves in SVG 2.0. But until that gets into browsers, we can already use it through the power of WebGL.</p>
<p><a href="http://labs.calyptus.eu/pagecurl/">Page Curl Effect</a></p>
<p><a href="http://labs.calyptus.eu/pagecurl/"><img alt="" src="http://blog.calyptus.eu/wp-content/uploads/page-curl.jpg" title="Page Curl" class="alignnone" width="512" height="512" /></a></p>
<p>A page curling effect can be implemented many different ways. However, to get a smooth realistic effect on dynamic content you really need some acceleration. This applies to many special effects that you can apply to 2D content.</p>
<p>Safari doesn’t implement SVG filters yet. Such filters can be implemented in terms of WebGL and extended further to advanced blending effects and crazy transforms.</p>
<h2 id="computation">Computation</h2>
<p>WebGL is a great complement to JavaScript. JavaScript can be used to implement business logic and UI components while WebGL is used for computationally expensive tasks. Many of the use cases of NaCi (Native Client) goes out the window. You can even use WebGL to performance non-graphics tasks such as audio signal analysis/transformation.</p>
<p>Now all we need is a JavaScript based OpenCL like API for running GPU accelerated workers. There are already several projects doing <a href="http://mathema.tician.de/software/pyopencl">something similar</a> with Python.</p>
<h2 id="learning-more-about-webgl">Learning more about WebGL</h2>
<p>The best way to learn WebGL is to head over to <a href="http://learningwebgl.com/">Learning WebGL</a> and check out <a href="http://twitter.com/gpjt">Giles Thomas</a>’ excellent introductory lessons.</p>
<h2 id="coming-up">Coming Up</h2>
<ul>
<li>Lessons Learned - WebGL</li>
<li>Lessons Learned - Diffusion Curves</li>
</ul>
<p><a href="http://blog.calyptus.eu/seb/2010/11/webgl-not-just-for-3d/">WebGL - Not just for 3D</a> was originally published by at <a href="http://blog.calyptus.eu">Calyptus</a> on November 18, 2010.</p>http://blog.calyptus.eu/seb/2010/11/javascript-proxies-leaky-this2010-11-17T23:31:17Z2010-11-17T23:31:17Zhttp://blog.calyptus.euinfo@calyptus.eu<p><em>You can introduce AOP interception patterns by either changing the original instance or by using additional proxy objects. The later can introduce confusion among instances if not treated correctly. Skip to “<a href="#leaky-this">Leaky this</a>” if you’re already familiar with proxies in ECMAScript Harmony.</em></p>
<h2 id="rewriting-vs-wrapping">Rewriting vs. Wrapping</h2>
<p>Intercepting normal program flows using aspect oriented and other meta-programming patterns can be used for a variety of advanced tricks. Object-relational mapping, logging, security, customized APIs etc.</p>
<p>You can do this by rewriting the methods of the original class/prototype/instance, effectively providing a single instance. You can also do this by providing wrapper objects. That way you have two or more instances.</p>
<p>Rewriting instances or prototypes is easy to do in JS by overriding the functions on existing objects with your own wrapper functions. Although it’s not always possible on the magical host objects. It can also be a problem when these objects have different requirements in various contexts. Such as in a secured sandbox vs. host environment.</p>
<p>Wrappers allow you to leave the original object intact while providing the wrapper to a specific context as if it’s the real object.</p>
<h2 id="wrappers-in-ecmascript-5">Wrappers in ECMAScript 5</h2>
<p>In <a href="http://www.ecma-international.org/publications/files/ECMA-ST/ECMA-262.pdf">ECMAScript 5</a> you can already do seemingly perfect wrappers by getting all the properties of an object. Then you define those properties on the wrapper - called a proxy. These are custom getters and setters that delegate to the original object - called the subject.</p>
<div class="highlight"><pre><code class="js"><span class="kd">var</span> <span class="nx">subject</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">Node</span><span class="p">();</span>
<span class="kd">var</span> <span class="nx">proxy</span> <span class="o">=</span> <span class="nb">Object</span><span class="p">.</span><span class="nx">create</span><span class="p">();</span>
<span class="nb">Object</span><span class="p">.</span><span class="nx">getOwnPropertyNames</span><span class="p">(</span><span class="nx">subject</span><span class="p">).</span><span class="nx">forEach</span><span class="p">(</span><span class="kd">function</span><span class="p">(</span><span class="nx">name</span><span class="p">){</span>
<span class="nb">Object</span><span class="p">.</span><span class="nx">defineProperty</span><span class="p">(</span><span class="nx">proxy</span><span class="p">,</span> <span class="nx">name</span><span class="p">,</span> <span class="p">{</span>
<span class="nx">get</span><span class="o">:</span> <span class="kd">function</span><span class="p">(</span><span class="nx">name</span><span class="p">){</span>
<span class="c1">// interception code</span>
<span class="k">return</span> <span class="nx">subject</span><span class="p">[</span><span class="nx">name</span><span class="p">];</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="p">});</span>
</code></pre></div>
<p>(For simplicity I’ve left out the prototype, setters, enumerations rules etc. Getters are enough to illustrate my point.)</p>
<p>These are fine for frozen objects. The problem is that they don’t represent later changes to the subject. E.g. adding or deleting properties. Additionally, property changes applied to the proxy aren’t propagated back into the subject.</p>
<h2 id="proxies-in-ecmascript-harmony">Proxies in ECMAScript Harmony</h2>
<p>It looks like the next version of ECMAScript will get support for <a href="http://wiki.ecmascript.org/doku.php?id=harmony:proxies">proxy object</a> that can delegate all operations dynamically - using so called “catch-all” functions.</p>
<div class="highlight"><pre><code class="js"><span class="kd">var</span> <span class="nx">subject</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">Node</span><span class="p">();</span>
<span class="kd">var</span> <span class="nx">proxy</span> <span class="o">=</span> <span class="nx">Proxy</span><span class="p">.</span><span class="nx">create</span><span class="p">({</span>
<span class="nx">get</span><span class="o">:</span> <span class="kd">function</span><span class="p">(</span><span class="nx">receiver</span><span class="p">,</span> <span class="nx">name</span><span class="p">){</span>
<span class="c1">// interception code</span>
<span class="k">return</span> <span class="nx">subject</span><span class="p">[</span><span class="nx">name</span><span class="p">];</span>
<span class="p">}</span>
<span class="p">});</span>
</code></pre></div>
<p>When we’re reading a property from the proxy instance, our get-function gets called with the receiver (the proxy instance itself) and the name of the property that should be loaded. We delegate this request to the subject by reading the same property. We can insert any intercepting code as we choose.</p>
<p>This is more efficient and allows for dynamic evaluation of intercepting code. Many ORM patterns - such as <a href="http://en.wikipedia.org/wiki/Active_record">Active Record</a> - often use this technique in dynamic environments.</p>
<p><a name="leaky-this"></a><strong>“Leaky this”</strong></p>
<p>However, a problem occurs when these two instances get confused. Imagine this scenario:</p>
<div class="highlight"><pre><code class="js"><span class="kd">var</span> <span class="nx">Node</span> <span class="o">=</span> <span class="kd">function</span><span class="p">(){</span>
<span class="kd">var</span> <span class="nx">self</span> <span class="o">=</span> <span class="k">this</span><span class="p">,</span> <span class="nx">children</span> <span class="o">=</span> <span class="p">[];</span>
<span class="nx">self</span><span class="p">.</span><span class="nx">addChild</span> <span class="o">=</span> <span class="kd">function</span><span class="p">(</span><span class="nx">child</span><span class="p">){</span>
<span class="nx">children</span><span class="p">.</span><span class="nx">push</span><span class="p">(</span><span class="nx">child</span><span class="p">);</span>
<span class="nx">child</span><span class="p">.</span><span class="nx">parent</span> <span class="o">=</span> <span class="nx">self</span><span class="p">;</span>
<span class="k">return</span> <span class="nx">self</span><span class="p">;</span>
<span class="p">};</span>
<span class="p">};</span>
</code></pre></div>
<p>If we wrap an instance of Node in a proxy. Then call:</p>
<div class="highlight"><pre><code class="js"><span class="nx">proxy</span><span class="p">.</span><span class="nx">addChild</span><span class="p">(</span><span class="nx">child</span><span class="p">).</span><span class="nx">addChild</span><span class="p">(</span><span class="nx">childOther</span><span class="p">);</span>
</code></pre></div>
<p>We first call addChild through the proxy instance. Our child.parent property is set to the subject instance. The subject instance is returned and we next call addChild directly on the subject instance. Bypassing the proxy.</p>
<p>This is a well-known problem in languages that strongly binds the “this” keyword to a specific instance. <a href="http://rogeralsing.com/">Roger Alsing</a> calls this “<a href="http://www.puzzleframework.com/forum/thread.aspx?Thread=828">leaky this</a>” which I find to be an appropriate term since the abstraction is leaking.</p>
<p>However, imagine this JavaScript code:</p>
<div class="highlight"><pre><code class="js"><span class="kd">var</span> <span class="nx">Node</span> <span class="o">=</span> <span class="kd">function</span><span class="p">(){</span>
<span class="k">this</span><span class="p">.</span><span class="nx">children</span> <span class="o">=</span> <span class="p">[];</span>
<span class="p">};</span>
<span class="nx">Node</span><span class="p">.</span><span class="nx">prototype</span><span class="p">.</span><span class="nx">addChild</span> <span class="o">=</span> <span class="kd">function</span><span class="p">(</span><span class="nx">child</span><span class="p">){</span>
<span class="k">this</span><span class="p">.</span><span class="nx">children</span><span class="p">.</span><span class="nx">push</span><span class="p">(</span><span class="nx">child</span><span class="p">);</span>
<span class="nx">child</span><span class="p">.</span><span class="nx">parent</span> <span class="o">=</span> <span class="k">this</span><span class="p">;</span>
<span class="k">return</span> <span class="k">this</span><span class="p">;</span>
<span class="p">};</span>
</code></pre></div>
<p>The addChild function now references the “this” keyword. If we repeat the same call in this scenario we get a different result. We call addChild while passing the proxy as the “this” reference. We set the parent property of child to the proxy instance. The proxy instance is returned, and the process is repeated on the second call. This shows the beauty of not having the “this” keyword bound to specific instances.</p>
<p>Although, sometimes you don’t want to use the proxy instance in internal functions. You could use this hybrid to use the subject for whatever use internally, while exposing any proxies externally:</p>
<div class="highlight"><pre><code class="js"><span class="kd">var</span> <span class="nx">Node</span> <span class="o">=</span> <span class="kd">function</span><span class="p">(){</span>
<span class="kd">var</span> <span class="nx">self</span> <span class="o">=</span> <span class="k">this</span><span class="p">,</span> <span class="nx">children</span> <span class="o">=</span> <span class="p">[];</span>
<span class="nx">self</span><span class="p">.</span><span class="nx">addChild</span> <span class="o">=</span> <span class="kd">function</span><span class="p">(</span><span class="nx">child</span><span class="p">){</span>
<span class="nx">children</span><span class="p">.</span><span class="nx">push</span><span class="p">(</span><span class="nx">child</span><span class="p">);</span>
<span class="nx">child</span><span class="p">.</span><span class="nx">parent</span> <span class="o">=</span> <span class="nx">self</span><span class="p">;</span>
<span class="k">return</span> <span class="k">this</span><span class="p">;</span>
<span class="p">};</span>
<span class="p">};</span>
</code></pre></div>
<h2 id="conclusion">Conclusion</h2>
<p><a href="http://brendaneich.com/">Brendan Eich</a> <a href="http://brendaneich.com/2010/11/proxy-inception/">points out</a> that unaware consumers of a proxy should be able to treat it just as any other object. This is only true if the provider of both the proxy and the subject have carefully considered the implications of two instances and made sure that no leaking occurs. Code that can be wrapped needs to carefully consider what instance it is currently working with.</p>
<p>Not all code can be wrapped using a proxy indiscriminately.</p>
<p><a href="http://blog.calyptus.eu/seb/2010/11/javascript-proxies-leaky-this/">JavaScript Proxies - "Leaky this"</a> was originally published by at <a href="http://blog.calyptus.eu">Calyptus</a> on November 17, 2010.</p>http://blog.calyptus.eu/seb/2010/03/rx-for-mootools2010-03-07T21:08:00Z2010-03-07T21:08:00Zhttp://blog.calyptus.euinfo@calyptus.eu<p><em>This is a follow up to my earlier post about the <a href="http://blog.calyptus.eu/seb/2010/03/rx-for-javascript/">Reactive Extensions (Rx) for JavaScript</a> by Microsoft’s DevLabs. This is also in response to Matthew Podwysocki’s post on <a href="http://weblogs.asp.net/podwysocki/archive/2010/03/05/introduction-to-the-reactive-extensions-for-javascript-jquery-integration.aspx">jQuery integration</a> (which deserves some credit for putting it out there).</em></p>
<p><em>I will assume some familiarity with Rx.</em></p>
<p>Just like any other DOM library, MooTools has a way of working with native and custom passive DOM events. We can easily give Element object and the Elements collection a method to provide these events as “Observables”. In the jQuery example the method name “ToObservable” was added to the jQuery object, accepting an event type parameter, which was my initial reaction as well. But I’m going to call mine <strong>getEvent</strong> as in “<em>getting a stream of events given the event type</em>”.</p>
<div class="highlight"><pre><code class="js"><span class="kd">var</span> <span class="nx">observableFromEvent</span> <span class="o">=</span> <span class="kd">function</span><span class="p">(</span><span class="nx">type</span><span class="p">){</span>
<span class="kd">var</span> <span class="nx">self</span> <span class="o">=</span> <span class="k">this</span><span class="p">;</span>
<span class="k">return</span> <span class="nx">Rx</span><span class="p">.</span><span class="nx">Observable</span><span class="p">.</span><span class="nx">Create</span><span class="p">(</span><span class="kd">function</span><span class="p">(</span><span class="nx">observer</span><span class="p">){</span>
<span class="kd">var</span> <span class="nx">fn</span> <span class="o">=</span> <span class="kd">function</span><span class="p">(</span><span class="nx">event</span><span class="p">){</span>
<span class="nx">observer</span><span class="p">.</span><span class="nx">OnNext</span><span class="p">(</span><span class="nx">event</span><span class="p">);</span>
<span class="p">};</span>
<span class="nx">self</span><span class="p">.</span><span class="nx">addEvent</span><span class="p">(</span><span class="nx">type</span><span class="p">,</span> <span class="nx">fn</span><span class="p">);</span>
<span class="k">return</span> <span class="kd">function</span><span class="p">(){</span>
<span class="nx">self</span><span class="p">.</span><span class="nx">removeEvent</span><span class="p">(</span><span class="nx">type</span><span class="p">,</span> <span class="nx">fn</span><span class="p">);</span>
<span class="p">};</span>
<span class="p">});</span>
<span class="p">};</span>
<span class="nx">Window</span><span class="p">.</span><span class="nx">implement</span><span class="p">(</span><span class="s1">'getEvent'</span><span class="p">,</span> <span class="nx">observableFromEvent</span><span class="p">);</span>
<span class="nx">Document</span><span class="p">.</span><span class="nx">implement</span><span class="p">(</span><span class="s1">'getEvent'</span><span class="p">,</span> <span class="nx">observableFromEvent</span><span class="p">);</span>
<span class="nx">Element</span><span class="p">.</span><span class="nx">implement</span><span class="p">(</span><span class="s1">'getEvent'</span><span class="p">,</span> <span class="nx">observableFromEvent</span><span class="p">);</span>
<span class="nx">Elements</span><span class="p">.</span><span class="nx">implement</span><span class="p">(</span><span class="s1">'getEvent'</span><span class="p">,</span> <span class="nx">observableFromEvent</span><span class="p">);</span>
</code></pre></div>
<p><em>These are infinite Observables but we could also make .destroy() trigger onComplete to make them finite as well.</em></p>
<h2 id="flickables-example">Flickables Example</h2>
<p>Instead of the canonical Drag and Drop example I thought I show a twist. Let’s say we want to listen to a mouse flick. The mouse position have to move over 100px in 200ms. Then we want the angle of the flick.</p>
<div class="highlight"><pre><code class="js"><span class="kd">var</span> <span class="nx">angleFromPosition</span> <span class="o">=</span> <span class="kd">function</span><span class="p">(</span><span class="nx">position</span><span class="p">,</span> <span class="nx">center</span><span class="p">){</span>
<span class="kd">var</span> <span class="nx">diffX</span> <span class="o">=</span> <span class="nx">position</span><span class="p">.</span><span class="nx">x</span> <span class="o">-</span> <span class="nx">center</span><span class="p">.</span><span class="nx">x</span><span class="p">,</span> <span class="nx">diffY</span> <span class="o">=</span> <span class="nx">position</span><span class="p">.</span><span class="nx">y</span> <span class="o">-</span> <span class="nx">center</span><span class="p">.</span><span class="nx">y</span><span class="p">;</span>
<span class="kd">var</span> <span class="nx">distance</span> <span class="o">=</span> <span class="nb">Math</span><span class="p">.</span><span class="nx">sqrt</span><span class="p">(</span><span class="nx">diffX</span> <span class="o">*</span> <span class="nx">diffX</span> <span class="o">+</span> <span class="nx">diffY</span> <span class="o">*</span> <span class="nx">diffY</span><span class="p">);</span>
<span class="kd">var</span> <span class="nx">angle</span> <span class="o">=</span> <span class="nb">Math</span><span class="p">.</span><span class="nx">atan2</span><span class="p">(</span><span class="nx">diffY</span> <span class="o">+</span> <span class="nx">distance</span><span class="p">,</span> <span class="nx">diffX</span><span class="p">)</span> <span class="o">*</span> <span class="mi">360</span> <span class="o">/</span> <span class="nb">Math</span><span class="p">.</span><span class="nx">PI</span><span class="p">;</span>
<span class="k">return</span> <span class="p">{</span> <span class="nx">distance</span><span class="o">:</span> <span class="nx">distance</span><span class="p">,</span> <span class="nx">angle</span><span class="o">:</span> <span class="nx">angle</span> <span class="p">};</span>
<span class="p">};</span>
<span class="kd">var</span> <span class="nx">distanceReached</span> <span class="o">=</span> <span class="kd">function</span><span class="p">(</span><span class="nx">angle</span><span class="p">){</span> <span class="k">return</span> <span class="nx">angle</span><span class="p">.</span><span class="nx">distance</span> <span class="o">></span> <span class="mi">100</span><span class="p">;</span> <span class="p">};</span>
<span class="kd">var</span> <span class="nx">timeLimit</span> <span class="o">=</span> <span class="nx">Rx</span><span class="p">.</span><span class="nx">Observable</span><span class="p">.</span><span class="nx">Timer</span><span class="p">(</span><span class="mi">200</span><span class="p">);</span>
<span class="kd">var</span> <span class="nx">mousePositions</span> <span class="o">=</span> <span class="nb">document</span><span class="p">.</span><span class="nx">getEvent</span><span class="p">(</span><span class="s1">'mousemove'</span><span class="p">)</span>
<span class="p">.</span><span class="nx">Select</span><span class="p">(</span><span class="kd">function</span><span class="p">(</span><span class="nx">event</span><span class="p">){</span> <span class="k">return</span> <span class="nx">event</span><span class="p">.</span><span class="nx">page</span><span class="p">;</span> <span class="p">});</span>
<span class="kd">var</span> <span class="nx">flicks</span> <span class="o">=</span> <span class="nb">document</span><span class="p">.</span><span class="nx">getElements</span><span class="p">(</span><span class="s1">'.flickable'</span><span class="p">)</span>
<span class="p">.</span><span class="nx">getEvent</span><span class="p">(</span><span class="s1">'mousedown'</span><span class="p">)</span>
<span class="p">.</span><span class="nx">SelectMany</span><span class="p">(</span><span class="kd">function</span><span class="p">(</span><span class="nx">event</span><span class="p">){</span>
<span class="k">return</span> <span class="nx">mousePositions</span>
<span class="p">.</span><span class="nx">Select</span><span class="p">(</span><span class="nx">angleFromPosition</span><span class="p">.</span><span class="nx">bindWithEvent</span><span class="p">(</span><span class="kc">null</span><span class="p">,</span> <span class="nx">event</span><span class="p">.</span><span class="nx">page</span><span class="p">))</span>
<span class="p">.</span><span class="nx">TakeUntil</span><span class="p">(</span><span class="nb">document</span><span class="p">.</span><span class="nx">getEvent</span><span class="p">(</span><span class="s1">'mouseup'</span><span class="p">))</span>
<span class="p">.</span><span class="nx">TakeUntil</span><span class="p">(</span><span class="nx">timeLimit</span><span class="p">)</span>
<span class="p">.</span><span class="nx">Where</span><span class="p">(</span><span class="nx">distanceReached</span><span class="p">)</span>
<span class="p">.</span><span class="nx">Take</span><span class="p">(</span><span class="mi">1</span><span class="p">);</span>
<span class="p">});</span>
<span class="c1">// ...</span>
<span class="nx">flicks</span><span class="p">.</span><span class="nx">Subscribe</span><span class="p">(</span><span class="kd">function</span><span class="p">(</span><span class="nx">current</span><span class="p">){</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="s1">'Flicked in direction: '</span> <span class="o">+</span> <span class="nx">current</span><span class="p">.</span><span class="nx">angle</span> <span class="o">+</span> <span class="s1">'°'</span><span class="p">);</span>
<span class="p">});</span>
</code></pre></div>
<h2 id="events-mixin">Events Mixin</h2>
<p>MooTools has a very strong benefit compared to many other libraries. The publish/subscribe pattern is made explicit even for custom classes, using the Events mixin. By implement our “getEvent” method on this class we can use Rx on all custom MooTools classes that provide passive events.</p>
<div class="highlight"><pre><code class="js"><span class="nx">Events</span><span class="p">.</span><span class="nx">implement</span><span class="p">(</span><span class="s1">'getEvent'</span><span class="p">,</span> <span class="nx">observableFromEvent</span><span class="p">);</span>
</code></pre></div>
<h2 id="side-effects">Side-effects</h2>
<p>Rx allows for the act of subscribing to an event to trigger an action/side-effect. Think of the Request object for example. You can use the act of subscribing to it, to issue a HTTP request. Then we can turn the subsequent events like success and failure into the Observable interface. This means that Request is a complete Observable in it self. This is what I was saving the conversion name <strong>toObservable</strong> for.</p>
<div class="highlight"><pre><code class="js"><span class="nx">Request</span><span class="p">.</span><span class="nx">implement</span><span class="p">({</span>
<span class="nx">toObservable</span><span class="o">:</span> <span class="kd">function</span><span class="p">(){</span>
<span class="kd">var</span> <span class="nx">self</span> <span class="o">=</span> <span class="k">this</span><span class="p">;</span>
<span class="k">return</span> <span class="nx">Rx</span><span class="p">.</span><span class="nx">Observable</span><span class="p">.</span><span class="nx">create</span><span class="p">(</span><span class="kd">function</span><span class="p">(</span><span class="nx">observer</span><span class="p">){</span>
<span class="kd">var</span> <span class="nx">listeners</span> <span class="o">=</span> <span class="p">{</span>
<span class="nx">success</span><span class="o">:</span> <span class="kd">function</span><span class="p">(</span><span class="nx">result</span><span class="p">){</span>
<span class="nx">self</span><span class="p">.</span><span class="nx">removeEvents</span><span class="p">(</span><span class="nx">listeners</span><span class="p">);</span>
<span class="nx">observer</span><span class="p">.</span><span class="nx">OnNext</span><span class="p">(</span><span class="nx">result</span><span class="p">);</span>
<span class="nx">observer</span><span class="p">.</span><span class="nx">OnCompleted</span><span class="p">();</span>
<span class="p">},</span>
<span class="nx">cancel</span><span class="o">:</span> <span class="kd">function</span><span class="p">(){</span>
<span class="nx">self</span><span class="p">.</span><span class="nx">removeEvents</span><span class="p">(</span><span class="nx">listeners</span><span class="p">);</span>
<span class="nx">observer</span><span class="p">.</span><span class="nx">OnCompleted</span><span class="p">();</span>
<span class="p">},</span>
<span class="nx">failure</span><span class="o">:</span> <span class="kd">function</span><span class="p">(</span><span class="nx">xhr</span><span class="p">){</span>
<span class="nx">self</span><span class="p">.</span><span class="nx">removeEvents</span><span class="p">(</span><span class="nx">listeners</span><span class="p">);</span>
<span class="nx">observer</span><span class="p">.</span><span class="nx">OnError</span><span class="p">(</span><span class="nx">xhr</span><span class="p">);</span>
<span class="p">}</span>
<span class="p">};</span>
<span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="nx">self</span><span class="p">.</span><span class="nx">running</span> <span class="o">||</span> <span class="nx">self</span><span class="p">.</span><span class="nx">options</span><span class="p">.</span><span class="nx">link</span> <span class="o">==</span> <span class="s1">'cancel'</span><span class="p">){</span>
<span class="nx">self</span><span class="p">.</span><span class="nx">addEvents</span><span class="p">(</span><span class="nx">listeners</span><span class="p">).</span><span class="nx">send</span><span class="p">();</span>
<span class="k">return</span> <span class="kd">function</span><span class="p">(){</span>
<span class="nx">self</span><span class="p">.</span><span class="nx">removeEvents</span><span class="p">(</span><span class="nx">listeners</span><span class="p">).</span><span class="nx">cancel</span><span class="p">();</span>
<span class="p">};</span>
<span class="p">}</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">self</span><span class="p">.</span><span class="nx">options</span><span class="p">.</span><span class="nx">link</span> <span class="o">==</span> <span class="s1">'chain'</span><span class="p">){</span>
<span class="kd">var</span> <span class="nx">disposed</span><span class="p">,</span> <span class="nx">running</span><span class="p">;</span>
<span class="nx">self</span><span class="p">.</span><span class="nx">chain</span><span class="p">(</span><span class="kd">function</span><span class="p">(){</span>
<span class="nx">running</span> <span class="o">=</span> <span class="kc">true</span><span class="p">;</span>
<span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="nx">disposed</span><span class="p">)</span> <span class="nx">self</span><span class="p">.</span><span class="nx">addEvents</span><span class="p">(</span><span class="nx">listeners</span><span class="p">).</span><span class="nx">send</span><span class="p">();</span>
<span class="p">});</span>
<span class="k">return</span> <span class="kd">function</span><span class="p">(){</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">running</span><span class="p">)</span> <span class="nx">self</span><span class="p">.</span><span class="nx">removeEvents</span><span class="p">(</span><span class="nx">listeners</span><span class="p">).</span><span class="nx">cancel</span><span class="p">();</span>
<span class="nx">disposed</span> <span class="o">=</span> <span class="kc">true</span><span class="p">;</span>
<span class="p">};</span>
<span class="p">}</span>
<span class="nx">observer</span><span class="p">.</span><span class="nx">OnComplete</span><span class="p">();</span>
<span class="k">return</span> <span class="kd">function</span><span class="p">(){};</span>
<span class="p">});</span>
<span class="p">}</span>
<span class="p">});</span>
</code></pre></div>
<p>This creates a finite stream of events - only one response to be exact. However, since the act of subscribing to it causes it to occur we can have it trigger repeatedly as part of a composite stream of events.</p>
<p>MooTools’ Fx provides a similar concept but slightly different. Even though we don’t get an event for each tick, we still get an asynchronous complete event. This means we can insert Fx as part of a composite stream of events.</p>
<p>Fx also requires from/to arguments to be passed at the start. So we add the option “defaultArgs” to allow us to pass those at initialization.</p>
<div class="highlight"><pre><code class="js"><span class="nx">Fx</span><span class="p">.</span><span class="nx">implement</span><span class="p">({</span>
<span class="nx">toObservable</span><span class="o">:</span> <span class="kd">function</span><span class="p">(){</span>
<span class="kd">var</span> <span class="nx">self</span> <span class="o">=</span> <span class="k">this</span><span class="p">;</span>
<span class="k">return</span> <span class="nx">Rx</span><span class="p">.</span><span class="nx">Observable</span><span class="p">.</span><span class="nx">create</span><span class="p">(</span><span class="kd">function</span><span class="p">(</span><span class="nx">observer</span><span class="p">){</span>
<span class="kd">var</span> <span class="nx">listeners</span> <span class="o">=</span> <span class="p">{</span>
<span class="nx">complete</span><span class="o">:</span> <span class="kd">function</span><span class="p">(){</span>
<span class="nx">self</span><span class="p">.</span><span class="nx">removeEvents</span><span class="p">(</span><span class="nx">listeners</span><span class="p">);</span>
<span class="nx">observer</span><span class="p">.</span><span class="nx">OnCompleted</span><span class="p">();</span>
<span class="p">},</span>
<span class="nx">cancel</span><span class="o">:</span> <span class="kd">function</span><span class="p">(){</span>
<span class="nx">self</span><span class="p">.</span><span class="nx">removeEvents</span><span class="p">(</span><span class="nx">listeners</span><span class="p">);</span>
<span class="nx">observer</span><span class="p">.</span><span class="nx">OnCompleted</span><span class="p">();</span>
<span class="p">}</span>
<span class="p">};</span>
<span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="nx">self</span><span class="p">.</span><span class="nx">running</span> <span class="o">||</span> <span class="nx">self</span><span class="p">.</span><span class="nx">options</span><span class="p">.</span><span class="nx">link</span> <span class="o">==</span> <span class="s1">'cancel'</span><span class="p">){</span>
<span class="nx">self</span><span class="p">.</span><span class="nx">addEvents</span><span class="p">(</span><span class="nx">listeners</span><span class="p">).</span><span class="nx">start</span><span class="p">.</span><span class="nx">run</span><span class="p">(</span><span class="nx">self</span><span class="p">.</span><span class="nx">options</span><span class="p">.</span><span class="nx">defaultArgs</span><span class="p">,</span> <span class="nx">self</span><span class="p">);</span>
<span class="k">return</span> <span class="kd">function</span><span class="p">(){</span>
<span class="nx">self</span><span class="p">.</span><span class="nx">removeEvents</span><span class="p">(</span><span class="nx">listeners</span><span class="p">).</span><span class="nx">cancel</span><span class="p">();</span>
<span class="p">};</span>
<span class="p">}</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">self</span><span class="p">.</span><span class="nx">options</span><span class="p">.</span><span class="nx">link</span> <span class="o">==</span> <span class="s1">'chain'</span><span class="p">){</span>
<span class="kd">var</span> <span class="nx">disposed</span><span class="p">,</span> <span class="nx">running</span><span class="p">;</span>
<span class="nx">self</span><span class="p">.</span><span class="nx">chain</span><span class="p">(</span><span class="kd">function</span><span class="p">(){</span>
<span class="nx">running</span> <span class="o">=</span> <span class="kc">true</span><span class="p">;</span>
<span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="nx">disposed</span><span class="p">)</span>
<span class="nx">self</span><span class="p">.</span><span class="nx">addEvents</span><span class="p">(</span><span class="nx">listeners</span><span class="p">).</span><span class="nx">start</span><span class="p">.</span><span class="nx">run</span><span class="p">(</span><span class="nx">self</span><span class="p">.</span><span class="nx">options</span><span class="p">.</span><span class="nx">defaultArgs</span><span class="p">,</span> <span class="nx">self</span><span class="p">);</span>
<span class="p">});</span>
<span class="k">return</span> <span class="kd">function</span><span class="p">(){</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">running</span><span class="p">)</span> <span class="nx">self</span><span class="p">.</span><span class="nx">removeEvents</span><span class="p">(</span><span class="nx">listeners</span><span class="p">).</span><span class="nx">cancel</span><span class="p">();</span>
<span class="nx">disposed</span> <span class="o">=</span> <span class="kc">true</span><span class="p">;</span>
<span class="p">};</span>
<span class="p">}</span>
<span class="nx">observer</span><span class="p">.</span><span class="nx">OnComplete</span><span class="p">();</span>
<span class="k">return</span> <span class="kd">function</span><span class="p">(){};</span>
<span class="p">});</span>
<span class="p">}</span>
<span class="p">});</span>
</code></pre></div>
<p>Of course since there are a lot of other classes extending the Request and Fx classes, you get the same benefits on them. This is one of the true benefits of MooTools’ modular extensibility.</p>
<p><em>That is one of the benefits of using the class(ical) pattern in JavaScript. More on that next time…</em></p>
<h2 id="side-effects-example">Side-effects Example</h2>
<div class="highlight"><pre><code class="js"><span class="kd">var</span> <span class="nx">popup</span> <span class="o">=</span> <span class="nb">document</span><span class="p">.</span><span class="nx">id</span><span class="p">(</span><span class="s1">'popup'</span><span class="p">);</span>
<span class="kd">var</span> <span class="nx">showPopup</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">Fx</span><span class="p">.</span><span class="nx">Morph</span><span class="p">(</span><span class="nx">popup</span><span class="p">,</span> <span class="p">{</span> <span class="nx">property</span><span class="o">:</span> <span class="s1">'opacity'</span><span class="p">,</span> <span class="nx">defaultArgs</span><span class="o">:</span> <span class="mi">1</span> <span class="p">});</span>
<span class="kd">var</span> <span class="nx">feed</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">Request</span><span class="p">.</span><span class="nx">JSON</span><span class="p">({</span> <span class="nx">url</span><span class="o">:</span> <span class="s1">'mydata.json'</span><span class="p">,</span> <span class="nx">method</span><span class="o">:</span> <span class="s1">'get'</span> <span class="p">});</span>
<span class="kd">var</span> <span class="nx">showFeed</span> <span class="o">=</span> <span class="nx">feed</span><span class="p">.</span><span class="nx">toObservable</span><span class="p">()</span>
<span class="p">.</span><span class="nx">Do</span><span class="p">(</span><span class="kd">function</span><span class="p">(</span><span class="nx">data</span><span class="p">){</span> <span class="nx">popup</span><span class="p">.</span><span class="nx">set</span><span class="p">(</span><span class="s1">'text'</span><span class="p">,</span> <span class="nx">data</span><span class="p">);</span> <span class="p">})</span>
<span class="p">.</span><span class="nx">Concat</span><span class="p">(</span><span class="nx">showPopup</span><span class="p">.</span><span class="nx">toObservable</span><span class="p">());</span>
<span class="c1">// ...</span>
<span class="nx">showFeed</span><span class="p">.</span><span class="nx">Subscribe</span><span class="p">();</span> <span class="c1">// loads mydata.json into #popup and displays it</span>
</code></pre></div>
<h2 id="using-arrays-in-unit-tests">Using Arrays in Unit Tests</h2>
<p>Since natives are allowed to be extended within the MooTools theorem, we can add a convenience method to turn an Array into an observable stream of content.</p>
<div class="highlight"><pre><code class="js"><span class="nb">Array</span><span class="p">.</span><span class="nx">implement</span><span class="p">(</span><span class="s1">'toObservable'</span><span class="p">,</span> <span class="kd">function</span><span class="p">(){</span><span class="k">return</span> <span class="nx">Rx</span><span class="p">.</span><span class="nx">Observable</span><span class="p">.</span><span class="nx">FromArray</span><span class="p">(</span><span class="k">this</span><span class="p">);});</span>
</code></pre></div>
<p>We can use this to fake the “flicks” event stream in our earlier example. We avoid having to include complex asynchronous tests or user action tests.</p>
<div class="highlight"><pre><code class="js"><span class="kd">var</span> <span class="nx">flicks</span> <span class="o">=</span> <span class="p">[</span>
<span class="p">{</span> <span class="nx">angle</span><span class="o">:</span> <span class="mi">0</span><span class="p">,</span> <span class="nx">distance</span><span class="o">:</span> <span class="mi">100</span> <span class="p">},</span>
<span class="p">{</span> <span class="nx">angle</span><span class="o">:</span> <span class="mi">45</span><span class="p">,</span> <span class="nx">distance</span><span class="o">:</span> <span class="mi">100</span> <span class="p">},</span>
<span class="p">{</span> <span class="nx">angle</span><span class="o">:</span> <span class="mi">90</span><span class="p">,</span> <span class="nx">distance</span><span class="o">:</span> <span class="mi">100</span> <span class="p">}</span>
<span class="p">].</span><span class="nx">toObservable</span><span class="p">();</span>
<span class="c1">// Unit tests</span>
<span class="c1">// Synchronously testing code that's depending on a flick event stream</span>
</code></pre></div>
<h2 id="web-sockets-and-web-workers">Web Sockets and Web Workers</h2>
<p>Now imagine this on a stream of events coming in from <a href="http://dev.w3.org/html5/websockets/">Web Sockets</a> or <a href="http://dev.w3.org/html5/workers/">Web Workers</a>.</p>
<p>You could set up a web socket to asynchronously feed you JSON objects, and easily hook that up to the rest of you UI just as easily as the Request example above.</p>
<p><a href="http://blog.calyptus.eu/seb/2010/03/rx-for-mootools/">The Reactive Extensions for JavaScript - MooTools Integration</a> was originally published by at <a href="http://blog.calyptus.eu">Calyptus</a> on March 07, 2010.</p>http://blog.calyptus.eu/seb/2010/03/rx-for-javascript2010-03-06T14:37:22Z2010-03-06T14:37:22Zhttp://blog.calyptus.euinfo@calyptus.eu<p>I’ve been following the work on the <a href="http://msdn.microsoft.com/en-us/devlabs/ee794896.aspx">Reactive Extensions for .NET</a> (Rx) by <a href="http://research.microsoft.com/en-us/um/people/emeijer/ErikMeijer.html">Eric Meijer</a> and others over at Microsoft. At first look I was intrigued but didn’t really understand the purpose of it. However, at a second look, I realized that it had the potential to solve every major problem I’ve had with advanced UI development in JavaScript.</p>
<h2 id="asynchronous-programming---composable-events">Asynchronous Programming - Composable Events</h2>
<p>Modern UI development forces us to use asynchronous patterns for user actions, animations and data load. But to make UI development easy you can think of each operation as sequential. You can even artificially lock down the user interface - by disabling or hiding UI elements - while an operation is occurring.</p>
<p>If you want to optimize your user experience, you will need to start dabble in the complicated art of event composition. The problem occurs when you have complex interactions that depend upon other interaction or state.</p>
<p>You can solve this using various state machine patterns. However, I think you will find it quite difficult at times. Even if you do solve it, it’s probably going to be for a specific purpose which is not easily generalizable nor extensible.</p>
<p>Various tools have tried using patterns like Futures and Promises. I think those patterns need to be applied at the language level to be really useful though.</p>
<h2 id="reactive-programming-in-an-object-oriented-world">Reactive Programming in an Object Oriented World</h2>
<p>JavaScript has introduced the map/filter/reduce methods on arrays to allow collection operations using a sequenced composition of functions.</p>
<p>There’s one minor thing that JavaScript developers should note. In <a href="http://linqjs.codeplex.com/">LINQ</a> these operations are lazy iterables. The map/filter operations aren’t actually executed until an .each() starts iterating over them. This avoids having to create duplicates of the result in memory. It also means that the underlying array can change after we call filter and map. This is similar to “live” collections in the DOM. But they can also be infinite in length just like <a href="https://developer.mozilla.org/en/Core_JavaScript_1.5_Guide/Iterators_and_Generators">Mozilla’s Iterators</a>. The .each() call is still essentially a synchronous operation though.</p>
<p><a href="http://channel9.msdn.com/shows/Going+Deep/Expert-to-Expert-Brian-Beckman-and-Erik-Meijer-Inside-the-NET-Reactive-Framework-Rx/">Erik Meijer and team</a> simply decided to make that iteration execution asynchronous.</p>
<p>This means that the source data can be asynchronous. So instead of thinking of Events as independent, think of them as a stream of data with an unknown length… (or an asynchronous list/array).</p>
<p>This means that you can now apply the same type of function composition to streams of events. Enter the <a href="http://msdn.microsoft.com/en-us/devlabs/ee794896.aspx">Reactive Extensions for .NET</a>.</p>
<p>Supposedly this could solve the problem of Event composition in the UI space.</p>
<h2 id="the-reactive-extensions-for-javascript">The Reactive Extensions for JavaScript</h2>
<p>To my surprise, <a href="http://weblogs.asp.net/podwysocki/">Matthew Podwysocki</a> recently started a <a href="http://weblogs.asp.net/podwysocki/archive/2010/02/16/introduction-to-the-reactive-extensions-to-javascript.aspx">blog series</a> about the Reactive Extensions for JavaScript (also Microsoft to be clear). Apparently the benefits of this tool in the JS world has not gone unnoticed.</p>
<p>There are no bits officially released yet. However, considering <a href="http://weblogs.asp.net/podwysocki/">Matthew Podwysocki’s</a> recent posts and <a href="http://live.visitmix.com/MIX10/Sessions/FTL01">Eric Meijer’s upcoming talk</a> at the Mix conference… I wouldn’t be surprised if something was released at Mix on March 17th.</p>
<h2 id="learning-more">Learning More</h2>
<p>I have since been interested in learning more about various alternative models. There’s a research project called <a href="http://www.cs.umd.edu/~jfoster/arrows.pdf">Arrows</a> which provides a different model that’s more purely functional. There’s also a framework called <a href="http://www.flapjax-lang.org/">Flapjax</a> which is more of a DOM library aiming to provide reactive concepts to JavaScript.</p>
<p>To learn more about the Reactive Extensions, take a look at the <a href="http://channel9.msdn.com/tags/Rx/">videos about the .NET version posted by the team over at Channel 9</a>.</p>
<h2 id="concerns">Concerns</h2>
<p>I’m not sure the first implementation of the Reactive Extensions is going to be the one to solve all these problems.</p>
<p>I think that many developers will have a difficult time thinking about these concepts in the terms of event streams. That could make it difficult to use the current method naming. In this sense, I think <a href="http://www.cs.umd.edu/~jfoster/arrows.pdf">Arrows</a> might be easier to get started with. It will allow you to think about events as sequential operations. However, I also see benefit in the model employed by the Reactive Extensions, IF we can all wrap our heads around it.</p>
<p>Another issue is the “Let” method. This may be difficult to know when to use for many developers. That’s true even for LINQ. However, in the Reactive Extensions I have a feeling those issues will become even more prevalent. Hopefully there will be better syntactical sugar to solve this issue.</p>
<p>Rx has yet to prove itself in real world complex applications that goes well beyond single subscription examples. I may try to extend the canonical drag and drop examples to my own <a href="http://github.com/calyptus/mootools-draggable">HTML5 based Drag and Drop</a> model and plugins to stress test it.</p>
<h2 id="naming-conventions">Naming Conventions</h2>
<p>There’s also the issue of upper camel case in method names. <a href="http://linqjs.codeplex.com/">LINQ for JavaScript</a> is also using this convention. I’m guessing they’re trying to be compatible. However, the JavaScript convention is to use lower camel case method names, which also the <a href="http://ajax.codeplex.com/">new ASP.NET AJAX</a> library is doing. So I don’t understand why.</p>
<p>In dynamic languages with limited auto-completion (IntelliSense) support, naming conventions are very important to follow.</p>
<p>Although, I do like the names Select/Where/OrderBy better than map/filter/sort since given the arguments, that tends to read better as a grammatical sentence.</p>
<h2 id="update-event-dsls">UPDATE: Event DSLs</h2>
<p>I should mention that the MooTools 2.0 team has been working on a DSL based on the CSS selector syntax. This is an extension of the <a href="http://mootools.net/docs/more/Element/Element.Delegation">Element.Delegation</a> plugin.</p>
<p>The idea is to use event names and pseudos in combination to create custom composite event listeners. This example would listen to the first click event:</p>
<div class="highlight"><pre><code class="js"><span class="nx">element</span><span class="p">.</span><span class="nx">addEvent</span><span class="p">(</span><span class="s1">'click:flash'</span><span class="p">,</span> <span class="nx">firstClick</span><span class="p">);</span>
</code></pre></div>
<p>This could enable a lot more powerful combinations of custom events. However, it doesn’t enable passing of parameters and the composability of Rx and Arrows.</p>
<p><a href="http://blog.calyptus.eu/seb/2010/03/rx-for-javascript/">The Reactive Extensions for JavaScript - Event Composition</a> was originally published by at <a href="http://blog.calyptus.eu">Calyptus</a> on March 06, 2010.</p>http://blog.calyptus.eu/seb/2009/11/the-performance-of-nodename2009-11-24T05:43:01Z2009-11-24T05:43:01Zhttp://blog.calyptus.euinfo@calyptus.eu<p>I was researching various options of traversing nodes for <a href="http://github.com/subtleGradient/slick">Slick</a> and the <a href="http://github.com/calyptus/mootools-more/tree/range/Source/Native/">DOM Range for MooTools</a>. I realized that the nodeName property is incredibly slow to access in WebKit browsers. This is because it is working with qualified names (with namespaces and stuff) internally.</p>
<div class="highlight"><pre><code class="js"><span class="k">if</span> <span class="p">(</span><span class="nx">node</span><span class="p">.</span><span class="nx">nodeName</span> <span class="o">==</span> <span class="s1">'A'</span><span class="p">)</span> <span class="c1">// do something with anchor tag</span>
</code></pre></div>
<p>If you add case insensitive matching to that it will be even slower.</p>
<p>Instead I decided to try to check the constructor of the node to determine what type it is. For example for the anchor (A) tag, modern browsers will use the prototype of HTMLAnchorElement. This can potentially speed up these checks if you’re looking for a known node type.</p>
<div class="highlight"><pre><code class="js"><span class="k">if</span> <span class="p">(</span><span class="nx">node</span><span class="p">.</span><span class="nx">constructor</span> <span class="o">===</span> <span class="nx">HTMLAnchorElement</span><span class="p">)</span> <span class="c1">// do something</span>
<span class="c1">// OR...</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">node</span> <span class="k">instanceof</span> <span class="nx">HTMLAnchorElement</span><span class="p">)</span> <span class="c1">// do something</span>
</code></pre></div>
<p>I ran <a href="http://labs.calyptus.eu/NodeName/performance-test.html">this performance test</a> in various browsers. It traverses all nodes in a large HTML documents and checks which ones are anchor nodes. It first does a blank run to eliminate any initialization quirks. Then it does a control run without the anchor check. Then it tests each of the above models.</p>
<p><strong>IE6 and IE7</strong> will obviously fail since they don’t support the HTMLAnchorElement constructor/prototype. For that case you would have to fall back to the nodeName property.</p>
<p><strong>IE8</strong> will be slightly slower with the constructor check than the nodeName check. But the difference is marginal in the overall scope of IE’s slowness.</p>
<p><strong>WebKit</strong> will gain significant performance using the constructor check. The difference is relatively small to the overhead of manually walking the tree. However, if you take the control value from the blank run into account, the difference of just the node type checks will be significant (several times faster). The slow part is the WebKit DOM API, so you will see this with both JavaScriptCore and V8 (Safari and Chrome respectively).</p>
<p><strong>Firefox</strong> will be slower on the first run for some weird reason. But in subsequent runs the constructor check will be faster than the nodeName check.</p>
<p><em>As a side note, node.tagName is no different. That is just an alias for node.nodeName.</em></p>
<p><em>In <a href="http://ejohn.org/blog/nodename-case-sensitivity/">John Resig’s case sensitivity</a> he discusses the case inconsistencies of the nodeName property in various contexts and the impact on performance. For example, in IE, the value of nodeName of unknown elements (like the new HTML5 elements) keeps it original case as in the markup.</em></p>
<p><em>This means that any proper CSS selector search for such elements would have to run a case-insensitive match against the nodeName property. Unfortunately the little trick I’ve shown above doesn’t remedy this problem because unknown elements will be lacking a known constructor. However, known Elements can still utilize this trick as a slight performance boost, while letting unknown element fallback to a case insensitive match.</em></p>
<p><em>UPDATE: I added a case-insensitive match to the <a href="http://labs.calyptus.eu/NodeName/performance-test.html">performance tests</a> using regular expressions - showing the added overhead compared to constructor checking.</em></p>
<p><a href="http://blog.calyptus.eu/seb/2009/11/the-performance-of-nodename/">The Performance of .nodeName</a> was originally published by at <a href="http://blog.calyptus.eu">Calyptus</a> on November 24, 2009.</p>http://blog.calyptus.eu/seb/2009/07/why-you-shouldnt-return-false-in-mootools-event-handlers2009-07-24T23:53:59Z2009-07-24T23:53:59Zhttp://blog.calyptus.euinfo@calyptus.eu<p>Let’s say I have a link (anchor tag with href), and I wish to attach an event listener to it.</p>
<div class="highlight"><pre><code class="xml"><span class="nt"><ul></span>
<span class="nt"><li><a</span> <span class="na">id=</span><span class="s">"mylink"</span> <span class="na">href=</span><span class="s">"http://..."</span><span class="nt">></span>my link<span class="nt"></a></li></span>
<span class="nt"></ul></span>
</code></pre></div>
<div class="highlight"><pre><code class="js"><span class="nb">document</span><span class="p">.</span><span class="nx">id</span><span class="p">(</span><span class="s1">'mylink'</span><span class="p">).</span><span class="nx">addEvent</span><span class="p">(</span><span class="s1">'click'</span><span class="p">,</span> <span class="kd">function</span><span class="p">(){</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="s1">'hello world'</span><span class="p">);</span>
<span class="p">});</span>
</code></pre></div>
<p>Now, if I click the link it will log the message but the browser window will also visit the location of the link. There are a bunch of such default behaviors to pretty much every event in the DOM. If we’re implementing custom behavior, we typically want to prevent this default behavior. A common practise is to have the method return false as such:</p>
<div class="highlight"><pre><code class="js"><span class="nb">document</span><span class="p">.</span><span class="nx">id</span><span class="p">(</span><span class="s1">'mylink'</span><span class="p">).</span><span class="nx">addEvent</span><span class="p">(</span><span class="s1">'click'</span><span class="p">,</span> <span class="kd">function</span><span class="p">(){</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="s1">'hello world'</span><span class="p">);</span>
<span class="k">return</span> <span class="kc">false</span><span class="p">;</span>
<span class="p">});</span>
</code></pre></div>
<p><strong>THIS IS BAD!</strong> Don’t. To understand the reason for this, you need to understand <a href="http://www.quirksmode.org/js/events_order.html">event bubbling</a> and the difference between <a href="http://mootools.net/docs/core/Native/Event#Event:preventDefault">preventDefault </a>and <a href="http://mootools.net/docs/core/Native/Event#Event:stopPropagation">stopPropagation</a>.</p>
<h2 id="event-bubbling-and-stoppropagation">Event bubbling and stopPropagation</h2>
<p>When an event is dispatched, it first fires the listeners of the ‘mylink’ element (not quite true, but we don’t use capture). But then it propagates (bubbles) up to the LI-element, UL-element, BODY-element etc. So for every click on any element, the ‘click’ event is triggered on the BODY-element. After all of that, the default behavior of the browser is triggered.</p>
<p><em>In most browsers bubbling continues to the document and window objects, but that’s not always true for IE.</em></p>
<p>This is a powerful model. It allows us to do things like <a href="http://www.clientcide.com/code-releases/event-delegation-for-mootools/">Event delegation</a>. You can place a listener on the UL-element to catch any events triggered on the LI-elements without adding listeners to all the existing or any new LI-elements.</p>
<p>Sometimes we don’t want bubbling to occur. Let’s say for example that I wanted to have a ‘click’ event handler on the UL-element that handles clicks on the UL area outside of any A-element. Then I could accept the <a href="http://mootools.net/docs/core/Native/Event">Event object</a> as the first parameter, use <a href="http://mootools.net/docs/core/Native/Event#Event:stopPropagation">stopPropagation</a> during the click event on the A-element to stop the event before it reaches the UL.</p>
<div class="highlight"><pre><code class="js"><span class="nb">document</span><span class="p">.</span><span class="nx">getElements</span><span class="p">(</span><span class="s1">'ul'</span><span class="p">).</span><span class="nx">addEvent</span><span class="p">(</span><span class="s1">'click'</span><span class="p">,</span> <span class="kd">function</span><span class="p">(){</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="s1">'You clicked within the UL but outside of any link.'</span><span class="p">);</span>
<span class="p">});</span>
<span class="nb">document</span><span class="p">.</span><span class="nx">getElements</span><span class="p">(</span><span class="s1">'a'</span><span class="p">).</span><span class="nx">addEvent</span><span class="p">(</span><span class="s1">'click'</span><span class="p">,</span> <span class="kd">function</span><span class="p">(</span><span class="nx">event</span><span class="p">){</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="s1">'You clicked a link.'</span><span class="p">);</span>
<span class="nx">event</span><span class="p">.</span><span class="nx">stopPropagation</span><span class="p">();</span>
<span class="p">});</span>
</code></pre></div>
<h2 id="preventdefault">preventDefault</h2>
<p>In my example above the browser would still visit the href of the link. Stopping propagation (bubbling) doesn’t actually prevent the default browser action. So we also need to call preventDefault during the click event to prevent the default operation of clicking a link.</p>
<div class="highlight"><pre><code class="js"><span class="nb">document</span><span class="p">.</span><span class="nx">getElements</span><span class="p">(</span><span class="s1">'a'</span><span class="p">).</span><span class="nx">addEvent</span><span class="p">(</span><span class="s1">'click'</span><span class="p">,</span> <span class="kd">function</span><span class="p">(</span><span class="nx">event</span><span class="p">){</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="s1">'You clicked a link.'</span><span class="p">);</span>
<span class="nx">event</span><span class="p">.</span><span class="nx">stopPropagation</span><span class="p">();</span>
<span class="nx">event</span><span class="p">.</span><span class="nx">preventDefault</span><span class="p">();</span>
<span class="p">});</span>
</code></pre></div>
<p>Now since this is fairly common MooTools has a shortcut for doing both stopPropagation AND preventDefault. Namely the stop() method:</p>
<div class="highlight"><pre><code class="js"><span class="nb">document</span><span class="p">.</span><span class="nx">getElements</span><span class="p">(</span><span class="s1">'a'</span><span class="p">).</span><span class="nx">addEvent</span><span class="p">(</span><span class="s1">'click'</span><span class="p">,</span> <span class="kd">function</span><span class="p">(</span><span class="nx">event</span><span class="p">){</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="s1">'You clicked a link.'</span><span class="p">);</span>
<span class="nx">event</span><span class="p">.</span><span class="nx">stop</span><span class="p">();</span>
<span class="p">});</span>
</code></pre></div>
<h2 id="so-why-is-return-false-bad">So, why is return false bad?</h2>
<p>In the standard browser DOM model it’s <strong>equivalent to calling <code>event.preventDefault();</code></strong> but <strong>in MooTools it’s equivalent to calling <code>event.stop();</code></strong> i.e. it also calls stopPropagation.</p>
<p>This is a problem. If you use this model routinely you may not notice that you actually prevent plugins attached to elements higher up in the bubbling chain.</p>
<p>Let’s say I want to use the ‘mouseleave’ event to hide the UL-element when the mouse leaves. If I also return false on the ‘mouseout’ event on the A-element, I may not get the ‘mouseleave’ event because the A-element stops it. OR maybe I have a plugin higher up that requires that my events bubble. It’ll be even more prevalent as more plugins makes use of <a href="http://www.clientcide.com/code-releases/event-delegation-for-mootools/">Event delegation</a>.</p>
<p>Therefore you need to be very explicit about when you stop propagation and not.</p>
<p>Second of all, the “return false” API doesn’t make sense. The function isn’t failing. It isn’t canceled. In fact, it’s canceling a DIFFERENT function.</p>
<p>Therefore you should ALWAYS be explicit by calling either event.preventDefault(), event.stopPropagation() or event.stop(); instead of relying on an implicit convention that differs between frameworks.</p>
<p><em>Returning a false value is a relic from the old days when we only had a single listener per event.</em></p>
<h2 id="binding-parameters">Binding Parameters</h2>
<p>Sometimes you need to bind parameters that you wish to pass to an event listener. A common practise is to use <a href="http://mootools.net/docs/core/Native/Function#Function:bind">bind</a>.</p>
<div class="highlight"><pre><code class="js"><span class="nb">document</span><span class="p">.</span><span class="nx">getElements</span><span class="p">(</span><span class="s1">'a'</span><span class="p">).</span><span class="nx">addEvent</span><span class="p">(</span><span class="s1">'click'</span><span class="p">,</span> <span class="kd">function</span><span class="p">(</span><span class="nx">paramA</span><span class="p">,</span> <span class="nx">paramB</span><span class="p">){</span>
<span class="c1">// do something with this, paramA and paramB</span>
<span class="k">return</span> <span class="kc">false</span><span class="p">;</span>
<span class="p">}.</span><span class="nx">bind</span><span class="p">(</span><span class="nx">someObj</span><span class="p">,</span> <span class="p">[</span><span class="nx">objA</span><span class="p">,</span> <span class="nx">objB</span><span class="p">]));</span>
</code></pre></div>
<p>In this case you can’t accept an <a href="http://mootools.net/docs/core/Native/Event">Event object</a> since you’ve bound your parameters to other objects. In this case you can use <a href="http://mootools.net/docs/core/Native/Function#Function:bindWithEvent">bindWithEvent</a> to let the first parameter (the event object) get through, while binding the remaining parameters.</p>
<div class="highlight"><pre><code class="js"><span class="nb">document</span><span class="p">.</span><span class="nx">getElements</span><span class="p">(</span><span class="s1">'a'</span><span class="p">).</span><span class="nx">addEvent</span><span class="p">(</span><span class="s1">'click'</span><span class="p">,</span> <span class="kd">function</span><span class="p">(</span><span class="nx">event</span><span class="p">,</span> <span class="nx">paramA</span><span class="p">,</span> <span class="nx">paramB</span><span class="p">){</span>
<span class="c1">// do something with this, paramA and paramB</span>
<span class="nx">event</span><span class="p">.</span><span class="nx">stop</span><span class="p">();</span>
<span class="p">}.</span><span class="nx">bindWithEvent</span><span class="p">(</span><span class="nx">someObj</span><span class="p">,</span> <span class="p">[</span><span class="nx">objA</span><span class="p">,</span> <span class="nx">objB</span><span class="p">]));</span>
</code></pre></div>
<h2 id="lambdafalse">$lambda(false)</h2>
<p><em>“But I don’t want to type out all of that just to stop an event. I like $lambda(false) to easily block events.”</em></p>
<p>People sometimes use the $lambda method to create a function that returns false to easily stop an event without doing anything else: el.addEvent(‘click’, $lambda(false));</p>
<p>So you need a method that does nothing other than accepts an Event object and calls preventDefault, stopPropagation or stop? Thanks to <a href="http://keetology.com/blog/2009/07/20/up-the-herd-ii-native-flora-and-fauna">MooTools generics</a> you can easily do that like this:</p>
<div class="highlight"><pre><code class="js"><span class="nx">element</span><span class="p">.</span><span class="nx">addEvent</span><span class="p">(</span><span class="s1">'click'</span><span class="p">,</span> <span class="nx">Event</span><span class="p">.</span><span class="nx">preventDefault</span><span class="p">);</span> <span class="c1">// OR...</span>
<span class="nx">element</span><span class="p">.</span><span class="nx">addEvent</span><span class="p">(</span><span class="s1">'click'</span><span class="p">,</span> <span class="nx">Event</span><span class="p">.</span><span class="nx">stopPropagation</span><span class="p">);</span> <span class="c1">// OR...</span>
<span class="nx">element</span><span class="p">.</span><span class="nx">addEvent</span><span class="p">(</span><span class="s1">'click'</span><span class="p">,</span> <span class="nx">Event</span><span class="p">.</span><span class="nx">stop</span><span class="p">);</span>
</code></pre></div>
<p><em>For you that think “return false;” saves bandwidth… “e,” and “e.stop();” is two bytes shorter.</em></p>
<h2 id="additional-event-listeners-on-the-same-element">Additional Event Listeners on the same Element</h2>
<p>Neither preventDefault or stopPropagation or <a href="http://dean.edwards.name/weblog/2009/03/callbacks-vs-events/">even an error</a> prevents any additional handlers/listeners on the same element. So if you have two handlers listening to the same event, then both will be triggered regardless of the result of either function.</p>
<p>That should be true for all Events, even Class events. More on that in MooTools 2.0…</p>
<p><a href="http://blog.calyptus.eu/seb/2009/07/why-you-shouldnt-return-false-in-mootools-event-handlers/">Why you shouldn't return false in MooTools event handlers</a> was originally published by at <a href="http://blog.calyptus.eu">Calyptus</a> on July 24, 2009.</p>http://blog.calyptus.eu/seb/2009/05/transitory_domain_object2009-05-30T18:53:46Z2009-05-30T18:53:46Zhttp://blog.calyptus.euinfo@calyptus.eu<p>A common problem with DDD is the injection of services to your domain model. Sometimes your domain relies on external services to do it’s job. You could do that by injecting your services directly to your entities using <a href="http://www.nhforge.org/doc/nh/en/index.html#manipulatingdata-interceptors">NHibernate Interceptors</a> or <a href="http://rogeralsing.com/2009/05/30/entity-framework-4-entity-dependency-injection/">ObjectStateManager for Entity Framework v4</a>.</p>
<p>There are many design issues with the POCOness of Entities when you keep references to external services within the Entities themselves. The reference itself is (usually) infrastructure and not really a persistence concern.</p>
<h2 id="double-dispatch-specifications-and-services">Double Dispatch, Specifications and Services</h2>
<p>The double dispatch pattern seems to be a popular approach. A better solution seem to be to move the logic in front of the Entites. Usually people seem to solve this by moving logic to <a href="http://devlicio.us/blogs/casey/archive/2009/02/17/ddd-services.aspx">services</a> or even <a href="http://devlicio.us/blogs/casey/archive/2009/03/02/ddd-the-specification-pattern.aspx">specifications</a>.</p>
<p>Moving domain logic to services is a big no, no. That’s a gateway to anemic domain models and bloated service implementations. Services should be a last resort for external concerns and should probably have a solid anti-corruption layer.</p>
<p>The double dispatch pattern is a pain, ugly and introduces lots of references to services where the ubiquitous language doesn’t dictate it. </p>
<p>The specification pattern is particularly ugly because that’s (usually) not how a domain expert would refer to the issue. We are violating the ubiquitous language.</p>
<h2 id="transitory-domain-objects">Transitory Domain Objects</h2>
<p>Recently I’ve started introducing unpersisted classes to my domain models. If you think about it, many domain models have transitory terms and concerns that are not really persisted.</p>
<p>Imagine that your domain model consists of an archive of home photography. Let’s call them Photos. Now, you want to work with a couple of them. You pick out all the ones that have a red lavish hue and start organizing, labeling them or other operations. Now you have a set of Photos.</p>
<p>You could claim that it is a UI or Controller concern. Given the right bounded context, that set of photos IS A Domain Concern! Your domain could have domain specific restrictions and operations occurring on those sets of photos. You can think about them as a workspace or extended units of work.</p>
<p>Now this set isn’t persisted. It’s not an entity, it’s not a value object. Your entities can’t refer to it. This transitory logic lies infront of your entities. Since it’s transitory it also means that it can contain references to repositories and external services. It makes reference management much easier.</p>
<p>Now we can change out our specification and double dispatch patterns:</p>
<div class="highlight"><pre><code class="c#"><span class="kt">var</span> <span class="n">redishPhotoList</span> <span class="p">=</span> <span class="n">photoRepository</span><span class="p">.</span><span class="n">Find</span><span class="p">(</span>
<span class="k">new</span> <span class="nf">HueSpecificiation</span><span class="p">(</span><span class="n">colorDetectorService</span><span class="p">,</span> <span class="n">Color</span><span class="p">.</span><span class="n">Red</span><span class="p">)</span>
<span class="p">);</span>
<span class="k">foreach</span><span class="p">(</span><span class="kt">var</span> <span class="n">photo</span> <span class="k">in</span> <span class="n">redishPhotoList</span><span class="p">){</span>
<span class="c1">//checks...</span>
<span class="n">photo</span><span class="p">.</span><span class="n">MarkWithMetaData</span><span class="p">(</span><span class="s">"RED"</span><span class="p">,</span> <span class="n">metaDataService</span><span class="p">);</span>
<span class="c1">//contraints...</span>
<span class="p">}</span>
</code></pre></div>
<p>To something more domain specific:</p>
<div class="highlight"><pre><code class="c#"><span class="kt">var</span> <span class="n">photoSet</span> <span class="p">=</span> <span class="k">new</span> <span class="n">PhotoSet</span><span class="p">(</span><span class="n">photoRepository</span><span class="p">,</span> <span class="n">colorDetectorService</span><span class="p">,</span> <span class="n">metaDataService</span><span class="p">);</span>
<span class="n">photoSet</span><span class="p">.</span><span class="n">UsingOnly</span><span class="p">(</span><span class="n">Color</span><span class="p">.</span><span class="n">Red</span><span class="p">).</span><span class="n">MarkWithMetaData</span><span class="p">(</span><span class="s">"RED"</span><span class="p">);</span>
</code></pre></div>
<p>We now have a domain object that we can easily pass around our application.</p>
<p>When you think about it you’re probably already using this pattern either as helpers or as “services”. But making the clear distinction that this is 1) A Domain Concern. 2) Temporary. Makes it easier to place your logic and apply constraints.</p>
<p><em>Achieving pure POCO is a pain from an infrastructure perspective but it’s worth it once it’s in place. I should be able to pass it to and from Db4O without any infrastructure concerns. Then you have a clear and solid domain model.</em></p>
<p><a href="http://blog.calyptus.eu/seb/2009/05/transitory_domain_object/">Transitory Domain Objects</a> was originally published by at <a href="http://blog.calyptus.eu">Calyptus</a> on May 30, 2009.</p>http://blog.calyptus.eu/seb/2009/05/png-parser-in-javascript2009-05-20T21:39:53Z2009-05-20T21:39:53Zhttp://blog.calyptus.euinfo@calyptus.eu<p><em>The other day <a href="http://www.davidwalsh.name/">David Walsh</a> was experimenting with rendering images in the browser using regular tags as pixels. <a href="http://mad4milk.net/">Valerio</a> picked up the idea and made some enhancements. A server-side script transformed PNG files into a JSON image format for easy parsing on the client. That raised the question… How difficult would it be to do that parsing on the client instead?</em></p>
<p>Why PNG? Well, other than becoming the new defacto standard for graphics it’s a very simple format. It’s also free of patents and uses only simple well known techniques. It makes it very easy to work with. This post is about parsing raw PNG image data in pure JavaScript. It has nothing to do with built in browser support for the format.</p>
<p><strong>Base64 Encoding</strong></p>
<p>JavaScript doesn’t allow us to work with binary data directly. Even with XHR we can’t work with the raw binary data because JavaScript doesn’t currently have a concept of raw bytes. Instead we have to get the bytes from a character representation of the data.</p>
<p>Luckily there’s already a standard transfer encoding already heavily in use in various places of the W3C standards… <a href="http://en.wikipedia.org/wiki/Base64">Base64</a>! You can use the <a href="http://tools.ietf.org/html/rfc2397">data:</a> URI scheme to embed image data in your HTML or CSS documents. It’s also heavily used for binary data in e-mails.</p>
<p>We can get the data either from an XHR request, from a src attribute or just statically embedded in your JavaScript file. So, now we have our data as Base64 encoded string.</p>
<p>To work with the raw data we need a way to represent bytes. If you’re working with ASCII data you can just stick to string representations. But since we’re going to be working binary data the most useful way seems to be simple Numbers. That allows you to do bitwise operations and easily convert them to and from ASCII. It’s also provides better performance than representing the bytes as Objects.</p>
<p>Now we need a parser. I went with <a href="http://www.codeproject.com/KB/scripting/Javascript_binaryenc.aspx">a sample parser by some guy named notmasteryet</a>. There are others but this seems like a pretty solid implementation and allows us to work with bytes as Numbers. It also works as a reader that lets us read our data piece by piece instead of filling our memory.</p>
<p><strong>DEFLATE</strong></p>
<p>The current PNG standard only uses the DEFLATE algorithm for compression. It’s the same algorithm used in ZIP, GZIP, zlib, etc. So it’s a very common format.</p>
<p>Luckily for us, <a href="http://www.codeproject.com/KB/scripting/Javascript_binaryenc.aspx">notmasteryet’s sample</a> also includes a DEFLATE decompressor. It also works as a piece by piece reader which makes it more memory efficient to work with. The reader pattern is a great way to read data in nested formats.</p>
<p><strong>PNG</strong></p>
<p>The PNG format consists of a set of named chunks. A set of “IDAT” chunks makes up the main image data. The total data stream is compressed using DEFLATE. The uncompressed data is filtered using one of 5 simple delta compression filters for each line of pixels.</p>
<p>Notice that we haven’t yet touched any image-processing specific logic. DEFLATE and delta compression is used for text and other data as much as anything else.</p>
<p>The raw data consists of a color for each pixel. This can be either grayscale, RGB or a reference to a palette color. This is what we really want.</p>
<p>The PNG format is open and <a href="http://www.libpng.org/pub/png/spec/1.1/PNG-Contents.html">well documented</a>. So I’m not going to cover it in any more detail.</p>
<p><strong>Proof of Concept</strong></p>
<p>Since we’re doing a lightweight JavaScript parser and probably have some control over the image data, we can skip some of the more outlandish features of the specification. We can also skip the verification parts. We’ll just skip the file headers and CRC checks.</p>
<p>I decided on an a simple API that reads each line of pixels as an array of RGB colors represented as a number.</p>
<div class="highlight"><pre><code class="js"><span class="kd">var</span> <span class="nx">image</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">PNG</span><span class="p">(</span><span class="nx">base64data</span><span class="p">);</span>
<span class="nx">image</span><span class="p">.</span><span class="nx">width</span><span class="p">;</span> <span class="c1">// Image width in pixels</span>
<span class="nx">image</span><span class="p">.</span><span class="nx">height</span><span class="p">;</span> <span class="c1">// Image height in pixels</span>
<span class="kd">var</span> <span class="nx">line</span><span class="p">;</span>
<span class="k">while</span><span class="p">(</span><span class="nx">line</span> <span class="o">=</span> <span class="nx">image</span><span class="p">.</span><span class="nx">readLine</span><span class="p">()){</span>
<span class="k">for</span><span class="p">(</span><span class="kd">var</span> <span class="nx">x</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span><span class="nx">x</span> <span class="o"><</span> <span class="nx">line</span><span class="p">.</span><span class="nx">length</span><span class="p">;</span><span class="nx">x</span><span class="o">++</span><span class="p">){</span>
<span class="kd">var</span> <span class="nx">px</span> <span class="o">=</span> <span class="nx">line</span><span class="p">[</span><span class="nx">x</span><span class="p">];</span> <span class="c1">// Pixel RGB color as a single numeric value</span>
<span class="c1">// white pixel == 0xFFFFFF</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div>
<p>I then took that RGB data and inserted the pixels into my document as DIV tags with a background-color.</p>
<p style="text-align: center"><a href="http://labs.calyptus.se/JSBin/Demo/Viewer.html"><img src="http://blog.calyptus.eu/wp-content/uploads/2009/05/gravatar.png" style="display: inline; margin: auto;" alt="Proof of Concept" /></a></p>
<p>In less than 3 hours I had a <a href="http://labs.calyptus.se/JSBin/Demo/Viewer.html">working Proof of Concept</a> of a format I had never worked with before.</p>
<p><em>I skipped interlacing, alpha and some of the filters for the demo. It’s not meant to be a fully working prototype nor a reference library in any way.</em></p>
<p><strong>Now What?</strong></p>
<p>You could…</p>
<ul>
<li>Display the image using a regular rendering method but use the PNG parser to extract colors using a Color Picker.
<li>Add obfuscation or cryptographic layers to render images that can't be easily ripped by bots or downloaded by users.
<li>Render embedded PNG images using VML in Internet Explorer (which lacks data: URI support) with full alpha support.
Don't expect this method to become the new hack for PNG or embedded images in Internet Explorer. The rendering methods here are probably too slow for that. You could do some nice stuff with CANVAS though.
However, I have demonstrated that it is possible to work with binary formats in JavaScript. We shouldn't be afraid of utilizing existing binary standards (PNG, GZIP, SVGZ, SWF, TTF...). We shouldn't always fallback to our comfortable old JSON format and reinvent the wheel for every client-side need.
<strong>Relevant Projects</strong>
The <a href="http://www.mootools.net/">MooTools</a> team is working on a tool set for vector graphics in the web browser, <a href="http://github.com/kamicane/art">A.R.T.</a> You could use binary formats to embed your vector based graphics in formats like... TrueType!
The <a href="http://www.ape-project.org/">APE (Ajax Push Engine) project</a> brings socket programming to the JavaScript platform.
Digg's <a href="http://blog.digg.com/?p=621">MXHR stream</a> parses multipart encoded data and extracts the parts for various uses. This could provide a packaging model for various widgets or data packets.
</li></li></li></ul>
<p><a href="http://blog.calyptus.eu/seb/2009/05/png-parser-in-javascript/">Parsing Base64 Encoded Binary PNG Images in JavaScript</a> was originally published by at <a href="http://blog.calyptus.eu">Calyptus</a> on May 20, 2009.</p>http://blog.calyptus.eu/seb/2009/05/client-side-dependency-strateg2009-05-17T12:06:00Z2009-05-17T12:06:00Zhttp://blog.calyptus.euinfo@calyptus.eu<h4 id="this-post-is-in-response-to-an-off-site-discussion-about-modular-dependency-strategies-but-i-figured-id-post-it-here-for-future-reference">This post is in response to an off-site discussion about modular dependency strategies. But I figured I’d post it here for future reference.</h4>
<p>The <a href="http://blog.calyptus.eu/source-code/">Calyptus Web Resource Manager</a> is a project that can on compile-time or on runtime handle your JavaScript, CSS, and other client-side dependencies. You can keep source code as separate files on the server or pre-compile packages (such as .ZIP, .DLL or .JAR). Currently source is available only on the .NET/Mono platforms but the concept is valid for all platforms. </p>
<h2 id="syntax">Syntax</h2>
<p>The syntax is largely inspired to be compatible with <a href="http://www.scriptdoc.org/">ScriptDoc</a> and <a href="http://www.ecmascript.org/es4/spec/overview.pdf">ECMAScript 4 Draft</a> import statement. In the top of your file you add the dependencies that your file relies on:</p>
<div class="highlight"><pre><code class="js"><span class="cm">/*</span>
<span class="cm">@import [package, ]filename</span>
<span class="cm">@include [package, ]filename</span>
<span class="cm">@build [package, ]filename</span>
<span class="cm">@compress [always|release|never]</span>
<span class="cm">*/</span>
</code></pre></div>
<p><em>Don’t worry, we’re not going to ruin your precious open-source project with inline docs. Read on.</em></p>
<p><strong>@import</strong> - Indicates that this file has a dependency on the referenced file and that it needs to be included in the final document (implicitly before this one). The other file may be a JavaScript file, CSS, image, Flash or something else. The project is fully extensible.</p>
<p><strong>@include</strong> - Same as @import but also indicates that the referenced file should be merged into this one on compile or runtime.</p>
<p><strong>@build</strong> - Same as @include but also merges any nested @import statements. Allowing you to create a single packaged file.</p>
<p><strong>@compress</strong> - Indicate whether the document should use a compression tool (such as YUI compressor) or not. Defaults to “release”, which means that it won’t compress during the debug stage.</p>
<p>You can reference a file by either filename/namespace or package + filename/namespace. You may include wild cards to reference an entire path or namespace. If you’re referencing another file in the same package, you can exclude the package name.</p>
<p>If you are running ASP.NET you can exclude the package if you’re referencing an assembly that is already referenced in your Web.config.</p>
<p>If you’re in a .js file, the filename will automatically look for files ending in .js.</p>
<p>This allows you to do a namespace like syntax on prepackaged files:</p>
<div class="highlight"><pre><code class="js"><span class="cm">/*</span>
<span class="cm">@import MooTools.Core.*</span>
<span class="cm">@import MooTools.More.URI</span>
<span class="cm">*/</span>
</code></pre></div>
<p>If you want to use the runtime view generating tools the syntax depends on what View Engine you’re running. For ASP.NET WebForms you can use the following controls:</p>
<div class="highlight"><pre><code class="xml"><span class="nt"><c:Import</span> <span class="na">src=</span><span class="s">"filename"</span> <span class="na">runat=</span><span class="s">"server"</span> <span class="nt">/></span>
<span class="nt"><c:Import</span> <span class="na">assembly=</span><span class="s">"package"</span> <span class="na">name=</span><span class="s">"namespace/filename"</span> <span class="na">runat=</span><span class="s">"server"</span> <span class="nt">/></span>
<span class="nt"><c:Include</span> <span class="err">...</span> <span class="nt">/></span>
<span class="nt"><c:Build</span> <span class="err">...</span> <span class="nt">/></span>
</code></pre></div>
<p>In the future this will be integrated into the ASP.NET ScriptManager as well. For other view engines the syntax would be much prettier.</p>
<h2 id="example">Example</h2>
<p><em>MyBaseStyle.css</em></p>
<div class="highlight"><pre><code class="css"><span class="nt">div</span><span class="nc">.BaseClassItem</span> <span class="p">{</span>
<span class="k">background-image</span><span class="o">:</span> <span class="sx">url(MyBaseImage.png)</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div>
<p><em>MyBaseClass.js</em></p>
<div class="highlight"><pre><code class="js"><span class="c1">// @import MyTheme.css</span>
<span class="kd">var</span> <span class="nx">MyBaseClass</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">Class</span><span class="p">({</span>
<span class="nx">initialize</span><span class="o">:</span> <span class="kd">function</span><span class="p">(){</span>
<span class="k">this</span><span class="p">.</span><span class="nx">element</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">Element</span><span class="p">(</span><span class="s1">'div'</span><span class="p">,</span> <span class="p">{</span> <span class="nx">className</span><span class="o">:</span> <span class="s1">'BaseClassItem'</span> <span class="p">});</span>
<span class="p">}</span>
<span class="p">});</span>
</code></pre></div>
<p><em>MyChildClass.js</em></p>
<div class="highlight"><pre><code class="js"><span class="c1">// @import MyBaseClass.js</span>
<span class="kd">var</span> <span class="nx">MyChildClass</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">Class</span><span class="p">({</span>
<span class="nx">Extends</span><span class="o">:</span> <span class="nx">MyBaseClass</span><span class="p">,</span>
<span class="p">...</span>
<span class="p">});</span>
</code></pre></div>
<p><em>MyView.aspx</em></p>
<div class="highlight"><pre><code class="xml"><span class="nt"><c:Include</span> <span class="na">src=</span><span class="s">"MyChildClass.js"</span> <span class="nt">/></span>
</code></pre></div>
<p><em>OUTPUT:</em></p>
<div class="highlight"><pre><code class="xml"><span class="nt"><link</span> <span class="na">href=</span><span class="s">"MyBaseStyle.css"</span> <span class="na">rel=</span><span class="s">"stylesheet"</span> <span class="na">type=</span><span class="s">"text/css"</span> <span class="nt">/></span>
<span class="nt"><script</span> <span class="na">src=</span><span class="s">"MyBaseClass.js"</span> <span class="na">type=</span><span class="s">"text/javascript"</span><span class="nt">></script></span>
<span class="nt"><script</span> <span class="na">type=</span><span class="s">"text/javascript"</span><span class="nt">></span>
var MyChildClass=new Class({Extends:MyBaseClass,...});
<span class="nt"></script></span>
</code></pre></div>
<p>Since I used the included command the file is included in the output document. All it’s dependencies are automatically added to the document through links.</p>
<p>Any referenced file is only added once to the output. So it’s no problem adding multiple references to the same resource in partial views or by indirect dependencies.</p>
<p><em>MyOtherView.aspx</em></p>
<div class="highlight"><pre><code class="xml"><span class="nt"><c:Import</span> <span class="na">src=</span><span class="s">"MyChildClass.js"</span> <span class="nt">/></span>
<span class="nt"><c:Import</span> <span class="na">src=</span><span class="s">"MyBaseClass.js"</span> <span class="nt">/></span>
</code></pre></div>
<p><em>OUTPUT:</em></p>
<div class="highlight"><pre><code class="xml"><span class="nt"><link</span> <span class="na">href=</span><span class="s">"MyBaseStyle.css"</span> <span class="na">rel=</span><span class="s">"stylesheet"</span> <span class="na">type=</span><span class="s">"text/css"</span> <span class="nt">/></span>
<span class="nt"><script</span> <span class="na">src=</span><span class="s">"MyBaseClass.js"</span> <span class="na">type=</span><span class="s">"text/javascript"</span><span class="nt">></script></span>
<span class="nt"><script</span> <span class="na">src=</span><span class="s">"MyChildClass.js"</span> <span class="na">type=</span><span class="s">"text/javascript"</span><span class="nt">></script></span>
</code></pre></div>
<p>In the sample above, I import the base class after the child class. Since the child is dependent on the base, it will be included first. Therefore the second reference to MyBaseClass.js is excluded.</p>
<h2 id="typical-work-flow---late-optimization">Typical Work Flow - Late Optimization</h2>
<p>Typically you would only use the @import statement in all your resources. You should only reference any direct resources that your code or style sheet uses. Indirect files are referenced by the referenced resources so that if a dependency changes, you don’t have to update all your reliers. Your views will only reference the direct resources that it is using by import statements as well.</p>
<p>This will generate a lot of <script> and <link> tags in your documents. This is not good for production where you want to minimize the overhead of multiple requests. That’s when you start building clusters.</p>
<p><em>Common.css</em></p>
<div class="highlight"><pre><code class="css"><span class="c">/*</span>
<span class="c">@build Headers.css</span>
<span class="c">@build Footers.css</span>
<span class="c">@build MyBaseStyle.css</span>
<span class="c">*/</span>
</code></pre></div>
<p><em>Common.js</em></p>
<div class="highlight"><pre><code class="js"><span class="cm">/*</span>
<span class="cm">@build MooTools.Core.Fx.Tween</span>
<span class="cm">@build MyChildClass.js</span>
<span class="cm">*/</span>
</code></pre></div>
<p>Now I can include the cluster Common.js in my view:</p>
<div class="highlight"><pre><code class="xml"><span class="nt"><c:Import</span> <span class="na">src=</span><span class="s">"Common.css"</span> <span class="nt">/></span>
<span class="nt"><c:Import</span> <span class="na">src=</span><span class="s">"Common.js"</span> <span class="nt">/></span>
...
<span class="nt"><c:Include</span> <span class="na">src=</span><span class="s">"MyChildClass.js"</span> <span class="nt">/></span>
</code></pre></div>
<p><em>OUTPUT:</em></p>
<div class="highlight"><pre><code class="xml"><span class="nt"><link</span> <span class="na">href=</span><span class="s">"Common.css"</span> <span class="na">rel=</span><span class="s">"stylesheet"</span> <span class="na">type=</span><span class="s">"text/css"</span> <span class="nt">/></span>
<span class="nt"><script</span> <span class="na">src=</span><span class="s">"Common.js"</span> <span class="na">type=</span><span class="s">"text/javascript"</span><span class="nt">></script></span>
</code></pre></div>
<p>The MyChildClass.js reference and all it’s dependencies are ignored since those file has already been included in the document by Common.css and Common.js. You can for example add these clusters to your Master view to automatically optimize all your partial views. If you remove a reference from your cluster it won’t break any of your code, since those files are individually added by your partial views to your document.</p>
<p>This pattern will allow you to do late optimization of your load-time by grouping only the files that are commonly used in to clusters. Leaving edge-case files into the outer branches of your site. To accomplish this I recommend that you use a modular framework such as <a href="http://www.mootools.net/">MooTools</a>.</p>
<p>Your clusters should be named and composed in relevant packages for your site, not in packages of JavaScript frameworks. For example, DON’T create a MooTools.js cluster that includes all MooTools files.</p>
<p><em>By default, @include and @build commands are evaluated as @import during the debug stage. That makes it easy to find the references to your source code with debugging tools such as FireBug.</em></p>
<h2 id="messing-up-your-beautiful-source-use-place-holders">Messing Up Your Beautiful Source? Use Place Holders</h2>
<p>If you’re working with a consultant project you can just put all your references in the source file. That makes it very easy to work with. But if you have an open-source project you may not want to mess up the source with dependency references. Instead, use place holder files that @include the original source and references the dependency place holders using @import.</p>
<p><em>Fx.js</em></p>
<div class="highlight"><pre><code class="js"><span class="cm">/*</span>
<span class="cm">@import Class.Extras.js</span>
<span class="cm">@include Real/Source/Fx/Fx.js</span>
<span class="cm">*/</span>
</code></pre></div>
<p><em>Fx.CSS.js</em></p>
<div class="highlight"><pre><code class="js"><span class="cm">/*</span>
<span class="cm">@import Fx.js</span>
<span class="cm">@import Element.Style.js</span>
<span class="cm">@include Real/Source/Fx/Fx.CSS.js</span>
<span class="cm">*/</span>
</code></pre></div>
<p>Now you can reference your place holders to get dependencies instead of the original source files.</p>
<h2 id="what-about-my-cdn">What about my CDN?</h2>
<p>You can use a CDN to store your clusters. Just reference the full URIs in your import statements. There is a pre-built class that does this with MooTools on Google. Just @import GoogleAPIs.MooTools.</p>
<p>I will add an <strong>@embedded</strong> syntax to reference other files that have already been included. That way you could write your own like this:</p>
<p><em>MooTools-Cluster-Google.js</em></p>
<div class="highlight"><pre><code class="js"><span class="cm">/*</span>
<span class="cm">@import http://ajax.googleapis.com/ajax/libs/mootools/1.2.2/mootools-yui-compressed.js</span>
<span class="cm">@embedded MooTools.Core.*</span>
<span class="cm">*/</span>
</code></pre></div>
<p>If you reference this cluster in your view, all references to your local MooTools files will be ignored since it they are already included in the Google cluster.</p>
<h2 id="include-on-images">@include on Images</h2>
<p>If @include filename.png is used in a style-sheet, every instance of url(filename.png) will automatically be replaced with base64 embedded data at runtime. This is only used on the runtime version since this content can’t be sent to IE browsers. IE browsers will get the url(filename.png) reference intact.</p>
<p>This also works with view/document Include commands. In that case an <img> tag is rendered with a link or embedded content depending on the browser capabilities.</p>
<p>This pattern allows you to do late load time optimization of image dependencies.</p>
<h2 id="getting-started">Getting Started</h2>
<p>As always, begin by <a href="http://blog.calyptus.eu/source-code/">checking out the source</a>.</p>
<p><a href="http://blog.calyptus.eu/seb/2009/05/client-side-dependency-strateg/">Client Side Dependency Strategy</a> was originally published by at <a href="http://blog.calyptus.eu">Calyptus</a> on May 17, 2009.</p>http://blog.calyptus.eu/seb/2009/03/large-object-storage-for-nhibernate-part-2-storage-options2009-03-29T03:24:06Z2009-03-29T03:24:06Zhttp://blog.calyptus.euinfo@calyptus.eu<p><em>This is part 2 of a series describing Large Object Storage (BLOB) in a Domain Driven fashion. Be sure to read <a href="http://blog.calyptus.eu/seb/2009/03/large-object-storage-for-nhibernate-and-ddd-part-1-blobs-clobs-and-xlobs/">Part 1 about the new base classes introduced by this project</a>.</em></p>
<h2 id="physical-storage-considerations">Physical Storage Considerations</h2>
<p>So, what are you options of storing large data objects in your relational database? This is actually not an easy problem to solve. Because a relational database is designed for small pieces of well structured data. Making a table, row or column too large will cause various problems with fragmenting, indexing and table scans.</p>
<p>Because of this, vendors have implemented data columns that store large data separately from the rest of the row. This typically means they can’t be used for indices or searches. They’re still internal to the RDMS and are fully covered by ACID transactions and backup procedures. You would typically keep the large object data on the same discs as the actual database itself. This can limit your overall performance and scalability. Additionally, the vendor API might support streaming for reading, while not supporting streaming for writing.</p>
<p>To remedy this situation vendors have come up with various ways of storing your data externally to your database (typically in a file system) while storing references to your data in the database and allowing you to access it and manage access control through your RDMS. This typically means that operations on these files are not covered by ACID transactions<sup>1</sup> and backup procedures. External storage allows you to save disc space, since multiple rows and tables can share a reference to the same data file in a true denormalized fashion. <a href="http://en.wikipedia.org/wiki/Content-addressable_storage">Content-addressable storage</a> (CAS) solutions are especially suitable for this kind of storage. You can use external storage with or without RDMS integration.</p>
<p>So to sum up your physical storage options:
* In-table RDMS storage
* Out-of-table RDMS storage
* External storage using RDMS integration
* External storage using NHibernate client</p>
<p>Because of these issues, various vendors have implemented more than one solution and there isn’t a consistent best-practise of working with large data. You have to chose the storage solution that is most appropriate for your particular requirements.</p>
<p><em><sup>1</sup> In this series I will only cover complete replacements of data rather than changes of data. This is done by exchanging one blob object for another as described in <a href="http://blog.calyptus.eu/seb/2009/03/large-object-storage-for-nhibernate-and-ddd-part-1-blobs-clobs-and-xlobs/">Part 1</a>. Therefore ACID transactions on individual data changes aren’t going to be important for external storage. The entire blob will be written to storage. If the entire transaction succeeds, the reference will be changed. Otherwise the reference will remain at the old data.</em></p>
<h2 id="data-transfer-considerations">Data Transfer Considerations</h2>
<p>Accessing in-row data is usually sent with the rest of the data result of the query. This means that it is not typically viable for streaming because the entire row is always read in to memory.</p>
<p>For out-of-table storage some vendors doesn’t send large object data with the rest of the row result. That means that it can be requested and streamed in pieces. However, because ADO.NET doesn’t offer a requirement and API for this, it is usually done in vendor specific implementations. Some vendors require the data reader to remain open while reading the stream. This makes it unsuitable for NHibernate since we would like to work with our entities in a disconnected fashion. So in this case, we would have to query that row and column again to open the data connection when needed (lazy loading).</p>
<p>When external storage is used, only a reference to the data is sent with the query result. The actual data transfer is usually done over a protocol completely separate to the RDMS connection. Sometimes it isn’t even communicating with the same machine as the database. This makes it a very scalable solution. It will also allow us to open that connection and stream the data without querying the row and column of the database again.</p>
<p>Because of the inconsistent ways of accessing the data, we will addressing this at the client level in a vendor specific fashion. More on this in Part 3 - NHibernate Mappings.</p>
<h2 id="small-in-table-data-types">Small In-Table Data Types</h2>
<p>All RDMS has small in-row data types. In-row binary and text data. Such as VARBINARY(size) or VARCHAR(size). These are typically limited to around 4000-8000 bytes of data and are therefore not suitable for large objects. You would typically just map these to memory using byte[] and string. If you currently only have small amounts of data but expect it to scale, you can start off using one of these small data types and map it to Blob and Clob objects and then scale as you need it.</p>
<p>Large object storage options are highly vendor specific. I’ll cover a few common vendors.</p>
<h2 id="microsoft-sql-server">Microsoft SQL Server</h2>
<p><a href="http://msdn.microsoft.com/en-us/library/ms188362.aspx">VARBINARY(MAX)</a>, <a href="http://msdn.microsoft.com/en-us/library/ms187993.aspx">VARCHAR(MAX)</a> - These types is used to store in-table binary and text data at up to 2 GB. The practical performance and scalability limitations involved in storing data in-table usually means that you want to keep data in these columns to a few MB. Using the <a href="http://msdn.microsoft.com/en-us/library/3517w44b(VS.80).aspx">UPDATETEXT</a> command in SQL Server you can write changes to the database in chunks.</p>
<p><a href="http://msdn.microsoft.com/en-us/library/ms187993.aspx">XML(DOCUMENT), XML(CONTENT)</a> - You can use the XML data type to store up to 2 GB of XML data per column. The same practical limitations as for VARBINARY and VARCHAR applies. You can specify either DOCUMENT or CONTENT to indicate whether the data has to comply to either a full XML document or an XML fragment. The Xlob base class allows for both complete documents and fragments.</p>
<p><em><a href="http://msdn.microsoft.com/en-us/library/ms187993.aspx">IMAGE, TEXT and NTEXT</a> - These data types are now deprecated and will be removed in future versions of SQL Server. Use VARCHAR(MAX) or VARBINARY(MAX) instead.</em></p>
<p><a href="http://technet.microsoft.com/en-us/library/bb933993.aspx">FILESTREAM</a> - If your data is more more than 1 MB on average, you should consider the new FILESTREAM data type introduced in SQL Server 2008. It stores data out-of-table in the NTFS file system. The data size is limited only by the local NTFS file system. Each row that uses a FILESTREAM column must have UNIQUEIDENTIFIER. The FILESTREAM column is completely integrated with SQL Server, it’s backup facilities and it’s client software.</p>
<p><a href="http://www.codeplex.com/sqlrbs">Microsoft SQL Remote Blob Storage (RBS)</a> - Microsoft has introduced a new plug-in API for external storage used together with SQL Server. This will allow any storage solution provider to hook into Microsoft’s common API. It’s installed both on the server and the client. The server handles garbage collecting and manages the references to various BLOBs in the storage solution. This is a flexible and highly scalable solution and it integrates nicely into the SQL Server product. If you want to leave the external storage API to the client, read on to External Storage.</p>
<h2 id="oracle">Oracle</h2>
<p><a href="http://download.oracle.com/docs/cd/B19306_01/server.102/b14220/datatype.htm#i3237">BLOB, CLOB, NCLOB</a> and <a href="http://download.oracle.com/docs/cd/B19306_01/server.102/b14220/datatype.htm#i13446">XMLType</a> - These are out-of-table data types for storing binary, text and XML data up to 4 GB. Oracle 10g and above supports up to 8 terabytes of storage depending on your CHUNK setting for the table. NCLOB stores text data in a Unicode national character set.</p>
<p>Oracles LOB types are all stored out-of-table and referenced using Lob locators. This makes them suitable for the disconnected environment used by NHibernate.</p>
<p>Oracle allows for XML operations to take place on the server which, in the future, could be used to speed up operations of a XmlReader generated by a Xlob.</p>
<p><em><a href="http://download.oracle.com/docs/cd/B19306_01/server.102/b14220/datatype.htm#i4146">LONG and LONG RAW</a> - These data types are now deprecated. They can store 2 GB of data. Use CLOB or BLOB instead.</em></p>
<p><a href="http://download.oracle.com/docs/cd/B19306_01/appdev.102/b14249/adlob_intro.htm#sthref36">BFILE</a> - Oracle has reference type that points to files on the local file system. You can read these files (up to 4GB) via the Oracle API. You can’t write to them though. You can change the reference to another file in the file system. So if you create a reference to an existing file using Blob.Create(“filepath”), the NHibernate mappings will be able to change out the reference to the new file. You can also open up a directory where NHibernate can store new files. In both cases, both the Oracle server and client will need access to this directory. BFILEs are an external storage solution. Oracle doesn’t handle write transactions, garbage-collecting of files nor backup procedures.</p>
<h2 id="postgresql">PostgreSQL</h2>
<p><a href="http://www.postgresql.org/docs/8.3/interactive/datatype-binary.html">BYTEA</a>, <a href="http://www.postgresql.org/docs/8.3/interactive/datatype-character.html">TEXT</a> and <a href="http://www.postgresql.org/docs/8.3/interactive/datatype-xml.html">XML</a> - Used for in-table binary, text and XML data respectively. Current APIs doesn’t support streaming of these types. They will have to be read in to memory all at once.</p>
<p><a href="http://www.postgresql.org/docs/8.3/interactive/storage-toast.html">TOAST</a> - PostgreSQL normally stores it’s data in tuples of 8 kb which doesn’t allow the above data types to be very large. Using TOAST large columns are automatically stored out-of-table. It also has mechanisms for compressing data and trying to fit it in to rows if possible. TOAST isn’t it’s own data type but can be used to expand BYTEA, TEXT and XML columns to a maximum of 1 GB.</p>
<p><a href="http://www.postgresql.org/docs/8.3/interactive/largeobjects.html">Large Objects</a> - PostgreSQL supports the notion of Large Objects. These are stored out of table but within the management of the RDMS itself. Each new object is given it’s own ID and it is this ID that is referenced in the data tables. These objects are read and manipulated using a special API. Each object can be referenced several times and across tables. As far as I know it is not garbage-collected nor handled by backup solutions. So this solution can be compared to other external solutions even though it is managed by PostgreSQL itself. Since this solution shares objects for the entire database you will have to incorporate your own custom garbage collecting solution.</p>
<p>Large Objects are useful when you need to store data larger than 1 GB. The documented limit is 2 GB but in practice you can store files of several GB depending on the file system. The Large Object API will also allow you to stream the data instead of reading it all into memory. Therefore this is the preferred solution for storing large data on PostgreSQL.</p>
<h2 id="mysql">MySQL</h2>
<p><a href="http://dev.mysql.com/doc/refman/5.1/en/blob.html">BLOB and TEXT</a> - These columns are used to store binary and text data up to 4 GB. MySQL doesn’t have a column for XML data. TEXT is the recommended column type for XML. These columns are stored out-of-table but MySQL doesn’t support streaming of data. This means that each object will have to be read into memory in it’s entirety.</p>
<p><a href="http://www.primebase.org/">PrimeBase Technologies</a> are currently working on a <a href="http://www.blobstreaming.org/">Blob streaming infrastructure</a> over HTTP to be integrated into MySQL. It uses their <a href="http://www.primebase.org/">XT storage engine</a>.</p>
<p>For other storage engines, you will need to look to external storage.</p>
<h2 id="external-storage">External Storage</h2>
<p>If you prefer to decouple your large object storage solution from the database you can use a completely external storage solution. In this case, you would store a reference to the data blob in your relational table. Usually as a fixed length binary or GUID/UUID. The data is stored in a completely external solution with no communication with the database. This makes this solution completely vendor independent and highly scalable.</p>
<p>The NHibernate.Lob client handles the communication with both the external storage solution as well as the database. Your client should on certain intervals (nightly?) let NHibernate.Lob scan all mapped tables for external references. It will then garbage collect the data blobs in the external storage that are no longer referenced.</p>
<p>The <a href="/source-code/">NHibernate.Lob</a> project includes a common API for external storage solutions for use with NHibernate. Included is also a file-system based <a href="http://en.wikipedia.org/wiki/Content-addressable_storage">CAS</a> storage option to get you started. High-end <a href="http://en.wikipedia.org/wiki/Content-addressable_storage">CAS</a> solutions such as EMC’s Centera or Caringo’s CAStor are very suitable for this kind of storage if you have extreme scalability or accessibility needs. They’re also useful if you need to comply with local regulations that require you to never delete data.</p>
<h2 id="text-and-xml-types">Text and XML Types</h2>
<p>Clob and Xlob are structured data since they have a specific format (Text and XML). These can be stored in various ways depending on your vendor’s specific data columns. Text can be stored in various different character sets. XML can be serialized as binary XML in storage or saved using various character sets. If your vendor does provide a specific Text or XML data type that is suitable for large objects I would recommend that you use it. This will allow the RDMS to handle the format and serialization constraints. Any compliant software can handle and display the data without further user interaction.</p>
<p>However, if you use external storage or want to utilize the various compression options mentioned in Part 5, you can store your Clob and Xlob data in any binary column as well. There by letting the client determine the serialization format.</p>
<h2 id="getting-started">Getting Started</h2>
<p>The full source code to Calyptus.Lob and appropriate NHibernate mappings are available at our <a href="http://github.com/calyptus/calyptus.lob/">Calyptus.Lob project at GitHub</a>.</p>
<h2 id="more-in-this-series">More in This Series</h2>
<p>In the next part of this series I’m going to describe how you can use the <a href="http://blog.calyptus.eu/source-code/">NHibernate.Lob</a> project to map up these storage options to your <a href="/seb/2009/03/large-object-storage-for-nhibernate-and-ddd-part-1-blobs-clobs-and-xlobs/">Blobs, Clobs and Xlobs</a> in your NHibernate Entities.</p>
<p><a href="/seb/2009/03/large-object-storage-for-nhibernate-and-ddd-part-1-blobs-clobs-and-xlobs/">Part 1 - BLOBs, CLOBs and XLOBs</a></p>
<ul>
<li><em>Part 2 - Storage Options</em></li>
<li>Part 3 - NHibernate Mappings</li>
<li>Part 4 - External Storage</li>
<li>Part 5 - Compression Options</li>
</ul>
<p><a href="http://blog.calyptus.eu/seb/2009/03/large-object-storage-for-nhibernate-part-2-storage-options/">Large Object Storage for NHibernate - Part 2 - Storage Options</a> was originally published by at <a href="http://blog.calyptus.eu">Calyptus</a> on March 29, 2009.</p>http://blog.calyptus.eu/seb/2009/03/html-5-current-browser-support-part-1-introduction2009-03-23T23:02:13Z2009-03-23T23:02:13Zhttp://blog.calyptus.euinfo@calyptus.eu<p>The <a href="http://www.whatwg.org/specs/web-apps/current-work/">HTML 5 working draft</a> is continuing it’s development of the future support for HTML 5. This includes new tags, attributes and a strong specification of how clients should interact with old and new elements. What I find even more intriguing, is the standardization of many advanced JavaScript DOM features (such as editable content, drag and drop). Most of which has been available to IE users for more than a decade. This is one area that standards has been particularly slow to adopt. With the current beta versions of Safari, Chrome and Firefox these new browsers are finally ready to leave IE behind (yes, even IE 8).</p>
<p>Many people are still frightened of implementing code according to a working draft. Especially since it’s <a href="http://wiki.whatwg.org/wiki/FAQ#When_will_HTML_5_be_finished.3F">not scheduled to be complete until 2012</a>. In my opinion, those fears are largely unfounded at this point. The primary reason for this is that many of the features have been available in IE for many years and the <a href="http://www.whatwg.org/specs/web-apps/current-work/">HTML 5 specification</a> centers around keeping some historical compliance. So the primary threat for lagging cross browser functionality has already been eliminated. It is also the <a href="http://www.whatwg.org/">WHATWG</a>’s estimate that browsers will have full compliance and people will have started utilizing this new standard long before it is finalized. For these reasons, by the time you read this, you may already be a late adopter.</p>
<p>However, there are still some quirks that you need to be aware of. I’ve been working on cross browser layers of the HTML 5 specifications since 2007 including backwards compatible code for older browsers. This code has been used in production and little of it has changed since mid-2008. Therefore I’ve <a href="http://github.com/calyptus">started work</a> on introducing these features to my JavaScript framework of choice, <a href="http://www.mootools.net/">MooTools</a>. While I refactor my code for this purpose I thought I might introduce some of the quirks that you might come across in your own endeavors.</p>
<h2 id="coming-up">Coming up</h2>
<ul>
<li>Part 2 - Drag and Drop, Copy and Paste</li>
<li>Part 3 - Range and Selection</li>
<li>Part 4 - ContentEditable and ExecCommand</li>
</ul>
<p><a href="http://blog.calyptus.eu/seb/2009/03/html-5-current-browser-support-part-1-introduction/">HTML 5 Current Browser Support - Part 1 - Introduction</a> was originally published by at <a href="http://blog.calyptus.eu">Calyptus</a> on March 23, 2009.</p>http://blog.calyptus.eu/seb/2009/03/large-object-storage-for-nhibernate-and-ddd-part-1-blobs-clobs-and-xlobs2009-03-12T01:25:38Z2009-03-12T01:25:38Zhttp://blog.calyptus.euinfo@calyptus.eu<h4 id="this-is-the-first-in-a-series-of-posts-describing-the-design-considerations-involved-with-storing-binary-large-object-blob-data-with-nhibernate-and-how-it-led-me-to-start-a-project-im-currently-calling-a-hrefhttpblogcalyptuseusource-codenhibernateloba">This is the first in a series of posts describing the design considerations involved with storing Binary Large OBject (BLOB) data with NHibernate and how it led me to start a project I’m currently calling <a href="http://blog.calyptus.eu/source-code/">NHibernate.Lob</a>.</h4>
<p><em>Note that the samples here are focused mainly on NHibernate but the pattern can be applied to many different persistence models. I’m considering support for DB4o for example.</em></p>
<h2 id="lazy-streaming-of-data">Lazy Streaming of Data</h2>
<p>The typical way to store binary data in NHibernate entities would be as a byte[] array. After all the basic premise of NHibernate entities is that the data is stored in-memory in the first level cache. For smaller binary data this is just fine. We don’t even really our columns to be lazy.</p>
<p>If we start adding larger data the first problem one might notice is that <em>the data is loaded every time the entity is loaded</em>. This is a common question around NHibernate user groups. This can quite easily be solved by separating it out to a lazy loaded entity or using lazy columns.</p>
<p>If we add even larger files we start wasting precious memory. This is especially problematic in high concurrency applications and web applications. A (very) common scenario would be to store image data together with an entity. At this point we shouldn’t ever keep the entire file in memory. Instead we should stream the data piece by piece from the persistent storage to whatever we want to use it for.</p>
<p>At this point, the actual data is never stored in the in-memory entity. Only pointer data is stored about where to find the information. It goes beyond the concept of lazy loading since only a piece of the data in available in memory at any point.</p>
<p><em>Note that this is NOT really related to the concept of a document database. We’re talking about large serialized objects (500 kb+ if I had to give a number) such as images, videos or large document files.</em></p>
<p><em>I also mention the term: pointers. In the context of this article series I don’t mean memory pointers but rather a reference to where one can find the real complete data. This may be in-memory, on disk, remote or distributed etc.</em></p>
<h2 id="streaming-data-types---the-current-state-of-adonet">Streaming Data Types - The Current State of ADO.NET</h2>
<p>So what data type will we use as the pointer to this data? Our domain model is suppose to be persistence ignorant so one of the common .NET types would be nice. There are typically three common structured types of large data stored in modern databases: Raw binary data, Text and XML. Binary Large OBjects are typically called BLOBs. Text or Character Large OBjects are sometimes called CLOBs. How you store the data and in which column types is very RDMS provider specific. From now on I will call these three types as just LOBs.</p>
<p>Now, the in-memory types for these would typically be byte[], string and XmlDocument. The streamed versions would be Stream, TextReader and XmlReader. However, this gives us some problems. The contract of these three abstract classes are more than just pointers to where to get the data. They also contain the current reading position of the stream. This means that we can only read from that entity ONCE during it’s life time. They also implement IDisposable and keep a data reading connection open and expect to be closed and disposed of.</p>
<p>There’s really no common way for working with streamed data in ADO.NET since everything. In fact, the typical example for dealing with LOBs in ADO.NET involves reading the full data into memory using IDataReader.GetBytes(…). Some providers have supplied there own solutions to this issue (such as <a href="http://msdn.microsoft.com/en-us/library/system.data.idatarecord.getbytes.aspx">OracleLob</a> and <a href="http://msdn.microsoft.com/en-us/library/system.data.idatarecord.getbytes.aspx">SqlBytes</a>). The most common solution seems to be to inherit Stream in their custom solutions. You can still read it several times by first cloning the Lob object but it isn’t really a nice solution for a domain model. They also imply that the connection is already open. What we really need is a type from which we can create readers.</p>
<p>Thankfully our friends on the Java end of things have already thought about this. In Java there are <a href="http://java.sun.com/j2se/1.4.2/docs/api/java/sql/Blob.html">Blob</a> and <a href="http://java.sun.com/j2se/1.4.2/docs/api/java/sql/Clob.html">Clob</a> interfaces which fits just this purpose. They can create both reader and writer streams. It is also nicely implemented in both JDBC and Hibernate.</p>
<p><em>Another issue with TextReader and XmlReader is that we have no way to write to them but this is not really an issue as I will describe at the end of this article.</em></p>
<h2 id="introducing-new-data-types---blob-clob-and-xlob">Introducing New Data Types - Blob, Clob and Xlob</h2>
<p>So to remedy this situation I’ve suggested that three new base classes are added to our .NET domain models. The contracts of these are pretty simple.</p>
<div class="highlight"><pre><code class="c#"><span class="k">namespace</span> <span class="nn">Calyptus.Lob</span>
<span class="p">{</span>
<span class="k">public</span> <span class="k">abstract</span> <span class="k">class</span> <span class="nc">Blob</span>
<span class="p">{</span>
<span class="k">public</span> <span class="k">abstract</span> <span class="n">Stream</span> <span class="nf">OpenReader</span><span class="p">();</span>
<span class="k">public</span> <span class="k">virtual</span> <span class="k">void</span> <span class="nf">WriteTo</span><span class="p">(</span><span class="n">Stream</span> <span class="n">output</span><span class="p">);</span>
<span class="p">}</span>
<span class="k">public</span> <span class="k">abstract</span> <span class="k">class</span> <span class="nc">Clob</span>
<span class="p">{</span>
<span class="k">public</span> <span class="k">abstract</span> <span class="n">TextReader</span> <span class="nf">OpenReader</span><span class="p">();</span>
<span class="k">public</span> <span class="k">virtual</span> <span class="k">void</span> <span class="nf">WriteTo</span><span class="p">(</span><span class="n">TextWriter</span> <span class="n">writer</span><span class="p">);</span>
<span class="k">public</span> <span class="k">virtual</span> <span class="k">void</span> <span class="nf">WriteTo</span><span class="p">(</span><span class="n">Stream</span> <span class="n">output</span><span class="p">,</span> <span class="n">Encoding</span> <span class="n">encoding</span><span class="p">);</span>
<span class="p">}</span>
<span class="k">public</span> <span class="k">abstract</span> <span class="k">class</span> <span class="nc">Xlob</span>
<span class="p">{</span>
<span class="k">public</span> <span class="k">abstract</span> <span class="n">XmlReader</span> <span class="nf">OpenReader</span><span class="p">();</span>
<span class="k">public</span> <span class="k">virtual</span> <span class="k">void</span> <span class="nf">WriteTo</span><span class="p">(</span><span class="n">TextWriter</span> <span class="n">writer</span><span class="p">);</span>
<span class="k">public</span> <span class="k">virtual</span> <span class="k">void</span> <span class="nf">WriteTo</span><span class="p">(</span><span class="n">XmlWriter</span> <span class="n">writer</span><span class="p">);</span>
<span class="k">public</span> <span class="k">virtual</span> <span class="k">void</span> <span class="nf">WriteTo</span><span class="p">(</span><span class="n">Stream</span> <span class="n">output</span><span class="p">,</span> <span class="n">Encoding</span> <span class="n">encoding</span><span class="p">);</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div>
<p>Basically there’s a PULL and a PUSH method to get the data from LOB. The WriteTo methods are NOT away to write data to the LOBs. It’s a way to PUSH the data from the LOB into a writer.</p>
<p>Why an abstract base class instead of an interface? This is a common debate in .NET. But since this pattern is overwhelmingly used most often in the .NET Framework (Stream, TextReader and XmlReader are a few examples) I figured it’d be best to keep that trend. It also allows for virtual methods to be added later (such as Java’s getBytes, position and length) without recompilation of inheritors.</p>
<p>I’m sure that these contracts are going to be very much debated since it involves the core of the domain model which, in the NHibernate world, should be persistence ignorant. You can still easily switch out the ORM and let that ORM handle these new types. The best would be if Microsoft’s Patterns and Practises team introduced these new types as a common practise and perhaps even into a System.Data.Lobs namespace.</p>
<p><em>Some of you may be thinking that this pattern makes the domain model aware of it’s repository. But it really doesn’t. No more than lazy loaded entities and collection does. You can even save it to another repository. More on that later.</em></p>
<h2 id="writing-to-blobs---dont">Writing to Blobs - Don’t</h2>
<p>You may have noticed that unlike the Java interface I didn’t put any way to write to the LOBs in the base contract. This is because you shouldn’t persist anything until a Flush (or SaveChanges) style event. If it’s a new entity, the row doesn’t exists and there may not be anything to write the data to. It could also not even be part of the row. It may be stored as it’s own “entity” and shared by multiple other entities. In this case you would override their data. Data should be written all together in an atomic manner.</p>
<p>So how do I change the data? You replace the LOB pointer (the Blob, Clob or Xlob objects) with something that points to some other data source with the new data. This can be from a file, a stream, memory, or maybe a custom implementation which combines or converts data on-the-fly. This will allow you to build pipelining patterns. It can even be an other LOB in your database. NHibernate will tell your LOB object when and where to write itself to.</p>
<p>This is also the same way Hibernate handles Blob and Clob in Java. It actually throws exceptions if you try to write to it’s Blobs or Clobs.</p>
<h2 id="finally-some-code">Finally Some Code</h2>
<p>Let’s start by defining a domain model. Let’s just stick to one single entity called Product. With a binary image file, a long description text and an XML file which contains further specifications.</p>
<div class="highlight"><pre><code class="c#"><span class="k">public</span> <span class="k">class</span> <span class="nc">Product</span>
<span class="p">{</span>
<span class="k">public</span> <span class="kt">int</span> <span class="n">ID</span> <span class="p">{</span> <span class="k">get</span><span class="p">;</span> <span class="k">set</span><span class="p">;</span> <span class="p">}</span>
<span class="k">public</span> <span class="kt">string</span> <span class="n">Title</span> <span class="p">{</span> <span class="k">get</span><span class="p">;</span> <span class="k">set</span><span class="p">;</span> <span class="p">}</span>
<span class="k">public</span> <span class="n">Blob</span> <span class="n">Image</span> <span class="p">{</span> <span class="k">get</span><span class="p">;</span> <span class="k">set</span><span class="p">;</span> <span class="p">}</span>
<span class="k">public</span> <span class="n">Clob</span> <span class="n">Description</span> <span class="p">{</span> <span class="k">get</span><span class="p">;</span> <span class="k">set</span><span class="p">;</span> <span class="p">}</span>
<span class="k">public</span> <span class="n">Xlob</span> <span class="n">Specifications</span> <span class="p">{</span> <span class="k">get</span><span class="p">;</span> <span class="k">set</span><span class="p">;</span> <span class="p">}</span>
<span class="p">}</span>
</code></pre></div>
<p>To read from these three LOBs you would use either the PULL or PUSH patterns (OpenReader or WriteTo). The following sample fetches a Product from the database. It then writes the image data to a HttpResponse. Then it writes the specifications to disk using a custom XmlWriter. Finally it reads the first line of the description.</p>
<div class="highlight"><pre><code class="c#"><span class="k">using</span> <span class="p">(</span><span class="n">ISession</span> <span class="n">session</span> <span class="p">=</span> <span class="n">sessionFactory</span><span class="p">.</span><span class="n">OpenSession</span><span class="p">())</span>
<span class="p">{</span>
<span class="n">Product</span> <span class="n">product</span> <span class="p">=</span> <span class="n">session</span><span class="p">.</span><span class="n">Get</span><span class="p"><</span><span class="n">Product</span><span class="p">>(</span><span class="m">100</span><span class="p">);</span>
<span class="n">Response</span><span class="p">.</span><span class="n">Clear</span><span class="p">();</span>
<span class="n">Response</span><span class="p">.</span><span class="n">ContentType</span> <span class="p">=</span> <span class="s">"image/jpeg"</span><span class="p">;</span>
<span class="n">Response</span><span class="p">.</span><span class="n">BufferOutput</span> <span class="p">=</span> <span class="k">false</span><span class="p">;</span>
<span class="n">product</span><span class="p">.</span><span class="n">Image</span><span class="p">.</span><span class="n">WriteTo</span><span class="p">(</span><span class="n">Response</span><span class="p">.</span><span class="n">OutputStream</span><span class="p">);</span>
<span class="k">using</span> <span class="p">(</span><span class="n">XmlWriter</span> <span class="n">writer</span> <span class="p">=</span> <span class="n">XmlWriter</span><span class="p">.</span><span class="n">Create</span><span class="p">(</span><span class="s">@"C:\MyFiles\SomeData.xml"</span><span class="p">))</span>
<span class="p">{</span>
<span class="n">product</span><span class="p">.</span><span class="n">Specifications</span><span class="p">.</span><span class="n">WriteTo</span><span class="p">(</span><span class="n">writer</span><span class="p">);</span>
<span class="p">}</span>
<span class="k">using</span> <span class="p">(</span><span class="n">TextReader</span> <span class="n">reader</span> <span class="p">=</span> <span class="n">product</span><span class="p">.</span><span class="n">Description</span><span class="p">.</span><span class="n">OpenReader</span><span class="p">())</span>
<span class="p">{</span>
<span class="kt">string</span> <span class="n">firstLine</span> <span class="p">=</span> <span class="n">reader</span><span class="p">.</span><span class="n">ReadLine</span><span class="p">();</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div>
<p>Changing the LOB data involves replacing the instance with another one. You can do this by using one of the built-in implementations using the static overloaded Blob.Create(), Clob.Create() and Xlob.Create() methods. The data can come from files, streams, memory, the web or your own implementations. You could for example create your own implementation which combines two files into one on the fly as it is written to the database.</p>
<p>The following sample loads a product from the database, replaces the image with a file from disk, replaces the description with an in-memory string, replaces the specifications with one from the web and then saves it all to the database.</p>
<div class="highlight"><pre><code class="c#"><span class="k">using</span> <span class="p">(</span><span class="n">ISession</span> <span class="n">session</span> <span class="p">=</span> <span class="n">sessionFactory</span><span class="p">.</span><span class="n">OpenSession</span><span class="p">())</span>
<span class="k">using</span> <span class="p">(</span><span class="n">ITransaction</span> <span class="n">transaction</span> <span class="p">=</span> <span class="n">session</span><span class="p">.</span><span class="n">BeginTransaction</span><span class="p">())</span>
<span class="p">{</span>
<span class="n">Product</span> <span class="n">product</span> <span class="p">=</span> <span class="n">session</span><span class="p">.</span><span class="n">Get</span><span class="p"><</span><span class="n">Product</span><span class="p">>(</span><span class="m">100</span><span class="p">);</span>
<span class="n">product</span><span class="p">.</span><span class="n">Image</span> <span class="p">=</span> <span class="n">Blob</span><span class="p">.</span><span class="n">Create</span><span class="p">(</span><span class="s">@"C:\MyFolder\MyImage.jpg"</span><span class="p">);</span>
<span class="n">product</span><span class="p">.</span><span class="n">Description</span> <span class="p">=</span> <span class="n">Clob</span><span class="p">.</span><span class="n">Create</span><span class="p">(</span><span class="s">"My short description."</span><span class="p">);</span>
<span class="n">product</span><span class="p">.</span><span class="n">Specifications</span> <span class="p">=</span> <span class="n">Xlob</span><span class="p">.</span><span class="n">Create</span><span class="p">(</span><span class="k">new</span> <span class="n">Uri</span><span class="p">(</span><span class="s">"http://domain/document.xml"</span><span class="p">));</span>
<span class="n">transaction</span><span class="p">.</span><span class="n">Commit</span><span class="p">();</span>
<span class="p">}</span>
</code></pre></div>
<p>Note that in the above sample it’s not a reference to the file and the web that is stored in the database. The actual data is read and stored. <em>Depending on your application, the use of WebRequests could be prohibited or a potential security issue to load unknown remote XML documents.</em></p>
<p>Note that you can also use the implicit casting of the LOB types to implicitly cast some known types. There are also Blob.Empty, Clob.Empty and Xlob.Empty singletons that you can use to insert empty data. This is not null. The following sample implicitly casts a Stream to a Blob, a String to a Clob and removes the product’s specification by replacing it with an empty one.</p>
<div class="highlight"><pre><code class="c#"><span class="k">using</span> <span class="p">(</span><span class="n">ISession</span> <span class="n">session</span> <span class="p">=</span> <span class="n">sessionFactory</span><span class="p">.</span><span class="n">OpenSession</span><span class="p">())</span>
<span class="k">using</span> <span class="p">(</span><span class="n">ITransaction</span> <span class="n">transaction</span> <span class="p">=</span> <span class="n">session</span><span class="p">.</span><span class="n">BeginTransaction</span><span class="p">())</span>
<span class="p">{</span>
<span class="n">Product</span> <span class="n">product</span> <span class="p">=</span> <span class="n">session</span><span class="p">.</span><span class="n">Get</span><span class="p"><</span><span class="n">Product</span><span class="p">>(</span><span class="m">100</span><span class="p">);</span>
<span class="n">product</span><span class="p">.</span><span class="n">Image</span> <span class="p">=</span> <span class="n">Request</span><span class="p">.</span><span class="n">Files</span><span class="p">[</span><span class="s">"uploadedImage"</span><span class="p">].</span><span class="n">InputStream</span><span class="p">;</span>
<span class="n">product</span><span class="p">.</span><span class="n">Description</span> <span class="p">=</span> <span class="n">Request</span><span class="p">.</span><span class="n">Form</span><span class="p">[</span><span class="s">"description"</span><span class="p">];</span>
<span class="n">product</span><span class="p">.</span><span class="n">Specifications</span> <span class="p">=</span> <span class="n">Xlob</span><span class="p">.</span><span class="n">Empty</span><span class="p">;</span>
<span class="n">transaction</span><span class="p">.</span><span class="n">Commit</span><span class="p">();</span>
<span class="p">}</span>
</code></pre></div>
<p>If you prefer a less anemic domain model you could keep the LOB internal to the class and do reads and writes with custom logic.</p>
<div class="highlight"><pre><code class="c#"><span class="k">public</span> <span class="k">class</span> <span class="nc">Product</span>
<span class="p">{</span>
<span class="k">public</span> <span class="kt">int</span> <span class="n">ID</span> <span class="p">{</span> <span class="k">get</span><span class="p">;</span> <span class="k">set</span><span class="p">;</span> <span class="p">}</span>
<span class="k">private</span> <span class="n">Blob</span> <span class="n">image</span><span class="p">;</span>
<span class="k">public</span> <span class="k">void</span> <span class="nf">ChangeImage</span><span class="p">(</span><span class="n">Stream</span> <span class="n">input</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">this</span><span class="p">.</span><span class="n">image</span> <span class="p">=</span> <span class="n">input</span><span class="p">;</span>
<span class="p">}</span>
<span class="k">public</span> <span class="k">void</span> <span class="nf">CopyImageFrom</span><span class="p">(</span><span class="n">Product</span> <span class="n">product</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">this</span><span class="p">.</span><span class="n">image</span> <span class="p">=</span> <span class="n">product</span><span class="p">.</span><span class="n">image</span><span class="p">;</span>
<span class="p">}</span>
<span class="k">public</span> <span class="k">void</span> <span class="nf">WriteImageTo</span><span class="p">(</span><span class="n">Stream</span> <span class="n">output</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">this</span><span class="p">.</span><span class="n">image</span><span class="p">.</span><span class="n">WriteTo</span><span class="p">(</span><span class="n">output</span><span class="p">);</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div>
<p>Note that the stream used in ChangeImage() will not be read and disposed of until your Session is Flushed. Depending on your application design this pattern may not be useful.</p>
<p><em>In the current version, the StreamBlob class which wraps a Stream as a Blob can only be read once if the Stream is not seekable. Therefore each instance can only be saved to one entity and not reused. In future versions it may replace the internal stream pointer to the one in the repository once the first one is saved. The same goes for the TextReader and XmlReader wrappers.</em></p>
<h2 id="getting-started">Getting Started</h2>
<p>The full source code to Calyptus.Lob and appropriate NHibernate mappings are available at our <a href="http://github.com/calyptus/calyptus.lob/">Calyptus.Lob project on GitHub</a>. I’ll make some official builds once the code stabilizes.</p>
<h2 id="coming-up">Coming up</h2>
<ul>
<li><a href="http://blog.calyptus.eu/seb/2009/03/large-object-storage-for-nhibernate-part-2-storage-options/">Part 2 - Storage Options</a></li>
<li>Part 3 - NHibernate Mappings</li>
<li>Part 4 - External Storage</li>
<li>Part 5 - Compression Options</li>
</ul>
<p><a href="http://blog.calyptus.eu/seb/2009/03/large-object-storage-for-nhibernate-and-ddd-part-1-blobs-clobs-and-xlobs/">Large Object Storage for NHibernate - Part 1 - BLOBs, CLOBs and XLOBs</a> was originally published by at <a href="http://blog.calyptus.eu">Calyptus</a> on March 12, 2009.</p>http://blog.calyptus.eu/seb/2009/03/welcome-to-the-calyptus-life-blog2009-03-11T12:40:29Z2009-03-11T12:40:29Zhttp://blog.calyptus.euinfo@calyptus.eu<p>Today is the start of our effort to share some of the thoughts and ideas we have about advanced web application development at Calyptus Life.</p>
<p>We will soon introduce a Wiki for the various open-source projects we have going right now. Until them, feel free to poke around our <a href="http://blog.calyptus.eu/source-code/">source code</a>.</p>
<p><a href="http://blog.calyptus.eu/seb/2009/03/welcome-to-the-calyptus-life-blog/">Welcome to the Calyptus Life blog!</a> was originally published by at <a href="http://blog.calyptus.eu">Calyptus</a> on March 11, 2009.</p>