Download

Go here to download ready-to-run and source distributions of Jangaroo. more...

Showing posts with label JavaScript. Show all posts
Showing posts with label JavaScript. Show all posts

Friday, June 8, 2012

Managing JavaScript Libraries with Maven

Jangaroo's mission is to create and apply Enterprise software development tools to Web development. While Jangaroo is known for its ActionScript-to-JavaScript compiler jooc, we also care about "native" JavaScript development. In the process of creating a Maven-based build process for compiled ActionScript, we were baffled that there was not even a Maven-based build process for JavaScript. In fact, there is no version-, dependency- and artifact-managing build process for JavaScript at all (as far as we know).

In the course of creating Jangaroo 2, and utilizing a new feature of Java Servlet 3, we can now present a light-weight and easy way to use Maven for managing JavaScript library versions and their dependencies and adding them to your Web application.

Approach and Related Work

Note that we are not the only ones who provide Maven JAR artifacts containing repackaged JavaScript libraries. Independently, James Ward created a project called WebJars, which does something quite similar as proposed here. He also uses JARs and Maven, but the difference is that he is targeting other Web servers and languages, essentially using the JAR as a ZIP. Our approach is to use the Servlet 3 JAR layout, essentially putting all Web resources under the path META-INF/resources. The main advantage is that every Servlet 3 compliant application server (Jetty 8, Tomcat 7, Glassfish 3, Resin 4, WebSphere 8, WebLogic 12) understands this standardized format (JSR-315) immediately without any plugin, Servlet or other extension. Another benefit is that you can add files to the JAR that are not supposed to be Web resources, e.g. other meta data and of course Java classes, so that you could for example unite code implementing the client- and server-part of some service in a single JAR.

An organizational advantage we have over James' artifacts is that we deploy all repackaged artifacts of Open Source libraries (almost all JavaScript libraries are Open Source!) directly to the Maven Central repository, saving you the need to add <repository> sections to your Maven POM and leaving you independent from our Maven repository being online.

Try It Out!

The first available artifact is a repackaging of Ext JS 3.4.0 (yes, it is an old version, but take it as an example of the mechanism) and a very light-weight example project using this artifact. The following steps are needed once to have an environment to develop Maven-managed JavaScript Web applications:

  1. Install a recent Java SDK like 6 or 7
  2. Install Apache Maven, current version 3.0.4
  3. [Install git]
Checkpoint: calling mvn -v on a shell should give you output similar to this:
> mvn -v

Apache Maven 3.0.4 (r1232337; 2012-01-17 09:44:56+0100)

Maven home: ...
Java version: 1.6.0_25, vendor: Sun Microsystems Inc.
Java home: ...
Default locale: ...
OS name: ...


To try the Ext JS example project, download or clone from github:
> git clone git://github.com/fwienber/webjars-extjs-example.git

Then, all you need to do is build the project (which downloads all needed stuff from Maven Central when invoked for the first time) and start a Jetty Web server via Maven, simply via
> mvn jetty:start
The default port is 8080, so you can watch the resulting Web app in any browser using http://localhost:8080. To prove that Ext JS has been loaded and works, the app just opens an Ext alert box.

Only three small text files are needed to
  • download the desired Ext JS library,
  • download  Jetty webserver,
  • configure Jetty to serve the contents of the Ext JS library and the local project resources, and
  • start Jetty in development mode on localhost port 8080.
The three files are
  1. pom.xml ―The Maven "Project Object Model" that configures the build,
  2. src/main/webapp/index.html ―the HTML file containing the simply JavaScript application,
  3. src/main/webapp/WEB-INF/web.xml ―Java Web configuration file.
Let me give you a walk-through of these three files.
pom.xml is the key to letting Maven create and manage your application. Fortunately, this POM is quite simple, since we take advantage of conventions, and Maven prefers convention over configuration.

<?xml version='1.0' encoding='UTF-8'?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>net.jangaroo.webjars.examples</groupId>
  <artifactId>webjars-extjs-example</artifactId>
  <version>0.1-SNAPSHOT</version>
  <packaging>war</packaging>

  <name>webjars-extjs-example</name>
  <description>...</description>

  <build>
    <plugins>
      <plugin>
        <groupId>org.mortbay.jetty</groupId>
        <artifactId>jetty-maven-plugin</artifactId>
        <version>8.1.3.v20120416</version>
      </plugin>
    </plugins>
  </build>

  <dependencies>
    <dependency>
      <groupId>net.jangaroo.com.sencha</groupId>
      <artifactId>ext-js</artifactId>
      <version>3.4.0</version>
    </dependency>
  </dependencies>
</project>

After the inevitable XML header (which is always the same), every Maven project model needs to specify a unique combination of groupId and artifactId, and a version. For this test project, they really do not matter, since the generated artifact will never be deployed to a repository. The packaging type war tells Maven that we are building a Web application (Java terminology: Web ARchive). name and description are just for human readability.
The only plugin that needs to be mentioned explicitly is jetty-maven-plugin, and you also need to specify the exact version you want to use. Well, it's still only a couple of lines to download and start a Web server, isn't it?
To download a serve all Ext JS 3.4.0 resources, a simple dependency suffices, thanks to the standard-conform JAR layout and central deployment of the artifact.

index.html is the simplest HTML page you could think of that starts an Ext JS application. The only interesting thing here is how the resource paths have been chosen, because you do not actually "see" the Ext JS resources―they are hidden in a JAR somewhere in your local Maven repository (see below). For every JavaScript library repackaging, we have to agree upon a standard path where it is served in the Web application. For Ext JS, we chose /ext-js/ as the base path. The file structure below this base path is exactly as in the ZIP that can be obtained from Sencha's Ext JS 3 download page.

web.xml is only needed to disable a Jetty feature ("file locking") that hinders development, and to satisfy the Maven WAR plugin, which would have to be configured to not <failOnMissingWebXml> otherwise.

The Jetty Maven plugin configures Jetty in a clever way: library dependencies like Ext JS, which have been downloaded to your local Maven repository automatically (usually ~/.m2/repository), are served directly from that JAR. That means no files are extracted, not even the JARs needed by your Web app are copied. All local project Web resources are taken directly from src/main/webapp. Try changing index.html (e.g. the alert message) and reload in the browser: it works without further build or deployment!

If you want a complete Web app that could be deployed into a remote Java application server, you can invoke mvn package, and a directory target/webjars-extjs-example-0.1-SNAPSHOT is created which contains all needed resources. This directory is also zipped as target/webjars-extjs-example-0.1-SNAPSHOT.war.

Unpacking all Web resources contained in JARs, e.g. to deploy them to a static Web server like Apache HTTP Server, is also possible via Maven, but I'll cover that in a later update.

Mighty Maven

What the simple example does not show is that, if ext-js 3.4.0 had a dependency on some other artifact, say ext-core 1.0, you wouldn't have to care about it. Simply specifying a dependency on ext-js lets Maven collect all transitive dependencies and add the corresponding artifacts to your Web application. Furthermore, if two libraries you use depend upon the same third library, Maven takes care of the third library being included only once, and offers dependency management to resolve possible version conflicts.

All in all, given the complexity of the problem it solves, Maven provides a light-weight way to manage complex JavaScript Web applications, making it easy to keep track of libraries used, their versions, and their dependencies. Furthermore, Java Servlet 3 helps structure a Web application as a composite of modules instead of a huge bunch of files, which speeds up build process and test system start-up.

What Next?

Next is to repackage and deploy or release several JavaScript libraries. There are several options, so here we need your vote and maybe even your help. We plan to repackage the libraries and versions we (plan to) use at CoreMedia, for example jQuery 1.7, Ext JS 4.1.0, RequireJS 2.0.1, and CKEditor 3.6.2. If you need other JavaScript libraries or other versions, you can simply create an issue at JooTrack/LIBS. We need the exact location of a download package or all single download links, and if you could provide additional information which files are usually needed in an actual deployment (e.g. no examples, no documentation, ...), it would help a lot. So start refactoring your large-scale JavaScript project today―clean up the JavaScript mess!

Monday, January 30, 2012

Simulating ActionScript Rest Parameter in JavaScript

Let's continue the series on simulating ActionScript language features in JavaScript with something similar to optional parameters, namely the ... (rest) parameter.
ActionScript allows you to specify that an arbitrary number of parameters follow the usual named and optional parameters by prefixing the last parameter with three dots (see "The ... (rest) parameter"):

 1 public function join(separator:String, first:String,
                        ...strings:Array):String {
 2   strings.splice(0, 0, first);
 3   return strings.join(separator);
 4 }

As you can see, all formal parameters following the named parameters are combined into an array. Since the ... (rest) parameter is always of type Array, you can safely skip the type annotation.

The arguments variable in JavaScript
Fortunately, JavaScript offers access to all formal parameters of a function call, even if they are not declared as named parameters of the function, through the arguments variable. For some reason, this variable does not actually hold an Array. Instead, it is an array-like Object with indexed properties and a length property. This means that we cannot invoke Array methods directly on arguments, but there is a trick: Call the slice function retrieved from the Array prototype, as described e.g. here, and the result is a "real" Array:

Array.prototype.slice.call(arguments)

At the same time, we can cut off the named parameters, since the first parameter to slice is the index where to start slicing from the source array. Thus, the resulting JavaScript code generated by the Jangaroo compiler jooc is the following:

 1 function join(separator, first) {
     var strings = Array.prototype.slice.call(arguments,2);
 2   strings.splice(0, 0, first);
 3   return strings.join(separator);
 4 }

Note the slice parameter value 2, which takes care of cutting off the two named parameters. Also note how Jangaroo only adds an auxiliary statement, but keeps all other statements exactly the same, which makes debugging a lot easier.

Side note: arguments is an array is AS3
The only thing Jangaroo currently does not yet implement correctly is that in ActionScript, the arguments variable is actually always an array! Thus, instead of writing

 1 public function joinComma() {
 2   return arguments.join(",");
 3 }

which would be fine in ActionScript, in Jangaroo you currently have to use

 1 public function joinComma(...args) {
 2   return args.join(",");
 3 }

This is to save the overhead of always converting arguments into an array. Of course, the compiler could analyze whether arguments is used as an Array at all and only add the conversion code then. I just filed the bug as issue JOO-12.

Saturday, January 28, 2012

Simulating ActionScript Parameter Default Values in JavaScript

To continue our series on simulating ActionScript language features in JavaScript, this episode is about parameter default values.
In contrast to JavaScript, ActionScript allows to specify a default value for a function (or method) parameter, like so:

 1 public function insult(s = "fool") {
 2   return "you " + s;
 3 }

The idea is that when the method is called without providing a value for parameter s, the default value "fool" will be used. Details on what additional rules hold for declaring parameter default values are given in the Adobe documentation on function parameters and a bit less reference-like e.g. in a tutorial on Ntt.CC.

Why, it's easy, isn't it?
A straight-forward implementation in JavaScript (similar to the solution suggested by Bernd Paradies for FalconJS) would be to replace undefined paramter values by their default value:

 1 function insult(s) {
     if (s === undefined) {
       s = "fool";
     }
 2   return "you " + s;
 3 }

(We could now start a discussion on whether it shouldn't be
     if (typeof s === "undefined") {
because in JavaScript, undefined may be redefined, but anybody who does so is, excuse me, a fool.)
In JavaScript, when you omit a parameter when calling a function, the parameter value indeed is undefined. But it is a fallacy to assume that every undefined parameter must be replaced by its default value! Consider the following ActionScript class:

 1 public class DefaultParameterTest {
 2
 3   public function DefaultParameterTest() {
 4     trace("1. " +  insult ("nerd"));
 5     trace("2. " +  insult ());
 6     trace("3. " +  insult (undefined));
 7   }
 8
 9   public function insult(s = "fool") {
10     return "you " + s;
11   }
12 }

What do you think will be traced? Try it out, and you'll see that explicitly handing in undefined does not trigger the default value! Thus, the result is
1. you nerd
2. you fool
3. you undefined

(Careful, never call a JavaScript programmer "you undefined"!)
Using the straight-forward JavaScript implementation, the result would be
1. you nerd
2. you fool
3. you fool

As you can see, using parameter default values depends on the number of formal arguments, not on their value. This is the reason why they are also called optional parameters, and why after a parameter with a default value, all following parameters must also specify a default value.

Getting it right
Thus, the correct JavaScript equivalent (as generated by the Jangaroo compiler jooc) is to check the number of formal parameters. Fortunately, this can be realized by checking arguments.length:

 1 function insult(s) {
     if (arguments.length < 1) {
       s = "fool";
     }
 2   return "you " + s;
 3 }

In fact, this solution even provides better runtime performance when using multiple optional parameters, since multiple arguments.length checks can be nested, while the undefined checks in the straight-forward solution would be sequential. Consider the following example:

 1 public function foo(p1, p2, p3 = 3, p4 = 4) {
 2   return p1 + p2 + p3 + p4;
 3 }

The JavaScript fragment generated by Jangaroo looks like so:

 1 function foo(p1, p2, p3, p4) {
     if (arguments.length < 4) {
       if (arguments.length < 3) {
         p3 = 3;
       }
       p4 = 4;
     }
 2   return p1 + p2 + p3 + p4;
 3 }

Jangaroo even optimizes undefined default values, since left out actual parameters are undefined in JavaScript, anyway.

Alternative Solutions
We also thought of using a switch statement, generating code like the following:

 1 function foo(p1, p2, p3, p4) {
     switch (arguments.length) {
       case 0:
       case 1:
       case 2:
         p3 = 3;
         // fall through
       case 3:
         p4 = 4;
     }
 2   return p1 + p2 + p3 + p4;
 3 }

At first sight, this solution seems more efficient, because arguments.length is only evaluated once. The reason why we still chose the nested if code layout is that the switch solution becomes either long or non-robust when using many non-optional parameters. Note the lines case 0: and case 1: in the example above: although the method should never be called with less than two parameters (since the first two are not optional), we want to handle that case, too, since Jangaroo code may be called from JavaScript, where no function signature check is performed. When the function is called with fewer parameters than required, you would still expect the default values to pop in. So the case statements would pile up when there are many non-optional parameters. However, I could imagine a mixed solution like the following:

 1 function foo(p1, p2, p3, p4) {
     switch (Math.max(arguments.length, 2)) {
       case 2:
         p3 = 3;
         // fall through
       case 3:
         p4 = 4;
     }
 2   return p1 + p2 + p3 + p4;
 3 }

The most intelligent solution would be to dynamically decide whether the prior length check pays off against adding several case statements. I guess I'll add that to the Jangaroo backlog!

Tuesday, December 6, 2011

Simulating ActionScript in JavaScript: Private Members Cont'd

In the first blog post about simulating private members in ActionScript, I compared different solutions to represent private members in JavaScript. The solution implemented by Jangaroo is to rename private members, so that they do not name-clash with private members of the same class, defined on a different inheritance level.
Bernd Paradies asked how Jangaroo solves untyped access to private members. Since the answer is rather extensive, I'll dedicate this follow-up post to the topic.

The short answer is: it's the nature of the beast. Untyped access to private members cannot be detected at compile time. However, potential access to private members can be detected at compile time and generate code that repeats the check at runtime, although Jangaroo does not implement this at the moment.

Typed Private Member Access
Let's recall the typical way to access private members, which can easily be detected by the Jangaroo compiler:

 1 public class Foo {
 2   private var foo:String = "bar";
 3   public function getFoo():String {
 4     return foo;
 5   }
 6 }

As foo in line 4 is resolved to a field of the class, and there is no implicit this access in JavaScript, the Jangaroo compiler adds this. before foo. Also, as field foo is declared private, the compiler renames it to foo$1 (see previous blog post) and thus generates the following JavaScript code for the body of method getFoo():

 4     return this.foo$1;

Where is My Type?
Now consider the following code (which does not really make sense, but illustrates the point):

 1 public class Foo {
 2   private var foo:String = "bar";
 3   public function getFoo():String {
 4     var untypedThis:Object = this;
 5     return untypedThis.foo;
 6   }
 7 }

Of course, in this simple example it would still be possible to determine the runtime type of untypedThis statically, but it is easy to imagine a situation where this is not possible. In the current Jangaroo implementation, the following JavaScript code would be generated for the body of method getFoo():

 4     var untypedThis = this;
 5     return untypedThis.foo;

As you can see, the compiler fails to detect that foo actually refers to the private member, does not rename the access to foo$1, and thus the undefined value of untypedThis.foo would be returned.

A (partial) solution is to let the compiler detect any expression of the form untyped.private-member and generate code that takes into account the runtime type of untyped, like so:

 5     return untyped[untyped instanceof Foo ? 'foo$1' : 'foo'];

Trusting today's JavaScript JIT compilers' optimizations, this code should be moved to a utility function to avoid double evaluation of the untyped expression (which could have side effects):

 5     return Foo.$class.get(untyped, 'foo');
 
where Foo.$class provides access to Jangaroo's "meta class" of Foo, and get() would be implemented there like this:

  public function get(object:Object, property:String):* {
    return object[object instanceof this.publicConstructor ?
      property + this.inheritanceLevel : property];
  }

The Weak and the Wicked
Looking at ActionScript carefully, you'll notice that the identifier or expression left of the dot does not even have to be untyped, but may just be typed with a more general type, and the same scenario may apply. For example, assume our class Foo is a subclass of dynamic class Bar, we could replace Object by Bar, and the example would still work! Even if Bar is not dynamic, we could access the private member foo through a local variable of type Bar using square brackets, like so:

public class Foo extends Bar {
  private var foo:String = "bar";
  public function getFoo():String {
    var weaklyTypedThis:Bar = this;
    return weaklyTypedThis['foo'];
  }
}

The Flex compiler does not complain, and at runtime, in Flash, indeed the private member is accessed. Compare this to the following example, where we try to access the private member of a superclass:

public class Baz extends Foo {
  public function Baz() {
    super();
    var superThis:Foo = this;
    trace(superThis['foo']); // is undefined, not "bar"!
  }
}

Maybe you'll be surprised that the result is not any kind of access violation, but simply undefined—the private member of the superclass is simply not visible for the subclass! This is well emulated by Jangaroo through renaming private members. The Jangaroo compiler just has to take care to detect every correct access of a private member and rename the property upon access.

Turing Strikes Again
Things get even nastier when the property is a dynamic expression. Consider

  return this[veryComplicatedComputation()];

Here, too, we'd have to generate

  return Foo.$class.get(this, veryComplicatedComputation());

and extend the utility function by a runtime check whether the property is actually a private member name:

  public function get(object:Object, property:String):* {
    return object[object instanceof this.publicConstructor &&
      this.getMemberDeclaration("private", property) ?
        property + this.inheritanceLevel : property];
  }

Summary
Taken together, the important criterion when to generate code for possible private member access is not whether the left-hand expression of the dot is untyped, but the compiler has to check whether the complete property access is not typed. This is the case exactly when the property is not a compile-time constant or when the property cannot be resolved statically within the type of the expression, and in any case, the type of the expression must be a supertype of the current class or the current class itself.

To wrap up, the compiler would have to generate a call to the dynamic access function if and only if
  1. the left-hand expression could have a runtime type compatible to the current class and 
  2. the property expression could be one of the current class's private members.
Here, could means that it may be, but is not certain at compile time. If both conditions are certain at compile time, the compiler knows for sure that the code represents access to a private member, and can simply use the renamed property.

One more thing: I have been talking about read access of private members. The same strategy would have to be applied when writing members, resulting in another utility function set(object, property, value).

Optimize?
Of course, we could optimize runtime performance by using specific utility functions for the following three different cases:
  1. The property is definitely a private member, but the left-hand expression may or may not be of the current class: check instanceof only.
  2. The left-hand expression is definitely of the current class, but the property may or may not be a private member: check the property only.
  3. Both the left-hand expression may or may not be of the current class, and the property may or may not be a private member: check instanceof and the property.
However, my guess is that this optimization is not necessary. Firstly, checking instanceof and whether some string is contained in a fixed and not very large set are cheap operations. Secondly, when using ActionScript (and not untyped JavaScript), you should avoid untyped access of properties whenever possible. Thus, the situation is a rare case, and would only be implemented for full ActionScript semantic compatibility. If the developer wants better performance, she can either insert a type cast to convert the property access to a typed one, or move dynamic properties to a dedicated object whose type is statically incompatible with the current class (e.g. use Dictionary), and thus no runtime check would be generated.
If this solution proves to inflict significant runtime overhead, the compiler should issue a warning when it has to generate runtime checks.

Wednesday, November 30, 2011

Simulating ActionScript in JavaScript: Private Members

Inspired by Adobe's Bernd Paradies, member of the FalconJS / FlashRT team, blogging about his ideas on "Compiling ActionScript to JavaScript", I thought it was about time to give more details about how Jangaroo implements ActionScript language features. Because there are so many of them, blogging about them one at a time seems like a good idea. Let's start with "private members".

The Early Days
When people started simulating class-based OOP in JavaScript, many thought that it is not possible to define private members that support the principle of information hiding, because in JavaScript, all properties of an object are publicly accessible. So in many cases, the convention was used to let private members start with an underscore ("_"), meaning "don't use this member, best assume it is not existing at all".

The Reference Solution
As we all know in the meantime, it is possible to implement real information hiding in JavaScript, namely by using closures / lexical scoping. Douglas Crockford (who else?) has written a nice summary of this approach back in 2001: Private Members in JavaScript
So it looks like all we have to do to simulate ActionScript's private members in JavaScript is to use this pattern, right? Of course, it is not that easy, as in practice, there are two problems with private members à la Crockford:
  1. Runtime overhead
  2. Source-code placement
While the first is a general problem, the second is Jangaroo-specific.
The runtime overhead of closures results from the fact that in Crockford's solution, private members and privileged methods (methods that use private members) have to be defined inside the constructor. This means that the corresponding function objects are created for every instance of the class! This results in a performance penalty as well as in increased memory usage. Of course, this is not an issue for classes with few instances (extreme: singletons), but for classes like events, where you create many, many instances, this can quickly become a major problem.
The second problem, source-code placement, results from a specific Jangaroo feature. In debug mode, the Jangaroo compiler generates JavaScript code that keeps every piece of executable code at exactly the same line as in the ActionScript source file. This is a central feature of Jangaroo and very important for source-level debugging with a free choice of JavaScript debugger. So how can we place private and privileged methods inside the constructor, when in the ActionScript source code, they are not?

The Super-Secret Private This
For solving the source-code placement issue, I came up with the idea of a "shared private this" object. It turned out that this approach makes things a bit more complicated and only mitigates (but not removes) the runtime overhead, so it did not actually make it into Jangaroo, but still I'd like to present it here.
The basic idea is that private members, as opposed to being local variables in constructor scope, are properties of a single object that is defined inside the constructor. To be shared, this "private this" object has to be "injected" into, i.e. put in the surrounding lexical scope of every privileged method. Consider the following class, which is the ActionScript version of the example used by Douglas Crockford:

 1 public class Container {
 2   public var member:String;
 3   private var secret:int;

 4
 5   public function Container(param:String) {
 6     member = param;

 7     secret = 3;
 8   }
 9
10   public function stamp(string:String):String {
11     return member + string;
12   }
13
14   private function dec():Boolean {
15     if (secret > 0) {
16       secret -= 1;
17       return true;
18     } else {
19       return false;
20     }
21   }
22
23   public function service():String {
24     return dec() ? member : null;
25   }
26 }
To inject the "private this" into all privileged methods, we wrap these inside functions. What Douglas calls that, we'll call this$, and all other helper idenfiers are suffixed with $. Here is the resulting JavaScript code:

 1 Container = (function() {
 2   //public var member:String;
 3   //private var secret:int;
 4
 5   var Container = function(param) { var this$ = init$(this);
 6     this.member = param;
 7     this$.secret = 3;
 8   };
 9
10   Container.prototype.stamp = function(string) {
11     return this.member + string;
12   };
13
14   var dec = function() {
15     if (this.secret > 0) {
16       this.secret -= 1;
17       return true;
18     } else {
19       return false;
20     }
21   };
22
23   var service$ = function(this$){return function(){
24     return this$.dec() ? this.member : null;
25   };};
26
27   var init$ = function(o) {
28     var this$ =
{
29       dec: dec
30     };

31     o.service = service$(this$);
32     return this$;
33   }
34
35   return Container;
36 })();
Note how all executable code stays in exactly the same line, and how similar it can be expressed in JavaScript!
The following variants of Douglas' patterns were used:
  • Private methods are only defined once (reduces runtime overhead!), and are supposed to be called on the "private this". If they need to access public members, too, we could easily extend the "private this" by a reference to the "public this".
  • Privileged methods are wrapped in a function that receives the "private this", so that the current "private this" is in lexical scope. 
  • To keep all source code at its original location, all additional initialization is moved to a generated method init$. It creates the "private this" this$ with all private members, creates instances of privileged methods handing in this$, and assigns these to the "public this".

The Pragmatic Solution
As said above, this solution still introduces runtime overhead and a bit complexity, so we looked for a more efficient and simpler alternative. I mentioned the naive approach to use a simple naming convention. The real flaw in this approach is not that you can access "private" members when you should not be able to. When compiling ActionScript to JavaScript, we can check access rights on the ActionScript source, so this is not an issue. But what this approach breaks is that private members are also used to avoid name clashes between subclass and superclass!
Image you define a framework, and provide some base class that is supposed to be extended by clients of the framework. This base class usually provides members with different visibility. The private members (and the "internal" ones, which we neglect for now) can not be seen by a subclass. A client's subclass could define its own private members any way they like, as long as they don't name-clash with public or protected members of the superclass.
Now you update the framework and add a private method, as you think this should not change the framework API. But if we use the naive naming convention of prefixing all private members with an underscore (or the like), there is now the chance that the new private member of the superclass name-clashes with an existing private member of the client's subclass!
Jangaroo's pragmatic solution to separate private members of different inheritance levels within one class is to suffix private member names with a $ followed by the numeric inheritance level. Object has inheritance level 0, and a class extending X has the inheritance level of X plus one. This effectively prevents name-clashes between private members of the same name, but defined on different inheritance levels, and thus solves the framework update problem described above.
We used to compute the inheritance level of a class at runtime, but this made class loading and initialization more complex. Thus, we later decided to let the compiler compute the inheritance level. Of course this reduces "binary" compatibility, meaning that when you refactor a framework class e.g. by introducing an intermediate class in the inheritance hierarchy, clients will have to recompile their code, or the inheritance level of their subclass will be incorrect. But recompiling (without changing the source code) after updating a framework should not really be a problem.
To give a concrete example, here is the simplified generated JavaScript code for the example above. It is not exactly what Jangaroo would produce, as there are many other features covered by the Jangaroo Runtime, which I'll elaborate on in upcoming blog posts. Also, it does not illustrate the inheritance issue, but just imagine a subclass that also defines a private field secret, which would then be renamed secret$2.
 1 Container = (function() {
 2   //public var member:String;
 3   //private var secret:int;
 4
 5   var Container = function(param) {
 6     this.member = param;
 7     this.secret$1 = 3;
 8   };
 9
10   Container.prototype.stamp = function(string) {
11     return this.member + string;
12   };
13
14   Container.prototype.dec$1 = function() {
15     if (this.secret$1 > 0) {
16       this.secret$1 -= 1;
17       return true;
18     } else {
19       return false;
20     }
21   };
22
23   Container.prototype.service = function() {
24     return this.dec$1() ? this.member : null;
25   };
26
27  
return Container;
28 })();
Wrap-Up
Let me conclude with an overview of the four solutions for private members in JavaScript discussed here.
naive naming convention private members
(D. Crockford)
private this inheritance level suffix
(used in Jangaroo)
information hiding ✕ ✓ ✓ ✕
avoiding name-clashes ✕ ✓ ✓ ✓
no runtime overhead ✓ ✕ ✕ ✓
keep source lines ✓ ✕ ✓ ✓
As you can see in the table, there is no perfect solution (no column with checkmarks in every row), so for Jangaroo, we chose the pragmatic one that ensures good performance, while having some drawbacks on information hiding.

Outlook
Since performance is the only argument against the "private this" solution, we should investigate in performance analysis of today's JavaScript engines to quantify the time and space overhead introduced by the appoach. Maybe it is not that bad after all.
With all modern browsers supporting the JavaScript API Object.defineProperty(),we can improve Jangaroo's pragmatic solution's information hiding by defining all private members to not be enumerable. Taking a closer look at ActionScript semantics, actually, all class members are not enumerable, i.e. they are all not visible in a for ... in loop (only dynamic properties are).
There are several possible improvements to approximate ActionScript semantics more closely when relying on features of a modern JavaScript engine (ECMAScript 5). I'll come to these when discussing further ActionScript language features.

Thursday, November 10, 2011

Enterprise JavaScript with Jangaroo at PLASTIC / SPLASH 2011

While almost one year ago, we talked about "Jangaroo - AS3 w/o FlashPlayer", we thought it was about time to stress the other story, namely using Jangaroo for creating large-scale JavaScript applications. Because that's what we do very successfully at CoreMedia.
Several people (thanks Florian, Martin!) pointed us at a very fitting call for papers of the PLASTIC workshop at the SPLASH 2011. It is about Programming Languages And Systems Technologies for Internet Clients, with a focus on "compilation and runtime techniques for Internet client programming languages". Andreas kicked off the paper, Olaf and Dennis supported with proof-reading, LaTeX support and helpful comments, and I fleshed out the text and extended the parts of my expertise, namely the runtime, libraries, and build process. The paper, entitled Enterprise JavaScript with Jangaroo --
Using ActionScript 3 for JavaScript ”Programming in the Large”
, was accepted and can be downloaded here.
On 24th of October, I presented the corresponding talk at the PLASTIC workshop in Portland, Oregon. To not bore the part of the audience who already read the paper (maybe the empty set?), the slides show a third category of "how to execute another programming language in the browser", since providing a client-side interpreter seems to be a hot topic (mostly because of Google swiffy, but there are others, too). But have a look at the slides for yourself.
Before starting with the slides, I showed off the demos Box2D and Open Flash Charts, which I admit are a bit off-topic, since they rather belong to the "port Flash applications to HTML5" story than to "Enterprise JavaScript", but hey, they are always good to get people's attention ;-).
I hope I could get across what Jangaroo is able to, and got some interesting feedback. Adam Welc (Co-Chair of the workshop, in his other life working on Flash at Adobe) pointed me to another Flash-related Open Source projects, lightspark, which might help in getting on with Jangaroo's Flash re-implementation JooFlash.
People liked the way Jangaroo keeps the generated code close to the source, even keeping line numbers to allow source-level debugging.
The most motivating part when visiting a workshop or conference is of course about meeting people. I remembered Google's Mark Miller from the Ajax Experience 2009 in Boston, who here at PLASTIC co-authored a paper about traits.js: recommended read! Christian Hammer, Andreas Gampe, Aiman Erbad and several others also presented and we had interesting discussions. Was nice getting to know you guys!
Another JavaScript guru I also met first on Ajax Experience is Rik Arends, then at ajax.org, who now does a wonderful job over at Cloud9IDE. I tell you he's one of the good guys (even if we disagree on where to use typed languages and compilers)!
Finally, I loved the key note of David Ungar, Everything You Know (about Parallel Programming) Is Wrong! A Wild Screed about the Future; he really is a great presenter! Using Romeo and Juliet as an example of parallel actions that went wrong (and might have ended up "hunky-dory" in a different sequentialization) is really funky...

Wednesday, July 1, 2009

Jangaroo 0.3: Amplified ActionScript 3 Compatibility

It's been quite a while, but eventually, I am proud to announce Jangaroo 0.3, featuring highly improved ActionScript 3 compatibility and better build and deployment support:
  • interfaces, is
  • type casts (as and support for the legacy syntax)
  • bound methods (get this right!)
  • no more need to always use this.x instead of x
  • *-imports
  • fully AS3 compatible Array class
  • include directive
  • annotations (syntax only)
  • getter and setter methods (unfortunately not in IE)
  • automatic loading of dependent classes
  • Maven plugin provides Jangaroo with a flexible and powerful build system
Actually, many of these features were already introduced with the 0.2.x versions, but these were only released in the Jangaroo Maven repository. Now, the main effort was to round things off and update all documentation, so please go ahead and re-visit the Jangaroo Web site to find an updated story ("JavaScript 2 is dead, long live ActionScript 3!") and many new technical details.
Jangaroo Tool Support
Jangaroo has always used a subset of AS3, so that all AS3 tools like asdoc and IDEs can be used with Jangaroo code. As mentioned before, IntelliJ IDEA 8 supports ActionScript 3 and thus Jangaroo very well. To top that, we have implemented a Jangaroo IDEA plugin that will be released in the near future.
Porting AS3 Libraries to Jangaroo
Now that we have added all these nice AS3 language features, it is possible to translate most existing AS3 code with the Jangaroo compiler and run it in the browser (without any Flash plug-in!).
For example, we work on a port of FlexUnit and, with very few code changes, can already run its self-test-suite successfully in the browser. Our FlexUnit variant (of course called JooUnit!) will be released shortly, and some of the changes (missing semicolons, mixed DOS/UNIX line end encodings, asdoc glitches) will be submitted back to FlexUnit's main branch.
Of course, making the browser understand AS3 natively is only half the truth: Most AS3 code relies on standard libraries, namely Adobe's Flash API. Thus, another running project is to re-implement the Adobe Flash 9 API in Jangaroo as an adapter to native browser APIs (including canvas for drawing), so that most Flash code and even programs that draw and animate can be run, and as always when using Jangaroo: without a Flash plugin, thus seamlessly embedded into any HTML page!
Based on the Flash API, we recently also managed to compile the whole open source Flex 3.3 framework after some Jangaroo-friendly changes, but it is still quite some way to go until it would actually work, so we hope to get you ActionScript whiz people to contribute soon...
Have fun with a new way of experiencing JavaScript and ActionScript 3! We can't wait for your feedback!

Friday, July 18, 2008

Former Life and Birth of Jangaroo

Today, CoreMedia released a first version of the Jangaroo language and tools as Open Source. This was a wonderful moment for me, because this little project has a long history and played an important underpart in my professional life at CoreMedia over the last years. Please read about it, download, apply, spread the word! Be prepared to develop JavaScript like you never did before! The new Web site should offer you everything for a quick start. If not, feel free to ask! In this first blog post, however, you are going to hear Jangaroo's story, which covers its birth, coming of age, and reincarnation, if you want.
Trying Not to Be Eclectic
When in 2003, I had my first deeper dive into JavaScript, I never had thought it would become a passion. As most Java developers, I despised the style of eclectic programming that was then common among JavaScript hackers: To implement some requirement, they didn't use frameworks or libraries, but grabbed code snippets from other Web pages or forums and somehow assembled them to make them work on their Web pages. My impression was that it is hardly possible to implement serious business in such a patchwork language. But I had to, since a new Web application for CMS content entry waited to be designed and implemented.
Patching a Patchwork Language
My colleague Andreas Gawecki surely agreed on that and decided that, before even starting to dirty our hands on pure JavaScript, we needed a tool to transfer at least some of the amenities of Java to the JavaScript world. But because that was in the days before the Web 2.0 hype and the JavaScript Renaissance, there was no such tool. If you know Andreas, you may be able to imagine what happened: Within a week or so, he had gone through the JavaScript 2 / ECMAScript 4 standard proposals and came up with a compiler that could translate a subset of JavaScript 2 language features to JavaScript 1.x! Of course, Andreas had done research on how classes are usually simulated in JavaScript, and of course he had implemented those JS2-features a Java developer would need most desperately when having to program in JavaScript. I was one of his first Guinea Pigs, and also his primary sparring partner for the language. We started developing tons of JavaScript 2 code, because there were not even JavaScript 1 frameworks available, and we had to do all the cross-browser compatibility code ourselves. Andreas even implemented a Rich Text Editor that had features like table editing that were rare in 2003 - and all that using the JavaScript 2 subset and his compiler.
One Thing Leads to Another
This first shot of the compiler was simply named jscc (JavaScript Class Compiler), and the source file extension was .jsc, so the language was also called JSC. Soon, more tools and libraries followed: Holger Tewis implemented a javadoc-counterpart, naturally called jscdoc, that tricked Java doclets into generating HTML API documentation for JSC programs. I did a port of JsUnit to JSC, needless to say called JscUnit, in about two days (and of course its self-tests were green then). We experimented with writing IDE plug-ins for the unfinished language. Having learned from the WebEditor, we implemented another UI framework, which we so far only used for popping up a context menu on preview Web pages to let the user invoke content-related actions (sounds a bit like breaking a fly on the wheel, doesn't it?).
I loved working with JSC, but to be honest it had some serious drawbacks:
  • There was no real IDE support.
  • No type checker, types are just remarks used for API documentation.
  • Bugs and request were not fairly tracked and never really tackled.
  • No class dependencies or automatic class loading: each class file had to be listed, and even in the right sequence before all its depending classes.
Worst, there was almost no documentation and the build process was tied to our primary product. JSC was suffering the typical fate of an internal development tool that had no (paying) customer, although a lot of value or at least potential. At that time, our company grew, and even other projects in the very same company did not use JSC for their JavaScript development. I felt there was something going wrong.
Time for a Change
CoreMedia went through some changes, which opened up new possiblities. We evolved to a learning company, where all members are given time for "peer group" work, i.e. work in small groups of people who pursue a common goal and handle a topic of individual choice. Luckily, I found fellows who were, like me, interested in reanimating JSC. The new spirit and transparency of our company was the perfect breeding ground for the idea to release JSC to the public, as Open Source. It was the perfect candidate:
  • En vogue topic.
  • Focussed, general purpose tool.
  • Not too big to handle.
  • Nothing anyone would want to sell: development tools are hard to sell, anyway, and these are not our company's focus.
  • We had to document and clean up, anyway, so hardly any additional cost.
The main goal of the peer group, consisting of Andreas Gawecki, Olaf Kummer, Matthias Buse, Mark Michaelis, and myself, was to prepare a decision memo convincing CoreMedia to spend resources on our first real Open Source project.
At that time, JavaScript really enjoyed a renaissance and AJAX was a hype. Interesting new articles about JavaScript mushroomed and inspired me to create a new JSC runtime supporting features like private and protected members, lazy class initializing, and even better readable compiled code. After several iterations, we decided to skip "really private" and protected members for performance reasons, but nevertheless found a solution that does the job to avoid unexpected name clashes. We also updated the language syntax to the latest standard proposals. Suddenly, IDEs started to support ActionScript 3, which is also an ECMAScript 4 language, so we could benefit from that, too.
We were overcoming most disadvantages of JSC. In other words, we had even more to offer than what we had used successfully as an internal tool for years.
The peer-group found that JSC needed a proper name: it should be Web-2.0-ish, cool, foolish, looney (add your favorite adjectives containing oo-s), and artificial, so that we could still grab the domain, the blogger subdomain (sic!), the user ID for community xyz, and so on. All our peer groups have rather silly animal names, and we were True Kangaroo. Replacing the first letter by a JavaScript J and keeping the popular oo-s finally led to Jangaroo. I guess the cool name must have been the critical factor when CoreMedia decided to give us a go.
Walking the Talk
In the execution phase, Masiar Bostanipoor (Web site) and Dennis Homann (build infrastructure, tutorial) joined the team. Andreas and Olaf rounded off compiler (now coherently called jooc) and runtime (where I helped a bit). Olaf wrote most of the language and compiler documentation. Kudos to numerous CoreMedians who helped us with the project, in no particular order and surely not complete: Thomas Stegmann, Gunnar Klauberg, Uli Henningsen, Tobias Baier, Jan Brauer, Carsten Böttcher, Stefanie Wegener, and Christian Pesch.
We used the project time CoreMedia granted us mainly to come up with a stable infrastructure and an appealing presentation (at least we hope so), not to create more tools. We got the domain jangaroo.net, created a Web home for Jangaroo, started this blog, wrote documentation, set up a proper build process involving a Maven repository, cared about license issues (German license jurisdiction is a nightmare!), and so on. Of course, some infrastructure is still missing, most importantly a forum and a bug tracker -- we are working on that, please bear with us! Until then, please comment in this blog for public discussions, and e-mail for direct communication. We also have a twitter user Jangaroo, feel free to follow if you like! Many of the tools and libraries mentioned here may follow jooc, so stay tuned!
I hope you have enjoyed hearing about Jangaroo's first life as JSC and its rebirth as an Open Source project. What I described is just the way I personally remember and feel about things. In the future, be prepared to find a bit more technological facts in my postings...