Category Archives: programming

NetBeans on GraalVM

Leave a reply

Last Devoxx I attended an interesting talk about GraalVM, an alternative JVM from Oracle. Among other features, it offers better performance in many cases. So, why not try to run NetBeans on it? The setup is quite easy, and works on my machine (MacBook Pro, running Mojave).

    1. Download GraalVM. I use the community edition. Note that, on a Mac, you may also need to install xquartz. I had it already, and it doesn’t run, but the important part is that it makes some required libraries available.
    2. Unzip and put somewhere. I put it with the rest of the JVMs, at /Library/Java/JavaVirtualMachines/
    3. Get to NetBeans’ configuration file. On a Mac, this is done by right-clicking the application icon, selecting “show package contents”, and navigating to Contents/Resources/NetBeans/etc/netbeans.conf
    4. Open the conf file, and make it point to the Home directory of Graal, by adding the below line:
      netbeans_jdkhome="/Library/Java/JavaVirtualMachines/graalvm-ce-1.0.0-rc9/Contents/Home"
    5. Save, close, relaunch NetBeans.

DONE:

The Curious Case of the Maze Walker and the Cow’s Tail

Leave a reply

The Backslash Incident

I guess it’s never a good time to find out your code has bugs. But there are exceptionally bad times to do so. A middle of a live demo is one such example. Immediately after said demo, when you’ve invited the attendees to take your code for a spin, is not too good either.

Case in point: I had the honor of presenting BPjs at a seminar at Harvard School of Engineering and Applied Sciences (SEAS). BPjs is a tool suite for running and verifying programs written in JavaScript, using the Behavioral Programming programming paradigm. In short (very very very short): It’s a paradigm that focuses on how the system should behave, rather than on how its implemented (e.g. objects, functions, or procedures). Because behavioral programs (“b-programs” for short) can be verified against a set of formal specification, BPjs allows writing verified systems in JavaScript. Which is, admittedly, weird. But in a good way.

As part of the above mentioned demo, I’ve created VisualRunningExamples, an application that allows playing with a behavioral program for a maze walker. Users can load and modify the walker’s behavioral model and formal specification, and then run or verify it, using a friendly GUI. It also serves as an example of how to build a program around a BPjs model.

VisualRunningExamples – Maze solution obtained by formal verification of the maze and walker model.

One fun part of VisualRunningExample’s UI is the selection of the maze the walker would roam in. My favorite maze has a cow (created using cowsay, of course). During the presentation I let the walker walk around the cow a bit, and I thought I saw it demonstrating a bad behavior – it seemed like it was going through a wall. I’m not an expert in maze walking, but I assume maze walkers should not do that.

Immediately after the presentation, I looked into this in more depth. It seemed like the walker was moving diagonally. Which it should never do, since no part of the program requests such a move. It ostensibly went from point X to point Y:

_______
<BP(moo) >     @@@@@@@@@   t
-------
        \Y  ^__^    @@@@@@@@
@ @@@@@ X\  (oo)_______
s           (__)~~~~~~~)\/\
@ @@@@@        ||----w~|
               ||     ||

Was this really happening? Using VisualRunningExamples, I’ve added the following b-threads, and verified the program against it. The idea behind this code is that whenever the maze walker gets into a cell, a set of b-threads that look for diagonal entries is spawned. These b-threads blow the program up if they detect a diagonal mode. Pretty neat pattern, I think (reminiscent of LSC’s symbolic charts used to forbid events):

bp.registerBThread("no-diagonal", function(){
    while (true) {
      var crd=getCoords(bp.sync({waitFor:anyEntrance}).name);
    	noEntryNextMove(crd.col-1, crd.row-1);
    	noEntryNextMove(crd.col-1, crd.row+1);
    	noEntryNextMove(crd.col+1, crd.row-1);
    	noEntryNextMove(crd.col+1, crd.row-+1);
    }
});
 
function noEntryNextMove(col, row){
    bp.registerBThread("nenv(" + col + "," + row + ")", function(){
        var evt = bp.sync({waitFor:anyEntrance});
        var newCoords = getCoords(evt.name);
        bp.ASSERT( newCoords.col !== col || newCoords.row !== row, "Diagonal Movement detected");
    });
}

Verification did not find anything wrong, yet that parallel movement is happening, live, in front of my eyes. So, verification is broken too. Highly embarrassing, given that I just presented the tools at Harvard SEAS’ SPYS group. Time to do some runtime verification.

Runtime verification really boils down to adding these b-threads to the b-program (“model”), and running it. Easily done, and we know the runtime works. So, yay, next time mr. maze walker wants to do a diagonal jump, one of these b-threads will catch it red-handed. With an evil grin, I click the Run button.

Nothing. The walker does diagonal jumps like a boss. But only between points X and Y. Luckily, VisualRunningExamples’s UI is fun to play with, so I change the \ to *.

No more parallel jumps, which never really happened. Good news: BPjs’ verification works. As does its runtime.

What Went Wrong

What really happened was that when creating the source Javascript b-program, the Java layer was copying the maze into an array of strings, verbatim. This meant that \ was really an escaped space, which translates to a single space in JavaScript:

jjs$ var s = "\ "
jjs$ "|" + s + "|"
| |
jjs$

The Java UI, on the other hand, displayed the backslash in the maze monitor table. In short: the JavaScript b-program saw a space there, whereas Java saw a backslash. The ostensible diagonal jumps were just the maze walker going from X to Y through an empty cell, that was marked by the Java layer as a \ cell.

Lessons Learned

  • Need to be careful when constructing code from user input (OK, that’s not a very new lesson, more of a reminder really. But still).
  • If you have a suspicion something is wrong in a b-program, BPjs makes it easy to verify.
  • Tooling and UI that make the b-program easy to play with are very important. All in all, I went from having a suspicion to verifying the code in about an hour of casual coding on the couches at an empty student center somewhere in the Harvard campus.
  • Since BPjs uses the same structures for verification and for modeling/execution (namely, b-threads written in JavaScript), it is trivial to share code between the b-program and its formal specification. That’s important, e.g., when one wants to extract data from strings.

My experience with this whole BPjs as a verification platform, is that it enables “casual verification” – it is very easy to create and run verification code, all the tools are there already, and you’re probably in the mindset as well. Compared to re-creating the system by modeling it in, say, PROMELA, it’s much more casual. Not to mention translation errors between model and actual software.

VisualRunningExamples now has a shiny new cow that does not contain any backslashes. I kept the old cow too, but it’s now called “cow (bad)”. Escaping the maze text properly is followed by issue #1.

Some remarks on The Repeated Deaths of OOP

Leave a reply

I like people that heartily dislike a programing language paradigm. First, this means they think about how programming is done, rather than about programs only. Second, as far as hate go, hating a paradigm is much better than, say, hating people for being born with the wrong skin color, faith or sexual preference. You can trust me on this one – I’m from the middle east.

But if one is to hate a programing language paradigm, one should do it in an informed way. Case in point: @Loup’s recent post, “the repeated deaths of OOP” (@Greptilian sent me a link to this post not long ago, which was quite nice of him). While this post has some interesting ideas, presented with an envy-inducing eloquence, I’d like to constructively disagree on a few issues. Before I do, let me stress that some of the mistakes in the post are very common, which might have misinformed the fact-checking stage of it. This is the reason I write a post of my own, rather than just an internal answer email. And thanks @Greptilian for suggesting that I write one. Quite nice of you.

OK. Let’s start. (Or, to quote Pascal language, BEGIN)

The post begins by stating that “Current mainstream programming languages all support OOP — except maybe C, and it is no longer that mainstream”. There’s some echo chamber effect in the PL blogosphere, and we should be aware of it. C is very mainstream. It’s currently #2 in the TIOBE index. It has more than double the rating of language #3 – C++. It is true, however, that it is not mainstream while writing web applications. I guess it’s just harder to blog when you have a daytime job (note to self: revisit this statement once I get a daytime job).

The post continues by stating that the it is unclear what “OOP” means, and then describes the eight “fundamental building blocks”, constructed from a survey done by Deborah J. Armstrong. That’s a classic straw-man argument. Why would a survey of a population that can’t agree on anything end up in a definition coherent enough to base a very successful programming paradigm on?

When I learned OOP – Based on Timothy Budd’s “An Introduction to Object Oriented Programming” (current edition), we were presented with four blocks that make sense, instead of eight that don’t. It was argued that an object is a 4-tuple:

  • structure – what the object looks like in memory
  • state – the values the object currently holds
  • protocol – the set of messages a given object can “understand”
  • behavior – what said object does when it gets a message from its protocol

Note the lack of classes and inheritance, which are a way of code reuse within an object oriented context. Abstraction, encapsulation and polymorphism naturally flow from these four principles. Budd’s book is pretty popular and I assume I’m not the only one that remembers his presentation on the subject (and the cute Platypus on the cover). At any event, when looking for a definition, it’s better to go to the books rather than the allegedly confused public. I mean, imagine how monads would look if we applied this approach to them (here’s Gilad Bracha on this subject).

“Then Java came. It was presented as an “easier C++”. Same syntax, similar abstractions, garbage collection… At that point Smalltalk was hardly used any more, and “OOP” basically meant Java and C++.”

One does not simply conflate C++’s version of OOP – multiple inheritance, supermarket principle, no GC – with Java’s. Java has a direct influence from SmallTalk and Objective-C, and the original staff hated C++. And played hockey with Scott McNealy. Here’s Patrick Naughton, a member of the original team that worked on Java (“Oak”, at the time):

… I thought Objective-C was the coolest thing since sliced bread, and I hated C++.
So, naturally when I stayed to start the (eventually) Java project, Obj-C
had a big influence. James Gosling, being much older than I was, he had
lots of experience with SmallTalk and Simula68, which we also borrowed
from liberally.

source

It is important to stress that the garbage collection is key in an object-oriented language, rather than a utility that makes life easier. With no garbage collector, any object creation has to be matched with precisely one object destruction (and of the same object as well). This means objects are not “first class citizens” in the language – the programmer cannot create anonymous ones. Java has to have garbage collection, because of the exact same reasons LISP has to have it when it allows programmers to create anonymous functions.

C++, on the other hand, is governed by the “supermarket principle” – the language accommodates many programming styles (including OOP, modular, procedural, and to some extent, functional. See Bjarne Stroustrup’s introduction to “The C++ Programming Language“), but you only pay for what you use. Garbage collection runs against this principle, as it has a heavy price tag.

Onward, to the second death.

“suddenly we hardly needed Object any more.”

Dude, you realize where toString(), hashcode() equals() and come from, right?

Onward – I’m skipping the “third death”, as I disagree that the grim reaper operates on a programming paradigm by adding more languages that support it.

“Lambdas are everywhere…”

A lambda is an object with a single method. Since it’s a single method, it does not need to have a name. When the need to name it rises, it is commonly called “apply”. In fact, OOP is a generalization of FP – FP only allows objects with a single method; OOP allows many methods. FP objects – “lambdas” or, more accurately, “closures” – can even have state: captured values from the internal calculations that created said closure. That state may even be mutable, e.g when it contains box values. So, yay FP, but FP and OOP are not mutually exclusive. You can do both. At the same time. In the same language. We’ll get to that.

Yet from a popular perspective, first class functions are not OO at all: they come from functional programming. Yet they managed to invade every single mainstream OOP language.

They didn’t invade. They where there at the beginning. SmallTalk had them. And again, FP was one of the two main influences on SmallTalk. Here’s Alan Kay, reflecting on his work (from ACM Queue)

We had two ideas, really. One of them we got from *Lisp*: late binding. The other one was the idea of objects.
source

By the way – this is a really good read. Long – but good.

“There is an easier way: don’t mutate state. Always construct new stuff, never throw away anything. Then let your smart pointers or your garbage collector take care of unreachable references.”

And I assume the garbage collector does everything in O(1), and was sent by to us from the heavens? More to the point – I think we can turn the hubris knob down ever so slightly, and construct the following argument from the above quote (at the risk of being accused of constructing a straw man myself):

“Application programming can be done with no mutable state. Code that mutates state is inherently low-level, and can be left to system programmers, the same way we treat assembly code today”

Now that’s a statement I see the merit of. I don’t agree, though – garbage collection has a real cost, and sometimes it does make sense to minimize it. There are programming models that accommodate mutability, while being safe and simple even in concurrent contexts. The actor model is probably the most popular, but there are others (shameless plug – my ongoing PhD thesis works with behavioral programming, developed by Harel, Marron and Weiss, which also offers that, in its very experimental way).

I agree that Entity-component-systems (ECS) is an interesting approach. Also, indeed, relational databases and OOP don’t mesh well, once OOP starts to use stateful inheritance. I guess this impedance contributes to the growing popularity of NoSQL databases (should ECS support that?).

Functional Reactive Programming is very interesting too. But do note that, at least in the web context (Elm, React) it is used to change the DOM, which is one of the poster boys of OOP (inheritance, composite design pattern, the works). This may actually be a good case for the aforementioned statement, where higher level programming is done in FP, while the underlying layer is done in OOP.

So is OOP not dead yet?

Prophecy, as the old proverb goes, was given to the fools. Well, I’ve been called a fool numerous times, and thus am happy to offer my opinion of the future of OOP: It’s not going anywhere. It will mutates (or rather, has mutated) to include FP. We already see this in Scala and Javascript – these languages model functions as objects, and take it from there. Martin Odersky calls this “postfunctional programming“. The term seems to be taking off. I think in the forthcoming years we’ll see OOP gobbling up logic programming as well. Some sort of merging between Prolog and Java/Scala. LP has been quiet for too long now.

P.S.

Oh, and regarding the game industry leaving OOP – I have no idea if the game industry is indeed leaving OOP. Some hard numbers are in order. But even if that’s true, and even if we accept that “games are simulations” – they are simulations that have to be done fast enough to be rendered, smoke, explosions, blood, gore, background music and all. In real time. And at least partially on GPUs, rather than CPUs. I’m not sure how strong a classifier the game industry is, given these traits. But hey, interesting development nonetheless (if true. Braaaaains! I mean, Nuuuuuuuumbers!).

Print a filled rectangle in Scala

1 Reply

@puppies , over at StackOverflow, had a question about how to print a filled rectangle in Scala. I guess s/he has an imperative background, because a) we all do, and b) s/he made an attempt to do this using loops. But in Scala, you can also do this with mapping and list concatenation.

Here’s my initial take (all code is printed on the scala REPL):


val ec="@"
val cc="X"
val cols=8
val rows=5

((ec*(cols+2)) +: Range(0,rows).map( _ => ec+cc*cols+ec) :+ (ec*(cols+2)) ).foreach( println(_) )

Which results in

@@@@@@@@@@
@XXXXXXXX@
@XXXXXXXX@
@XXXXXXXX@
@XXXXXXXX@
@XXXXXXXX@
@@@@@@@@@@

So then, Ben Reich suggested using until in stead of Range. Also, we can replace the side-effect in foreach with mkString:


((ec*(cols+2)) +: (0 until rows).map( _ => ec+cc*cols+ec) :+ (ec*(cols+2)) ).mkString("\n")

Can you print this rectangle using a shorter snippet? Post your version in the comments.

Uninheritance Using Undefined in Javascript

Leave a reply

Javascript has a value called undefined. It’s an interesting little fella, which is the value of everything that is not defined. One common use of it is trolling your javascript console:


> defined
ReferenceError: defined is not defined
> var o = {}
undefined
> o.defined == undefined
true

Ha ha, silly javascript, not can’t even tell defined from it’s opposite. Except that it can. And it’s also very elegant.

See, unlike other dynamic languages that requires programs to test the symbol table for symbol presence (e.g. ruby’s defined?() method), javascript does not have to test the table, since every valid expression (such as a variable reference) has a value. It just may be that this is undefined.

Common inheritance models allows programmers to refine behavior (when overriding methods) or to extend them (when adding new methods). But javascript, with it’s undefined and its prototype-based inheritance, has a new type of inheritance – inheritance with method deletion.

Suppose we want to model vehicles. Vehicles can go, er, I mean go(), and they also have a number of doors. Easy enough, here’s the vehicle object, and a car and a truck.


var vehicle = Object.create({});
vehicle.go = function() { return this.name + " is going"; };
vehicle.doors = function() { return this.doorCount; };
// make a truck
var truck = Object.create( vehicle );
truck.doorCount = 2;
truck.name="Truck";
// make a car
var car = Object.create( vehicle );
car.doorCount = 5;
car.name="Car";

This works fine (all examples here were executed using Firefox Developer Edition):

> var aCar = Object.create(car);
undefined
> aCar.name="MyLittlePony";
"MyLittlePony"
> aCar.go()
"MyLittlePony is going"

Nothing new so far. But how do you go about modeling a broken car? When using languages that use class-based inheritance, such as Java, one might result to throwing an IllegalStateException, since brokenCar is a vehicle and thus must have a go() method. In javascript, however, there’s another option:


> var brokenCar = Object.create( car )
undefined
> brokenCar.go = undefined;

And now:

> brokenCar.doors()
5

But

> brokenCar.go()
TypeError: brokenCar.go is not a function

In other words, brokenCar is a car that can’t go, just like in real life, where a broken car is a car that can’t go.

Achievement “modeling bliss” unlocked.

Anorm and Duplicate Column Names

Leave a reply

A bit of a technical post, just in case someone else hits this problem, e.g. while working through Play for Scala book (which I heartily recommend). That’s how I stumbled on this issue.

Anorm is an interesting alternative to ORMs. This is pretty much going against the flow of “let us write your SQL for you”, so they have some explainin’ to do. Anorm sticks to native SQL, and make it easier to parse result rows into objects. One way of making it easy are SQL row parser combinators.

The idea is this: parsers take column values from an SQL result row, and return a Scala object. Combining the parsers yields tuples, that can later be parsed into Scala objects (normally, some case class).

So that’s all fine and dandy, but the code below produces unwanted results:


val prodsAndItems =  SQL("""
    SELECT p.*, s.* FROM products p INNER JOIN stock_items s 
    ON p.id = s.product_id
    ORDER BY p.id
    """)

  val productParser: RowParser[Product] = {
    long("id")~long("ean")~str("name")~str("description") map {
      case id~ean~name~description => Product(id, ean, name, description)
    }
  }
  
  val productsParser: ResultSetParser[List[Product]] = { 
    productParser *
  }

  val stockItemParser: RowParser[StockItem] = {
    long("id")~long("product_id")~long("warehouse_id")~long("quantity") map {
      case id~prodId~wareHouseId~quantity => StockItem(id,prodId,wareHouseId,quantity)
    }
  }

  val stockItemsParser: ResultSetParser[List[StockItem]] = {
    stockItemParser*
  }
  
  val productStockItemParser: RowParser[(Product, StockItem)] = {
    productParser ~ stockItemParser map (flatten)
  }

  def selectAllProductsAndItems: Map[Product,Seq[StockItem]] = DB.withConnection{ implicit c =>
    val res:List[(Product, StockItem)] = prodsAndItems.as( productStockItemParser * )
    res.groupBy( _._1 ).mapValues( _.map(_._2) )
  }

This is a classic case of products, warehouses and stock items (product quantities that are stocked in warehouses). You would expect the call to selectAllProductsAndItems to return a map object mapping products to their stock items. Instead, it returns a set of new products that don’t exist in the database, and maps a single stock item to each one.

Now, I love creating new products. I just don’t want my data access layer to do it for me. Enough of this “machines replacing people” zeitgeist.

Why was Anorm doing this? (spoiler alert from here on)
Both stock_items and products have a column called “id”. The basic parsers (defined in stockItemParser and productParser) call the columns by their names, and so get ambiguous results.

The solution turns out to be pretty simple (as always) – reference the column by its full name: “products.id” instead of “id”. Over to you, Peg:

P.S commit link on github

A Playful Eye for the Java EE Guy

2 Replies

Here’s a link for a presentation I gave at the IQSS (an awesome place I’m lucky to be part of). It’s yet another “intro to Play” talk, geared towards people with background in Java EE.

There’s a twist, though – the presentation is done in Play.

It’s available at github, and there’s also a PDF transcript available at the IQSS site, in case you don’t want to install the software needed for it to run.

As always, big thanks to IQSS’ data science team for making this possible.