Author Archives: Michael

Anorm and Duplicate Column Names

A bit of a technical post, just in case someone else hits this problem, e.g. while working through Play for Scala book (which I heartily recommend). That’s how I stumbled on this issue.

Anorm is an interesting alternative to ORMs. This is pretty much going against the flow of “let us write your SQL for you”, so they have some explainin’ to do. Anorm sticks to native SQL, and make it easier to parse result rows into objects. One way of making it easy are SQL row parser combinators.

The idea is this: parsers take column values from an SQL result row, and return a Scala object. Combining the parsers yields tuples, that can later be parsed into Scala objects (normally, some case class).

So that’s all fine and dandy, but the code below produces unwanted results:


val prodsAndItems =  SQL("""
    SELECT p.*, s.* FROM products p INNER JOIN stock_items s 
    ON p.id = s.product_id
    ORDER BY p.id
    """)

  val productParser: RowParser[Product] = {
    long("id")~long("ean")~str("name")~str("description") map {
      case id~ean~name~description => Product(id, ean, name, description)
    }
  }
  
  val productsParser: ResultSetParser[List[Product]] = { 
    productParser *
  }

  val stockItemParser: RowParser[StockItem] = {
    long("id")~long("product_id")~long("warehouse_id")~long("quantity") map {
      case id~prodId~wareHouseId~quantity => StockItem(id,prodId,wareHouseId,quantity)
    }
  }

  val stockItemsParser: ResultSetParser[List[StockItem]] = {
    stockItemParser*
  }
  
  val productStockItemParser: RowParser[(Product, StockItem)] = {
    productParser ~ stockItemParser map (flatten)
  }

  def selectAllProductsAndItems: Map[Product,Seq[StockItem]] = DB.withConnection{ implicit c =>
    val res:List[(Product, StockItem)] = prodsAndItems.as( productStockItemParser * )
    res.groupBy( _._1 ).mapValues( _.map(_._2) )
  }

This is a classic case of products, warehouses and stock items (product quantities that are stocked in warehouses). You would expect the call to selectAllProductsAndItems to return a map object mapping products to their stock items. Instead, it returns a set of new products that don’t exist in the database, and maps a single stock item to each one.

Now, I love creating new products. I just don’t want my data access layer to do it for me. Enough of this “machines replacing people” zeitgeist.

Why was Anorm doing this? (spoiler alert from here on)
Both stock_items and products have a column called “id”. The basic parsers (defined in stockItemParser and productParser) call the columns by their names, and so get ambiguous results.

The solution turns out to be pretty simple (as always) – reference the column by its full name: “products.id” instead of “id”. Over to you, Peg:

P.S commit link on github

A Playful Eye for the Java EE Guy

Here’s a link for a presentation I gave at the IQSS (an awesome place I’m lucky to be part of). It’s yet another “intro to Play” talk, geared towards people with background in Java EE.

There’s a twist, though – the presentation is done in Play.

It’s available at github, and there’s also a PDF transcript available at the IQSS site, in case you don’t want to install the software needed for it to run.

As always, big thanks to IQSS’ data science team for making this possible.

Lessons learned by a frugal hacktivist

There’s a compulsory biometric database being built in Israel. It’s bad, but it’s not the point of this post. I’m taking part in some civilian effort to stop it, and part of this is having a website. So we built one. For free. Here it is. And here is what I learned, divided to non-technical and rather technical points.

Non-technical:

  • We wanted a good looking, fast website. As the government employes paid commenters, we didn’t want any talkback options.
  • We have some people that can build websites (me, but not just me). However, we’re all busy.
  • The website had to be updated periodically, but not too often.
  • We are a voluntary organization, and we don’t have any money.

While we were aware of standard options, such as opening a wordpress blog or a Facebook page, we decided to go with a static web site, hosted as a github project page. The motivation was that this would generate a very fast website, and we could get all the design flexibility we need (canned solutions are not that flexible, really).

However, keep in mind that github project pages come with some strings attached. The website has to be open-source, or pay for the hosting, and no server-side processing can occur. This means any updates to the site require some technical skills.

For hacktivism, the open source string may be a real issue. If you spend a lot of time creating a web site, and you don’t want the other side to just fork it and create their own version. You may want to keep your source closed. We didn’t have this issue, as the biometric database already has web presence, as well as TV commercials, posters, etc.

So far, which is not very far really, the choice for static website seems good. Here are the technicalities. The main goal was to get something done fast, but at a high quality.

Site generator

We went with harp.js. While jekyll looks like a more natural choice for github, it is not as flexible as harp. Jekyll assume you are writing a blog. Harp has a more holistic approach, and also supports lesscss (and many other languages). I already had experience with jekyll (earned while doing this site) but decided to try harp and have jekyll as a “plan b”. All in all this worked.

Front end

Bootstrap3. You can’t beat the king.

Content Generation

Here’s where Jekyll is better than Harp – we wanted to generate a list of links, display a list of them and give each one a page. Jekyll would have done this out of the box, if we would have treated each link as a post. Harp requires having a file for each page in the site, which meant we had to maintain the json map and a set of files, and make sure each key in the map has a file named after it. Not fun and error prone. Well, bash to the rescue:


cat _data.json | jq keys | grep \\w | cut -d\" -f2 | xargs -I{} touch {}.md

This line ensures that each key has a file with corresponding name. If the file already exists, it is not overridden, which is important if someone wanted to add additional data to the to a link’s page.

Issues

  • Contact forms require a back-end to process the input, and static sites don’t have any back-end. There are ways of doing them, e.g. using google forms. We address this soon, but currently we’re happy to use github issues, which also works. Also, you can use a mailto link, but you’ll be facing another set of issues there.
  • As always, collaborating using github is a pleasure.
  • When other people fork your site, they shouldn’t call their branch “gh-pages” – it makes the CNAME files collide. Only have a single branch with this name.

Conclusion

When needing a website that does not take data from users, static websites may be a good choice. If you can keep your source open, hosting them as github project pages is a fast solution that’s free and easy, as long as you have people with technical skills available. If you have some cause you’re trying to push, you can fork us on gihub.

Oh, and if you want to learn about the biometric data base in Israel and you don’t read Hebrew – we have a site for this.

Draw More, Work Less

Below are the slides for a talk I gave at IQSS about software tools that generate diagrams. Two examples are shown – GraphViz and PlantUML.
I was hoping to show that focusing on the semantic level of the diagram (e.g. structure, relations) and letting the computer decide on the actual layout can make life easier. I know I use these tool quite often.

Anyway, the slides are below. The files for most diagrams can be found here.

Geek Dad #12FFA: Cowsay

The summer vacation is upon us, and with it, the need for kids to not be bored (or is it the need for the parents to have kids that are not bored?). Coloring pictures is a nice pastime, and you can always find a lot of coloring images by going to google images and limiting the search to line art only.

But, does it educate your kids to be better geeks? Does it tickle their imagination? Not really, no. Well, what is a geek dad to do?

 _________________________ 
< Lets color ascii art!!  >
 ------------------------- 
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

Cowsay is a lovely little program written by Tony Monroe. It allows you to quickly create ascii art of cows saying things. And it’s not limited to cows, there is also tux, a bunny, a koala and various other images, which can be listed with cowsay --list. Do not pipe the --list result to a loop and create all the images, as some are NSFW, (or, NSFK, in this case).
Also, it’s not limited to saying things either: there’s also cowthink.

And thus:
10 Kid: Dad, can I color a cow?
20 Dad: cowsay please color me\! | lpr
30 Kid: <coloring happily>
40 GOTO 10

cowsay is available for most platforms. On linux it’s available through the package manager. On OS X, it’s available through brew.

Invitation to Scala

Here is the material for the talk I gave at the Harvard IQSS’s “Tech talk” lecture series. This is a gentle introduction to the Scala language and to functional programming. As this lecture had quite a few demonstrations, you need to follow the lecture notes and try the commands shown there on a scala REPL in order to get the most of it.

Very big “Thank You” for all the IQSS staff that attended – it was great fun*!

Update:

SlideShare featured award

SlideShare’s featured award

This presentation just got featured by SlideShare’s editorial team. Thanks, guys!

 

 

* well, for me, at least.